I think you are correct
I think the pioneering work of virology was shoddy science (eg physical damage causing plant illness in Tobacco Mosaic Virus research)
I think the definition of virus has changed several times without finding a physical thing to nail down what exactly it is
I think strict germ theory a la Koch's postulates (in which a pathogen is both necessary and sufficient to cause illness) doesn't hold water; this is shown by Koch's own work on cholera as well as other experiments such as the attempts to spread 'Spanish' influenza in a controlled setting
All that said, I think contagion seems to be a real phenomenon (although strictly speaking what we really observe is clusters or sequences of illness) and this requires something like a pathogen to explain
I am inclined to think that some "viruses" are psychosomatic, environmental toxins, etc. but probably some are just smaller bacteria
On the other hand, the PCR-based method of "identifying" viruses introduces concern about whether you're really identifying the thing you think you're identifying