In the span where I gained 50 follows, other people gained 5000 or 50000 follows. You're saying that only 1 out of 1k or 10k newbies are interested in what I post?

Unlikely.