Cats are picky. People says cats are low maintenance but cats get sad or bored in small places. They need lots of space and fun.
I have cats visit me all the time from the neighborhood. They show up everyday and I feed them. I'm sure someone is feeding you cats and it is ok. Maybe it will show up someday.