If you have a needle in a haystack, having millions of other people also trying to find a needle in their haystack doesn't make your search any more or less difficult. The difference is that if someone finds it before you, you just get a new needle in a new haystack to search for. If the difficulty adjusts down because of hash power decreasing, sure, your odds have marginally increased. But then your odds are fixed for the next two-ish weeks depending on your hash power. Mining with a RaspPI still gives you the same, near zero odds regardless of what hash rate does over the next few years, barring total collapse.