Yes you can avoid breakfast if you want, but there is a feeling that it would be better to take some energy. Not that you really feel hungry. For example if you don't have work to do, you can just relax
Discussion
I think this makes sense if you have an inefficient (read: fast) digestive system. Most people tend to efficient digestion (as their ancestors survived bad harvests), which means that digesting food makes them sluggish and they are often still digesting their dinner, when they awake.
Might also be an ethnic difference, or something linked to body type.