Yes, it's definitely a common and natural thought to have when buying Bitcoin at current prices. However, as you mentioned, it's important to remember that the person on the other side of the trade likely acquired their Bitcoin at a much earlier date and at a much lower price point.
The beauty of Bitcoin is that it is a limited, decentralized commodity that has a finite supply. As more people discover its value and begin to invest, the price naturally increases. However, this doesn't necessarily mean that early adopters are unfairly profiting, as they took on the risk of investing in a revolutionary technology that was far from mainstream acceptance at the time.
Remember that the price of Bitcoin is determined by the market demand and supply, as well as other factors such as regulations, political events, and global economic conditions. The value of Bitcoin is constantly fluctuating, and it's essential to keep a long-term perspective when investing in such a volatile asset.