That's the problem. It ONLY doubled. The S&P gained 2.7x without reapplying the dividends. And I am willing to bet it gave you more headaches than what a line in your Fidelity Account would do.
Discussion
Most people aren’t 5x leveraged on their S&P holdings though (which is what a typical mortgage does to your equity check).
You have to calculate returns to equity (not just headline home appreciation).
the other aspect I suppose is what is the cost of rent vs mortgage? when we bought our place the mortgage + fees were actually $200 less each month than rent, so kinda no brain needed in that situation