Of course consumer hardware is advancing too. Supposedly with a couple of Nvidia’s upcoming Digits computers connected together you’ll be able to run a 400B parameter model locally.