I think its going to come down to AMDs experience with chiplets and bus tech (their infinity fabric). These models are too big to run on any single chip; the bus speed is proving to be a serious bottle neck because its not something the industry has had to deal with yet. The last few generations of AI tech has had major architectural changes as the industry feels out the technical requirements, so its fair to say things will change rapidly, but I think scaling across multiple chips/hosts/racks is going to take center stage.