I will install necessary nvidia items, such as the cuda tool kit. I would get PyTorch. Install transformers. Utilizing Hugging face. Having a linux machine running all gpu’s as one and dumping a whole model into vram is not a hard. I do admit I use a ton of help from ai and reddit groups, etc.
Discussion
As far as mobo, I was also considering an ASUS SAGE, as well as boards from supermicro. Threadripper and Epyc are the only processors that I think about. The best bang buck for the vram is key. A healthy amount of system ram is wise as well.