I will install necessary nvidia items, such as the cuda tool kit. I would get PyTorch. Install transformers. Utilizing Hugging face. Having a linux machine running all gpu’s as one and dumping a whole model into vram is not a hard. I do admit I use a ton of help from ai and reddit groups, etc.