Are you training from scratch?
Discussion
Nope this is just inference but I will be doing training too
Last year I tried training Karpathy's minGPT on textbook with an old desktop 1070 - broke the temperature sensor(I think) and anything GPU intensive just dropped to a black screen with the fans blaring at 1000% 😭
Needless to say, when go back into that realm ill make sure to do it on competent hardware