I have yet to see a phone run an LLM at a decent rate. anyone using an LLM on a phone is either paying someone else, or running their own on more powerful hardware.
Discussion
not to mention it’s only able to handle severely quantized models.
I have yet to see a phone run an LLM at a decent rate. anyone using an LLM on a phone is either paying someone else, or running their own on more powerful hardware.
not to mention it’s only able to handle severely quantized models.