that's one I don't hear often. it does have the memory to run 14b from what I can tell, so I'd say try that.
you are right, 14b works decently, I should switch to vllm from ollama and that should speed it up even more
Please Login to reply.
No replies yet.
