Colleague tipped me off that now you can run distilled LLMs on your desktop CPU!

https://ollama.com/

https://github.com/gergap/vim-ollama/