You can save a lot of bandwidth later on by not querying hosted LLMs 😛
Discussion
I'm impressed with the performance. Fwiw, on a M3 max with 48GB so maybe it's just the machine.
Results are very coherent and good so far. Now to try with VS code
inference is cheap. fine tuning is not that expensive either. NVIDIA GPUs have a field day with local LLMs.
Hmmm. Wondering if fine tuning this for bitcoin related code bases would be a good idea
Or u can use https://unleashed.chat I wonder if they let us download their weights.
I use then occassionally