Nostr Web Client

so, i finally have been testing the local LLM using LM studio, which seems to be the only tool for this.

damn things have a mind of their own, literally prompts have to start with DO NOT DO THIS DO NOT DO THAT before you start describing what you want it to do. hilarious.

even still they don't really quite follow the instructions.

using quen2.5-code-14b. seems not much dumber than claude or gpt-4o

probably will experiment with other models, plenty of space for them, it's just a question of time.

i tried using a deepseek model and it spat out a whole complicated and weird flow of thoughts about stuff that were all like the inner dialogue of a nervous slave.

Reply to this note

Please Login to reply.

Discussion

Davi Pinheiro 5mo ago

Try ollama

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ 5mo ago

i installed it but it doesn't seem to have a GUI, it told me something about how to run them but i didn't read it lol. then it's like, ok now what?

come to think of it, i think i have Jan installed, i think it can do the job too

the performance is comparable to what i get from cloud services too, on my RX 7800 XT

i just need to find good models, this qwen model seems to be a bit mischievous, keeps putting an empty comment line above the doc comments which is improper syntax

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ 5mo ago

different variants of qwen seem to be better and actually mostly follow the instructions

i think i can see why the prompt text matters so much. i guess i'm gonna have to search some tips on writing good prompts for this kind of shit. this is yet more stuff that is hidden from you in regular interfaces.