Avatar
Max 2y ago

Running Vicuna chat bot on my Qubes laptop with only a CPU.

It works!!

But incredibly slow, about one minute per token.

https://vicuna.lmsys.org/

Reply to this note

Please Login to reply.

Discussion

Avatar
Tony 2y ago

Still leaves much to be desired, but it’s a step in the right direction.

FOSS rocks ✊

#[0]

Avatar
PetertheFox 2y ago

Nice, I'm looking forward to more updates from you!

Avatar
Max 2y ago

It runs substantially faster on a bare metal Debian machine, now it's actually usable, the responses are solid.

Avatar
Erik 2y ago

On what specs do you have it running on?