Nostr Web Client

I have a new micro-project.

It allows to use venice.ai lifetime Pro account with local apps that communicate over the ollama API (open-webui, or continue.dev your VS Code IDE / JetBrains IDE).

Check out the demo video, it's quite nice.

I can also provide a pro account if you don't want to fiddle with MOR tokens.

I started doing this because llama-3.1-405b is really a great model (I think better than ChatGPT for many coding tasks), but I can't run it locally on my laptop.

With this, I have everything set up to make it work locally, with the best open-source model available today.

https://pay.cypherpunk.today/apps/26zEBNn6FGAkzvVVuDMz3SXrKJLU/crowdfund

Reply to this note

Please Login to reply.

Discussion

hit with a hint of s 1y ago

So I can just start ollama on my crappy laptop and connect to lifetime venice instead of openai?

Juraj 1y ago

Ollama runs your model locally

This project is a code different from Ollama, it pretends it is ollama but uses models through Venice.

So any app that would talk to local model through ollama can use Venice instead.

Juraj 1y ago

Here's a demo video of how you can use it.

Use the best state of the art FOSS AI models without a GPU.

https://vimeo.com/999915399

hit with a hint of s 1y ago

your crowdfunding is using email. Would we not prefer something modern? Just sent you an invitation on 0xchat

Juraj 1y ago

I'm not going to change btcpayserver crowdfunding module, but write yournpub@npub.cash as an email.

I'll write you on 0xchat

hit with a hint of s 1y ago

I am trying to run it headless in docker, and failing miserably. help would be nice.

Should i forget docker?

would LXC work?

Juraj 1y ago

I just run it in venv. What about just creating a new user and a python venv?

Headless is default, so that should not be a problem.