so much for pay per query. $11 to use o3
is #routstr a scam? guess I should continue working on https://jb55.com/lmzap/
also it forced me to use cashu with no NWC option...
Discussion
o3 is good at code but does not interact well with tools, I heard thats what makes it so expensive.
I learned it the hard way too with payperq.
Use other models.
Oh wait you used it via the GUI only?
just tried the routstr gui yeah, can't use it without depositing 11k sats
I really enjoy o3 when I'm thinking through some complex technical problems. But haven't tried it with tool use yet.
perhaps this is the main issue? nostr:npub130mznv74rxs032peqym6g3wqavh472623mt3z5w73xq9r6qqdufs7ql29s nostr:npub16g4umvwj2pduqc8kt2rv6heq2vhvtulyrsr2a20d4suldwnkl4hquekv4h
nostr:note1psg5l3mqnc44uvsvynf3x0v0x5nhu228w05vtv4648k5lacjxglq0jyp5v
but 10k sat sticker shock is pretty brutal
Exactly. The average query is probably not consuming the entire 10k sats, but they (routstr) need that in reserve just in case it does. In most cases, I think they are issuing you change on a per query basis.
The question is whether all of that engineering of 10k deposit sticker shock, issuing change, increased latency etc is being justified by some level of inceased UX. I'm still unsure.
yeah i would take the risk to get users earlier on and accept some possibility of loss. You can always button up once you scale up?
Possibly yea.
Additionally, there are ways that one can predict the cost of a query based on the input. If those methods are accurate, it could serve to make the buffer amount they require to be more dynamic. Basic questions like the one you asked above might demand a much lower buffer than more advanced questions.
But I didn't look too deeply into that stuff. I just wanted to ship AI products more and figured that the top up method was "good enough" for the time being.
May revisit in the near future though. Excited to see what routstr might come up with. And always happy to ideate on collabing with routstr in some way.
I joined routstr team recently.
Did myself worked on such solution
https://github.com/9qeklajc/ecash-402-client
Where u pay ecash per request.
This method ist for sure as u mentioned it, not perfekt, but its a decision the provider make and risk loses ..
Wrote also some ideas on the topic, about a smart client and dynamic provider.
https://github.com/ecash-402/ecash-402-specs
Feedback is welcome
Hmm I wonder if PPQ could adopt this once you polished it up a bit? The choice between topping up and paying per query would still be up to your users but seems like this option does have some possible advantages.
It would also be really cool to see an official doc written up on potential advantages of this model vs topping up. Then just show that in the readme of this repo.
Yeah I'm currently cleaning up and improving some stuff, but basically the code is about the client that the user runs locally and hook up to any ai driven software that use the openai api protocol (like. Goose, roocode, cline, etc.).
will soon release the client with more docs and features..
Ppq or other provider should only implement the interface based on the provider-cliemt spec which as u mentioned still not well documented yet (working on it 😉)
Did some work and summarized all in this project..
It’s just the maximum amount a query can potentially cost! You will get a refund after the request of the spare balance it’s pure openai api pricing
would be good to make that clear ... the ui didn't make that obvious
thanks for the feedback we’ll try to make it more clear!
pricing with LLMs is unpredictable before you make the request since the model will dynamically decide how long your answer will be so we need to charge the full amount upfront to not get scammed and lose sats
we have NWC on our todo list!