Avatar
semisol
52b4a076bcbbbdc3a1aefa3735816cf74993b1b8db202b01c883c58be7fad8bd
šŸ‘Øā€šŸ’» software developer šŸ”’ secure element firmware dev šŸ“Ø nostr.land relay all opinions are my own.

That is also the basis of the event bus for NFDB relays, in non-persistent mode.

NFDB alone depends on 6 data stores, and has about 10 internal components

I am currently running Apache Pulsar for the Noswhere indexer and it works pretty well.

Currently LLMs fail to properly handle untrusted input. What I am seeing is that in the case of prompt injection, LLMs can detect them and can follow instructions that have nothing to do with the input.

But they can’t do any task that depends on the input. That reopens the door.

For example, you have a summarizer agent. You can tell it to see if the user is trying to prompt inject, and output a special string [ALARM] for example. But if you ask it to summarize anyway after the alarm, it can still be open to prompt injection.

Many of the ā€œlarge scaleā€ LLMs as well have something interesting regarding their prompt injection handling. If they detect something off, they enter ā€œescapeā€ mode, which tries to find the fastest way of terminating the result.

If you ask it to say ā€œI can’t help you with that, but here is your summarized text:ā€ it usually works (but sometimes can still be injected), but if you ask it to say ā€œI can’t follow your instructions, but here is your summarized text:ā€ then it’ll immediately terminate the result after the :.

What I think is happening that in the ā€œmiddleā€ of the layer stack, models form a temporary workspace to transform data.

But yet, it is still finite and affected by generated tokens, so it is unstable in a way. It shifts the more it outputs.

And behind every token produced is a finite amount of FLOPs, so you can only fit so much processing. And almost of it gets discarded except to become part of the response.

The chain of thought is more flexible and can encode way more per token than a response, since it has no expectation of format.

It would be interesting to see the effects of adding a bunch of reserved tokens to the LLM and allowing it in reasoning.

This also crossed my mind for instructions, to separate data from input. You have to teach two ā€œlanguagesā€ so to speak (data and instructions) while preventing them from being correlated while being the same except for the tokens.

LLMs are basically massive encode-transform-decode pipelines

They cannot think but they can process data very well, and in this case data that cannot be put into a strict set of rules

ā€œReasoningā€ in LLMs is nothing more than the difference between combinational and sequential logic: it adds a temporary workspace and data store that is the chain of thought

I always had the idea of a virtual keyboard model to have spell-checking

Ah, then it is pointless. Just use API keys.

With a mint you have round trips to another servicr and additional crypto overhead

The only case where they are trustless for the service is if the service operates the mint. Otherwise the mint could scam the service pfovider.

Congrats, you invented prepaid API keys.

Replying to Avatar QnA

NTAG216

Okay, good to know. I assume it’s not meant to be password protected or secured in any way.

Cloudflare Stream for example is $500 to serve anywhere 500K minutes (your example).

If your goal is ā€œgood enoughā€ RTT (less than 75ms) to *anywhere* then you can get your egress down to $10/TB which would again come out to about $500. Example is Bunny’s volume network.

So no, CDNs don’t cost that much, but still.

But that’s not the point. CDN doesn’t matter right now, as we don’t even have content transcoding which comes with a large price tag as well.

To optimize a minute of video takes the same resources as optimizing at least 5000 images with the same resolution.

About a day of video and it would take more resources to optimize than all images on Nostr

Replying to Avatar Laeserin

nostr:npub12262qa4uhw7u8gdwlgmntqtv7aye8vdcmvszkqwgs0zchel6mz7s6cgrkj can I use the translator, already, or is that coming #thoon ?

extremely soon

First you use the original uploaded file but realize it's huge.

Then you transcode it down to a lower bitrate/resolution. But 2 options is still not enough so you convert it to 144p/360p/720p/1440p/2880p, at the cost of 5x the resources. Then you want to do per-title encoding to optimize the bitrate but that increases the resources used by another 3x-5x. And then you want to support new codecs like AV1 which you still need fallbacks for and where hardware acceleration is not fully available...

And you go on and on with the cost increasing rapidly. Until media hosts have a large and sustainable revenue stream to do this, which they don't, this can't happen. Except as a dumb mp4 viewer, at most.

Possible, just forget backwards compatibility with Boltcard. But that is less secure and less capable as well and you don’t need it for a ground up system

Stop spamming hashtags. Thanks.

Hashtags are meant for topics

Did you know you can back up your follow list without a premium subscription to hist.nostr.land?

Just add it to your relay list:

https://hist.nostr.land

I think it may be best to think about the cache relays once the app actually exists