The key behind vLLM is stuffing the GPU as full of prompts as possible. So there's one model and many prompts. As prompts get finished they're refilled with more prompts

Didn’t we already know this though? It’s literally anyone not in their cult who is a tErRoRiSt! nostr:note1nxcgz2t9vc2cytc0rmquv63jnsggrfw2n89cjxnc0zz7t9t33t2qpfh6j0

vLLM is nutty

Before I was doing naive queuing with text-gen-webui for parsing documents and getting 1 page/sec thinking it was good (way better than human speed)

An 8b on a 3090 handles 60 reqs/second with vLLM

Bruh I'm going insane I just want to emit db objects to client as they're made this shouldn't be this hard

ecash sat stamp🎯 nostr:note19dsnzrh25nfwn5tcvdzqfsttapssf2xs6pmp5h8m4vz9cemcqyzsydkxw4

I’m more afraid of human NPC’s than AI.

Much bigger threat.

AGI isn't going to be centralized for basically the same reasons that central planning cannot produce prosperity. It's a fools errand.

OpenAI is the next FTX

Lots of chatter on the airwaves and in print lately probing the limits of free speech, the justifications for censorship and the future viability of the US constitution. It was a talking point in last night's VP debate as well, during which Walz recited the folk-loresy understanding of the issue: "you can't yell fire in a crowded theater."

Here's a PSA on this little gem a bit in case you ever find yourself up against it.

The phrase comes from supreme court Justice Oliver Wendell Holmes Jr during the 1919 case Schenck v United States. Anyone unfamiliar with this case might be shocked to learn that it had nothing to do with shouting fire, or with any kind of mischievous, anti-social trouble-making whatsoever. At stake was the printing and mailing of pamphlets to draft-eligible men that made the case that the draft was against their constitutional rights and inviting citizens to peacefully abstain. In other words, what was at stake in the trial were the defendants right to criticize perceived government transgression of the constitution, exactly what the first amendment was designed to ensure.

Amazingly, what was passed down to our collective memories about this episode is not the hard truth that governments will attempt to violate constitutional rights for their own, arguably nefarious, ends (a conceptual inheritance that would strengthen civic support for free speech), but rather the wildly inaccurate analogy of the crowded theater (an Orwellian description of the case that suggests government censorship is there to protect us and keep us safe). And even though the precedent of Schenck was overturned in 1969, the "fire in a crowded theater" litmus test still lingers as the operative heuristic for a great many Americans, including, apparently, one of the candidates for Vice President of the country.

If the Affordable Care Act (Obamacare) is so great, why does our health insurance system still absolutely suck ass?

it’s almost been a year since Israel experienced 6,000,000 9/11s in one day. time flies when you