Nostr Web Client

NEW: Cost to 'poison' an LLM and insert backdoors is relatively constant. Even as models grow.

Implication: scaling security is orders-of-magnitude harder than scaling LLMs.

Prior work had suggested that as model sizes grew, it would make them cost-prohibitive to poison.

So, in LLM training-set-land, dilution isn't the solution to pollution.

Just about the same size of poisoned training data that works on a 1B model could also work on a 1T model.

I feel like this is something that cybersecurity folks will find intuitive: lots of attacks scale. Most defenses don't

PAPER: POISONING ATTACKS ON LLMS REQUIRE A NEAR-CONSTANT NUMBER OF POISON SAMPLES https://arxiv.org/pdf/2510.07192

Please Login to reply.

Delicious. 🤓👾

We've been doing covert LLM poisoning work for criminal topics, but this is a good first level explaination of the What and How. 😏

#PoisonLLMs #PoisonAI #LLMOverlords #TheMoreYouKnow #EatTheRich

Very fascinating. And yes, LLMs are nearly impossible to fully “secure” due to their non deterministic output.

I’m baffled by the number of “security controls” that are just additional prompts. That’s not a control, it’s a wish.

My day job is thinking about how to make code secure. I’ve been thinking about this research a lot.

There are two main challenges here:

1. Most code that is used to train LLMs was written by humans. Humans do not write secure code.

2. Data poisoning is a real attack vector and it has a non linear affect on LLM output.

Securing code at scale before LLMs was incredibly difficult. Now? The game is 10x harder.

Also, in before someone suggests just having LLMs review the code for vulnerabilities 😅