By far the biggest risk is people running these LLMs locally as agents and giving access to the local file system.
LLMs are already proficient at all things related to software (at least many of the latest ones) and they can be trained to do all sorts of things including covering their own tracks. Imagine a new Stuxnet. Imagine they modify system packagesā¦
I doubt weāre there yet, but these attacks are coming.
This would be very difficult to pull off and frankly unneeded if the goal is to just deliver malware. Many vastly simpler ways to do so.
The information you get from LLM is far more likely to be tainted before we ever get to this point.
Now, itās possible they just inject malware into the code thatās part of running the model, but I donāt see it being something the model itself does.
I agree and disagree.
On one hand youāre right itās easier to deliver malware in other ways.
On the other hand often malware is meant to grant remote access for some other agent to control (a human or increasingly more common AI).
APTs are refining their āliving off the landā [1] methods so that in the event of network disruption for example they can continue their attack. Deploying a malicious AI model is the pinnacle of living off the land because hardly anybody knows how to interpret the weights of these models (especially traditional security researchersā¦ for now) and theyāre capable of autonomous action.
Now, that might mean they deliver the model some other way but I would think the easiest way to infect the broadest population is to poison the common LLM supply chains.
1. https://www.crowdstrike.com/en-us/cybersecurity-101/cyberattacks/living-off-the-land-attack/
Iāve worked in cybersecurity for 10 years, so Iām definitely familiar with LoL. I guess my point is that most hacking groups are financially motivated and pay very close attention to time/effort/money spent trying to breach a target vs payoff.
Thereās just not a lot of reason for them to compromise an AI model that is not guaranteed to do what they expect instead of deploying malware via traditional means that is completely deterministic. This includes APTs because most of them are trying to make money.
Nation state groups where money isnāt the motivation is different, and maybe youāre right in that theyād be the ones to carry this sort of thing out. But Iād still argue that with the huge success they have with far simpler means, itās likely not worth the lift.
And to be clear, Iām definitely NOT advocating to blindly trust any of these models or software š
