But the vulnerabilities definitely exist, whether they're being actively exploited much or not. And it's going to be *very* hard to inspect an on-die black box like that. Few people have the tools to even attempt it, and I can think of a bunch of ways offhand to make it more difficult to detect that it's doing shady stuff.
I'm not super concerned about it myself, but if I was a terrorist or something I would take the threat seriously. It's bad enough that the hardware companies don't prioritize user security, but the fact that they outright hide and lie about functionality (on multiple levels) in their hardware is very disturbing.
I agree that the government is not super competent in general, but we can't ignore the possibility that in some cases they're simply lying about how they built their case. Parallel construction happens all the time and we know all these big companies cooperate with the government. Some of them surely have teams of spooks actively working within them, whether the companies actually know about it or not. How many of these "hardware exploits" are true accidents, and how many are sneaky exploits some NSA dorks dreamed up and slipped into the spec?