If I had to guess, the biggest challenge is dealing with legacy infra creap
Discussion
How fast could AI replace it all using OSS?
Umm, seems difficult lol. Gov regs, embedded interests, AI data provenance issues, OSS maintenance, etc.
Palantir has pioneered something similar to what you're thinking with their Forward Deployed SWE concept. Albeit, with their own SW stack.
This was informative: