That's pretty much been my experience as well.

OK at "coding" and providing known answers faster than searching google or stack overflow.

But completely unable to do "programming" work and can't help me design or build something new.

I don't think they will ever do "programming" well, but I'd at least settle for a finely tuned model to help me do "coding" faster. I get stuck and held up a lot on stupid little things 😂