Maybe it could bootstrap from a smaller training set of well coded projects… then train on synthetic data after that. Definitely possible in theory
Discussion
Yeah this was my thinking too, also the dataset of excellent code could continue to grow as LLM architecture continues to improve, with a side of Moores law.
Further, someone who can tell the difference between excellent code and average code could simply instruct the LLM on the differences as they see them arise in a project, and a sort of half vibe-coding thing with current tech and probably come out with some above average code more quickly than having written it themselves.
