As predicted here in March… Open Source AI catches up with GPT-4 for writing code. 😎

Model is called CodeLlama-34B-Python

It achieved 69.5% pass@1 on HumanEval, GPT-4 achieved 67%

NB: 34 billion is significantly smaller than GPT-4 it’s approx 1/10th the size.

Again I confidently predict that many small and open source specialists will destroy monolithic general models, when it comes to creating something akin to AGI.

The large forest of super narrow specialists will win. It won’t even be close. Monoliths are a dead end.