The wait is over! As the leading AI code review tool, CodeRabbit was given early access to OpenAI's GPT-5 model to evaluate the LLM's ability to reason through and find errors in complex codebases! Our evals found GPT-5 performed up to 190% better than other leading models!
As part of our GPT-5 testing, we conducted extensive evals to uncover the model’s technical nuances, capabilities, and use cases around common code review tasks using over 300 carefully selected PRs.
Across the whole dataset, GPT-5 outperformed Opus-4, Sonnet-4, and OpenAI's O3 on a battery of 300 varying difficulty, error diverse pull requests – representing a 22%-30% improvement over other models
We then tested GPT-5 on the hardest 200 PRs to see how much better it did on particularly hard to spot issues and bugs. It found 157 out of 200 bugs where other models found between 108 and 117. That represents a 34%-45% improvement!
On our 25 hardest PRs from our evaluation dataset, GPT-5 achieved the highest ever overall pass rate (77.3%), representing a 190% improvement over Sonnet-4, 132% over Opus-4, and 76% over O3.
Our results show that GPT-5 represents a significant improvement in the ability to reason through a codebase and find issues – thanks to some new capabilities.
Check out our in-depth testing process with detailed results and examples in our latest blog. coderabbit.ai/blog/benchmark…
United States Trendy
- 1. #RomanEmpireByBangChan 10.4K posts
- 2. ROMAN EMPIRE OUT NOW 9,442 posts
- 3. Jayden Daniels 24.9K posts
- 4. jungkook 608K posts
- 5. Dan Quinn 7,036 posts
- 6. #River 7,371 posts
- 7. #STARDOM 4,662 posts
- 8. Perle Labs 5,280 posts
- 9. Seahawks 38.7K posts
- 10. #MondayMotivation 25K posts
- 11. Commanders 50.5K posts
- 12. Jake LaRavia 6,037 posts
- 13. Sam Darnold 15.2K posts
- 14. #RaiseHail 8,842 posts
- 15. Godzilla 44.1K posts
- 16. 60 Minutes 77.1K posts
- 17. Bronny 15.3K posts
- 18. Washington 130K posts
- 19. Snopes 3,416 posts
- 20. Jaxson Hayes 3,376 posts
Something went wrong.
Something went wrong.