Tweet Overview
View this X/Twitter post from @augmentcode published on April 15, 2025 at 12:02 AM. This post contains 1 images.
๐๐๐-๐.๐ ๐๐ฅ๐ฆ๐จ๐ฌ๐ญ ๐ญ๐จ๐ฉ๐ฌ ๐๐ฅ๐๐ฎ๐๐ ๐.๐ ๐จ๐ง ๐๐จ๐๐ข๐ง๐ ?! New eval dropping using our #1 SWE-bench coding agent! - GPT-4.1 beats Gemini 2.5 Pro and almost tops Claude 3.7 Sonnet! - Even GPT-4.1 mini matches Claude 3.5 Sonnet V2 performance. It was the top model just 2mo ago!
The evaluation is done through our proprietary codebase understanding benchmark AugmentQA. You can learn more at: https://www.augmentcode.com/blog/you-make-your-evals-then-your-evals-make-you-introducing-augmentqa Try our agent yourself at: http://www.augmentcode.com.
