On Thursday, Google announced the rollout of an updated Gemini 2.5 Pro model, building on the version introduced at the I/O 2025 event in May.
Gemini 2.5 Pro (Preview 06-05 Thinking)
The new Gemini 2.5 Pro shows a 24-point Elo score increase on LMArena, holding a top score of 1470 and maintaining its leaderboard position. It also gained 35 Elo points on WebDevArena, reaching 1443 and leading the rankings there.
The model leads in coding benchmarks like Aider Polyglot and performs well on tests such as GPQA and Humanity’s Last Exam (HLE), which measure math, science, reasoning, and knowledge.
Tulsee Doshi, Senior Director of Product Management at Google, said the update reflects user feedback, with better creativity and improved response formatting. She added that “thinking budgets” have been introduced to help developers control cost and latency.
Performance, Benchmarks, and Pricing
Gemini 2.5 Pro ranks high across major AI benchmarks, outperforming OpenAI’s o3 and o4-mini, Claude Opus 4, Grok 3 Beta, and DeepSeek R1. Key scores include:
- LMArena: 1470 (↑24 Elo)
- WebDevArena: 1443 (↑35 Elo)
- GPQA (science): 86.4%
- AIME 2025 (math): 88.0%
- LiveCodeBench (code generation): 69.0%
- Aider Polyglot (code editing): 82.2%
- SWE-bench (agentic coding): 59.6% (single), 67.2% (multiple)
- HLE (reasoning/knowledge): 21.6%
- SimpleQA (factuality): 54.0%
- FACTS Grounding: 87.8%
- MMMU (visual reasoning): 82.0%
- Vibe-Eval (image): 67.2%
- VideoMMMU (video): 83.6%
- MRCR v2 (long context): 58.0% at 128K, 16.4% at 1M
- Global MMLU (multilingual): 89.2%
Gemini ranks among the highest performers across these categories. Its usage costs are also lower than many competitors. The rate is $1.25 per million input tokens, with caching adjustments ranging between $0.50 and $2.00, and $10.00 per million output tokens, with caching varying from $1.00 to $15.00. In comparison:
- OpenAI o3: $10 input / $40 output
- Claude Opus 4: $15 input / $75 output
- DeepSeek R1: $0.55 input / $2.19 output (lower cost but weaker performance)
Availability
Google plans to release Gemini 2.5 Pro as a stable, generally available model within a few weeks, aimed at enterprise-scale use.
Meanwhile, developers can access the upgraded preview through Gemini API via Google AI Studio and Vertex AI. The Gemini app also started rolling out this version on June 5, 2025.