Leaderboard
MathOdyssey tests the mathematical reasoning ability of large language models. The dataset was created by human professionals to ensure high quality and has not been seen by the models before. It is designed exclusively for testing purposes. We welcome you to contribute results from your model evaluations, or we can evaluate your model and add it to the leaderboard. Please contact us at hello (at) agiodyssey.org and MathOdyssey (at) gmail.com.
| Rank | Foundation models | Method | Score |
|---|---|---|---|
| 1 Sep, 2024 |
OpenAI o1 OpenAI |
- | 65.12 |
| 2 June, 2024 |
Gemini Math-Specialized 1.5 Pro Gemini Team, Google |
- | 55.8 |
| 3 2024 |
Qwen2-72B-Instruct Qwen Team, Alibaba Group |
Step-DPO | 50.1 |
| 4 2024 |
GPT-4 Turbo OpenAI |
MCT Self-Refine | 49.1 |
| 5 2024 |
GPT-4 Turbo (gpt-4-turbo-2024-04-09) OpenAI |
CoT | 47.0 |
| 6 2024 |
Qwen2-72B-Instruct Qwen Team, Alibaba Group |
- | 47.0 |
| 7 June, 2024 |
Gemini 1.5 Pro Gemini Team, Google |
- | 45.0 |
Data Explorer
| PROBLEM SOURCE | MATH Algebra Level 1 |
| QUESTION | If a drip of water is equivalent to $\frac{1}{4}$ of a milliliter, how many drips are in a liter of water? Note: 1 liter = 1000 milliliters. |
| REFERENCE ANSWER | If a drip of water is equivalent to $\frac{1}{4}$ of a milliliter, then $4$ drips of water must be equivalent to $1$ milliliter of water. Since there are $1000$ milliliters in a liter, it follows that there are $4 \times 1000 = \boxed{4000}$ drips in a liter of water. |