Stanford HAI "2025 AI Index Report" Highlights

Original Title: The 2025 AI Index Report

Original Author: Stanford HAI (Stanford Institute for Human-Centered Artificial Intelligence)

Original Translator: Felix, PANews

Stanford HAI recently released the 456-page 2025 AI Index Report. Here are some key points regarding artificial intelligence trends:

1. Artificial Intelligence Has Become Much More Powerful Than Imagined

In the new benchmark tests MMMU, GPQA, and SWE-bench, the performance of artificial intelligence has seen significant improvements: scores increased by 18.8%, 48.9%, and 67.3%, respectively. In addition to benchmark tests, AI systems have made significant progress in generating high-quality videos, and in some cases, large language models (LLMs) have even outperformed humans in timed programming tasks.

Notes:

· MMMU is a carefully designed new benchmark tailored for university-level interdisciplinary multimodal understanding and reasoning, aiming to evaluate the expert-level multimodal understanding ability of base models across a wide range of tasks.

· GPQA is a challenging dataset containing 448 highly curated multiple-choice questions authored by domain experts from various fields. The accuracy rate of experts with or pursuing a doctoral degree in the respective field is only 65%, while highly skilled non-expert validators, despite spending an average of over 30 minutes and having unrestricted internet access, achieve an accuracy rate of only 34%.

· SWE-bench is a benchmark for evaluating large language models (LLMs) on real-world software problems collected from GitHub.

2. Artificial Intelligence Is More Efficient, Accessible, and Cost-Effective

The capabilities of smaller AI models with fewer parameters are increasingly enhanced: in just two years, the parameter count has decreased by approximately 100-fold, yet their scores in large-scale multitask multilingual understanding (MMLU) tests still exceed 60%.

The performance gap between open-source and closed-source models is also narrowing, with performance gaps in some benchmark tests decreasing from 8% to just 1.7%.

In addition, from November 2022 to October 2024, the inference cost of systems reaching the level of GPT-3.5 has decreased by over 280 times. At the hardware level, costs are decreasing by 30% annually, while energy efficiency is improving by 40% annually.

The threshold for advanced artificial intelligence is rapidly decreasing. Not to mention the development of sparse models like DeepSeek, where in an Expert-Mixed (MoE) structure, only relevant parameters are activated to answer user queries, making the whole process more efficient.

Indeed, with the emergence of smaller yet more powerful AI models, the requirements for AI model training have decreased, and cost-effective distributed training is expected to become mainstream in the next decade. Currently, some leading projects are conducting related research based on different theoretical frameworks.

3. Artificial Intelligence is increasingly integrating into daily life

In 2023, the U.S. Food and Drug Administration (FDA) approved 223 AI-assisted medical devices, compared to only 6 in 2015. On the roads, autonomous vehicles are no longer experimental: one of the largest operators in the U.S., Waymo, provides over 150,000 autonomous driving services per week, while Baidu's Apollo Go self-driving taxi fleet is now operational in multiple cities in China.

4. Enterprise investment in the field of artificial intelligence has significantly increased, driving record investments and applications

The commercial applications of artificial intelligence are also accelerating: by 2024, 78% of organizations are using artificial intelligence, up from 55% the previous year. At the same time, more and more research confirms that artificial intelligence can enhance productivity and help narrow the skills gap across the entire workforce.

Indeed, as artificial intelligence leads to an exponential increase in customer expectations, existing solutions can quickly become outdated overnight, leading to missed adaptation opportunities for existing businesses, and more frequent occurrences of product-market fit failures.

5. While global optimism about artificial intelligence is rising, Asians are more optimistic about artificial intelligence

In countries like China (83%), Indonesia (80%), and Thailand (77%), most people believe that the benefits of AI products and services outweigh the drawbacks. In contrast, the optimism in places like Canada (40%), the United States (39%), and the Netherlands (36%) remains significantly lower.

However, this attitude is changing: since 2022, the optimism in some previously skeptical countries has significantly increased, including Germany (10% increase), France (10% increase), Canada (8% increase), the UK (8% increase), and the U.S. (4% increase).

6. The Increasing Impact of Artificial Intelligence on Scientific Research

The growing importance of artificial intelligence in research is becoming a key driver of scientific advancement. This is evidenced in major scientific awards: two Nobel Prizes were awarded for contributions to deep learning (in physics) and its application to protein folding (in chemistry), while the Turing Award recognized groundbreaking contributions to reinforcement learning.

Evidently, artificial intelligence is rapidly advancing at an exponential and unexpected pace, which is significant for most people. Therefore, AI safety is also becoming increasingly important. While AI makes forgery easier, cryptography makes forgery more difficult. There is anticipation for cryptographic projects that can leverage the native properties of blockchain (verifiability and transparency) to build practical solutions in this area.

Original Article Link