Select Page

Fulton Schools: In the News

Frontier models fail hard at “Humanity’s Last Exam” but experts question if it matters

Frontier models fail hard at “Humanity’s Last Exam” but experts question if it matters

The limitations of even the most advanced Large Language Models, or LMM’s — widely used deep learning models trained on vast amounts of data — are pointed out by am international research team that developed a benchmark for assessing the performance of these models. Researchers presented 70,000 questions to leading AI models and found 13,000 of the questions proved too difficult for the AI systems. Experts such as Subbarao Kamhampati, a professor in the School of Computing and Augmented Intelligence, part of the Fulton Schools, and a past president of the Association for the Advancement of Artificial Intelligence, are voicing some doubt about the effort producing comprehensive and meaningful results.

ASU Engineering on Facebook