If Your AI Scores 100%, Your Eval Is Wrong - Ramakrishnan Lokanathan, eslams.com

하이라이트

• 95% of organizations fail to achieve returns on generative AI because production evaluations lack the structure required to deliver core product promises.

• Achieving a 100% score on an AI evaluation indicates a flawed and insufficiently ambitious test framework.

• eSlams is an open-source Python package supporting over 50 games and 90 providers to evaluate long-chain agent reasoning through competitive gameplay.

• Transitioning Google Assistant to Gemini caused a regression where basic functionality like setting reminders temporarily ceased to operate.

• Production evaluation systems must incorporate automated CI checks, human review, and adversarial testing rather than relying on a single aggregate score.

타임라인

1. 00:00 - 05:33

   First-Principles Definition of AI Evaluations

   Evaluations draw direct parallels to human examinations like the SAT, GMAT, or bar exams, acting as bounded tasks and standards. Over-indexing on a single test score creates a false sense of security regarding model capability. Production environments demand rigorous validation to ensure that deployed models consistently fulfill promises made to end-users.

   • Evaluations function as standardized examinations to measure whether AI systems meet expected performance standards.
   • Individual tests like the SAT serve as single signals of readiness rather than comprehensive measures of total capability.
   • 95% of organizations fail to capture returns on generative AI investments due to insufficient evaluation structures in production.
2. 05:33 - 09:44

   Core Principles for Designing High-Stake Evaluations

   Building effective evaluations begins by establishing the most ambitious long-term outcomes and answering boundary questions about required and forbidden behaviors. Testing must target specific workflow components, such as tool-calling accuracy, rather than merely inspecting final outputs. Holistic evaluation suites combine automated CI pipeline checks, human reviews, and adversarial testing to uncover edge-case vulnerabilities.

   • Defining ambitious success targets based on hypothetical AGI capabilities sets the appropriate scope for testing.
   • Evaluations yielding a score of 100 out of 100 indicate a lack of ambition and insufficient stress testing.
   • Testing every step within a multi-step workflow prevents hidden failures that simple input-output checks miss.
3. 09:44 - 12:07

   Evaluating Agent Reasoning Through Games

   General-purpose language models require evaluation paradigms beyond static benchmarks like SWE-bench. Drawing parallels to human chess and Go expertise, the eSlams framework deploys agents into interactive environments such as chess, poker, and arcade games. Observing how models adapt strategies and manage liabilities during gameplay provides deeper insight into their multi-step reasoning capabilities.

   • eSlams is an open-source Python library that uses 50 games across 90 providers to test long-chain agent reasoning.
   • Competitive gameplay evaluates strategic planning, hidden state navigation, cooperation, and recovery from blunders.
   • AI models playing against each other reveal decision-making patterns that standard single-output benchmarks fail to capture.
4. 12:07 - 18:36

   Evolving Test Suites and Incorporating Real-World Feedback

   Continuous evaluation requires tracking regression cases to ensure upgraded systems preserve previously stable features, such as reminder creation during assistant migrations. As model performance improves, evaluation benchmarks must be systematically hardened to push operational frontiers. Integrating user observability data, bug reports, and human reviews completes the feedback loop necessary for robust production deployments.

   • Platform migrations risk feature regressions when past functionalities are omitted from continuous evaluation pipelines.
   • Reaching score thresholds of 85 or 90 signals the necessity to increase test difficulty and raise standards.
   • Real-world customer failures and user feedback serve as essential inputs for expanding future test suites.