Without Caching, Your AI Agent Costs 10x More - George Cameron, Artificial Analysis

Highlights

• Artificial Analysis benchmarks models across the full AI stack, covering the agent layer, language models, image, video, speech, and music models, as well as inference hardware.

• Prompt caching reduces input token costs by roughly 10x, and running without cache hits can spike agent task costs from $21 to over $200.

• Accessing the same level of intelligence drops in cost by an order of magnitude (10x) within approximately one year.

• Task costs across models span multiple orders of magnitude, ranging from above $5 to $0.05 or less per task.

• Adjusting reasoning effort on models like Opus 5.5 changes task costs from $1 at low effort to $21 at max effort.

Timeline

1. 00:00 - 02:49

   Introduction to AI Benchmarking and Full-Stack Trade-offs

   Making deployment decisions requires evaluating multiple layers simultaneously, including access methods, pricing, speeds, and cache hit rates. Rather than focusing solely on raw intelligence bar charts, developers must understand the operational trade-offs across the stack. Leading labs regularly utilize these cross-layer insights when moving projects into production environments.

   • Artificial Analysis operates as an independent AI benchmarking company evaluating the entire AI stack.
   • Evaluations span agents, language, image, video, speech, and music models, alongside hardware and inference layers.
   • Industry leaders reference these benchmarks to navigate trade-offs during production rollouts rather than just raw intelligence metrics.
2. 02:49 - 06:53

   The AI Landscape and Intelligence Index

   The intelligence index serves as a single metric to help developers select models for specific use cases rather than tracking theoretical progress toward AGI. While generalist scores provide a baseline, specific benchmarks like the AA analyst agent reveal specialized strengths, such as Gemini models navigating complex spreadsheets. The fast-paced release cycle means open-weights alternatives remain competitive options alongside proprietary options.

   • The intelligence index synthesizes 10 different evaluations covering knowledge work, scientific reasoning, coding, and hallucination rates.
   • Anthropic's Opus 5.5 currently leads the intelligence index among over 500 cataloged models.
   • Open-weights models consistently stay within three to nine months of leading proprietary models.
3. 06:53 - 15:35

   Cost Drivers and Economic Dynamics of AI Agents

   As agents execute long-horizon knowledge work tasks across thousands of files, costs scale significantly based on architectural parameters. A task averaging $21 with caching enabled can skyrocket past $200 if prompt caching fails or remains disabled. Furthermore, adjusting reasoning effort settings dramatically alters token consumption and overall task expenses.

   • Individual AI tasks now cost anywhere from over $5 down to $0.05 or less depending on the model.
   • Reaching a specific tier of intelligence drops by 10x in cost within roughly one year.
   • Input token counts, cache hit percentages, and the number of execution turns per task dictate agent costs.
4. 15:35 - 19:32

   Navigating Trade-Off Curves and Production Decision-Making

   Selecting models requires balancing intelligence against cost by analyzing Pareto frontiers. Limited intelligence tasks gain nothing from excessive model power, allowing developers to optimize strictly for cost efficiency. Advanced users deploying production systems should construct tailored benchmarks or utilize tools like Optima to map out precise trade-off curves.

   • Tasks divide into unlimited intelligence tasks where smarter models always win, and limited intelligence tasks where performance caps out.
   • Production workloads benefit from creating custom benchmarks using shortlisted model alternatives.
   • The Optima product generates custom trade-off curves matching specific user contexts and cost constraints.