How a Trading Firm Ships New AI Models the Same Day - Noyan Tokgozoglu, Optiver

Highlights

• Optiver processes 10 million plus trades daily, prices more than 1 million instruments, and handles petabytes of data daily.

• Changes deployed to traders and quants show P&L impact within hours, as the firm zeroes out its balance sheet at the top of every day.

• New frontier models are integrated into production flows on the exact same day they become publicly available.

• Token spend spiked significantly after summer frontier model releases before being brought back under control via tool search and caching optimizations.

• A self-service web portal with session management, a personal assistant, and agent workbenches gives all employees unified access to AI capabilities.

Timeline

1. 00:00 - 02:52

   Context and Immediate Feedback Loops in Trading

   The high-frequency trading environment demands immediate responsiveness because the balance sheet is zeroed out daily. Because trading rewards volume rather than competing for a finite amount of user attention, the economic incentives differ fundamentally from typical software services.

   • Optiver operates as a market maker trading with its own money across more than 10 global locations.
   • Unlike traditional consumer tech companies that measure engagement metrics over months and quarters, trading decisions show P&L impact within hours.
   • Missing a market opportunity carries a high cost, requiring a strong bias for action and rapid decision-making.
2. 02:52 - 07:05

   Core Principles of the Internal AI Platform

   To maintain speed without sacrificing control, the engineering team establishes internal benchmarks that reflect proprietary data distributions. While services remain self-service, forward-deployed engineers bridge the gap between technical implementation and trading workflows.

   • The internal AI platform relies on modularity to rapidly ingest new models and paradigms.
   • Forward-deployed AI engineers work directly with traders to build self-service tools with reliable default settings.
   • All infrastructure runs strictly on-premises to protect intellectual property and comply with regulatory requirements.
3. 07:05 - 09:54

   Model Evaluation and Gateway Infrastructure

   Rigorous evaluation harnesses run new models automatically against internal data to verify actual production value. The inference gateway ensures operational resilience by allowing seamless switching between providers if an outage occurs.

   • New frontier models are benchmarked immediately on golden sets provided by key trading teams.
   • An inference gateway routes traffic between third-party providers and on-prem models while tracking costs and budgets.
   • A meta-harness layer allows teams to configure their experiences and maintain portability across different model harnesses.
4. 09:54 - 13:56

   Token Cost Optimization and Usage Controls

   Following a surge in spending driven by summer frontier model releases, technical adjustments reduced token bloat. Additional cost controls include utilizing competitive open-source models and dynamic prompt routing between different model tiers.

   • Unoptimized default sessions frequently accumulated over 200k tokens of context due to bulk context dumping.
   • Implementing Model Context Protocol (MCP) tool search and optimized caching parameters significantly reduced token consumption.
   • Providing teams with transparent cost visibility and alerting empowers them to self-manage budgets effectively.
5. 13:56 - 23:13

   Unified User Interfaces and Multi-Agent Orchestration

   High-agency traders routinely build bespoke workflows if official tools are absent, prompting the engineering team to deliver polished internal equivalents. Rather than expanding a single agent to handle complex overlapping responsibilities, the architecture relies on orchestrating multiple specialized agents through a unified ledger.

   • A centralized web portal provides session management, chat windows, and collaborative document editing for all employees.
   • An agent gateway acts as a single ledger for all corporate agents, facilitating multi-agent communication.
   • Scoped agents prevent single-agent bloat by separating distinct tasks into modular, easily measurable components.