How Revolut Cut AI Coding Costs 90% With Self-Hosted Open Models - David Abitbol, Revolut

하이라이트

• Revolut deployed 56 B200 GPUs to self-host the GLM open-source model for 2,000 engineers.

• Self-hosting open-source coding models lowered token costs to $0.30 per million, reducing inference expenses to 10% of commercial provider fees.

• Engineering lead time improved by 70% and coding time decreased by 60% after adopting self-hosted open-source models.

• Optimizing inference parameters improved time-to-first-token performance by 88% while handling 76% higher request volume on the same hardware footprint.

• Restricting quantization to FP4 on Blackwell GPUs yields a 40% reduction in GPU footprint without any loss in model precision.

타임라인

1. 00:00 - 03:32

   Introduction and Initial AI Adoption at Revolut

   Revolut maintains a strong emphasis on shipping products rapidly and maintaining engineering excellence. Initial rollouts of commercial tools generated high expenses with limited financial return, leading the engineering team to cap budgets and test open-source alternatives like GLM.

   • Revolut operates as a European digital bank supporting 80 million customers across 45 markets.
   • Initial deployments of Gemini and commercial cloud coding tools yielded modest efficiency gains alongside high subscription costs.
   • Strict budget limitations prompted an exploration into self-hosted open-source models for coding tasks.
2. 03:32 - 05:45

   Hardware Metrics and Cost Savings of Self-Hosting

   Self-hosting shifts the financial structure from unpredictable variable billing to flat infrastructure costs. Engineers no longer ration tokens for critical tasks, utilizing parallel task execution and off-peak hours to maximize throughput.

   • A cluster of 56 B200 GPUs successfully serves 2,000 internal engineers using the GLM model.
   • Targeting a time-to-first-token latency under three seconds prevents developer disruption and Slack alerts.
   • Blended token costs settle at $0.30 per million, producing invoices equivalent to 10% of Anthropic and OpenAI spending.
3. 05:45 - 08:13

   Inference Optimization and Parameter Tuning

   Raw model weights require extensive configuration adjustments to avoid poor inference performance. Resisting the urge to scale hardware prematurely forces deeper optimization of quantization, tensor parallelism, and context length parameters.

   • System performance increased by 88% while handling 76% higher request volume on the exact same GPU count.
   • Limiting context windows to 256k tokens and using FP4 quantization on Blackwell GPUs maximizes efficiency.
   • Maintaining cache hit rates above 90% is required for effective coding task execution.
4. 08:13 - 11:44

   Production Setup, Compliance, and Governance

   Deploying open-source models within a heavily regulated banking environment requires robust data privacy frameworks, zero data retention policies from GPU providers, and comprehensive open telemetry logging to track all operational incidents.

   • Local routers direct image recognition tasks to multimodal models while retaining GLM for primary text coding.
   • Hosting model weights on US-based servers satisfies data sharing compliance despite using open-source models originating abroad.
   • Sandboxed execution environments enforce strict tool permissions to satisfy strict regulatory requirements for human-in-the-loop oversight.