Our Smaller Model Beat Opus on Finance Tasks - Alexander Shevchenko, Ramp

Highlights

• LLM use cases at Ramp increased by 123% over the past year.

• Ramp partnered with Prime Intellect to post-train a 35-billion parameter QAN 3.5 model that outperformed Opus 4.6 on financial task accuracy while matching Haiku-level latencies.

• Cartridges compress up to 30 times more context into the KV cache offline before financial professionals begin month-end close processes.

• Ramp utilized 10,000 parallel cloud agents via Ramp Inspect to pen-test infrastructure, discovering and patching seven high-severity vulnerabilities.

• Open-weight models used within agentic swarms achieve detection rates comparable to frontier reasoning models while costing up to five times less.

Timeline

1. 00:12 - 03:10

   Introduction to Applied AI and the Bitter Lesson at Ramp

   Ramp applies LLMs across multiple surfaces to optimize financial products. While frontier models continuously drop in cost and increase in capability, maintaining an internal applied research function allows the company to solve specific domain problems ahead of standard API releases.

   • Ramp operates with an applied research function despite rapid advancements in foundation models.
   • Agent development in 2023 required strict guardrails, prompt engineering, and minimal context windows.
   • Modern agents leverage Model Context Protocols (MCPs) and handle broader context seamlessly.
2. 03:10 - 06:01

   Specialist Models and Financial Question Answering

   To address friction in processing multiple spreadsheets for tasks like discounted cash flow (DCF) modeling, Ramp partnered with Prime Intellect. They post-trained a specialist financial question-answering model that operates alongside a frontier orchestrator at high speed.

   • Ramp Sheets launched as an early spreadsheet agent before frontier labs released comparable tools.
   • Financial planning tasks previously created bottlenecks due to the slow speed and high token costs of Opus 4.6.
   • Post-training a 35-billion parameter QAN 3.5 model yielded higher exact-task accuracy than Opus 4.6.
3. 06:01 - 10:09

   KV Cache Cartridges and Agentic Swarms for Security

   Cartridges pre-prepare necessary context overnight—such as ledgers and aging reports—so agents possess complete situational awareness when financial professionals start work. Additionally, large-scale agentic swarms autonomously execute adversarial testing and deduplicate vulnerability tickets.

   • Cartridges compress up to 30 times more context offline into the KV cache prior to execution.
   • Ramp Inspect deployed 10,000 parallel agents to conduct automated penetration testing across infrastructure.
   • Automated swarms identified and patched seven high-severity vulnerabilities, including a complex email-based script injection.
4. 10:09 - 14:07

   Ramp Products and Research Blog Overview

   The insights derived from Ramp Labs directly influence commercial offerings like token spend tracking and model routers. Ongoing research focuses on amortizing post-training costs through latent projection and building self-improving software monitors.

   • Token spend management products provide semantic analysis of LLM usage linked to accounting categories.
   • Ramp Router integrates strategies like NVIDIA Switchyard to optimize latency and token costs.
   • Ramp Labs publishes research on cross-model weight transfer, automated Datadog monitoring, and semantic tagging.