Stop Giving Your Agent Tools. Give It a Sandbox - Mikyo King, Arize AI

하이라이트

• Transitioning from traditional tool calling to code mode reduces token usage by 43% and lowers costs by 31%.

• Traditional tool calling accumulates context window bloat, increasing latency, turns, and cost with every tool invocation.

• Code mode allows agents to execute multiple tools in a single turn and return only necessary data to the context window.

• Arise integrates bash, a Python sandbox, CRUD endpoints, and SQL writing capabilities into its Phoenix open-source observability platform.

• Running agents through the Harbor framework with network access disabled helps eliminate flakiness during evaluation.

타임라인

1. 00:00 - 01:58

   Evolution of Software and Agent Architectures

   Programming enables users to write short custom scripts to automate tasks like extracting email invoices, whereas non-technical users depend on restrictive software interfaces. Coding agents replicate this programmatic flexibility by dynamically generating software to meet specific user needs. Foundation model providers recognize this trend, driving training efforts toward software engineering benchmarks.

   • Software divides historically into two categories built for programmers and non-programmers.
   • Coding agents dissolve boundaries by dynamically building required software solutions.
   • Foundation models train extensively on coding benchmarks like SWE benchmark.
2. 01:58 - 04:36

   Limitations of Traditional Tool Calling and the Code Mode Alternative

   Conventional agents suffer from context bloat as tools and raw outputs accumulate across multiple interaction turns, increasing latency and cost. Code mode bypasses this limitation by giving agents a complete programming environment. Instead of feeding every tool definition upfront, the model writes code to call multiple tools simultaneously, filtering out unnecessary output before updating its context window.

   • Traditional tool-calling architectures push large tool catalogs into context windows iteratively.
   • Context rot, multi-turn overhead, and data pagination plague traditional tool execution.
   • Code mode provides a programming environment where agents search, compose, and execute tools sequentially or in parallel.
   • Code mode executes multiple tools in one turn and minimizes context window pollution.
3. 04:36 - 06:14

   Core Advantages of Code Mode

   Progressive tool disclosure allows agents to identify and understand only the tools required for a specific task. Composing tools in parallel or sequence enables complex operational workflows. Furthermore, agents can dynamically write custom tools to handle tasks outside their pre-built catalog, such as sanitizing sensitive personal data without leaking raw details into the context window.

   • Progressive tool disclosure eliminates the need to load giant tool catalogs upfront.
   • Composability allows chaining tools together in sequence or parallel for efficient execution.
   • Dynamic tool creation unlocks emergent behaviors like stripping personally identifiable information safely.
   • Token efficiency improves because models trim and search outputs before returning data to the context window.
4. 06:14 - 11:44

   Implementation in Phoenix and Architectural Challenges

   Arise embeds bash primitives, Python runtimes, CRUD operations, and SQL text generation inside Phoenix to support aggregate span queries and data manipulation. Binding a programming paradigm to a web worker enables complex browser UI interactions. However, adopting sandboxes brings operational complexity, requiring strict controls over network traffic, credential vaulting, overbroad permissions, and multi-layered execution parsing.

   • Arise integrates bash, Python sandboxes, CRUD endpoints, and SQL capabilities into the Phoenix observability platform.
   • A web worker-based programming paradigm allows agents to interact with frontend UI and state.
   • Sandboxing introduces engineering challenges involving resource management, network egress, secret vaulting, and permissions.
   • Double sandboxing occurs when code-mode harnesses interact with external protocols like MCP.
5. 11:44 - 14:30

   Evaluation Framework and Concrete Results

   Rigorous evaluation is critical due to the non-deterministic nature of LLM-generated code. By running automated test suites through the Harbor framework, Arise measures performance consistency and identifies instruction flaws. Empirical results confirm that code mode successfully decreases token counts by 43% and cuts costs by 31% while maintaining stable latency.

   • Arise evaluates agents using the Harbor framework inside isolated sandboxes with disabled network access.
   • Harbor utilizes binary verifiers and tracks pass rates across multiple permutations to counter system flakiness.
   • Deploying code mode reduces token consumption by 43% and lowers total costs by 31%.
   • Trajectory and evaluation data enable iterative prompt and instruction optimization.