Your AI Agents Shouldn't Care Where They Run - Nikhil Harithas, Factory

Highlights

• Factory's coding agent, Droid, encountered strict enterprise security and deployment constraints when engaging with large financial institutions like Blackstone.

• Open Compute is an upcoming open-source project designed to abstract the compute lifecycle for any arbitrary agent within a customer's environment.

• The Open Compute architecture runs entirely within a customer's Virtual Private Cloud (VPC) to ensure complete data control and security.

• Once initialized, the compute instance establishes an outward WebSocket connection to a control plane relay, eliminating the need for inward ingress connections.

• Open Compute drivers wrap around different compute environments such as Amazon EC2, Kubernetes, or sandbox providers like Daytona and Modal.

• The compute management layer supports essential lifecycle operations including pausing, resuming, destroying, and snapshotting sessions for multi-day agent workflows.

Timeline

1. 00:00 - 04:46

   Enterprise Deployment Challenges with Autonomous Agents

   Deploying coding agents like Droid within fractally complex enterprises such as Blackstone highlights the inadequacy of default deployment models. Strict regulations prevent running agents on local laptops or relying on shared sandbox identities. Enterprises demand complete single-tenant control, forcing engineering teams to adapt to diverse customer-preferred infrastructure types including Kubernetes, Google Cloud, and EC2 fleets.

   • Deploying autonomous software engineering agents in large enterprises reveals significant security and compliance hurdles.
   • Traditional agent architectures heavily rely on borrowing individual user identities and sandboxes, which fails to satisfy enterprise constraints.
   • Customers require the flexibility to bring their own compute, inference engines, identities, and single-tenant environments.
2. 04:46 - 11:40

   Introducing Open Compute and Architecture Patterns

   Managing the raw compute lifecycle manually remains overly complex for customers, prompting the creation of Open Compute. The system uses a compute gateway to communicate with a customer-side controller, which utilizes specific drivers to spin up compute instances like EC2 machines or Kubernetes pods. Security teams often restrict inward ingress traffic, so the sandbox initiates an outward WebSocket connection to a relay in the control plane to enable seamless back-and-forth communication.

   • Open Compute is developed to abstract the compute lifecycle for agents without placing the burden entirely on the customer.
   • The architecture relies on a compute gateway, a customer VPC-bound controller, drivers, and a control plane relay.
   • Outward-facing WebSocket connections bypass strict enterprise ingress restrictions while maintaining full security containment.
3. 11:40 - 17:08

   Compute Lifecycles, Drivers, and Future Open Sourcing

   Managing arbitrary compute introduces challenges such as handling dropped requests, bookkeeping state, and overcoming session duration caps like AWS micro-VM eight-hour limits. Snapshotting allows long-running missions to save state, fork tasks, and rehydrate across new instances seamlessly. By decoupling session logic from computer management, the open-source specification allows developers to easily write custom drivers for any compute provider.

   • Open Compute focuses on managing computers rather than specific application sessions to maximize reusability.
   • Snapshotting and forking capabilities support complex multi-agent swarms and multi-day orchestration workflows.
   • The project is scheduled to be open-sourced to allow broader community contributions and flexible driver integrations.