Observability for the AI Cloud

Malavika, a product manager at Vercel, opens by welcoming the conference audience and establishing the context for the discussion. She emphasizes that Vercel is a unified platform for building, deploying, and running intelligent applications and agents, with a core mission to abstract away infrastructure complexity and let developers focus on building amazing user experiences. The speaker acknowledges that the AI Cloud concept has been emphasized throughout the conference and sets up the discussion by noting that while Vercel has succeeded in abstracting complexity at build time, there is still significant work to be done at runtime. This introduction frames the need for better runtime observability and incident management as the central challenge Vercel aims to address.

The speaker highlights the substantial manual effort required for runtime application management, comparing it unfavorably to the streamlined build-time experience. She describes the incident response workflow: teams must set up alerts and monitors to identify problems, then deal with alert fatigue as they filter signal from noise, and finally spend hours debugging to identify root cause and apply fixes. Through an interactive audience engagement, Malavika reveals that approximately 50% of incident time is spent on identifying the root cause and determining ownership, which she emphasizes is wasteful given current AI capabilities. She poses the question: 'What if we could reduce that to seconds?' to introduce the solution Vercel is building, establishing the urgency and business value of addressing this pain point.

Vercel Agent is introduced as the company's solution to dramatically reduce debugging time through AI-powered anomaly detection and root cause analysis. The speaker explains that Vercel Agent monitors applications for suspicious activities out-of-the-box with no configuration required, and when unusual behavior is detected, the agent investigates automatically, performs root cause analysis, and diagnoses problems in seconds. A key competitive advantage is that Vercel has full context on applications since they build, deploy, and run them in production, positioning them uniquely to provide AI-native approaches to ensuring reliability, performance, and security. Vercel Agent investigations build on top of native observability tooling including runtime logs, opinionated dashboards, anomaly alerts, and a query tool that allows developers to explore vast amounts of collected metrics and answer detailed questions about their applications.

The speaker details the components of Vercel's native observability tooling designed to provide comprehensive visibility into runtime behavior while maintaining build-time context. Runtime logs offer granular visibility into application behavior and allow developers to trace HTTP requests from ingress into the Vercel network to the response returned to the client. Opinionated dashboards are provided out-of-the-box to help developers quickly understand application health, pinpoint issues, and optimize performance. Anomaly alerts enable active monitoring for unusual activity, allowing teams to identify and resolve issues quickly without waiting for users to report problems. The query tool empowers developers to explore metrics and author custom queries to answer specific questions, ranging from identifying bot traffic to quantifying performance metrics like P90 time to first token for various model providers, giving developers granular control over what they monitor.

The speaker articulates Vercel's vision for the future of the AI Cloud, emphasizing a fundamental shift from reactive monitoring and information delivery to proactive problem-solving. Rather than simply identifying and informing developers about issues, the goal is for the AI Cloud to actively repair and optimize applications by providing solutions, recommendations, pull requests, and automated actions. This represents a significant evolution beyond traditional observability platforms that only present problems, moving toward an autonomous system that can implement fixes and improvements. The vision acknowledges that the current generation of Vercel Agent represents a starting point, with automatic remediation and pull request generation as features the company is building toward, representing the ultimate goal of reducing developer toil in production incident management.

The speaker provides practical information about how developers can access Vercel Agent features and what tier of service includes which capabilities. Vercel Agent is in public beta with $100 in free credits offered to all Vercel users to try the new features. The code review skill launched previously and is available to all Vercel users. Agent investigations are available starting immediately and are accessible to pro and enterprise customers who have observability plus. Developers can access and configure these features by visiting the agent tab in the Vercel dashboard, making it straightforward to begin using the new capabilities without requiring additional setup or configuration.

The speaker transitions to discuss the importance of monitoring output quality in non-deterministic AI applications, noting that agents introduce even more complexity through their multi-step reasoning processes. She highlights that a large ecosystem of agent frameworks has emerged using OpenTelemetry to help developers monitor and debug agent workflows. To address this need while maintaining developer flexibility, Vercel has partnered with Brain Trust through the Vercel marketplace, allowing developers to automatically stream traces and evaluation data from Vercel to Brain Trust with just a few clicks. The company provides a demo app (an AI chatbot built with Next.js, AISDK, and AI Gateway with traces sent to Brain Trust) that developers can clone from the template library to quickly get started with evaluation monitoring. This approach reflects Vercel's philosophy of providing both native observability tooling and flexible integration with third-party tools through Vercel Drains, ensuring developers have freedom to choose their tools while being able to easily collect and export their data.