CopilotKit: The Problem with AI Chatbots in SaaS Apps

EnglishالعربيةDeutschEspañolFrançaisहिन्दीBahasa Indonesia日本語한국어PortuguêsРусский中文
BBetter Stack
Computing/SoftwareSmall Business/StartupsInternet Technology

Transcript

00:00:00most ai features and apps have the same problem they look good but really they're just a chat box
00:00:06slapped to the side of the screen you ask for something it gives you a markdown maybe a table
00:00:11maybe even a few paragraphs explaining the work you still have to do but what if the agent didn't
00:00:17just talk this is copilot kit it can render real components share live state with your app
00:00:23and stop to ask for approval before it changed anything so can this make an ai feature feel
00:00:29more like part of the product let's find out
00:00:37now first off a lot of people think they're adding ai to the app but what they're really adding is a
00:00:43second app inside their app your product is over there the ai is over there and the user has to
00:00:49copy context back and forth in their head to really get anything working now that's fine if all you
00:00:54really need is this basic q a structure but the second you want the agent to update state
00:01:00call tools or work with the user inside a real workflow we start to hit a wall now you're building
00:01:07streaming events state sync approval flows and everyone is rebuilding the same thing just slightly
00:01:14different copilot is going after this problem up front not how do we make another chatbot the better
00:01:20question is how do we make agents feel native inside the apps themselves so let's start with the demo then
00:01:26i'll break down what's real what's useful and what's just a little bit too much if you enjoy coding tools
00:01:32to speed up your workflow be sure to subscribe we have videos coming out all the time all right now
00:01:37watch this i'm starting from a clean terminal and we can just run our install i got from the copilot kit
00:01:42this scaffolds the app and gives me a working starting point just like any other project that we'd really
00:01:49start up not just an empty chat bubble but real pieces the front end pieces are already wired together
00:01:56enough that i can start testing the actual experience that we want now i jump in and i run the development
00:02:02mode and here's the app running locally now of course this part looks familiar there's a chat surface
00:02:09but that's not really the whole interesting part here the really cool part is what happens when
00:02:13the agent is connected to the ui i'm going to ask a question here in the interface okay there we go
00:02:21and notice the big thing here the agent doesn't just answer with some blob of text it can stream the
00:02:26response it could call tools and it could render an actual component inside the app that's the first
00:02:32really big change here with copilot kit the ai is not just describing the interface it's now
00:02:38a piece of all of it now let's push it a step further i'm going to ask it a follow-up question here
00:02:46okay and here's the part a lot of other agents are going to skip now our agent pauses it asks for
00:02:52approval and it asking for approval is the part that matters that's the part that i really like because
00:02:57in real software control is not really optional our users still need the final say in really what's
00:03:03happening now cool demo right now the easy way to explain copilot kit is really just this copilot kit
00:03:09is a front-end stack for agentic apps it's not just a chat component it's not just a wrapper around
00:03:15another llm api it gives you the pieces we need for ai experiences that live inside of the product there
00:03:23are four pieces to really understand here first is agui now agui is an open event based protocol
00:03:31for how agents talk to front ends right now agent tooling has connection problems you have land graph
00:03:37crew ai master custom agents and whatever new framework launched this morning we're trying to
00:03:43drop that in our app then you have react apps mobile apps dashboards all this other stuff that's
00:03:48happening that's a lot of different things to juggle all of these things are being juggled without a shared
00:03:55protocol every back end needs custom code for every front end agui is trying to become the shared
00:04:01language between the agent and the interface messages state updates tool calls ui events all moving
00:04:07through a common event stream then we have generative ui instead of the model only returning text the agent can
00:04:14trigger real components it's not random htl it's just your ui rendered at the right moment then is shared
00:04:22state or what copilot kit calls co-agents the front end and the agent back end can share state in both
00:04:29directions when the user changes something the agent can react when the agent updates something the ui can
00:04:35reflect it this is huge when we're building out tools dashboards anything where the user and the agent are
00:04:41working on the same object then finally is the whole human in the loop thing this is one of the most
00:04:47important parts with all of this a lot of agents a lot of agent demos can act like the best experience
00:04:52is full autonomy but really that's not always the case right in real products users want power and
00:04:59control confirm before sending approve before creating choose between options all of the stuff and copilot
00:05:05gives us things for just that now copilot is not the right choice for everyone right if you compare it
00:05:10to something like for cell ai sdk copilot kit is more batteries included if you want streaming chat
00:05:17generative ui shared state and human approved patterns right out of the box this is what copilot kit is
00:05:23getting closer to a the full product layer for cell ai sdk is lighter it gives you more low level control
00:05:31so if you want to own every part of the architecture then sure that's honestly a much better fit now if
00:05:38you compare copilot kit to building everything yourself this is where it gets pretty hard to beat
00:05:43because the hard part is not rendering out some chat bubble that's now becoming the easy part right the
00:05:49hard part is everything surrounding that but of course with any of this there is a trade-off it can feel
00:05:54heavier than a minimal ai sdk of course you are adopting copilot kits patterns also it's free to
00:06:02an extent honestly if you're playing around in solo okay it's super cool but it becomes annoying when
00:06:08you want to use this for scale because then all of that is not free also if you guys do know a complete
00:06:14open source alternative to this drop a comment and let me know because i'm searching for just that
00:06:19with copilot kit you do need to understand what is open source you need to understand what needs keys
00:06:25what's hosted what's paid this is not just a dunk on copilot saying it's all bad but you do need to
00:06:32understand what's free and what's not if you only need a basic support chatbot copilot kit is probably
00:06:38an overkill but if you want something that works really well for agentic ux where the agents work with
00:06:44your ui in your app state this is really cool so should you use copilot kit well give it a shot if
00:06:50you're building some serious in-app ai especially if you're in react or next js and you want the ai to
00:06:56feel like a part of the product not just a side panel this is really cool you can spin it up really
00:07:02fast and with all the components we have it makes it really practical to drop in the workflow if you
00:07:07already built your own streaming state tool call ui switching might not be worth it and if your
00:07:13feature is literally just ask a question get an answer copilot kit is going to be a big overkill for
00:07:19that you probably don't need a full agentic front end stack just use something lighter okay ship it
00:07:25move on get going if you enjoy coding tools like this be sure to subscribe to the better stack channel
00:07:30we'll see you in another video

Key Takeaway

CopilotKit enables AI to function as a native part of the product interface by providing an agentic front-end stack that supports generative UI, bidirectional state synchronization, and built-in human-in-the-loop approval workflows.

Highlights

  • Most AI features in SaaS apps function as isolated side panels that require users to manually transfer context between the app and the chat interface.

  • CopilotKit provides a front-end stack that renders real components, shares live application state, and handles human-in-the-loop approval workflows.

  • The AGUI protocol creates a shared event-based language for communication between AI agents and diverse front-end frameworks like React and mobile dashboards.

  • Generative UI allows AI agents to trigger specific application components directly rather than relying on plain text or Markdown responses.

  • Shared state (Co-agents) enables bidirectional synchronization, allowing the agent to react to user input and the UI to reflect agent-driven updates in real time.

  • CopilotKit is intended for complex, agentic workflows, while lighter libraries like the Vercel AI SDK are more suitable for basic question-and-answer functionality.

Timeline

Limitations of Standard AI Chatbots

  • AI features often function as disconnected apps trapped inside a side-panel chat box.
  • Users bear the cognitive load of manually syncing context between the main application and the AI interface.
  • Building sophisticated agent features from scratch requires complex, repetitive development of streaming, state sync, and approval logic.

Standard AI integration in SaaS often results in a decoupled experience where the user must copy information back and forth. While basic Q&A bots survive this model, any attempt to implement workflows, tool calls, or state updates forces developers to rebuild the same infrastructure. CopilotKit addresses this by focusing on making AI agents feel like native components of the software.

Core Architecture and Demo

  • The CopilotKit scaffolding provides a fully wired front-end environment rather than just a chat component.
  • AI agents can stream responses that render functional UI components directly into the application space.
  • Built-in approval flows ensure users maintain control by forcing the agent to pause before finalizing state changes.

The demonstration shows a local development environment where the AI agent does not return text blobs but instead triggers live components. A follow-up interaction illustrates the human-in-the-loop requirement, where the agent pauses for user authorization. This design prioritizes user control over full agent autonomy, which is a requirement for production-grade software.

Components of an Agentic Front-end Stack

  • AGUI acts as an open, event-based protocol for bridging the gap between back-end agent frameworks and front-end interfaces.
  • Generative UI allows agents to interact with application components instead of outputting static text.
  • Co-agents enable bidirectional state management between the agent back-end and the user interface.

CopilotKit offers four main pillars for agentic applications. The AGUI protocol standardizes communication across various agent frameworks like LangGraph or CrewAI. This allows for unified event streams covering tool calls and UI updates. Additionally, state synchronization ensures that changes made by the user or the agent are reflected immediately across the entire system.

Implementation Considerations

  • CopilotKit is a batteries-included solution, while the Vercel AI SDK provides more low-level, architectural control.
  • The framework includes associated costs and usage limits that become relevant at scale.
  • Basic Q&A features do not require a full agentic stack and should use lighter alternatives.

Choosing between frameworks depends on the desired level of abstraction and the complexity of the feature. CopilotKit offers a rapid development path for in-app agentic experiences, whereas more minimal SDKs allow for bespoke architecture. Developers should evaluate their specific needs for streaming and state management before committing to the heavier pattern of a full agentic stack.

Community Posts

No posts yet. Be the first to write about this video!

Write about this video