00:00:00(upbeat music)
00:00:02- Hi, thanks so much for having us.
00:00:11I'm Jack, and along with my colleague Nikita,
00:00:13who will be out here soon,
00:00:15we built Lightfield, an AI native CRM.
00:00:19We started using AI SDK V4 back in January,
00:00:22and adopted V5 as soon as it hit alpha in June.
00:00:26Today, we want to share with you
00:00:27how we built a production system
00:00:29where AI agents have secure full read-write access
00:00:32to customer data,
00:00:34how we handle human-in-the-loop workflows,
00:00:37and the architectural decisions that made it all work.
00:00:40We'll walk through the patterns we discovered,
00:00:42the trade-offs we made,
00:00:44and how the AI SDK enabled us to move fast
00:00:46without painting ourselves into a corner.
00:00:49But first, let's talk about why CRMs were broken,
00:00:53and why this matters.
00:00:54So, who's familiar with CRMs?
00:00:59Maybe?
00:01:00Some engineers?
00:01:01Yeah, so here's what's supposed to happen, right?
00:01:03You start talking to customers.
00:01:05Maybe you're a founder doing sales.
00:01:07Maybe you're on the sales team.
00:01:10At first, it seems manageable.
00:01:11You remember everyone.
00:01:13Every conversation is fresh in your mind.
00:01:16Then, you hit 10 customers, 20, 50,
00:01:20and someone on your sales team asks,
00:01:23"Hey, what did Sarah from Acme say about our pricing?
00:01:26Did she have concerns about the enterprise tier?"
00:01:29So, now you're searching Slack.
00:01:31You're searching your email.
00:01:32You're searching Google Docs.
00:01:34Maybe that Zoom recording that hasn't been transcribed yet.
00:01:38You eventually find it buried in a thread from two weeks ago,
00:01:41but you realize you never updated your spreadsheet again.
00:01:44So, you buy a CRM.
00:01:46It promises to be your single source of truth,
00:01:49but it just becomes another place you forget to update.
00:01:52Here's the problem.
00:01:54Traditional CRMs were built decades ago
00:01:56with the fundamental assumption
00:01:58that humans will do manual data entry.
00:02:01They gave you these rigid fields and predefined schemas,
00:02:04but the actual context, nuance of your conversation,
00:02:08it lives in your email, Slack,
00:02:11meeting notes, different places.
00:02:13And CRM just becomes a reporting tool for your VP of sales,
00:02:17not something that helps you sell.
00:02:20So, we thought there had to be a better way.
00:02:22What if the system could just remember?
00:02:24What if it captured everything intelligently
00:02:27and could actually take action on your behalf?
00:02:30That's Lightfield.
00:02:31So, Lightfield reimagines what a CRM should be.
00:02:35It's a system of memory and action for startups.
00:02:39So, it has automatic capture.
00:02:41Conversations, meetings, emails, they all get captured
00:02:46and structured without manual entry.
00:02:50It has lossless memory.
00:02:52We support schema lists and customizable schemas.
00:02:54You don't need to know what to track up front
00:02:56or pay a consultant to set it up for you.
00:02:58And it turns memory into action.
00:03:02Lightfield uses all that captured context,
00:03:05both the structured and the conversational data,
00:03:07to draft follow-ups, surface insights,
00:03:10and automate workflows for you.
00:03:11Now, traditionally CRMs are built
00:03:14for sales team tracking sales deals,
00:03:16but because Lightfield captures and structures
00:03:18all this conversational data,
00:03:20it becomes really powerful for anyone who needs to remember
00:03:24and act on customer context.
00:03:26What were the most requested features
00:03:29from last week's onboardings?
00:03:31Customer success teams understanding patterns
00:03:33across support conversations.
00:03:35Same system, different questions,
00:03:37but all powered by that same memory layer.
00:03:40That's the product.
00:03:41Let me show you what it actually looks like.
00:03:43So, here's an example of asking Lightfield agent.
00:03:48I think we're asking find five stalled ops
00:03:52and draft a personalized email to each.
00:03:55So, it can search across all of your customer information
00:03:58using an agent built on AI SDK.
00:04:02It can understand what are stalled ops,
00:04:05and then it can use that information
00:04:07to draft customizable emails
00:04:09to all of the people for those opportunities.
00:04:23So, here's an example.
00:04:25And then, you know, a user can,
00:04:27we can now send that email best for you.
00:04:29So, how does all this work?
00:04:34Let's walk and talk through
00:04:35what's happening underneath the hood.
00:04:37A user takes an action.
00:04:39So, this could be sending a chat message.
00:04:41This could be an external event, like a trigger,
00:04:43like an email or finishing a meeting.
00:04:47The agent immediately gets context.
00:04:50Where's the user in the app?
00:04:52What have they been doing recently?
00:04:54And what's their intent?
00:04:55What tools are available to them?
00:04:57Then Lightfield kicks in.
00:04:59It searches for relevant data,
00:05:01takes action in the CRM and updates records and response.
00:05:05All of this happens through the same unified data layer
00:05:08that powers the UI.
00:05:10Let me show you how we do this.
00:05:11Here's the architecture that makes this all work.
00:05:15Three different interfaces here.
00:05:19UI for humans, agents for natural language,
00:05:23and workflow jobs for automation.
00:05:26Here's the key.
00:05:27They all interact through the same unified layer,
00:05:30domain objects.
00:05:32So, they have the same permissions.
00:05:33The agent has the same permissions
00:05:35as the user enacting the agent.
00:05:37Same business logic and same data access patterns.
00:05:41There's no separate agent API
00:05:43with different rules or limited access.
00:05:45So, we bring together storage
00:05:49from a variety of systems here.
00:05:51So, structured data, object storage,
00:05:53and indexed in various search platforms.
00:05:57So, we provide the same capabilities
00:05:59and the same interface.
00:06:01So, one principle that we use to build our platform
00:06:08is agent UI parity.
00:06:10So, if a user can access it, an agent can access it.
00:06:14Full read, create, and update capabilities across all data.
00:06:19So, the same permissions, same visibility, same operations.
00:06:24Well, it's a product and architectural choice for us
00:06:26that we made from day one.
00:06:28It's why building AI native from the ground up
00:06:31beats bolting agents onto legacy systems.
00:06:34So, agents in Lightfield act on your behalf
00:06:37with the same permissions through the same data layer
00:06:40that powers the UI.
00:06:42They're just another interface to your data.
00:06:44So, when we are choosing tools to build Lightfield,
00:06:49we needed primitives that wouldn't force us
00:06:50into different architectures for agents versus users.
00:06:54That constraint influenced our whole stack,
00:06:56including the AI framework we picked.
00:06:58So, and for us, the thing about building AI products in 2025
00:07:06is nobody has the full playbook, right?
00:07:10So, we're trying to optimize for learning speed
00:07:13over perfection.
00:07:14So, we actually dog food this concept with Lightfield.
00:07:19When our engineering teams needs to understand
00:07:21the customer issue, they don't have to navigate through CRM.
00:07:25They can just ask it.
00:07:26So, natural language is really the interface
00:07:33that we want there.
00:07:35So, AISDK gave us the flexibility to iterate on this
00:07:39without rewriting everything.
00:07:41But the key was mindset.
00:07:43We focused on building features and solving real problems,
00:07:47not fighting frameworks or over-engineering abstractions.
00:07:50So, the key here is move fast and learn quickly.
00:07:53So, we kept coming back to this quote.
00:08:02"Duplication is far cheaper than the wrong abstraction"
00:08:06from Sandy Metz.
00:08:08And I think this is pretty prevalent
00:08:11in building AI products today.
00:08:13It's very fast to build software quickly now.
00:08:17It's even faster than it was a year ago.
00:08:19And making sure that the right framework exists
00:08:22is really important.
00:08:23And having the wrong abstraction can be even more costly.
00:08:27So, let's talk about this more in practice.
00:08:34So, as we've been building Lightfield,
00:08:38we started developing AISDK in January of this year.
00:08:43So, we adopted it to support model switching
00:08:51and started using the stream text kind of primitives.
00:08:54And so, we were able to ship early tasks
00:08:56to specific agents in weeks.
00:08:58So, we started building more and more agents
00:09:01in more and more chat features.
00:09:04And in June 2025, we started adopting the useChat API,
00:09:09specifically because of the custom transport options
00:09:15that were released.
00:09:16So, the main thing here is we've been able to adopt AISDK
00:09:22go from V4 to alpha V5.
00:09:25So, I guess it sounds like V6 will be released soon,
00:09:30pretty seamlessly with kind of moving fast.
00:09:34We have kind of a joke internally
00:09:40that we'll identify a feature we need from the AISDK
00:09:43and the next day we'll see a tweet from the AISDK team.
00:09:46And learning this morning, I guess, that Nico has an agent
00:09:49that just generates those tweets.
00:09:51So, it's pretty funny to see that.
00:09:53So, that's exactly what you want from a framework.
00:09:57It grows with you instead of forcing you
00:09:59to rewrite or slow down.
00:10:00So, here's an example of Lightfield in action here.
00:10:05So, in the chat here, I'm asking,
00:10:12I'm typing a question, what's next for this account?
00:10:16What did Jordan Lee mention on our last call?
00:10:19So, notice what the user didn't have to do.
00:10:21They didn't have to say the account is streamlined protocol
00:10:26or ask specifically about a certain meeting.
00:10:30So, we use AISDK to build this feature we have
00:10:34called Adaptive Context Building.
00:10:37So, it provides signals from the user
00:10:40combined with intelligent retrieval
00:10:42to figure out what actually matters for that.
00:10:45So, let me share some examples
00:10:47of how we use the SDK to do this.
00:10:48So, the SDK has a API called Data Parts
00:10:54and we use this to provide signals from the client
00:10:57to the server that's actually building the context.
00:11:01We can, on the client, we can use different entities
00:11:05and provide different signals using the Data Parts API
00:11:08and then we hydrate this fully on the server.
00:11:11I'm gonna let my colleague Nikita talk more
00:11:15about how we use Data Parts to build more features here.
00:11:19(upbeat music)
00:11:24(upbeat music)
00:11:27- Thanks so much, Jack.
00:11:28So, another example similar to Adaptive Context Building
00:11:32is how we inject files into the chat thread.
00:11:35The AISDK provides us with a really easy way to do this.
00:11:39We can simply use the send message function
00:11:42from the use chat hook,
00:11:43provide it with the user's query and the file list
00:11:47and it'll work with any provider right out of the box.
00:11:50But this brings up some practical concerns
00:11:52around scalability.
00:11:54For example, how do we make sure
00:11:56that we avoid persisting that data directly in the database
00:11:59if we're directly encoding the files?
00:12:01If we're using S3 URLs, how do we make sure
00:12:05that we don't accidentally expose
00:12:07that private user data to the public?
00:12:09Our solution to this is to instead have the client
00:12:14send the backend an internal ID referencing the uploaded file
00:12:19inside of our own data store.
00:12:21In the backend, we'll iterate through all of the file parts
00:12:25and replace those internal identifiers with signed S3 URLs.
00:12:30This enables the external LM providers
00:12:34to still view these attached files,
00:12:36but the expiration time on the signed URLs
00:12:39prevents unauthorized access.
00:12:41Another example of how we protect user data in Lightfield
00:12:46is through this concept of contextual tool collections.
00:12:50Whenever a user interacts with Lightfield's chat product,
00:12:54we'll dynamically construct a tool set
00:12:58that's specific to the user.
00:13:00We'll inject those dependencies directly into the tools.
00:13:03For example, in this data retrieval tool,
00:13:06we inject the user's IDs directly into the tool itself.
00:13:11The LLM never directly issues queries to the database.
00:13:15It always goes through the same unified data layer
00:13:19that the user would access
00:13:21through the rest of the CRM's interface.
00:13:23So we have this design philosophy of maintaining parity
00:13:30between the CRM's UI and the agent's capabilities.
00:13:34When the user can create CRM entities like accounts,
00:13:38opportunities, and contacts
00:13:40through this modal interface in the UI,
00:13:43we want them to be able to do the same thing
00:13:45through the chat-based interface.
00:13:48The LLM can issue a tool call to create these accounts
00:13:51and will render a form with the same inputs
00:13:54that are shown inside of the user interface.
00:13:57We built this by leveraging the AI SDK's
00:14:01human-in-the-loop abstractions.
00:14:03The way this basically works
00:14:04is that when the LLM issues a tool call
00:14:07that requires confirmation,
00:14:09it'll forward that tool call to the front-end client.
00:14:13The client will render an interface
00:14:16and append a tool result depending on the user's action.
00:14:20On the back-end, right before we submit that output
00:14:25to the LLM, we'll execute the functions
00:14:29depending on what the user submitted.
00:14:31A schema describing how did we do this is shown here.
00:14:37So the user's initial input is this tool call.
00:14:43The LLM suggests a set of input values,
00:14:46in this case an array of items
00:14:47representing the account names and their domains.
00:14:51After the user edits the values,
00:14:54the output becomes the user's edited values
00:14:57along with an additional field
00:14:59indicating whether they approved that particular item.
00:15:03After the actual function is executed,
00:15:06we append that result to the tool output
00:15:09before it's sent to the LLM.
00:15:11For example, was the account creation successful
00:15:15or did it fail for some reason,
00:15:16such as perhaps the account already exists in the CRM?
00:15:19This provides the LLM with full visibility
00:15:24into the history of the interaction.
00:15:26It can see the entire flow,
00:15:29the originally suggested values and the outputs.
00:15:33This provides it with an ability
00:15:35to appropriately suggest the next steps.
00:15:38So we also have this design principle
00:15:41of enabling the user to mold the CRM to fit their needs.
00:15:45Every business has unique aspects about themselves
00:15:50and unique sales processes.
00:15:52We want you to be able to customize the CRM
00:15:55and customize your experience with the agent
00:15:58to fit your specific needs.
00:16:00Inside of Lightfield, you can construct a custom data model
00:16:06for each of the CRM entities.
00:16:08For example, if you're a B2B business productivity tool
00:16:13trying to sell your coding tool to startups,
00:16:16you might be particularly interested
00:16:18in tracking your customer's tech stack,
00:16:20the size of the engineering team,
00:16:22and perhaps any mutual investors that you have with them.
00:16:26Inside of Lightfield,
00:16:27you can specify all of these typed fields.
00:16:30And you can specify how the agent
00:16:35should use these fields in its processes.
00:16:37You can provide additional instructions
00:16:40over the meanings of these fields
00:16:42and how it should use these fields
00:16:45when updating them in the various background workflows.
00:16:48For example, if you created a field,
00:16:52you can ask the agent to backfill it
00:16:56by doing deep research on the web
00:16:58and enrich these fields
00:17:00for all of the accounts in your system.
00:17:02Or you can ask it to backfill
00:17:05by searching through your CRM records,
00:17:07which include your meeting transcripts, emails,
00:17:10and other interactions with the account.
00:17:12The way this looks on the backend
00:17:18is we create this tool at runtime,
00:17:22which with a schema that's based on
00:17:24your company's particular configuration.
00:17:28The actual tool schema itself is derived from that database.
00:17:32And when the LLM suggests values,
00:17:33we'll validate the types to ensure that they match that schema.
00:17:38This enables us to build these really flexible
00:17:40and highly reliable tools.
00:17:42Inside of Lightfield,
00:17:45you can also configure this knowledge section
00:17:48where you can provide the LLM
00:17:50with additional context about your business.
00:17:53You can provide information about your company's products
00:17:57and also provide instructions
00:18:00for how the LLM should run background workflows,
00:18:05such as meeting prep.
00:18:06Before every meeting,
00:18:10Lightfield will prepare a document for you,
00:18:12preparing you for the discussion.
00:18:15It'll list the key attendees
00:18:16and additional information about them.
00:18:19It'll list information about the particular account
00:18:22that you're meeting with,
00:18:23and as well as other important key discussion points.
00:18:27After the meeting, it'll suggest follow-up action items
00:18:31and suggested field updates based on what you've discussed.
00:18:35All of these basic building blocks combine
00:18:39to unlock powerful new capabilities.
00:18:42Because Lightfield has the full context
00:18:44of all of your sales interactions
00:18:47and has a high degree of customized knowledge,
00:18:50it can collaborate with you to rapidly generate
00:18:53high-quality emails upon your behalf.
00:18:56For example, after a meeting,
00:18:58you can use this tool to access your Google Calendar
00:19:02to view your availability.
00:19:05When this draft email artifact is generated,
00:19:08it can appropriately suggest follow-up times
00:19:12based on your previous discussions.
00:19:14These draft emails are still gated behind user approval,
00:19:18so you can be confident that the LLM agent
00:19:21will never take action without your explicit approval.
00:19:25These follow-up action items and email drafts
00:19:28are prepared for you, and will send you notifications for,
00:19:33to help ensure that you stay on top
00:19:35of every deal that you're working on.
00:19:37All right, back to you, Jack, to bring this all together.
00:19:43- Yeah.
00:19:46(audience applauding)
00:19:50So, thanks Nikita.
00:19:53So, the core principles we discovered
00:19:56while building Lightfield with AI SDK.
00:19:59Principle one, agent UI secure parity.
00:20:03Designed for this from day one.
00:20:05Agents need full read-write access
00:20:07through the same data layer humans use.
00:20:09Don't build a separate agent API.
00:20:11You'll end up maintaining multiple systems,
00:20:13and neither will feel complete.
00:20:15Principle two, fast iteration over perfect abstraction.
00:20:19Optimize for learning speed early, not perfection up front.
00:20:23We had similar looking code across chat agents,
00:20:25API features, and background workflows.
00:20:28Some duplication is genuinely cheaper
00:20:30than the wrong abstraction,
00:20:32especially when conventions are forming.
00:20:35Principle three, human in the loop workflows users trust.
00:20:41People need to stay in control,
00:20:42especially for high-stakes interactions.
00:20:45We intercepted the tool layer.
00:20:48The agent sees the original suggestion,
00:20:50the user's edits, and the execution result.
00:20:52Full transparency, full history.
00:20:56That's what earns trust.
00:20:58Principle four, programmable systems by users and agents.
00:21:02Real customers need custom data models.
00:21:04Every business tracks things differently.
00:21:07Both users and agents can define new fields,
00:21:10and the system can adapt to it.
00:21:13This means your product molds
00:21:14to how customers structure their data,
00:21:16not the other way around it.
00:21:18It's more complex to build out,
00:21:20but it's the difference between a product people tolerate
00:21:22and one they can't live without.
00:21:24So we'd love to hear what you're building
00:21:26and what patterns you're discovering.
00:21:28Come find us after or check us out at lightfield.app
00:21:32to see these principles in action.
00:21:34Thank you.
00:21:35(upbeat music)