Your agents lack context: Here's how to fix "You're absolutely right!" — Brandon Waselnuk, Unblocked
AAI Engineer
컴퓨터/소프트웨어AI/미래기술
스크립트
00:00:00Good afternoon. I hope you're all having a lovely day here at AIE. We've had great
00:00:17weather though the UV has been like nine so hopefully you put your sunscreen on
00:00:20you're being appropriate adults. I'm here to talk to you about context
00:00:24engineering and I have the good fortune of following AJ from LinkedIn because he
00:00:27talked a lot about the system that we actually design and sell to other
00:00:29solutions and I'm gonna give you a bunch of open source tools so if you watch
00:00:33that last talk just before me you're gonna get a bunch of tool chains you can
00:00:36go mess around yourself and I'll teach you a bunch of techniques today. The goal
00:00:39of course is to fix you're absolutely right. I think they've taken that out of
00:00:44the prompts now so it just says you're right or other things but I'm sure you've
00:00:47all been there. So I'm Brandon I work at unblocked yes I have a coconut we've been
00:00:53giving these away for fresh context fresh fresh coconuts but the thing that I want
00:00:57to talk to you about is with these models especially with mytho class models
00:01:01I think Fable 5 is coming back today so they say you can watch my grain call
00:01:05recording try to book this we'll ignore it. But what I want you to do is to think
00:01:11about the fact that with these tools AI generated code should feel like it was
00:01:15written by someone who's been on your team for years. So to get in the right headspace for
00:01:23years you have to consider that you have been the context engine. How did you do
00:01:27that? You built context by going to work and asking questions shipping PRs and
00:01:34getting them rejected going to meetings and all this slowly over time built up the
00:01:39engine that is your brain. You understand how it works here you know how stuff gets
00:01:42shipped you're on call that night when you took prod down and why that happened. The
00:01:48problem is that these agents have this exact same problem every time you create
00:01:53a new terminal session with an agent in it it's very intelligent but it doesn't
00:01:57have any context on how your company operates so it needs to get that somehow. The
00:02:02problem is as you move these agents up in scale that cost compounds if you get it
00:02:07incorrect at the beginning. The leverage off context and content. We're just gonna fix this
00:02:13because I think people want to take some photos. Perfect. That context issue will
00:02:20compound. So at the far left we all remember the age-old time of two years
00:02:25ago where we had tab complete models that were pretty cool. What happened is it
00:02:29popped up and said hey do you want to tab this and quickly in your head with your
00:02:32context and you go no that's bad. Or you went oh sweet you hit tab. Nice. As we
00:02:37move along the agentic kind of adoption curve what happens is you are moving into
00:02:41more situations in which you have agents running without a human in the loop or at
00:02:46least you wish you didn't have to be in the loop. What they need is some way to be
00:02:50able to ask the questions they need when they hit walls in order to write code or
00:02:54solve or basically fix the issue and ultimately output code that's mergeable
00:02:58into your code base especially with many people here who actually work in brownfield
00:03:02code bases that have been around for a long time that run real revenue across
00:03:06them not just greenfield fun projects. So that cost of bad context compounding at
00:03:12the beginning is cheap if you think like shift left finding a defect or a bug you'll
00:03:17want to find it as early as possible. It's the same with context because as you move
00:03:22across you get into doom loops you usually ask your agent to do something it's like
00:03:26hey I did it you're like no man and then you correct and correct and correct.
00:03:29That's wasted search tokens it's also wasted rework time and that is not
00:03:34acceptable with the tokenomics we have coming and then as you move into parallel
00:03:39agents etc you start hitting a review tax so these AI code reviewers were trying to
00:03:43use but again key context is important there so that those code reviews are able
00:03:48to basically understand how the operations of the business are so it knows the
00:03:52business logic and more and then finally if your hope is to move all the way out of the
00:03:57loop you're like background agents get it done make no mistakes you really need to
00:04:02make sure that you have a context engine so those agents can query it and get all
00:04:05the answers they need so they can keep operating in an effective way.
00:04:08There are some common approaches that don't work they're basically like a local maxima two of the ones
00:04:16we see the most with our hundreds of enterprise clients and mid-market sized businesses is the
00:04:21curated context trap if you've ever sat down and taken a virtual file system or maybe a local file
00:04:26system you put some markdown files in it you're like here's all the context of this project it's
00:04:30how it works you then allow your agent to grep over that and it gets a bunch of good data and then it will perform better.
00:04:35The issue is first now you have to distribute that so maybe throw it up in the GitHub and your team can grab it but then the next is that repo is going to rot just like all the other docs you wrote down and then who at your org is the omnipotent one who has the taste to curate this file or repo for literally everyone in the org.
00:04:52So you start to hit these issues the next is the MCP plateau this one is pretty clear we have MCP's they're great you can give it to your agent and now it can basically get information from another source system the problem is of course based on how you write the server description the tool descriptions your agent may never call it even though it should have or if it does there's a known bias called the satisfaction of search bias what that means is the agent when it finds a
00:05:22the first piece of information that it thinks is correct it goes oh I have what I need and it proceeds and most organizations there's a slack conversation from last night that says you should be doing a instead of doing b and the agent will never find it if it found some architecture record first so it doesn't actually consider all the context.
00:05:40the problem here is access to information is not understanding so to deliver understanding to a model you have to do other techniques what I'm basically trying to say is what your agent can't see is everything below the waterline it can a hundred percent get code that compiles but that code that compiles is taking down prod and you have a p0 at one in the morning because it missed the fact that you have a certain rollout procedure you're supposed to turn off a feature flag whatever it might
00:06:10be. So your team needs a context engine because what it should do is understand who you are and where you work in an organization so if I say to you I want to get auth stood up it knows where I work it knows where my git commits are it knows who reviews those commits and it understands that my context it can focus me and then use that as a trigger point to find the rest of the information.
00:06:33it resolves conflicts as mentioned an old architecture diagram and last night slack convo with the CTO which one is right you need to use a bunch of techniques to determine that.
00:06:44Respects permissions and governance of course MCP allows us to use OAuth and other scopes and SSO but if someone asks a question over here who's not supposed to know about secret project A you need to make sure that doesn't leak into the response.
00:06:58And then finally deliver the right context at the right time to the model in a token optimized way.
00:07:03We have multiple surface areas because human engineers still talk to unblocked all the time to get information they need in slack or otherwise.
00:07:09But then you want token optimized responses if you're just speaking machine to machine in order to not waste a bunch of bold classes on your token spend.
00:07:17This is how an engine works. I'm going to be brief on this but basically on the left hand side you see all the data sources that are coming in.
00:07:26For us we focus on engineering teams and that's who uses us as well as the technically light teams around it like support sales and otherwise.
00:07:34You ingest all that data you get real-time data from tools like your instant management tool chain.
00:07:39It comes into the engine where that engine is it thinks at the bottom I'll expand on that slide in a moment.
00:07:45But basically it uses these six key characteristics and then on the right you output the context to the exact workflow in the manner that it is needed.
00:07:53Those six key points, as mentioned unified system context, you have to go across the whole thing.
00:08:01At large orgs, companies like LinkedIn scale, Workday, General Motors, whatever, they need this type of data.
00:08:08They need to understand everything that's happening.
00:08:10And Tharik this morning actually talking about Fable coming out potentially later today.
00:08:15He mentioned that you need to actually provide a map and then let Fable discover the territory.
00:08:21The way to help confine that is making sure that these models have access to all of the context because they will find your unknown unknowns.
00:08:29There are definitely things going on in your company that you're just unaware of but would be really helpful for the task you're trying to do.
00:08:35That will move faster.
00:08:37But the targeted retrieval, you should be able to if you provide a link quickly, unfurl it, get that document back and move along.
00:08:43So two tasks, deep research, go long, that's fine.
00:08:46But you also need speed when speed is required.
00:08:49Conflict resolution, we already talked about that.
00:08:51But one thing says do A, one thing says do B, who is right.
00:08:55Personalized relevance, who am I, where do I work, what am I working on.
00:08:59That token optimization, making sure the response is good and effective and doesn't bloat the window.
00:09:04And then permission enforcement, of course, OAuth, you shouldn't see it, you shouldn't see it.
00:09:10What we did with some tests is we actually ran the exact same prompt to the same model and one with context and one without.
00:09:17This is the wall clock time savings and then two hours, which is great.
00:09:21And then the tokens savings.
00:09:23So it was a sizable task.
00:09:25It took about 21 million tokens without and then 18, or sorry, 10.8 million tokens with it.
00:09:31This is the type of experience that you typically see when you're using a context engine because the majority of those wasted search tokens where it has to grep at the beginning of every session to understand and discover things are no longer there when it's hydrated with context.
00:09:45Hydrated.
00:09:47And then as you move forward, you get these types of outcomes.
00:09:5050% fewer tokens, faster triage, and the answer quality is actually better because it knew what was going on inside of the business.
00:09:57Now this next part, you'll probably want a photo.
00:10:01If you don't know, you can actually take a picture of a QR code and then later in photos, tap on it and then load the link so you don't need to float here because I'm going to give you three QR codes.
00:10:09This first one is for the social comment network.
00:10:12I'll pop that up so you can take a photo.
00:10:14But this is an open source tool that we've got that actually using all deterministic programming goes over your GitHub and understands who works on your team.
00:10:22This is my real team.
00:10:23We called Rasheen the machine because he ships like crazy.
00:10:26But on the right, you can see who he commits, where he commits, who's reviewing his work.
00:10:30And then in those tabs, you can find a distilled experts graph.
00:10:33You get full coverage of what's going on in your business.
00:10:35And if you optionally add one of the API keys for either OpenAI or Anthropic, it'll determine what your teams are by doing some labeling for you.
00:10:44It's a really cool tool to understand where your team works and get that social network in there in order to focus a context engine if you're going to be building these tools yourself.
00:10:52The next is called the repo rules agent.
00:10:55This is a sample from our real code base.
00:10:57I'm going to pop that up anyway, so you don't need to talk to the thing.
00:11:00But in short, what it does is discover all the places your team has written rules files, checks them all, and then tells you what severities you've given, what other things you've given.
00:11:11Should I just switch to this?
00:11:13It tells you what -- oh, hey.
00:11:15It's good to meet you all.
00:11:17Basically, it will find all the rules that are inside of your repo and then tell you if you have duplicate issues or other problems.
00:11:24And then you can grep over it as an index.
00:11:26So that index can be called and you can dedupe, and it'll help improve your retrieval of context.
00:11:32And then finally, on Monday, we delivered this workshop, which was going beyond RAG, and taught how to build a relational context engine from scratch.
00:11:40So if you scan that, you'll get the full workbook.
00:11:43It has six PRs stacked that teach you how to walk through doing this.
00:11:46But in short, RAG is an incredible technique, and you want that.
00:11:49But the other half of the problem is what people actually ask is,
00:11:53what are the open PRs that I worked on in the last week with authentication?
00:11:57RAG cannot answer that question alone.
00:12:00You need queries.
00:12:01So this shows you how to do a schema-less basically lookup that allows the agent to discover a schema,
00:12:08and then write queries against it deterministically in order to get that type of relational data out.
00:12:13Very useful technique.
00:12:15Use cases of a context engine, of course, do go beyond code generation.
00:12:21This is where we live a lot.
00:12:22A lot of our customers spend their time.
00:12:24But it's amazing to see what happens when a bunch of other people around the business start picking up these tools.
00:12:30Customer success people solving tickets right at the time that it comes in from a customer.
00:12:35We've got salespeople closing deals earlier in their quarter because they're able to just query the unblocked context engine on the fly while in the field.
00:12:44And so many more.
00:12:45What you can also do is if you saw that curve chart earlier where I talked about the levels,
00:12:51we've built a fun little tool where basically an LLM will quiz you and ask you about what's going on,
00:12:55and then it will map you to exactly where you are and then tell you some techniques about how to level up through that
00:13:01if you are looking to basically compound your capabilities and ship with AI tools at scale.
00:13:07It's readiness.getunblocked.com.
00:13:11The gap is not intelligence any longer.
00:13:13It's context.
00:13:14We will continue to get incredible models like Mythos as it's been grown by Anthropic,
00:13:19and I'm sure Sol, once I'm allowed to see it, I'll get it.
00:13:22Happy Canada Day, by the way.
00:13:24But what's happening is it's about the context you surround these models with in order for them to be effective
00:13:30and token efficient inside of your organization.
00:13:35So I have a question slide, but I'm not sure I'm allowed.
00:13:40Nope.
00:13:41So what you'll do is come meet me at booth P16.
00:13:44You can look for the coconut.
00:13:46It would be great to hang out with all of you and get into details here if you need it.
00:13:49Thank you for your time.
00:14:00Thank you.
커뮤니티 글
아직 글이 없습니다. 이 영상에 대한 첫 번째 글을 작성해 보세요!
이 영상에 대해 글쓰기