No Memory, No Harness: Why the Database Is the Last Line of Defense — Kay Malcolm, Oracle
AAI Engineer
Computing/SoftwareManagementInternet Technology
Transcript
00:00:00Tanya Cushman Reviewer: Peter van de Ven
00:00:12Everyone, are we having fun?
00:00:15Oh, you've got to give me way more than that.
00:00:18So let me tell you, my name is Kay Malcolm.
00:00:21I am a retired hip-hop instructor.
00:00:24So if I don't get more energy than that, we will start --
00:00:28are you having fun?
00:00:30Okay, all right.
00:00:32So here's what we're going to talk about today.
00:00:36Now, you guys have heard a lot about two letters.
00:00:39Does anyone want to guess what those two letters are
00:00:41that I'm going to talk about today?
00:00:43That was pretty good, DB.
00:00:48I'm going to talk about AI,
00:00:50but I'm specifically going to talk about agent harnesses.
00:00:54But before I do that, I want to introduce you all to a few people.
00:00:57Is that okay?
00:00:59Yes or yes?
00:01:00Is that okay?
00:01:01Yes.
00:01:02I gave choices.
00:01:04Yes?
00:01:04Anyway.
00:01:05All right.
00:01:05Okay, all right.
00:01:06This is my team.
00:01:09I run an outbound database product management team at Oracle.
00:01:14I've been at Oracle a really long time, 20 years.
00:01:17Funny story.
00:01:18I started when I was 12, so don't do the math
00:01:21and don't start adding in your head.
00:01:24And we've got a problem.
00:01:27That problem is I've got one group that does platform development.
00:01:33And then I have another group that does content development for Live Labs,
00:01:37a platform that I wrote myself.
00:01:40So, yeah, I'm an engineer, but I'm kind of a developer poser, too.
00:01:44And then I've got another group who does QA.
00:01:48And then I have another group who does my front end development.
00:01:52With AI, here's what I found out as a leader.
00:01:57Because in the token maxing era of 2025, 'cause you know we're not token maxing anymore, right?
00:02:05We are responsible AI-ing now.
00:02:08But in the token maxing era, the thing that I found out was while AI was making the individuals
00:02:15on my team faster, there was another problem it was creating.
00:02:23It wasn't making my team more productive.
00:02:28And the reason was when one team from the Netherlands checked in code at my 4:00 a.m. in the morning,
00:02:37because I've got half of my team that's in EMEA, and I have half of my team that are here in the United States.
00:02:43They checked in the code, but they didn't check in their context from codex.
00:02:49We use codex at Oracle.
00:02:51So then when the US team woke up,
00:02:56they got the code, but no information about the context.
00:03:00So, we used AI to solve a problem that AI created.
00:03:07And here's what we did.
00:03:08Oh, well, let me talk about this first.
00:03:10So, some of the issues.
00:03:12The context, like I said, wasn't shared.
00:03:15GitHub wasn't tracking that.
00:03:18I had repositories that were diverging.
00:03:21And I was asking the managers who work for me, what's happening to your teams?
00:03:26Why are we not going faster?
00:03:29We're spending all of this money on tokens.
00:03:31We're spending all this money on AI.
00:03:33Yet something is missing because we're still spending time
00:03:37doing testing and validation.
00:03:39So, our net net wasn't really working for us.
00:03:45Because Git records the code and not human intent.
00:03:49So, it's a problem.
00:03:50And even though code creation was no longer our problem, we still had a bottleneck.
00:04:03We needed a collaboration layer.
00:04:05Now, I do have members of my team in the audience.
00:04:10So, don't judge me.
00:04:12And you know who you are.
00:04:13I'm not saying that you all didn't collaborate.
00:04:17But now, we've got a new team member.
00:04:20And that new team member is AI.
00:04:23So, we needed to figure out how to track our progress and our next steps.
00:04:29How to rationalize decisions that the agent was making.
00:04:39We needed to figure out how to resolve questions and conflicts.
00:04:45Okay.
00:04:48Hold my problem.
00:04:49Will you all hold my problem for me?
00:04:51Right here.
00:04:51We're going to just tuck that in a little box.
00:04:54Let me define what a enterprise agent actually is.
00:04:59Now, most people think that an enterprise agent is the model and workflow.
00:05:05How many people agree with me?
00:05:08Man, this is a tough crowd.
00:05:10Okay, one person.
00:05:12Okay, the rest of you think it's a little bit more.
00:05:14Okay, let's see what.
00:05:18Could it be that a real enterprise agent has tools?
00:05:26Tools are how it does things.
00:05:30Context.
00:05:33The context, that's a context window.
00:05:35That's what's in the actual prompt.
00:05:39Memory.
00:05:43And if you're thinking, but wait, Kay, memory.
00:05:46You just said that the model is kind of like the brain of the operation.
00:05:51Hold tight.
00:05:52We're going to talk a little bit more about memory.
00:05:55Retrieval.
00:05:57Because you don't want to get everything back.
00:05:58So, that's being able to retrieve the right information back.
00:06:03And then, I know that there are a lot of developers here, and you all don't care about security.
00:06:11I care about security.
00:06:13Because I work for the most secure database company.
00:06:17And I used to work for an agency that has no name.
00:06:22But guardrails is also important.
00:06:25This is the harness.
00:06:27I speak in analogies, and I speak in stories.
00:06:31Because if I tell you this and Marvel, you know exactly what I'm talking about.
00:06:38So, the agent, think of it as the model little brain floating in a glass jar.
00:06:46Plus, this harness.
00:06:49This harness is the body.
00:06:52So, it's how the agent can actually do things and get things done.
00:06:58That memory, that's the part of the central nervous system.
00:07:03And you remember, the central nervous system connects the brain to the rest of the body.
00:07:08Legs, arms.
00:07:10That's the part of the central nervous system that carries context.
00:07:16So, you remember my problem with git?
00:07:20What I needed was memory.
00:07:21Okay.
00:07:22So, there are a number of memory types.
00:07:26I chose five, the five most common ones that people talk about.
00:07:29And these are the ones that I want you to remember.
00:07:32The first one is short-term memory.
00:07:34That's the session, right?
00:07:36And so, if you're storing memory of an AI process, that is, the short-term memory is,
00:07:44if you're with chat, cloud code, right, codex, pick your poison.
00:07:49The long-term memory is what persists across sessions.
00:07:55Episodic memory, hmm, what happened the last time I interacted with fill-in-the-blank?
00:08:06That's your episodic memory.
00:08:09Procedural memory.
00:08:11Tools.
00:08:12Steps that were taken.
00:08:15And then finally, semantic memory.
00:08:17And semantic memory, because we're talking enterprise agents.
00:08:21We're not talking the agent that I built, Sasha Fierce.
00:08:25Because remember, I told you guys that I'm a dancer.
00:08:28So, of course, my chief of staff is going to be called Sasha Fierce because that was Beyonce.
00:08:34Any Beyonce fans?
00:08:37Okay, I'm sorry.
00:08:38All right.
00:08:38We've got to focus.
00:08:40Okay.
00:08:40So, these are the memory types.
00:08:43Now, when you're defining this real enterprise agent in this memory,
00:08:49there's something you need to consider, where to store it.
00:08:53And so, I'm going to tell you guys a story.
00:08:55But when I tell you the story, you have to promise me that you're not going to judge me.
00:09:00Do you promise?
00:09:06Do you promise?
00:09:08You're not recording me, right?
00:09:10Because this doesn't paint me in a good light.
00:09:12Okay.
00:09:12All right.
00:09:12The world of data was one simple.
00:09:14I've been at Oracle a long time, but I came from a customer.
00:09:17That customer's name was Southern Company.
00:09:19It was a power company.
00:09:20I'm based out of Atlanta.
00:09:22And I was hired at Southern Company because I was a rock star performance tuner.
00:09:28You had a SQL query.
00:09:29I mean, I'm dating myself, but whatever.
00:09:31You had a SQL query.
00:09:32I knew all of the init.ora parameters.
00:09:34Even the ones when you called support and they said,
00:09:36don't remember these.
00:09:37Don't write them down.
00:09:38I wrote them down in my little notebook.
00:09:40I could tune a query within one inch of its life.
00:09:43Then, one of you came to my desk.
00:09:47Because, I mean, the world was rows and columns.
00:09:50It was a great time back in my Al Bundy days.
00:09:55And said, hey, I need to store data unstructured.
00:10:01Why?
00:10:02Why do you do that?
00:10:04And so, me being Kay, the diligent DBA, I was like,
00:10:09let me figure it out and get back to you.
00:10:13Did I get back to him?
00:10:16I didn't get back to him.
00:10:18Now, the thing you have to know about Southern Company was,
00:10:21for every database system that a DBA managed,
00:10:24I had to attend two meetings.
00:10:26Today, when I hear Sarbanes-Oxley,
00:10:28I still throw up a little bit in the back of my throat.
00:10:31So, I had to attend a security meeting and a patching meeting.
00:10:34Every week.
00:10:35Never failed.
00:10:37Now, because this developer installed a database that was specialized for unstructured.
00:10:44Okay, there are really smart people in the room.
00:10:46How many meetings am I going to now?
00:10:54Four.
00:10:55Okay, I'm a little annoyed, but I'm like, okay, we can do this.
00:10:59Then they said, Kay, since you're such a good tuner,
00:11:03I need you to figure out this relationship.
00:11:06Now, the way that Southern Company worked,
00:11:08there was this people could die application.
00:11:11And it was like a Nokia phone that people who were climbing the towers, right?
00:11:18So, you guys have been in a storm and the power goes out, right?
00:11:22And then you're pretty sure that within maybe an hour or two, the power will go on.
00:11:27Well, that system that would tell the people who are climbing those trees
00:11:32and risking their lives to turn the power back on sometimes would have false positives
00:11:37or false negatives.
00:11:39So, they wanted to look at all of the other polls in the area
00:11:46to try to get away from the false positive or the false negative.
00:11:50And so, I did that in a SQL query.
00:11:52And it was amazing.
00:11:53It was a five-nested union all statement.
00:11:57It was some of my best work.
00:11:59Now, it might have taken like 20 minutes to work, but it was like a predecessor to graph.
00:12:06Yeah, they installed Neo4j.
00:12:10So, now, how many meetings am I going to?
00:12:13Six.
00:12:14That's a problem.
00:12:16So, I – oh, let me – I got ahead of myself.
00:12:19So, you know what I did?
00:12:20I quit.
00:12:23I left and I came to Oracle because I was like, this is a problem and maybe I can go to Oracle
00:12:26to help solve it.
00:12:27So, then Joe Mundy called me and he said, "Hey, we are installing Redis.
00:12:33Oracle is late to the game.
00:12:35We've got a vector database."
00:12:37Okay.
00:12:40But here's the problem, Joe.
00:12:45Agents now need access to all of this data.
00:12:49So, if data is in an Oracle database, if then it's also in an unstructured JSON database,
00:12:55if it's in a graph database and it's in a vector database, where is your single source of the truth?
00:13:03The agent has to figure that out.
00:13:04Sometimes it'll get it right.
00:13:08Most times it'll get it wrong and it's going to burn up a whole bunch of tokens.
00:13:12And so, now, if you want to store your memory somewhere, you can store it in a file system.
00:13:20You can store it in Claude or ChatGPT because we all know about the memory.md file.
00:13:29But that's going to be a problem.
00:13:31Now, I want to illustrate this.
00:13:32I need four volunteers.
00:13:33I can see you.
00:13:34Raise your hand.
00:13:35One.
00:13:36Two.
00:13:37Okay, I can't.
00:13:38Three.
00:13:40I need a fourth.
00:13:41Ah, fourth in the back.
00:13:42Okay, fourth in the back.
00:13:44You are going to be our old reliable.
00:13:47You're going to be a relational database.
00:13:48Yes or yes?
00:13:50You got your, so you have your assignment?
00:13:52Okay, and then there was someone here.
00:13:54You're going to be my unstructured database.
00:13:57And then where was my other, ah, very good.
00:14:00You're going to be my graph database.
00:14:02You good?
00:14:02Relationship guy.
00:14:03You look like a relationship guy.
00:14:05All right, very good.
00:14:06Fourth.
00:14:06Where was my fourth?
00:14:09Was it you?
00:14:09Yes.
00:14:10Yeah, you are my vector database, okay?
00:14:15Now, everybody be really, really quiet.
00:14:20For my four volunteers, I need you all.
00:14:24I'm going to say something to you,
00:14:27and I need you all to decide how you're going to store it,
00:14:31and who's going to have the single source of the truth.
00:14:33You can't get up from your seats, and you have to whisper,
00:14:37because if you talk loud, that's five extra tokens for you.
00:14:40Yes?
00:14:41Okay, are we ready?
00:14:42All right.
00:14:44The cow jumped over the moon.
00:14:48Go.
00:14:51Hmm.
00:14:53It doesn't really work, does it?
00:14:55That's a problem.
00:14:56Okay.
00:14:58Oracle, and if you don't forget one,
00:15:02if you forget everything I say, and you remember one thing,
00:15:06Oracle is not the Oracle that you think.
00:15:09That is why I am here today.
00:15:11How many of you knew that Oracle could natively,
00:15:15in the same table, down to the same partition,
00:15:19store JSON, graph, vector, my vector friend over there,
00:15:24my JSON friend, spatial?
00:15:27You want your memory to be immutable?
00:15:30No blockchain in the same database?
00:15:32Raise your hand.
00:15:34Yeah.
00:15:36We have a marketing problem.
00:15:39So, any data type can be stored in a 26AI database.
00:15:44Any workload.
00:15:46Anywhere.
00:15:47AWS, GCP, Azure, OCI, on-prem.
00:15:52Choice and flexibility.
00:15:54So, now, when we take this and we talk about the agent,
00:15:57I want to be able to store my long-term and procedural memory in relational.
00:16:02In JSON, I want to store my short-term and my long-term memory.
00:16:05Graph, I want to store procedural, because procedural, that's how I figure out the relationships,
00:16:09right, the steps.
00:16:11My episodic and semantic memory, I need to do some vector and then store it also as text.
00:16:17Now, if I have four different databases, you all saw, they can't talk to each other.
00:16:25It's going to be a problem.
00:16:27And so, what I'm saying to you today is the Oracle AI database is the best place to store this agent
00:16:35memory that's going to power your harness.
00:16:38Remember, your harness is your body, and that memory is your central nervous system.
00:16:44Okay, back to Poly.
00:16:46So, the problem that I had, we solved it with a memory broker named Poly.
00:16:51We used agent memory.
00:16:54We got out of that automatic continuity.
00:16:57So, with my team, they were able to share not just their code, but Poly also kept track of the context.
00:17:07So, if one context window had procedural memory, episodic memory, information about the long-term memory,
00:17:15that was then shared with the other folks on the team.
00:17:19You could call them agents, if you will.
00:17:21They're just human agents.
00:17:23Shared across forks.
00:17:26The developers on the team remained in control while Poly was able to create the context,
00:17:35figure out which fork and branch it belonged to and which commit it belonged to.
00:17:41Now, this is a very simplistic example, but when you take this to the enterprise, here's what happens.
00:17:49Memory is the thing that becomes non-negotiable in an agent's harness.
00:17:53Now, these are three papers that I read on the airplane.
00:17:59This first one is from OpenAI, and it's about its in-house data agent.
00:18:03And the thing that it says is it is saying that its in-house data agent actually needs memory.
00:18:11Memory was crucially important to ensure that its agent was able to filter correctly instead of trying to string match.
00:18:19Harrison Chase said, "Your harness, your memory, and if you don't own your harness, you don't own your memory," which is key.
00:18:27And then, I'm sure you all are wondering, "Well, Claude has memory. Why can't I use that?"
00:18:32Well, it's kind of like file system memory, and it works with one, but just like in my example,
00:18:38when you scale past one, and you're going to scale past one in the enterprise, it creates a problem.
00:18:45So, Oracle has a Oracle Agent Memory Package.
00:18:51PIP install Oracle Agent Memory, you get access to it, and this memory is, this SDK that we have,
00:19:00is the thing that will hold your live conversations, your memories, your facts,
00:19:05and figure out what is worth keeping. So, if we look at Poly, now, Kevin can share his context with Poly,
00:19:14our memory broker. We use the Oracle Agent Memory SDK. It's stored in an Oracle Autonomous Database.
00:19:22We can use the LLM of our choice, or we can use a local model through the Oracle Private AI Services
00:19:28container. And then, Linda, who's actually sitting right here,
00:19:34can interact and work with Kevin, no issues. So, yes, AI makes individuals faster.
00:19:42Shared memory on an Oracle AI database makes teams faster. So, I don't want you all to compromise.
00:19:49In the age of AI, what 26AI does is you can choose and pick what's best. For agent memory,
00:19:58file systems stored in a database file system or in the database. If you need to do data modeling,
00:20:04you've got JSON, you've got relational, we've got choice.
00:20:09Okay, I've got some goodies for you. The Oracle AI Developer Hub, that's where you guys can get coding
00:20:16materials, the applications, what I talked about today. LiveLabs.oracle.com. If you've done any of
00:20:22our workshops today, that happens to be something that I wrote myself about six years ago and 40 million
00:20:28users ago, spend my OCI tenancy money, kick the tires on any Oracle technology for six hours,
00:20:3812 hours, however long you need. And then, I'm giving you all a Mac mini. No, I'm just kidding.
00:20:45I'm giving you an OCI mini. So, I don't know if you knew, but there's an always free OCI. It is the most
00:20:51generous of any of the hyperscalers where you can get a free Oracle database, free compute. You can send
00:20:573,000 emails a month, 200 gig in storage. And if you click on that, you can get access to it. Or just
00:21:05search Google for Oracle Cloud, always free. Connect with me. If you build something, will you all message
00:21:14me and let me know? Yes or yes? Thank you.
00:21:35We'll see you next time.
Community Posts
No posts yet. Be the first to write about this video!
Write about this video