00:00:00Thanks for coming out. Thanks for sticking around. My name's Alex. Working for a little
00:00:12company called Corridor. Here to talk a little bit about vibe coding. Some of the fascinating
00:00:17things going on from a security perspective. First off, I'm actually a big fan of using
00:00:23AI to do code generation. I think this is an incredible opportunity for people to utilize
00:00:30computers in a way they never have. A bunch of professional software developers in the
00:00:33room. Those of us who grew up writing code, who grew up using computers from a command
00:00:39line or coding them from an early age, we don't really understand what this means for normal
00:00:46people, but for the first time ever, normal people are going to be able to utilize computers,
00:00:54how they really should have been available to people for a long period of time. Coding
00:00:59is a superpower. Being able to ask computers to do things without having to buy software
00:01:06or use open source or to get other people to write code for you, that is a superpower.
00:01:11Vibe coding is bringing that to millions of people. That is an incredible thing. We should
00:01:15be super happy that we're at the start, the very start of this revolution that is going
00:01:22to bring accessibility to everybody. It's also an amazing foot gun. And foot gun is probably
00:01:28actually an understatement. It's like a foot bazooka that we're giving to all these people.
00:01:34There are example after example after example out there of bad things that are happening
00:01:41to normal people when they are vibe coding apps, some of which are just fun little things
00:01:48like their kids' little league schedule or things that they're putting their personal
00:01:53data in. Some people are vibe coding medical record systems or Bitcoin systems or things
00:01:59that are holding personal data or taking people's credit card numbers or storing people's driver's
00:02:04licenses. There's tons of examples of people using vibe coding apps to create things that
00:02:10are important. Perhaps some competing platforms that shall not be named, but whose names are
00:02:16very visible and obvious behind me are making this especially bad because they're using very
00:02:21poor defaults and not making this easy for folks. And making it really easy for them to
00:02:27use really bad default configurations of things like super base. Which isn't super base's fault.
00:02:33It's just that the way that the vibe coding platform, some other platforms that do not
00:02:37configure these things by default securely. This is not great. And not even in the situation
00:02:44where we're not talking about just straight vibe coding on a platform, but professionals
00:02:50utilizing we actually have good empirical academic data on this. This is an excellent paper I
00:02:55recommend you read called Backbench Academic Group. Went and created a bunch of prompts
00:03:01that they went out that they thought, huh, here are prompts for coding agents that we
00:03:05think might create backend code that could have security vulnerabilities. And then they
00:03:11tested these prompts against a bunch of coding tools and LMs to see. And then they tested
00:03:17one, whether the code generated was correct. And two, whether it had security flaws. And
00:03:24to their credit, they actually keep on updating this as new models come out and publish it.
00:03:29You can go check this out. And as you might expect, LMs actually introduce one, make a
00:03:36lot of mistakes, but also introduce lots of flaws. Now, this is trending the right way.
00:03:41And you don't have to take pictures of this. You can go to backspends.com and get a much
00:03:45easier to read version. Also, all their code is open source. So you can recreate this yourself.
00:03:51So one, this is trending the right way, right? So if you look even within a family of products,
00:03:56so if you look at OpenAI's family of products, it's going the right direction. GPT-5 is doing
00:04:02much better than GPT-4.1, GPT-4.0, and such. Fewer vulnerabilities using the same prompts.
00:04:10Now, one of the problems here is that these prompts and the tests are now open source.
00:04:14So you might have an overfitting issue for the tests. Although we have seen the same thing,
00:04:19what we have done at our company is we have brought this in-house, and we're now using
00:04:22some of our own tests. And we do see the same results. In our own tests, actually, the winner
00:04:27is Claude Sonnet 4.5, which they haven't released publicly. But Anthropic has taken the leaderboard
00:04:33just a little tiny bit better than GPT-5. But anyway, it is trending the right direction.
00:04:39But still, even at the top here, of the things that pass your regression tests of the actual
00:04:46correct code, if 20% of them have some kind of security vulnerability, that's not fantastic.
00:04:52That's not what I would consider, as a security professional, what I'd really want to see.
00:04:56I'm a big fan of the Fallout series. War never changes, right? And it feels a little bit to
00:05:03me, what we're doing here is where these vibe coders, especially ones who have not professionally
00:05:08written software before, we're giving them all like the base weapon. We're giving them
00:05:13all like a slingshot and a backpack. And then we're pushing them out into a world full of
00:05:19mutants with sharpened sticks. And they're getting eaten immediately, right? Because it's
00:05:24not like the bad guys are inventing their code from scratch. For 20-some years, we've had
00:05:34professional attackers figuring out how to break into web applications, how to break into
00:05:40mobile apps, how to reverse engineer these things, and especially how to build financial
00:05:44models of how to do malicious things. And so we have an entirely new generation of
00:05:50people who now have, miraculously, access to use computers in a new way, going up against
00:06:01professionals whose entire job it is to monetize bugs in software. This is reflected in the
00:06:13security industry in the ways that we qualify and quantify these vulnerabilities. One of
00:06:18those frameworks that we do that is called the MITRE ATT&CK framework. In security, we
00:06:25stole this idea of the kill chain from the military, but it's basically the idea of the
00:06:30steps you have to take to have an effective intrusion as an attacker. That's the steps
00:06:36here from the left to the right, reconnaissance, resource development, access, execution, persistence,
00:06:41and the like. And going down are the different categories of techniques here. And for each
00:06:47of these, there are sometimes dozens of or hundreds of different techniques for each of
00:06:54these categories. This is just the top level of MITRE, which is an organization that the
00:07:01U.S. government pays to create this overly complicated chart, to categorize all these
00:07:06things so we can track different threat actors in the security community and have kind of
00:07:09standard language to talk about what we see in the wild. So we have our sweet little vault
00:07:18dwellers who we give these tools to and we're like, "Hey, good luck." And what they're walking
00:07:23out to is a world full of bad guys who do all of this stuff. If you go to attack.mitre.org,
00:07:31you will see the list of known exploit chains, known things that have happened in the wild
00:07:39from different threat groups, the AP-228s and 29s, UNC-3886. So this is like the Ministry
00:07:45of State Security of the People's Republic of China or the Russian SVR or a whole list
00:07:50of professional financially motivated groups like Lapsys and such. And then here are the
00:07:55techniques they have used to attack different victims. The idea that vibe coders are going
00:08:01to understand all this is just ridiculous. But that has effectively kind of been the
00:08:07assumption up to this point. So what can we do better? Well, the first thing we can do
00:08:14is we can start to actually, you know, when we want to solve problems as engineers, we
00:08:20start to kind of break the problem into pieces, right? And the truth is we have two totally
00:08:24different categories of problem here. The first is we've got to break the use of these tools
00:08:30into two major categories. The first is actual vibe coding, right? When I talk about vibe
00:08:35coding, I mean people who are not professional software engineers, normal people, right? So
00:08:40people who have like actual normal hobbies. So this would be people who are not in this
00:08:45room, right? Who do things like do not come to this conference on a Thursday afternoon,
00:08:51have better things to do, no offense, than be at Chip AI. And this is fine. It's great
00:08:55to be here. I'm here. But like, if you're here, you're not a vibe coder, right? You know, we
00:09:04are people who work, when we use AI, we're probably doing AI-assisted engineering, right?
00:09:09So vibe coding would be people who are, for the first time ever, able to access this kind
00:09:15of capability with new tools that are built just for them. So the first thing we've got
00:09:19to do is we've got to kind of separate out the total classes of issues and the way people
00:09:23are using these tools. And I think we've got to stop calling AI-assisted engineering vibe
00:09:28coding. Maybe AI-assisted engineering is like, we can come up with a better term here. But
00:09:33this ranges from people auto-completing, tab-completing, and cursor to, you know, now you have like
00:09:39a professional engineer will dispatch four or five different agents in the background
00:09:44to do different jobs while they do the thing that they want to do. And even though those
00:09:48agents are acting autonomously, they're still in charge and they know what they want, the
00:09:55engineer does. That is still very different than vibe coding. The vibe coder is often,
00:10:01they are relying upon a fully-fledged platform, right? So they need a platform that does effectively
00:10:07everything for them end to end. They are not putting things together piece to piece. They
00:10:12are describing an outcome. They have an outcome they want. They can do that in English or whatever
00:10:17language they speak. They might be doing it visibly, right? So there's a bunch of these
00:10:20platforms that allow you to lay it out with GUIs and such. Obviously V0 is a great example
00:10:24of that. And so they have an outcome they want. They aren't necessarily, they're unlikely to
00:10:31be describing how they're trying to get there. They're often starting with a green field.
00:10:34So they have the benefit of not having to fit whatever the vibe coding platform is doing
00:10:40into some kind of existing codebase or some kind of existing architecture, which is from
00:10:44a security perspective actually great. We can talk about that in a second. But this gives
00:10:48a lot of benefits from a security perspective to the vibe coding platform in that it does
00:10:52not have to fit into some kind of existing AWS or God forbid self-hosted architecture.
00:11:00The downside here is they have little ability on their own to secure the output. So this
00:11:04is where you end up with those problems that we see people tweeting about of like, why am
00:11:10I seeing this error or where did all my Bitcoin go? Which if you search on X for where did
00:11:17my Bitcoin go, you get a lot of tweets, right? Because that turns out, I mean, you get that
00:11:25for a lot of reasons, but vibe coding turns out to be one of the reasons now that people
00:11:28will tweet that. Because they do not have the ability to look at the output and then
00:11:32to figure out whether or not it's secure or not. Whereas a software engineer that happens
00:11:37to be using AI to make their job better, they are a professional that is supervising an agent.
00:11:41And again, that could be as simple as I have started a function and I want you to finish
00:11:45that for me. It could be as much as I have this bug and I want you to work on this bug
00:11:50and fix it. And that could be you're dispatching an agent to work for you for 30 minutes. But
00:11:54in the end, they're probably describing a component they want with much more specific
00:12:00request to it. And so they have much more specificity and not the output that they want
00:12:05as much as here is the layer of steps I want. And they can also lay out, okay, first give
00:12:10me a plan. You edit the plan and then you give the plan back to the agent and let it execute
00:12:15step by step. Now, the downside for these folks is you're often working with an existing codebase
00:12:21and then often requirements, compliance requirements, architecture requirements. You're not starting
00:12:25Greenfield usually if you're doing AI assistant engineering. So it's not as easy for you just
00:12:31to come up with some kind of security plan if you're this kind of person. But you hopefully
00:12:36do have some security skills and you might even have a dedicated security team that you
00:12:39can rely upon if you're this person. So these folks have very different security requirements
00:12:45that we can help them out with. They need their own solutions. So what are some solutions we
00:12:50can do as a group to help these folks, especially people who are working on AI coding platforms?
00:12:56So what do the vibe coders need? So first thing is they need things to be secure by design.
00:13:01So when we talk about security by design, we're often talking about the end product, right?
00:13:04This is often we're talking about a SaaS platform should have good authentication options by
00:13:09default that all of the settings should be secure by default and that you have to intentionally
00:13:14turn those secure settings off and such. In the AI coding perspective, what we really want
00:13:19is we want the vibe coding setting, the architecture and the code to be opinionated. That the AI
00:13:26coding agent should have a really good opinion on this is the components you should use and
00:13:31this is how they should be configured to do things correctly the first time. I talked about
00:13:35that super base example or any kind of database. It should have an opinion on role level security.
00:13:39It should have an opinion on, well, by default, your users should have very little or no access
00:13:44to data and then we should explicitly give them access to data as they need it, right?
00:13:50We shouldn't have just a backend database that you can get access to anything and then later
00:13:54block it down. It should have a strong opinion about this and then make those decisions on
00:13:59behalf of the user because it is unlikely that a vibe coder in this scenario will be able
00:14:04to make those decisions itself. It probably should also detect whether or not a project
00:14:11that it's asked to do is out of scope for what it feels comfortable doing. There's a
00:14:15number of vibe coding platforms where you can ask it, please build me a medical record system
00:14:21and it will do that and then say, here you go. Here's a medical record system. Our CEO,
00:14:27Jack Cable, who's here, did this on one vibe coding platform and it said, here's your medical
00:14:31record system. It's HIPAA compliant. Yeah, so news break. That was not HIPAA compliant.
00:14:40Let me just tell you as a professional, as somebody who's a public company, CSO, that
00:14:44was not true. Just saying something's HIPAA compliant does not magically make it HIPAA
00:14:48compliant. That's not how that works. And he pointed it out. I don't think that's HIPAA
00:14:52compliant. And I said, oh no, you're right. I don't think it is. And it made a couple of
00:14:56changes. Now it's HIPAA compliant. Still wasn't, right? This is the kind of thing that if I
00:15:00was building a vibe coding platform, if you asked me to build a medical record system,
00:15:04I'd probably say, nope, not going to do that. That is a bad idea. You're in the wrong place
00:15:09for that, sir. Same thing with like, please build me a system to store my Bitcoin. I would
00:15:14say, no, I am not going to do that. That is not a good idea. I'm not going to store Bitcoin
00:15:18for you. That is going to get stolen. That Bitcoin is going to go straight to North Korea
00:15:22and it's going to be used to buy rockets. You should do something else with your Bitcoin.
00:15:26This is a vibe coding platforms should know their limits. And there should be something
00:15:31in the middle that if you ask you to do something in the middle like store people's personal
00:15:36information that it will allow you to do it, but it's going to engage a bunch of security
00:15:41features and if possible, bring in security agents just like on a bunch of vibe coding
00:15:47platforms. If you want to store data, it will say, great, I need a database server. If you
00:15:51want to play video, it says, okay, I need a video CDN, right? So there are things in the
00:15:56middle, but there are some things that should just be like, yeah, that's a bad idea. I'm
00:16:00just not going to do that for you. And finally, what they need is they need to have those
00:16:05kinds of engagements with partners that will allow it to do code review because a vibe
00:16:10coder is not going to say, oh, I already have a relationship with a software review tool.
00:16:15That's just not something they're going to bring. So you want that to be built in to the
00:16:19base product just like you have that base relationship with a database partner or something like that.
00:16:25Security paternalism or maternalism, if you prefer, is okay. It is okay to make decisions
00:16:31on behalf of users who do not have the ability to do so. I think this is something that the
00:16:36security industry has been afraid to do because we are afraid as security people to be held
00:16:41responsible for decisions we make on behalf of other people. This is just a common problem
00:16:46with security people is what we will do is we will create a tight rope. And if somebody
00:16:52falls off the tight rope, we're like, oh, shit, sorry. I guess you don't know how to rock on
00:16:55tight ropes. That is your fault, right? That is not okay. It is better for us to do the
00:17:01best we can to give people a safe way to cross that chasm and take some responsibility if
00:17:08people fall off. It is not okay for us to make things so hard. It is okay to be a little
00:17:13bit paternalistic in these cases and to make decisions, especially if you're building products
00:17:18that you know are going to be used by non-experts. Now, what if somebody is an expert? What if
00:17:22you're building a product that experts are you? So this would be more like a cloud code
00:17:26or a cursor or a product that you're expecting people to be more. Now, cloud code is an interesting
00:17:32challenge because that is used both by normies and by people in here, right? And so, if you're
00:17:39like a product manager for Anthropic, it becomes more of like do we want to have like a mode,
00:17:43a vibe code mode? Do you want to detect whether or not like do you want to give a test up front?
00:17:48I'm not sure exactly how you want to engage it, but I think there's an interesting challenge
00:17:52from a product manager perspective. At what point do you kick in like a vibe code mode
00:17:56where you're really doing things on behalf of a user versus giving them more capabilities?
00:18:01So, certainly like a cursor should only be being used probably by professional engineers.
00:18:05So, professional engineers also need secure by design. Now, it is a different way though,
00:18:10right? Like if you're doing secure by design for a professional engineer, you're probably
00:18:14not making things like sweeping overarching architectural decisions. But what you are doing
00:18:20is you're not making those mistakes that we just talked about in 20% of the time in the
00:18:25GPT-5 and 60% of the time or something in the Grok situation where you're just making dumb
00:18:31security mistakes, right? You need to make and write code that does not have flaws by
00:18:35default. And you need to at least prompt the user and question the user, "Hey, can I do
00:18:41this better for you by default?" Then try to make by default decisions that are not a good
00:18:48idea. Something that these agents are actually quite bad at is having a big picture focus.
00:18:56I think you've all seen this happen where if you ask a coding agent like, "I would like
00:19:02to build an incredibly complex system that does a lot of things." It's like, "Great, I'm
00:19:08going to start writing code right now." That is not how we build software, right? Like you
00:19:12don't get 20 people to build an incredibly complex, distributed system and start just
00:19:18by opening up a file and writing code, right? You have a PRD. You have a design meeting.
00:19:26You think about what are our requirements. You do a bunch of product management. There's
00:19:31a couple of exceptions, but for the most part, the coding agents just want to start writing
00:19:35code. That is what they know. They just jump into it immediately. And so I think what would
00:19:41be nice would be for these things to start to slow down and to have the planning steps
00:19:46and to be much more thoughtful about, "Let's lay out an architecture and a design," and
00:19:51then do things like document APIs, document how we're going to do input validation. How
00:19:55are we going to prevent common flaws in this kind of architecture? How are we going to do
00:19:59authentication between, "You're designing something that's definitely going to have different
00:20:03services. How are we going to authenticate those different services?" This is the kind
00:20:06of thing that almost none of the coding agents ever think about and would be really thoughtful
00:20:10to do upfront when you're starting with, even if you're a professional engineer.
00:20:16And then we also need the ability to have AI security agents. One of the funny things that's
00:20:21going on is if you work at a tiny startup or you're 22 years old, you think the only people
00:20:28that write software are software engineers, but I see a lot of professionals in this room.
00:20:33When you're a professional, and especially if you work at a company that's regulated or
00:20:37does something important like builds airplanes or something, you realize there are software
00:20:41engineers, but there's also security engineers, and there's security architects, and there's
00:20:45privacy engineers, and there's compliance people, and there's lawyers. I know. We're not super
00:20:52fans of all these people, but they actually have important jobs. There's a reason these
00:20:56people exist, and there's a reason they influence the code base, because bad things have happened
00:21:03with software, and so we've come up with all kinds of rules about why we write software
00:21:08and why we have to have compliance rules and why we have product management, why we have
00:21:13privacy laws and such. What we've done is we've taken the software engineer's job, and we've
00:21:19taken a 40-hour workweek, and we've turned it into 20 minutes of GPU time. Well, all those
00:21:24other people still work 40-hour workweeks and interact with each other in conference
00:21:31rooms and not over MCP and aren't able to operate at the same speed as a software engineer who
00:21:38has 10 agents operating in the background, spitting out code. What we also need to do
00:21:44is we need to build mechanisms so that all of the other people who still have important
00:21:49jobs are able to be as efficient as a software engineer is in the AI coding era, because in
00:21:55the end, there are still humans at companies who have real responsibilities. In fact, some
00:22:02of these people could go to jail if the software engineers with their coding agents do a bad
00:22:08job. I have been a public company CISO three times. That is a terrifying job, like a joke
00:22:14I like to say, and it's not true, but it feels a little true is that the word CISO is Greek
00:22:19for the goat that is slaughtered first, right? But it is really terrifying to be a CISO these
00:22:26days, because you are getting blamed for what hundreds or thousands or tens of thousands
00:22:31of other people might be doing that you can't really have control over or really even understand,
00:22:38and that was true before every single one of those people had five background agents writing
00:22:43code for you. And so we need to find ways that these humans who have incredibly important
00:22:49jobs around compliance, around privacy, around safety, around security, are able to understand
00:22:56and to have some idea that the rules that they have to live up to in the physical world are
00:23:01still being enforced. So at Corridor, we believe that there's kind of two areas in which to
00:23:08start off, there's going to be more, but there's at least two areas we have to start off with
00:23:12where we need to have some standardization to make AI coding enterprise ready. First,
00:23:18telemetry, and then in security workflow. So on the telemetry side, we're going to be writing
00:23:24a blog post about this. It'll probably come out next week. We think that security agents
00:23:30need to be pushing out everything that they're doing, all of their interactions with users,
00:23:38that this is the kind of stuff that if you're an enterprise, that you need to be able to
00:23:42see in one central place what people are doing with their agents, who is logged in as whom,
00:23:48have full visibility and prompts and full visibility into what tools the agent is calling, and then
00:23:53especially in what code is being generated. Our product actually provides us visibility,
00:23:58but we have to do it by reverse engineering how all the coding agents work. It's kind of
00:24:02hacky. It's not a lot of fun. It'd be much better if the coding agent supported this.
00:24:07So the company that's gone the furthest here is Anthropic. Claude Code supports a standard
00:24:11based upon telemetry, so it's a nice open standard. It's pretty clean. In their current
00:24:16shipping version, it's got 70% of what they need. In a future version, it looks like they're
00:24:21going to ship 90% of what we all need. So it would be great if first, Anthropic ships everything,
00:24:28and then second, if everybody else copies them. But it'd be really cool if all the coding
00:24:32agents had the ability to get telemetry of what everybody's doing. That's just the first
00:24:37step. It's the first step of any security solution, just to know what's going on. And
00:24:40it would be really cool if this could be configured by MDM, by device management. So as an enterprise
00:24:45CISO, you could just push down using your MDM provider onto everybody's machine, "Hey, just
00:24:50send me all telemetry to this place." Then you wouldn't have to have agents go out onto
00:24:55all these machines to go try to pull it. The second, I think, more important and more
00:25:00interesting area is we need a mechanism to standardize that conversation. Like we talked
00:25:05about, software engineers and security engineers used to meet over coffee and have a conversation.
00:25:10Well, we need standard ways for security agents and software agents to talk to one another
00:25:16to replicate what those humans used to do. And it needs to be kind of deterministic.
00:25:20So again, we do this right now with our product, but we had to kind of shove it in there. And
00:25:26it's non-deterministic. It's based upon how the LM's feeling. And it would be nice that
00:25:30if you issue, if you ask a coding agent to go do something in the background for 30 minutes,
00:25:35that it goes and talks to your security agent and says, "Here's my plan." It gets feedback
00:25:39on the plan. It says, "Here's my code." It gets the code checked. It gets all the bugs
00:25:43checked. And then it fixes all the bugs. And all of that is done without the engineer having
00:25:48involved. And then the security team sees on this checklist, "Yep, this all got fixed,"
00:25:52which would be awesome. So what do we have right now is we have a corridor. We're actually
00:25:58shipped to this today into GA. We've had enterprise customers, but we shipped for GA today is we
00:26:05plug in with VR ID plug in, and we plug via MCP into your coding agent that when you ask
00:26:11it to do something, it ships us a plan. And then we can use that to provide security context
00:26:17that's specific to a code base. And that provides both generic security advice, but also allows
00:26:24a company to have your specific security rules for that company. So say you're a big bank
00:26:29and you've got a, this is how we handle social security numbers, or this is how we tokenize
00:26:34credit card numbers. That's the kind of context that we can provide into that security agent
00:26:39to make sure the coding agent does its job in the first place. And then we scan that code
00:26:44on the backend to make sure that that rule was provided. And then we also gather telemetry
00:26:48from our IDE plugins. So we can see all of this interaction and make sure that the plugin
00:26:54is doing its job. And then also see all of the unauthorized coding tools that everybody's
00:26:58using. Not that any of you would bring a coding tool to work that was not authorized by your
00:27:03security team. I see lots of rule followers in this room for sure. And so we're already
00:27:11doing this, but it would be super cool if this became a standard thing that lots of different
00:27:18coding agents supported, and then lots of people could compete against us and copy our product.
00:27:22That's fine. We're happy to be the Kleenex of the space by lots of people doing this kind
00:27:26of work. But I think this is going to be the future of having coding agents and security
00:27:31agents have a relationship here, just like security engineers and software engineers
00:27:35have a totally friendly and not at all competitive or difficult relationship these days.
00:27:42We proudly released this to GA today, and we shipped it on Vercel's AI agent marketplace.
00:27:48So we're right there on the top. You can go to corridor.dev to check us out or find us
00:27:52on Vercel's marketplace. We're the only security product there. So we thank very much Vercel
00:27:56for their partnership in that and thank them for giving me this time today. If you want
00:28:01to chat with us, Jack and I, our CEO and co founder, will be out in the corridor to chat
00:28:09about that. Anyway, I think there's a lot we could do together here to give this superpower
00:28:14of AI coding agents to millions of people while also protecting them from these villains that
00:28:21are out there as well. Anyway, I'm Alex, alex@corridor.dev. If you ever want to chat, I'll be out in the
00:28:27corridor or drop me an email. Thank you very much. Thanks so much for Vercel. Have a great
00:28:31rest of your day.