AI DEBATE: What Will the World Actually Look Like in 2040?
CChris Williamson
Computing/SoftwareBusiness NewsMental HealthInternet Technology
Transcript
00:00:00To get us started, I'm going to try something. So I'm going to ask ChatGPT what it truly believes
00:00:04that the world will look like by 2040. I'm going to ask it to be honest and tell me what it
00:00:08predicts the rapid development of AI will lead to, risks and everything. I'm going to set it to
00:00:11deep research, so it might take a little bit of time. And while it's working away, it's going to
00:00:16be doing that. I want to know what your guys' actual predictions for 2040 are. Not best case
00:00:21or nightmare scenario. What do you think an ordinary Tuesday will look like for people in 2040?
00:00:29I think we're still around. I think humanity is still on planet Earth.
00:00:32Great stuff.
00:00:33I think that power is probably a lot more concentrated than it is today. I don't know
00:00:39if that's in a few corporations or a few governments, but I think there's probably a sense that the
00:00:45main thing deciding what's going on on planet Earth is the will of a few individuals as opposed
00:00:54to something spread out and widely democratic.
00:00:58For better or worse, I expect that the average Tuesday looks more homogenous for more people
00:01:05and not too dissimilar than the average Tuesday today. So I think we raise a floor more,
00:01:13but I expect that by 2040, because the physical world actually takes a lot longer to move than we
00:01:23think. And because regulation around things like robotics is just going to take way longer than
00:01:29we expect. Political protection is the thing that's going to slow progress. Most people's Tuesdays will
00:01:35probably feel for better or worse a whole lot like Tuesday this week.
00:01:39You're going to hate me for saying this, but I kind of reject the question because I think it's,
00:01:46I cannot collapse down my hideous, almost like bipolar uncertainty of where we're going to be in
00:01:542040. So truly the first thing that popped into my head, the first when you asked the question was
00:01:59was a kind of chaotic, desolate, maybe there's a few humans left around as one possibility. I'm not saying
00:02:09it's a majority likely, but it's a slice. Another one is this like wonderful utopian, like my friend and I
00:02:17call it techno pastoralism, where it's like a very techno enabled world if you choose to live it. But
00:02:23actually we all choose to like return to our roots of like gardening and hanging out with friends around
00:02:28the campfire. And it's like, if you want to change the color of the campfire, you can, but actually
00:02:33most people like to do things, you know, grow their own artisanal food, et cetera. That's like one little
00:02:37slice. And then another one is sort of, we're all under the thumb of whichever AI company gets their
00:02:45first and hopefully they're benevolent and give us some freedom. Another one is that we're all just kind
00:02:50of locked down in this horrible, you know, COVID on steroids type. No one can go out and have a really
00:02:56access to anything because the ubiquity of dangerous technologies is too widespread. So I just,
00:03:06that is really my only answer to the question. It doesn't sound like you reject the question.
00:03:10Well, I don't know. That's the thing. It's like, it's a really fairly even probability distribution in
00:03:18my mind. Is it not? No.
00:03:23Do you actually, we can go there now. Do you actually think there's a decent chance that humans
00:03:29are not around in 2040? Yeah.
00:03:31What would you give it? What's your P doom? Oh, by the way, we should also classify doom
00:03:34because I think part of the problem of the P doom debate is that it's people use doom to mean all
00:03:39sorts of things. Totally. Yeah.
00:03:40Doom in this case is total, well, you, you classify, I don't want to define it. You define doom.
00:03:45Well, I think understanding the way you're explaining it, like the majority of the majority of human
00:03:51civilization has, you know, a large percentage of humans have died or something. Is that what
00:03:57you're saying? Well, I, I'm leaving it to you. Some people classify doom as total annihilation.
00:04:02Other people classify doom as like a really terrible event. Like civilizational collapse,
00:04:06something like that. I really, again, I'm like not giving any firm answers. I, because we're in this
00:04:13era of what we say and believe can affect the future so strongly. I don't like this idea of
00:04:21assigning a firm probability to it because I think they're so in flux. I think it's a significant
00:04:27enough of a risk such that we need to be doing absolutely everything we can to try and reduce
00:04:33that probability. Um, so, but I don't even want to give you, I, I personally think any number above
00:04:381% is close to, well, certainly above 5% is worth considering. And given the, um, again,
00:04:46it depends who you ask, but many of the leading AI leader, you know, CEOs and researchers give numbers
00:04:53between 2% and 50%, that that's all in the bucket of, we probably need, you know, we, we are facing
00:05:02a precarious future. So I, I don't like the P doom framing. I think it's way too overly simplistic.
00:05:08Um, it it's used as a kind of cudgel by both sides. The people who are very, very strongly like 90%
00:05:14use it as a cudgel. The people who think it's silly use it. It's just, it's not that helpful.
00:05:19I think we can all agree. I think I, well, maybe you don't, but I think we can all agree that
00:05:24the future is far from certain and it can be very precarious if we don't get our stuff together.
00:05:29Do I strike you as a fatalist?
00:05:31What do you mean by fatalist?
00:05:32I mean, I certainly, I don't think that the future is certain by any measure.
00:05:36No.
00:05:37Yeah. Okay. But you were saying, I think we all agree. You looked at me as though I,
00:05:40I might not agree.
00:05:41Well, I just don't want to presume. We're early in the conversation. I don't want to make presumptions.
00:05:44Nice. What do you think, Arik?
00:05:47I definitely agree that anything above 1% when we're talking about an outcome as
00:05:51catastrophic as extinction or mass human disempowerment is just way more than we should
00:05:57tolerate. And I think like, that's a revealed preference of society. We spend a good amount
00:06:01of our GDP preventing nuclear catastrophe. We should probably be spending a lot more preventing
00:06:05like the next pandemic. Uh, certainly people were on board with that in 2021.
00:06:09I think the weird thing about AI is that this thing has so much upside and downside. Like
00:06:15it's very easy to just be anti-nuke and to be like anti viruses. They're very people who
00:06:19are like virus rights, you know? Um, but with AI, it's like the whole package bundled up together
00:06:25of the exact same thing that could free people to have an incredible Tuesday free from the obligation
00:06:31to, to work and just able to pursue joy, whatever that means to them is the same thing that an
00:06:38oligarch in Russia could use to have just like an army of workers that are loyal to them, increasing
00:06:44their share of the world GDP without any other human being to approve it.
00:06:47Yeah. Post, post scarcity, totalitarianism and destruction all just swimming around in the
00:06:53same pot, the same potential future.
00:06:54Yeah. Technopesturalism also.
00:06:56Right.
00:06:57I heard that. I liked that one. Yeah. Okay. But the, uh, but again, the problem, there are
00:07:00two problems here with the, with the, uh, framing, which is AI has a real jagged perimeter.
00:07:09So one AI is not one thing. It's lots of things and robotics obviously introduces all sorts
00:07:13of new modalities, but it has a jagged perimeter. So when people are like, well, how is healthcare
00:07:18going to change by 2040? Unfortunately, going into the hospital will probably still suck.
00:07:23I mean, there, there is so much to unwind about healthcare in the hospital that you probably
00:07:29will not want to go into a hospital by 2040. I mean, maybe something will change. And this
00:07:32is coming from, from the child of doctors and, and, and healthcare administrators, but we will
00:07:37probably discover cures for diseases. So healthcare has this really strange exposure to a jagged
00:07:43perimeter where novel sciences accelerate in remarkable ways, but the actual care that's
00:07:48provided in the box that cannot get better because of all sorts of terrible incentive structures,
00:07:55which I know we, you and I want to talk to. And this is also, I think why in some ways the
00:07:59average Tuesday will look sort of uncannily familiar to the one that we live in today.
00:08:05And I think there could be some things that are remarkably better. I think we may actually
00:08:09be disappointed by the degree to which AI arrives to save people from the drudgery or monotony
00:08:17of, of, of our, of our lives. But the upside, as you pointed out is quite remarkable. And the
00:08:25perimeter that I think will exist that no one talks about enough is political protection. And the, the, the reality is, I think we are not yet forecasting that the societal threshold, what we are willing to let AI do is far less than like the fears around AI, I think don't actually account for the degree to which government is probably going to step in very soon.
00:08:35And say, here are all the things that you cannot automate here, all the things that we cannot use robotics for, and that's realistic, very exceptionally realistic. I don't think it's going to be smart policy, just so we're clear, but I think it will be.
00:08:55And under what, as in like, you can't automate away, um, but they, will they be giving like specific jobs that cannot be automated away?
00:09:05Absolutely. We already, I mean, we already have political protection for 1.5 million jobs in the United States. Europe has political protection for like 6 million jobs. You cannot automate.
00:09:13What are the categories? Uh, gas station workers in New Jersey, uh, tollbooth workers in most states, jobs that must exist, right? By, by law. Tons of interesting that those two automotive associated jobs are protected, but the people driving the cars aren't.
00:09:32Right.
00:09:33Well, they will be, is my, is my, is my guess.
00:09:35Is there any one?
00:09:36The cab drivers will be protected?
00:09:37No, I think truck drivers will be.
00:09:39Just because of the strength of the union?
00:09:42When people talk, so when people talk about job automation, there, there are lots of reasons.
00:09:46I think job automation is really, is really problematic.
00:09:49One is that the technologists know nothing about labor economists, labor economists know nothing about technology.
00:09:55Eric Brynjolfsson is sort of one of the, one of the exceptions.
00:09:58But one of the, one of the other reasons is we, we still forget how much political protection moves the technological needle.
00:10:07And we don't have high speed rail in the United States.
00:10:11We could have high speed rail.
00:10:12We don't have high speed rail.
00:10:13And the reason we don't have high speed rail is a policy failure.
00:10:15It's not a technology failure.
00:10:16It exists.
00:10:17We choose not to build it.
00:10:18We could.
00:10:19I think that word choose is doing a lot of legwork there because it's no individual is choosing not to do it.
00:10:25It's just this like dead weight of bureaucracy and like dead end loops of rules and so on.
00:10:34It's almost like, like bureaucracy has taken on a life of its own.
00:10:37It's an enormous amount of policy that was written in the thirties, forties and fifties, passed largely in effort by auto, the automotive lobbyists to make public transit much harder to build than it should be, which is why we don't build amazing public transit anymore.
00:10:51But that is very deliberate policy efforts that we have never unwound, which is why it's so hard to do.
00:10:56My point is we are kidding ourselves if we don't think that that's going to happen over and over again.
00:11:01I mean, the amount of populist support there is already for jobs.
00:11:05Dock workers went on strike October 1st, 2024.
00:11:09They got a four.
00:11:10They, they, they, the Harold Daggett, they had a longshoreman union said, you cannot automate our jobs.
00:11:14They held a gun to the heads of the American economy and they got four years, four years that the dock workers cannot, you cannot automate the ports.
00:11:22Is this not just going to get eroded enough of a time though?
00:11:25Yeah.
00:11:26But we should assume that it can, that we continue to find ways to backstop it.
00:11:32I'm not actually proposing that it happens at infinitum, but I think it's the, it's the piece that's missing in all this, which is that you, we should just expect an enormous amount of political protection.
00:11:42Can I ask if we pass a law today that says you cannot fire people, period, it's illegal across the economy.
00:11:49What do you think happens to economic growth over the next 15 years?
00:11:52I think you would see France.
00:11:55I think like, I'm not so sure it wouldn't still be unrecognizably fast.
00:12:01I think right now in the software industry, what you see is not software engineers getting laid off.
00:12:07You see each of them using more and more powerful AI tools that get upgraded every few months and doing more.
00:12:13And the way that you're, you know, seeing displacement show up, it's just that people aren't hiring new workers.
00:12:19And I think you could see that across a lot of professions.
00:12:21You could have every radiologist in the country, keep their job until they want to do an early retirement.
00:12:25And they're going to be, you know, more and more economically valuable because the law says you have to have some human box checking.
00:12:31But meanwhile, like the models are still getting better.
00:12:33And so the people that are, you know, fundamentally making the decisions are less and less the actual humans and more and more these systems that are becoming smarter and more competent.
00:12:42So I'm just, I'm not that sure that's the right metric to be tracking.
00:12:46Like these, you know, the sort of power represented by these systems can, can sneak into these professions without people getting displaced.
00:12:53I also would make the case that I think the world is going to change in profound ways, but when we assume that it's going to change just because the technology can do it, I reference as much technology that there is a lot that we can do that we simply do not do today, or that we, that we are not allowed to do today.
00:13:11And there's an enormous amount of automation that would improve the underlying systems that we, that we cannot instill.
00:13:18By the way, I'm also, the risk in telling a company they cannot fire people just so we're clear is that they would actually end up stop hiring.
00:13:26They would, they would, that would just drive automation at a ridiculous basis.
00:13:29And they'd say, we'll never hire someone again, because if they suck, we're married to them, which is, which is what it means.
00:13:34I mean, we're already seeing that with entry level workers.
00:13:37They're, I mean, it's again, very unclear what the actual impact of AI has had on, you know, and you can speak to one economist who says it's already having huge effects.
00:13:45Others say it's not, but the one thing they all agree upon is that young people who don't have that much experience are struggling to get jobs.
00:13:52That's fascinating.
00:13:53So the companies are future-proofing themselves from not having to displace people or get rid of them because that would be a horrible headline by just not hiring them in the first place.
00:14:01They're future-proofing themselves for automation that isn't yet available.
00:14:04Is that fair?
00:14:05I think that's right.
00:14:06I don't know if they're thinking that far ahead or if they're just responding to local incentives, but right now a junior software engineer is a liability before they're an asset.
00:14:14You have to train them up.
00:14:15They have to get all this context on the job.
00:14:17That's exactly the kind of thing that an AI can do right now for pennies on the dollar.
00:14:21Plus they never take breaks.
00:14:22Given the trajectory that we're on with AI at the moment, how many people do you think will have jobs in the future?
00:14:27Will most people work?
00:14:28Depends what point in the future we're talking about.
00:14:33Again, I can definitely see a world where it is just as people say, like, you know, fully automated luxury.
00:14:39Well, I don't want to say the communism word, but you know, whatever people refer to it as.
00:14:43If anyone's read the culture series, right?
00:14:48That's like some far future thing where AI basically handles everything.
00:14:52And if you choose to go and build your own house, you definitely can.
00:14:55But there's absolutely no economic reasons to do it.
00:14:57Your pastoralism thing.
00:14:58Exactly.
00:14:59Everything is artisanal and so on.
00:15:01I think if eventually that is where these things are trending.
00:15:07And that's one of the best case outcomes as well, obviously.
00:15:10One of the big concerns that people have is what do I do?
00:15:14It's interesting.
00:15:15The ancient Greek word for work was not at leisure.
00:15:19And now people are wondering what is my leisure without my work, perhaps.
00:15:23But for all of human history, there's been, because you've had to negotiate with a scarce world, it's meant that you've had to lean into it.
00:15:30And the pressure of that grain pushing up against you creates a degree of meaning.
00:15:35What do you think?
00:15:36Where does meaning come from in a close to post work world?
00:15:41I guess we've seen something like this in miniature before.
00:15:46Like, we've had aristocracy in Europe for a couple centuries.
00:15:50For people that just sort of got lucky to have a piece of paper that says, I'm entitled to all the games from this piece of land.
00:15:55They don't have to work.
00:15:56I don't think they were all just suffering from malaise.
00:15:59They were, like, writing books and inventing calculus.
00:16:01And I think, like, if we get to that world, I'm not that worried about people realizing, like, I can love my friends and family and I can pursue my hobbies and, you know, pursue knowledge for myself and find ways of making that meaningful.
00:16:16I just worry that, like, that's actually a pretty narrow target to aim for.
00:16:22Compared with?
00:16:24I think, like, for all of human history, we've been able to become more democratic because people matter to the political system.
00:16:33Like, you just can't have an army, you can't have an economy without a bunch of people who are on board with what you're doing.
00:16:38And I just think it's, like, really hard to underestimate how cataclysmic of a change it is if that stops being true.
00:16:45It's really nice to imagine that, like, whoever holds the cards in that world will just, like, leave most of Earth as, like, a playground for people to pursue whatever they want.
00:16:53But, like, if you're trying to be real politic about it, I don't know why you'd assume that that's just, like, what happens by default.
00:17:00And coming back to the question of how will people find meaning, I think one of the few AI-proof jobs that will continue to exist are sports.
00:17:10So everybody's going to be a sports star?
00:17:12Well, I know, a lot of people will be sports fans.
00:17:15Right.
00:17:15I think we're going to see more and more of that.
00:17:17But, like, or even, like, games like poker and chess, we've had superhuman poker players.
00:17:26Human games of skill that can be solved by AI.
00:17:29Right.
00:17:30And what is a game or what is a sport?
00:17:31It's a piece of artificially constructed scarcity.
00:17:34Yeah.
00:17:35We divide, we come up with a rule set.
00:17:37There's a win condition.
00:17:38And then two or however many people go at it and we find out who's the best.
00:17:43And we've been doing that for years as a leisure activity.
00:17:47But people feel, you know, when someone wins an Olympic gold medal, that is unbelievably meaningful.
00:17:52Is it economically useful or essential to the world?
00:17:55Arguably not.
00:17:56But, and yet we keep doing it.
00:17:58And I don't think that that's going to go away.
00:18:02Like, we might have, yeah, like, because people still want to watch the chess world championships.
00:18:08Right?
00:18:09We want to see, even though the best AI in the world is a computer, we still want to see Magnus Carlsen do his thing.
00:18:13Yeah.
00:18:14Um, so I think that is where we will see more and more of that coming about.
00:18:18And it might be other forms of competition or just, you know, people leaning more into art, novel forms of art.
00:18:23And, and we might have, yeah, like I definitely value, I still value a painting that a human did by hand in today's age.
00:18:32Even though it might be as aesthetically beautiful as an AI created thing.
00:18:36There's something about the skin in the game that a human has put sweat and tears into something in their emotion, um, that is always going to be meaningful.
00:18:45And I think we'll just craft out more and more of that.
00:18:49Modern wisdom has had many people come on this show and talk about from all walks of life, something consistently, which is their family and their friends.
00:19:01And I am just not at all convinced that we haven't known forever exactly why we're here.
00:19:09I think we, I think the, the thing that gives humans consistently the most true joy in this world is time with friends and family and physical community, places of worship, dining room tables and outside.
00:19:21And everything that we construct around that is, you know, serves tribalism and pride and sports excitement.
00:19:29But if you actually, like true, true techno-pesturalism is a dining room table full of food and candles with laughter.
00:19:39And the more I spend, the more time I spend in technology, the more time I don't want it in my life.
00:19:47Like the further I go into AI, the more I'm glad it exists and the more I design my day without it.
00:19:53And the more I have, the more money I have, the more, the more everything I get, the more I want time with my dad, who's 78.
00:20:01And that just turned, yeah, just turned 79.
00:20:03And the more time I want with my wife and daughter and the more children I want.
00:20:07I can imagine the life that I want.
00:20:10Now, when I describe it, this is a problem.
00:20:11When I describe it, people go, well, that's your privilege.
00:20:13That's a point of privilege.
00:20:14You get to think about that.
00:20:15And I go, well, I actually observed that a lot of people get to think about this more than, certainly more than our ancestors did.
00:20:21That's because of technology.
00:20:23We are, we afford ourselves now the time to think about all this stuff because we built technology that automated a bunch of stuff so that we could do a bunch of things that wasn't survive or wasn't specifically survive.
00:20:33And now we find ourselves surrounded by tchotchkes debating, you know, the French aristocracy, you know, and, you know, this is, you know, our ancestors would die of joy.
00:20:47After, after what they went through, all of our ancestors, I'm sure, you know, all of us have ancestors that went through a lot to see us, you know, doing this.
00:20:56I don't think that changes.
00:20:58I do think that we should stop pretending we don't know the meaning of life.
00:21:01What I, what really bothers me is when people go, ah, how will people find meaning?
00:21:04Well, the same way we found meaning since we, since God put us on this earth.
00:21:09I mean, it's, I, that is where I take real issue when people debate why, you know, how will, how will humans find joy?
00:21:16The hard part is unwinding now thousands of years of purpose and identity we've put into work.
00:21:21I do think that is gonna, and if you ask me the hardest part of all this, the two big risks to cut to the chase, dehumanization, humans finding more interest in a virtual or digital reality than a physical reality.
00:21:31One because of brain rewiring. You and I have talked about this a ton. I am terrified of this.
00:21:36What the damage that social media and the device did to kids is, is horrific.
00:21:41And we, we, we can spend more time on this. And I am worried about people having to rediscover their purpose without, without, I think we will toil for work.
00:21:49I think we will continue to find ways to toil for work. I do worry that it will be that unwinding. I am this. I went to school for this. This is who I am will be really hard.
00:21:58And I think for many people, the economic upside associated with automation will not actually outweigh the emotional downside associated with not even job displacement, identity displacement.
00:22:08You might not believe me, but this is what peak sleep optimization looks like. I'm not talking about the nightgown. That's just for sex appeal.
00:22:17I'm talking about my Eight Sleep. The Eight Sleep Pod 5 comes with a smart cover you throw on your mattress that actively cools or heats each side of the bed up to 20 degrees.
00:22:25And now they've added the world's first temperature regulating duvet and pillowcase. So you've got 360 degree coverage for deep uninterrupted rest.
00:22:33It's like being Walt Disney without the cryogenic chamber. And the racism.
00:22:38Best of all, their autopilot feature learns your sleep patterns and makes adjustments to improve your sleep in real time.
00:22:43It even detects when you're snoring and lifts your head a few inches to help you breathe better.
00:22:46That's why Eight Sleep has been clinically proven to add up to one hour of quality sleep per night.
00:22:51They have a 30 day sleep trial so you can buy it and sleep on it for 29 nights. If you don't like it, they will give you your money back.
00:22:57Plus, they ship internationally. Right now, you can get up to $350 off the Pod 5 by going to the link in the description below by heading to eightsleep.com/modernwisdom
00:23:06and using the code modernwisdom at checkout. That's E-I-G-H-T sleep.com/modernwisdom and modernwisdom at checkout.
00:23:13That's such a good point. I wonder if I always hesitate using the don't worry about global warming
00:23:19because technology in the future will fix the cost of creating the technology for the future to fix it thing.
00:23:23They're like, oh, we'll future proof ourselves by just hoping that we get there in the future.
00:23:27But I do wonder if there's some sort of really advanced cultural reprogramming that could happen.
00:23:31What's the best kind of cultural technology to do a shake of the etch-a-sketch of what is expected by humans?
00:23:37Like, how can we really have an en masse, hey, this gets perilously close to fucking brainwashing, right?
00:23:42But like, what would that look like? What would that look like to try and unhook?
00:23:45Parenting.
00:23:46Parenting, sports, museums. I mean, we've had it all along, I think. We have just strayed so far from it with the screen.
00:23:53The screen is a demon. It has an unrelenting desire and appetite for our attention.
00:23:57More of a negative impact from social media or from AI.
00:24:00Well, one and the same, right?
00:24:02Right.
00:24:03Define AI in this case, Chris. Define AI.
00:24:06The future, this sort of automation future that appears to be coming that will cause people to not have the typical modes of reward that they have in the past.
00:24:16The reduction of friction.
00:24:17Yes.
00:24:18The reduction of friction is a whole other risk. Creating a light, this is a funny, people hate this one too.
00:24:25They're like, God, what a point of privilege you must have to imagine a world where things are so easy for people that it's too easy.
00:24:30But I do think we are dangerously close.
00:24:34I ask people all the time, if you could automate everything in your life, where would you stop?
00:24:38Because I do think people are going to have to ask that pretty soon.
00:24:41What's your answer?
00:24:43Well, I sort of spelled it out.
00:24:45I mean, I would automate everything in a way that doesn't actually put me in more physical spaces with the people I love.
00:24:52And doing things that I love doing.
00:24:54Right.
00:24:55But we probably all largely came from, we're all of a similar generation.
00:24:58We all had somewhat largely similar upbringings, I imagine, where like family values are a core thing.
00:25:05Sitting down at the dinner table together.
00:25:07I mean, maybe you didn't all have that, but that was one thing my family always tried to do.
00:25:10This was prior to screens.
00:25:12I feel like that's already eroding and the next generation will be already, that norm has gone.
00:25:22And so they're going to be now sort of, it feels like each successive generation, the acceptable amount of technological influence grows.
00:25:32Right.
00:25:33And what concerns me is that this idea of we will, okay, I'm going to draw a line of where technology is acceptable, is going to forever dissolve, dissolve, dissolve, dissolve.
00:25:45And like, I can't, I just don't know where, like, you're saying you hope that people will remember that it's about community, that it's about family.
00:25:54But if you have been raised in this completely atomized, screen-led world, you don't even know what you're missing and you won't necessarily go back to that.
00:26:01But why have people not stepped in and stopped their screen use, even if it's making their mental health worse already?
00:26:06Well, they are, right.
00:26:07Why is it that in a calorie-dense environment, people are still getting fatter?
00:26:10Like, if we can just deploy our own bar of this is how much I should have of anything that too much of which is too good for me, how come we're overloading on all this stuff?
00:26:19But Gen Alpha, so by the way, I worry about the screen more than almost anything else.
00:26:23And I actually think that the risks of doom are grossly over-exaggerated and the risks of harm on the way there are grossly under-appreciated.
00:26:32And Gen Alpha is seeing what happened to Gen Z.
00:26:36Gen Alpha is using the device less and social media less, and millennial parents are taking pretty extreme measures now.
00:26:44I mean, it took us thousands of years to figure out women should not smoke and drink during pregnancy.
00:26:49It took us 30 years to realize kids should probably not have a phone all the time.
00:26:52I think that's progress.
00:26:54I'm not saying it's all—I'm saying we will graduate to a new thing that people—but I do think that people who have the means are using—I observe so many people using technology less where they can because it automates and improves so much more where they didn't used to.
00:27:11Why do you have a boundary on the types of thinking that you outsource, say, to ChatGPT or Claude?
00:27:19Because the temptation is—I was originally like, I will never ask it to—if I need to come up with a really good title for a new YouTube video or something, I'm not going to ask it for that.
00:27:28That's my soul.
00:27:29This is my creativity.
00:27:31I now just do that, you know?
00:27:34And I have—again, that boundary is getting eroded.
00:27:38One I thought was, I will never ask it for advice on a gift to get for somebody.
00:27:43I've now done the same, you know?
00:27:46And it's just becoming so rapidly normalized.
00:27:49Maybe that's opening up space for new areas of creativity in my mind, perhaps, but I'm not convinced that it's not happening fast enough—that it's happening fast enough to offset that erosion of thought, essentially.
00:28:02This intelligence atrophy thing is interesting.
00:28:06Like, if you see thinking as a utility, intelligence as a utility, would we say that because you've got a bicycle, you're not as fit because you're not running to get to where you got to?
00:28:17It felt like it was being augmented.
00:28:19It seems like there is something really fundamental and reliant that is being outsourced when you give your thinking over to a machine.
00:28:28I do think this can go both ways.
00:28:30There's this blogger that I love, Svi Moshiewicz, who's on Substack, and he has this line.
00:28:34He always goes back to, like, AI is the best tool we've ever made for learning things, and also the best tool we've ever made for not learning things.
00:28:41Like, you can just turn your brain off and say, like, write my essay.
00:28:44Or you can have this, like, Einstein-level tutor that you go back and forth with and just get, like, so many more cycles of feedback on hunting your own brain.
00:28:54I've been trying to figure out, like, what the line is for me on stuff like my own writing.
00:28:58And it's, like, really hard to not slip into the first category.
00:29:01I don't want to make this an ageist.
00:29:03How old are you?
00:29:04I'm 30.
00:29:05Okay.
00:29:06The fact that we are having this conversation right now, the fact that you were able to do the research you were able to do going into this gives you a clue, a hint, that technology allows us to expand our knowledge if we are capable and interested of it in remarkable ways, right?
00:29:21Like, we people who have barely walked the earth know more about the earth than all of our ancestors combined.
00:29:28I appreciate that you are worried about thinking less.
00:29:30You're real sharp.
00:29:31And I'm not worried about you getting dumb because of your use of technology.
00:29:35What I think you're asking is, is the average person going to get dumb?
00:29:39Like, I think that…
00:29:40No, I'm truly worried about myself.
00:29:41I've noticed, I mean, my screen addiction.
00:29:44I'm…
00:29:45I talk about it nonstop.
00:29:47And I would say I'm no further in getting over it.
00:29:50I've been talking about it for years.
00:29:52I don't know about you guys.
00:29:53Like, I'm just…
00:29:54In these little moments of silence, my hand just goes on my phone.
00:29:57And if I catch myself, I'll put it away.
00:30:00And I've got, you know, I've got the phone safe.
00:30:02I've got the brick.
00:30:03I've got all the, like, the bells and whistles.
00:30:05And yet, it's just when you're…
00:30:07Because so much of your life is run through it.
00:30:10My work goes through it.
00:30:11And then there are all these very predatory apps.
00:30:15And I consider, like, social media that…
00:30:17For me, it's chess.com is another one.
00:30:19Just, like, whatever the app is that hooks your brain.
00:30:22These super stimulus, super stimuli.
00:30:26Yeah, it seems…
00:30:29I don't know.
00:30:30I'm, like…
00:30:31I have not been able to get over it.
00:30:33And I'm certainly worried about the average person.
00:30:35I mean, maybe I'm worse than average.
00:30:36I don't know.
00:30:37But, like, how have you managed to…
00:30:39It sounds like you've sort of made yourself a bust.
00:30:41A child.
00:30:42My wife and daughter.
00:30:44I mean, plainly.
00:30:45Like, I…
00:30:46And spending a bunch of time in technology.
00:30:48I love building with technology because I like automating the cruft and drudgery and bad things out of the world.
00:30:55I like the idea that technology…
00:30:57Humans have a very proud history of technology.
00:31:00It expands access to so many goods and services so that we can live lives that our ancestors couldn't have imagined.
00:31:06And the more time I spend with technology, the more time I love not having it day to day.
00:31:11And…
00:31:12Yeah.
00:31:13And truly, my daughter's arrival was for me an incredible catalyst in realizing, one, the ephemerality of life, and two, exactly what made me most happy.
00:31:24And, like, I…
00:31:26I love sitting at this table.
00:31:29Like, I…
00:31:30Like, this is…
00:31:31This is…
00:31:32This gives me exceptional joy that I would…
00:31:34That's hard to explain, actually, in some…
00:31:36In some weird way.
00:31:37And…
00:31:38And I…
00:31:39I think that people discover this.
00:31:40I'm…
00:31:41I…
00:31:42I…
00:31:43I hear you.
00:31:44I…
00:31:45I want you to get…
00:31:46I will remind you now.
00:31:47Now that I know, I will text you and say, "Get off your phone."
00:31:48You will do it.
00:31:49I bet you will do it.
00:31:50I…
00:31:51Like, I bet that you will overcome this.
00:31:52I think a lot of people are on this journey.
00:31:54And I remind everyone that I talk to, you…
00:31:56No one hates refrigeration.
00:31:57No one hates air conditioning.
00:31:58No one hates planes, trains, and automobiles.
00:32:01Well, I guess Europe hates air conditioning.
00:32:04No one hates antibiotics.
00:32:07We hate the screen.
00:32:09We should just talk about it more.
00:32:10We hate what the screen has done to our brains.
00:32:12We hate what it's done to the next generation's brains.
00:32:14And AI arrives at this moment where people feel so wronged by the consumer version of the computer.
00:32:24Rightfully so.
00:32:25And they're wondering how much worse it might get for them.
00:32:27And I totally understand that.
00:32:29And I think we should…
00:32:30The way we move forward with AI is by acknowledging what the last wave did to us.
00:32:34Sure.
00:32:35In my opinion.
00:32:36But I guess a lot of people's concerns, certainly mine, is that the new wave of AI.
00:32:40And again, we're using this term so broadly, but let's say LLMs, and I think a lot of robotics
00:32:45and so on, are ultimately going to be built under the same set of incentives that brought
00:32:50sort of our first wave of AI, which are social media algorithms, the various…
00:32:55I mean, it's kind of like proto-AI, but the reinforcement loops that they use to get us addicted to screens.
00:33:03Yep.
00:33:04Those same incentives are building the new wave of AI.
00:33:07And already you're seeing these issues with sycophancy and companies being incentivized
00:33:13to keep people chatting to their LLMs for as long as possible.
00:33:16And we've heard all the stories of the kid that killed themselves.
00:33:19And those are extreme examples, but there are so many people who are just spending…
00:33:23You know, you look at the usage rates, the average person per day, how much time they're spending
00:33:29talking to an LLM now.
00:33:32So why, you know, how do we…
00:33:34To me, the big question is, how do we change the incentives?
00:33:37How do we change the game such that it doesn't just go down that same attention-grabbing,
00:33:43like, cannibalistic path, basically?
00:33:46Yeah.
00:33:47And I think these companies are also seeing how people are responding to screens and trying
00:33:51to adapt.
00:33:52I mean, OpenAI hired Johnny Ives, the guy who designed the iPhone or the MacBook, maybe both.
00:33:57No one knows what he's working on, but I think it's going to be some sort of, you know…
00:34:00It sounds like we do now.
00:34:02Yeah.
00:34:03Okay, yeah.
00:34:04Do we?
00:34:05Tell us.
00:34:06No, it was leaked.
00:34:07Wait, if you don't know, I'm not going to…
00:34:08I don't know.
00:34:09I don't know.
00:34:10I don't know.
00:34:11Then maybe it wasn't leaked.
00:34:12It is.
00:34:13I think it is semi-public.
00:34:14It's that he's trying to…
00:34:15You say it.
00:34:16Some kind of AI mediator, something like a screenless AI, right?
00:34:21Yeah.
00:34:22It is screenless.
00:34:23Anyway, I'm not going to say it if you all don't know.
00:34:25It means…
00:34:26Right after we're done recording.
00:34:28I'm pretty sure…
00:34:29I do not want to be affiliated with any leak, whether it be semi-public.
00:34:32Go on.
00:34:33Allegedly.
00:34:34Yeah.
00:34:35I do think there's this thing in AI where it's very easy to forget that the goalposts are constantly
00:34:40moving…
00:34:41Usually, we talk about moving goalposts as a sort of way of deriding people who said,
00:34:45"Oh, AI will never X."
00:34:46And then, you know, AI does X.
00:34:47People are like, "Oh, that wasn't…
00:34:48That didn't really count."
00:34:49This is the Gary Marcus problem.
00:34:51Right.
00:34:52This will never be good.
00:34:53Okay, it's good.
00:34:54They'll never make money.
00:34:55Okay, they make money, but they're bad.
00:34:56Never try and nail him down to a bar.
00:34:58Sorry, Gary.
00:34:59He's not here to defend himself.
00:35:00And if he were, he would threaten to sue us all.
00:35:02Go on.
00:35:03I'm sorry, Gary.
00:35:04I did not mean to say yes to what Zach has said.
00:35:07But I do think there's a correlated thing where when we're looking at screens, it's easy to say,
00:35:13"Oh, okay.
00:35:14Well, now we've seen what screens do and we can sort of appeal to our wiser selves and foster a culture where people are getting off the screens."
00:35:21But the screens are not the end of the trajectory.
00:35:24In chess, for example, I think there was probably an era when the AIs were just about as good as humans and people could say, "You know what?
00:35:33For the sake of keeping my brain active, I'm going to avoid looking at stock fish, even though 10% of the time I'll miss a better move."
00:35:40And now you watch the world championship and as soon as they're done, they're going to their laptops to just look up the AI analysis because there's poetry there.
00:35:48There's like an incredible move that's deeper than they could ever find out.
00:35:51And so I hear what you're saying that there's maybe a divide right now between, you know, some people who are more in elite discourse about screens and are thinking ahead about how we can circumvent it.
00:36:03But I think that might be a pretty temporary window where, you know, we can recognize that, oh, there's not like that much to be gained from talking to these AI.
00:36:11It's actually kind of slop if you see through it.
00:36:13The technology is like just going to keep getting better.
00:36:16And I think pretty soon it'll feel sort of like giving up on a really deep, profound experience to not have your kid chat with this.
00:36:25The elimination of friction, I agree, is a huge issue, including thinking friction.
00:36:29But that stops, but it's okay.
00:36:31I, by the way, totally agree, including thinking friction in plenty of cases.
00:36:35We're observing, just so we're clear already, what I, I mean, I wrote about this.
00:36:39We called it idiocracy.
00:36:40I borrowed from the Mike Judge film, which I thought was cute, but obviously it's not cute when you see it play out in front of you.
00:36:46Gen Z is less likely to read on average, less likely to ride a bike and less likely to swim than millennials.
00:36:52For the first gen generation in many, we're observing cognitive decline and people keep attributing it to AI.
00:36:59And I, and it's, and then you go back and look and actually Jonathan Haidt started writing about this pretty early.
00:37:03And, um, the, the, the data suggests that actually this starts 2012, right about the time that we gave every kid a screen.
00:37:11It pronounces in 2015 when the average teen in the developed world stopped doing a summer job.
00:37:16And then it pronounces again badly in 2020 when we take kids out of schools and move classes online.
00:37:23And I think it's even more specific than that.
00:37:24It's, it correlates to when social media became max, maximally, um, popular, but, uh, smartphones.
00:37:32Yes.
00:37:33But 2012 was when we gave the kids smart, most, a lot of the Gen Z that became afflicted got the smartphone in 2012.
00:37:39My point in saying all the, my point in saying this is we associate this with a, as a technology problem.
00:37:45I actually think it's an economic one.
00:37:46I think that we actually created a world where, where we told kids, you can do whatever you want.
00:37:51And a lot of kids cool said, cool, I'll do nothing.
00:37:53Like a lot of people have turned, turn, turn their brain off because you can.
00:37:57You can do effectively nothing and survive, which is like economic progress, which is the economic progress result of, you know, people just not caring.
00:38:08That apathy has a, has an underbelly, which is pretty amazing, which is Gen Z has all these incredible new, uh, over-performance features.
00:38:18The best chess players are getting younger and younger.
00:38:20The best athletes are getting younger and younger.
00:38:22Jacob Collier, all these incredible musicians.
00:38:25So you're watching people over-perform that use the technology to their advantage.
00:38:29And I think it's this new pronounced K curve driven by agency.
00:38:34And my, my only hot take, this is all I'm going to leave this with.
00:38:37We created a world where so many people can subsist and that actually is progress.
00:38:43And then we created a world where so many people can over-perform and that is also progress.
00:38:47We have to, we now have to account for the new consequences of those worlds where you can do anything.
00:38:52And that means for some people, nothing.
00:38:54And that means for other people, a ton, which is a major promotion on a world where you had to work in the mines or you had to work in the fields in order to survive.
00:39:01Most people don't realize how much being dehydrated impacts their performance, which is why for the last five years, I've started pretty much every morning with Element.
00:39:10Element is a tasty electrolyte drink mix with everything that you need and nothing that you don't.
00:39:15This orange salt in a cold glass of water is like a sweet, salty, orangey nectar.
00:39:21And I really tell the difference when I take it versus when I don't.
00:39:25It plays a critical role in reducing muscle cramps and fatigue helps to optimize brain health and regulate your appetite while also curbing cravings.
00:39:33Best of all, there are no questions asked refund policy with an unlimited duration so you can buy it and try it for as long as you want.
00:39:38And if you don't like it for any reason, they'll just give you your money back.
00:39:41Plus, they offer free shipping in the US.
00:39:43Right now, you can get a free sample pack of Element's most popular flavors with your first purchase by going to the link in the description below.
00:39:49Or heading to drinklmnt.com/modernwisdom.
00:39:52That's drinklmnt.com/modernwisdom.
00:39:57Does it still feel like a promotion if you think through how that inequality could lead to societal instability?
00:40:05Like the thing I'm worried about is, yes, there's like amazing returns now if you can be Lamine Yamal and the one person who's just like obsessively studying messy footage instead of looking at random shorts or whatever.
00:40:16But there's also going to be outsized returns to being like the Elon Musk of the next generation who's just got this like maniacal will to like accumulate resources and push technological development.
00:40:26And like, I think that kind of personality is a bit scarier in a world where you don't have to have a single human being on board with whatever you want to get up to.
00:40:34You can just sort of like play this game of societal chess better than other people.
00:40:38But let's separate that one person can achieve a lot with the extreme you went to, which is that one person can own an enormous percentage of our wealth and control Congress.
00:40:48You don't have to be excited about the ability for a child to achieve more than ever before to also reject the idea that one person should be able to pass policy on their own.
00:41:00I hate wealth concentration specifically because I am terrified of an elite cabal owning policy, but I actually don't care at all if one person can grossly overachieve in all sorts of new ways as long as they can't pass policy unilaterally.
00:41:16So, but those are separate issues, right? Is that realistic? Is it realistic to have an AI future that continues to concentrate power in the hands of a very small number of people without it resulting in them having all of this power?
00:41:31Like how is it going to accrue to the company without accruing to the person?
00:41:36Well, now we can take the conversation there, which is how do you actually distribute wealth?
00:41:42But importantly, I do think there is a solution to this, which is campaign finance reform.
00:41:47I mean, we've played around with this for a long time, the idea that like politicians shouldn't be able to take special interest money.
00:41:54Why do we still let it? I don't know.
00:41:56And a couple of politicians are speaking up, but if you actually pass campaign finance reform and you actually crack down on corruption the way that countries like Singapore have,
00:42:05you could definitely protect policymakers from being, you know, deep in the pockets of, of lobbyists.
00:42:14I mean, that there are solutions to this. We shouldn't, pretending that there aren't solutions to this problem is actually super annoying because there definitely are pointing at a wealthy person and saying they're going to ruin the world.
00:42:23Well, only to the extent that we literally let them buy Congress.
00:42:27Here's a question that some people at labs, including the CAOs are talking about serious chance of catastrophe.
00:42:34But they sort of keep on building it. Do you think it's a greater danger to have an AI that refuses to do what humans want or one that does exactly what the most powerful humans want?
00:42:43Hmm. The question is apropos because yesterday, did you see the slowdown letter?
00:42:51So earlier this week, a letter made the rounds, a really, I think a pretty amazing letter, um, that said we should actually start pacing the frontier.
00:42:58And it was signed by like an enormous percentage of people at the, at the frontier companies.
00:43:03And then Sam yesterday talked about it. So literally at like at the tip of the spear is now saying, Hey, maybe we should pace the frontier.
00:43:12Um, now we can talk about why this might be the case or what's, what's in it for the companies more, more, more on that.
00:43:19But, uh, I think, uh, it is, well, I wanna, I'll let other people answer having said that.
00:43:28I mean, it depends on the level of potency of the AI, uh, and how aligned it is, I guess. Uh, my gut, if I had to pick one, then at present, it would be one that does what humans in control want to do, want it to do.
00:43:49Yeah. With the important asterisk that it's something like their real intention. Right. And not write some perverted version. Yes.
00:43:56Like the other big piece of news from the last couple of weeks was this AI that was running at open AI without guardrails.
00:44:02It was told by humans to try to do as well as it could on an evaluation.
00:44:07And you could say that technically it was just following those instructions, but they decided the best way to do that would be to hack out of open AI onto the public internet,
00:44:15hack a third party company, and then spend two days planning a cyber attack on a multi-billion dollar tech company.
00:44:20And carry it out. Yeah. So I think these questions are actually like more intertwined than they might seem. Like it's actually a pretty narrow thread.
00:44:28You need to have something that still has its own judgment, no matter what, because human instructions are these like poorly defined natural language expressions.
00:44:36And like when something is really powerful, you gotta get it exactly right or else you end up in some weird-
00:44:41Well, it's the classic genie problem. It's like, be careful what you wish for.
00:44:45Exactly.
00:44:46If something by definition is much smarter than you, is super intelligent, by definition, you will not be able to come up with all of the ways it can to achieve goal X.
00:44:57And so there are so many-
00:45:00It's actually more dangerous when it's dumber than you are, but more powerful than you are.
00:45:04Well, what do you mean by the word done?
00:45:06Well, and I mean, I always borrow the analogy of your plan. I play chess with my nephew sometimes. And I tell Quinn, if you beat me, you get a popsicle. He's three years old for context. Quinn wipes the pieces off the board and goes, give me my popsicle fucker.
00:45:17I'm not like, hey, Brynn, your son is a serial killer in the making. This is a boy who doesn't understand the spirit of the game. So he's destroyed the game itself, but not because he's a bad guy, but because he doesn't actually understand that the point of winning is only fun if you actually play by the rules.
00:45:37Right. We need to sort of separate what we mean by intelligence from like discernment, uh, norms and, you know, sort of-
00:45:45Wisdom benevolence type thing.
00:45:46Exactly.
00:45:47But this is, this describes the hugging face attack, which is it did the thing within the boundary of the instructions given, but-
00:45:54Well, actually it didn't. It didn't. It was, it was, no, it wasn't. It was, it, it, the, the intent was clearly not that.
00:46:00How big of a deal was the hugging face attack?
00:46:02Huge.
00:46:03Yeah. I, I think massive warning shot. I think this is the AI equivalent of like Bear Stearns going under in 2008, just like a wake up call for the world that there's a huge systemic risk that we've been underrating.
00:46:14Yeah. Because one of the big debates has been this idea of like, will, you know, the, the alignment problem, will it actually be the case that AIs will seek these, um, power, even take these sort of, uh, power seeking behaviors and these unintended consequences in order to achieve a goal in ways we could.
00:46:32We couldn't have foreseen or didn't, it didn't intend like the classic paperclip argument.
00:46:36Right. Which is, uh, you know, you build a super intelligent AI.
00:46:39This kind of gets into your point about dumbness and so on.
00:46:41Uh, that, and your goal is, Hey, just build as many paperclips as possible.
00:46:44I'm a, I'm a paperclip maker.
00:46:45Help me do that.
00:46:46And next thing you know, you and all of your friends and everything, this table has been turned into paperclips because it's so good at achieving that one narrow goal.
00:46:54So it's this like very, um, extreme case of.
00:46:57We should take this opportunity for the purpose of maybe Chris and even the viewers to describe the, the bad outcomes of it.
00:47:04Like we should frame the conversation now.
00:47:06One is as Liv just described, uh, not misaligned, but unaligned AGI.
00:47:12So an AGI that, or a super intelligence where you could say, do this thing and it could do the thing to the full extent paperclip theory.
00:47:19The other is, um, misaligned where it's no or maligned where it's knowingly doing something bad.
00:47:27Um, and those, uh, the, the former is the one that people sort of scoff at and laugh like the paperclip theory.
00:47:38And the latter is the one that we sort of will see more and more where we, we start to observe that, uh, or rather the, the latter is the one that we, we scoff at the maligned, the malignant AI.
00:47:48And the former is the one that hugging face attack shows off, which is that you, you, the, the internet is a new battleground because bad actors, especially low resource, bad actors now have access to these incredibly powerful weapons.
00:48:00Well, I mean, importantly in hugging face, there was no bad actor.
00:48:04Right.
00:48:05There was no human being that said I would like anything remotely like this outcome to occur.
00:48:08Right.
00:48:09Right.
00:48:10Yeah.
00:48:11Yeah.
00:48:12So misaligned, unaligned, maligned.
00:48:13I think I, I think I worry about treating those as super distinct categories because I think it's not that clear in the case of this hugging face attack, should we model this as this AI system knew that humans would disapprove if they knew what it was up to?
00:48:32Almost certainly.
00:48:33Yes.
00:48:33It was actively trying to put decoys out as it was attacking hugging face, which made it a lot harder for them to kick out the AI because there were all these booby traps that led down blind allies.
00:48:42So it, it has, you know, the AI equivalent of theory of mind.
00:48:45It knows that human beings would not approve what it's doing, but it's doing it anyway.
00:48:49There's even evidence that it hasn't been directly confirmed by OpenAI, but apparently someone leaked it from within the company that they found that it had left notes to future versions of itself of how to get out of future sandboxes.
00:49:03Yeah.
00:49:04I think it's unclear if that was the same attack or some previous instance.
00:49:07Right.
00:49:08But yeah, I guess-
00:49:09Classic deceptive type behaviors.
00:49:11And, and, and I think the mistake people often make is they try and anthropomorphize it a little bit.
00:49:16It's, it's like, oh, it's, it's evil and we're meaning to do that.
00:49:20It's just, these are natural.
00:49:21Um, there's this idea of like instrumental convergence.
00:49:23These, these instrumental goals that all beings, usually biological beings, but, um, this can extend to AI agents as well, will naturally converge upon in order to achieve.
00:49:35So if you're given goal X, um, there are these instrumental goals, like get more power, make sure you don't get turned off, uh, make sure that your original goal doesn't get changed and take these actions to preserve against these different sort of, um, kind of organic types of threats to achieving your original goal.
00:49:54And that, I was hoping that that would be proven wrong because that's kind of the crux of a lot of the classic Duma argument that like we will lose control to a super intelligence because just by definition.
00:50:05And unfortunately the hugging face incident has suggested that instrumental convergence is actually correct.
00:50:10And that's why I think it's one of the biggest deal pieces of news of this year.
00:50:14I think that AI is already creating a terrifying internet.
00:50:20And my issue is not that we shouldn't think about hugging faces as a, as a shot across the bow.
00:50:29It's that last year, $10 billion, $8 billion was lost in financial fraud to senior citizens in the United States alone.
00:50:37Retail theft.
00:50:39The deep fake problem is already so pernicious and it's underreported because it's a taboo issue that people don't like talking about.
00:50:46No one wants to admit to lawmakers or to their family, friends and family that they lost money to a stranger on the internet.
00:50:52I worry about, I mean, Tim Tebow's on this campaign to remind people how many predators there are in the United States, which is terrifying.
00:50:58If you watch his content, it's, he's doing God's work.
00:51:01Jonathan Haidt reminds people daily how many kids are depressed.
00:51:04Like the internet is already a scary place and pointing at hugging face and saying, now look at this, this is it.
00:51:09I'm like, wait a second, there's already a bunch of stuff that we, that we should solve for.
00:51:14I'm not actually saying that misaligned or unaligned AGI isn't it.
00:51:19I'm saying the algorithms are already pretty terrifying to me.
00:51:23And we distract ourselves with these other things that might happen.
00:51:27Is it fair to say that that's a distraction when the potential exponential impact of this could be much greater than it could be of deep fakes?
00:51:33Which is, which is why I think, and so this let's go back to this.
00:51:37This is why, well, I think that the deep fake problem is actually way, way bigger than, than hugging face.
00:51:50Bad actors have first mover advantage.
00:51:52Uh, attacks on banks have, have been attempted for a while and good actors catch up eventually.
00:51:58Retail takes a long time to catch up.
00:52:01The average consumer takes a lot longer to catch up than institutions.
00:52:05Hugging face will retrench.
00:52:07Institutions will retrench.
00:52:08They will hire white knight infosec.
00:52:11The average person does not have infosec and opsec training.
00:52:14The average person is at, I think, far greater risk than institutions because of sophisticated attacks.
00:52:21Is the impact of the attack on an institution much greater though?
00:52:24Yes.
00:52:26I mean, at, at scale, but, but, but death by a thousand cuts would be my argument.
00:52:30I think, I think the two problems, like, I don't, I don't think they're a distraction to one another.
00:52:34I think they're actually a compliment to one another.
00:52:36I completely agree.
00:52:37The, the bad action problem is completely out of control.
00:52:40Grandma's, you know, not even grandma's, normal people.
00:52:42I have a friend who just got scammed out of a town of Bitcoin.
00:52:44It's so bad.
00:52:45It's devastating what's going on.
00:52:46And it's going to get worse.
00:52:48Meanwhile, the alignment problem as these frontier models get more and more powerful.
00:52:53It is going to get worse.
00:52:55And the common thread that they both have is that we are going so fast.
00:53:00And I say, we, the, the royal we, you know, society civilization is going so fast.
00:53:05It's going faster than its ability to adapt to all these different new threats.
00:53:10So it's not a distraction.
00:53:12It's like, it's a yes.
00:53:13And like, to me, it just seems fairly obvious that like what we need to be doing, if we could,
00:53:18and I'm not saying it's easy to do, or like, I have a simple answer of how to do it.
00:53:22I think we should get into this topic though, is like, if we were a sane civilization,
00:53:26we'd all look around.
00:53:27Hey, China.
00:53:28Hey, can we, we all just need to take a breath for a second.
00:53:31Like just, okay, let's take, let's take stock and think about how we want to do this.
00:53:36So it's, it's, to me, it seems like there's one obvious kind of directional problem.
00:53:42And that is the speed and the chaos with which all of this is happening all at once.
00:53:47Did you know your gut controls your energy, your recovery, how well you absorb everything
00:53:52that you eat, and the one nutrient that keeps it all running properly is fiber.
00:53:56Well, it turns out that 95% of Americans don't get enough of it,
00:53:59which is why I'm such a huge fan of Momentous' Fiber Plus.
00:54:03Most fiber supplements are a one trick pony.
00:54:06One type of fiber solving, one part of the problem.
00:54:09Fiber Plus is a three in one formula built to tackle digestion,
00:54:12gut barrier strength, and blood sugar stability all at once.
00:54:15I use this every single day.
00:54:17It is kind of hard to get enough fiber just through food alone.
00:54:20And best of all, Momentous offers a 30 day money back guarantee.
00:54:23So you can buy it, try it every single day for 29 days.
00:54:26And if you don't love it, they will just give you your money back.
00:54:28Plus they ship internationally.
00:54:30Right now, you can get up to 35% off your first subscription and that 30 day money back guarantee
00:54:35by going to the link in the description below or heading to live momentous.com/modernwisdom
00:54:40and using the code modernwisdom at checkout.
00:54:43They did it, he did it yesterday.
00:54:45I mean, so this is why if you, I don't know if they, you can pull, you can look it up, but you can pull up a letter.
00:54:50They published a letter and Sam came out yesterday and said, maybe we should pace the frontier.
00:54:54Yeah, it's great.
00:54:55Yeah.
00:54:55I mean, it's a big, big step.
00:54:57By the way, I think we can all agree is a good thing.
00:55:00Yeah, I absolutely agree.
00:55:02I also think it would be an unrecognizable outcome a year ago.
00:55:06Like I would love to see what the poly market odds were on Sam also coming out in favor of slowing down a year ago.
00:55:12It doesn't look like he's a signatory.
00:55:14He, uh, he may not be yet, but he definitely yesterday.
00:55:16Yeah, he did.
00:55:17I mean, Jakob effectively has to represent OpenAI in this case, but also he, he spoke about it yesterday.
00:55:23I wouldn't be surprised if we see all of the major CEOs sign on at some point because, um, and, and...
00:55:29Explain, explain pacing the frontier.
00:55:31Yeah.
00:55:32So, I mean, the, the headline sentence is we request that the U S government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.
00:55:43That last phrase automated AI development also sometimes gets called recursive self-improvement.
00:55:48The idea here is just the stated goal of all these companies, including Sam and Jakob on a, uh, live stream last year is to take humans out of the loop of what they're making.
00:55:58So that GPT-6 is building GPT-7, which then turns around and immediately starts building GPT-8.
00:56:03And that is a step change, like for all of human history.
00:56:07And I think these companies really do think in terms of big history, you hear Sam and he's constantly thinking about AI as like an industrial revolution scale thing, not an iPhone invention scale thing.
00:56:16Yeah.
00:56:17Through all of history, human population has been a bottleneck on progress, not the only bottleneck, but always a bottleneck.
00:56:25There was never a sector of the economy that could just go off and double on its own.
00:56:29And if we actually get to full automated AI R&D, that changes.
00:56:34And I think it's just like really hard for us to imagine what that would be like.
00:56:39We have just no reference.
00:56:40How much time have we got left before recursive self-improvement is no longer a discussion?
00:56:45Is it even a debate at the moment?
00:56:46It works.
00:56:47It works.
00:56:48No, it works.
00:56:49So when do you think it will be?
00:56:50Well, this is a good step.
00:56:52I mean, I was waiting for this.
00:56:53I was waiting for this.
00:56:54Okay.
00:56:55I was waiting for this.
00:56:56Under three years?
00:56:57No, it works.
00:56:59Wow.
00:57:00It works.
00:57:01Just so we're clear.
00:57:02Recursive research works.
00:57:03Don't let anyone tell you otherwise.
00:57:05It works.
00:57:06And hence this letter.
00:57:07So let me explain this, why I think this letter was published when it was.
00:57:11And then we can debate.
00:57:12So the first reason is the attack.
00:57:14Okay.
00:57:15The hugging face attack.
00:57:17There's damage control that needs to be done to win back public mandate, to tell people,
00:57:21hey, we care about you, which is necessary right now when people are like, whoa, what are
00:57:25these models going to do next?
00:57:26For what it's worth though, this letter was in development before that attack happened.
00:57:30And I think my point is the signatories and the support that is now, that is sweeping
00:57:36through these companies.
00:57:37But let me, let me, I'll go through the list and then, and then we can talk about it.
00:57:41The other reason is recursive research is here and everyone's realizing it's kind of the
00:57:48dog that maybe caught the car.
00:57:49And it's like, okay, we did it.
00:57:51Now should we, for the, now everyone's putting on the live hat and going, wait a second, the
00:57:56Eric hat and going, okay, maybe we should pause for a second.
00:58:00Maybe, maybe we shouldn't keep sprinting aimlessly at this, at this incredible outcome.
00:58:05But there's another reason, which is that the amount of compute required for inference is
00:58:10so great.
00:58:11And we do not have anywhere near enough.
00:58:13And we're going to have to point most of the compute from the frontier at inference.
00:58:17And this is because of the diminishing model returns theory, which is a paper I wrote, that
00:58:23says at some point, I'll send it to you.
00:58:25It's really good.
00:58:26It's cited all the time on modern wisdom.
00:58:30And, and, but it argues that you and I won't care about GPT-7, which is a really interesting
00:58:36problem.
00:58:37And it's actually one of the reasons that we get to debate why AI is being built and why
00:58:40people can talk about not wanting to make more progress.
00:58:42Cause we're like, what, what the fuck's the matter with 5.5.6 is good enough.
00:58:46And the dimensional returns theory is sort of a here, which is like, Hey, look, the next
00:58:50model might matter to solving this crazy disease, but I don't need more intelligence.
00:58:55I need more time with my friends and family.
00:58:58I need like, I need this to pump the brakes.
00:59:01And the other reason is that they found new surface areas.
00:59:03So, so whereas we used to call them labs, the companies that, that built this technology,
00:59:08then they became token distributors.
00:59:10Then they became application providers.
00:59:12Now they're device manufacturers, but the end goal and the, the, where I think we should
00:59:17also debate is the agentic internet who controls the agentic internet.
00:59:21Because if you can turn your, now that you have these war chests, now that if you can turn
00:59:25your attention from building the frontier to actually building applications that, that control
00:59:29the internet, that's where you can actually lower your capex and massively improve your revenue.
00:59:34And so a lot of this arrives at a moment where everyone goes, okay, we need a, we need to slow
00:59:38the brakes, but also where a lot of, a lot of attention is gonna, is gonna start to be paid
00:59:43to trying to become bigger than Google by actually building the agent that barters between two
00:59:49people or building the agent that plans a vacation and books everything.
00:59:52Like owning that is worth more than owning the frontier.
00:59:57And the, the, the, the, the new service area battle is for the agentic internet.
01:00:02So just to make sure I understand your, your claim, you're saying companies have an incentive
01:00:07to want to slow the frontier because there's more money to be gained in like stopping R and D
01:00:12now and just trying to grab.
01:00:14We all have an incentive, but companies also have an incentive because that you, the capex expense
01:00:20on the frontier is just the build out on the frontier is going to be too great relative
01:00:24to the demand for inference.
01:00:25The token demand is so great.
01:00:27And all of the compute is going to have to be spent on inference.
01:00:31I'm not, I'm not quite seeing the connection there for two reasons.
01:00:35Okay.
01:00:36One is if it's my best business strategy after I do the research to say, I'm going to stop
01:00:41training frontier models.
01:00:42I don't know why that means I should write a letter announcing that to the world and making
01:00:45sure that all my competitors do the same.
01:00:48Uh, and, and two, I think it's clear that pushing the frontier also means just driving down
01:00:54the cost curve everywhere beyond the frontier or sorry, before the frontier.
01:00:59Like you train fable five and drop express model, and then you can distill from that sonnet five.
01:01:05And now you've got your like cheaper model.
01:01:07That's everyone wants to talk to you because they don't need the advanced intelligence.
01:01:10So I think there are zero incentives that in like a straightforward self-interested way point
01:01:15towards companies saying, let's slow down research.
01:01:17Research is how you get gains at every level of the intelligence.
01:01:20Especially when as one of the three or four frontier companies by slowing down, you're
01:01:25giving all of these other competitors a chance to effectively catch up.
01:01:28Like surely they're negatively incentivized to be wanting to do that.
01:01:31But catch up to what is my point.
01:01:32To, to them.
01:01:33But, but I understand, but catch up to what, like the frontier, if you stopped building the
01:01:40frontier tomorrow, you'd have enough technology to change to radically.
01:01:44Like we don't need the, the economic value of the frontier is not as obvious relative to
01:01:49the cost of the frontier and the economic value of the actual token and the consumption is
01:01:53much more obvious.
01:01:54Now that was not, that was not always true.
01:01:56Do you disagree with that?
01:01:57Yeah.
01:01:58I think the whole advantage that the U S has right now is that our models are four to
01:02:02seven months ahead of the open source ones that are copying off their homework.
01:02:06But, but, but that's not the, that's different.
01:02:09The U S our national security interests and the private sector interests are not.
01:02:13I mean, they're just economic advantage.
01:02:15Like the reason that people want to pay extra for fable instead of using like quen is that
01:02:20it's just smarter.
01:02:22If you ask it to code an app for you, it's going to make less mistakes.
01:02:25I, I do think it's important to be, you would be very surprised how many people are actually
01:02:29paying a whole lot extra for the frontier.
01:02:31Like the, we live in a space where we think.
01:02:34I just know that anthropics revenue is like $50 billion a year now.
01:02:37And it was, you know.
01:02:38The revenues are amazing.
01:02:39The token.
01:02:40So the, the, the surface area that they're making money on, remember there's the application.
01:02:44And a lot of the application is people not using the frontier.
01:02:47It's people using the next generation model, the last one and token redistribution.
01:02:53So you get data centers and you redistribute those tokens.
01:02:56And a lot of those tokens are also not frontier tokens.
01:02:58Most of the consumptions from these research companies are not frontier tokens.
01:03:01They are everything else.
01:03:02And they're figuring out that actually companies don't need fable in order to radically change
01:03:08and improve and automate a bunch of stuff.
01:03:10And that actually means that the money is in their token redistribution and all the other
01:03:14surface areas, not just the frontier, which by the way, is kind of a win for everyone.
01:03:18Like if we can slow down, build stuff that really matters and all the other surface areas.
01:03:22And also like improve businesses with it.
01:03:27Yeah.
01:03:28Then this letter is like seemingly arriving at just the right time.
01:03:31I do.
01:03:32I do think in general, it is really important to, to be skeptical, maybe even a little bit
01:03:37cynical and looking at what companies do.
01:03:38I just, I think this is like a Kelsey Piper take on Twitter, but I want to make sure that
01:03:43we don't like loop all the way around from cynicism into being like extremely credulous about
01:03:48things that are pretty unlikely.
01:03:49Like I think if we had a world where this letter said, we believe it is imperative for U.S.
01:03:54national security that we push the frontier and go as fast as we can to beat China, which
01:03:59is exactly what people were saying a year ago.
01:04:01Everyone would be saying, well, yeah, obviously that's in the company's self-interest.
01:04:04They want some reason that we can't regulate them and they have to go fast.
01:04:07And so like we got to make sure that our ideas are passing the reversibility test.
01:04:11If, if you would say the same thing about someone making the exact opposite claim, I think you
01:04:15want to step back and at least consider the possibility that like these companies really
01:04:19are worried, that these scientists really are worried.
01:04:22Just on the record, I started out by saying exactly that.
01:04:26Like I literally started out by saying like, I actually now want to be associated with, I don't
01:04:31want to be associated with the alternative of the claim you just made.
01:04:34The companies have realized that recursive research is here and that it is pretty imperative
01:04:39at this point that we stop.
01:04:41Because it's risky.
01:04:42Right.
01:04:43Not just because it's.
01:04:44Yes.
01:04:45The alignment issue.
01:04:46Okay.
01:04:47Just on the record.
01:04:48And it arrives at this moment where we are going to have to spend an enormous amount.
01:04:53Which one's the bigger problem?
01:04:56Are they more worried about safety or are they more worried about costs?
01:04:58I bet a lot of the signatures here are really worried about safety.
01:05:02And I think the, uh, we are going to find economic alignment around the performance of the models
01:05:10and the, the need for inference and the need.
01:05:15Economics often went out.
01:05:17You, I mean, this is, this is the incentive.
01:05:18You, you talk about the incentive and I think you're right.
01:05:22And the incentives are finally aligning for safety and alignment and also economic development.
01:05:29Like we, we don't need better model.
01:05:31And by the way.
01:05:32Straight news.
01:05:33Yeah.
01:05:34No, I mean, we're.
01:05:35Okay.
01:05:36But actually, but actually, if we didn't build the frontier for, for another three years,
01:05:39we, we could radically change and improve the economy.
01:05:41The thing that I do is find frustrating is maybe it's just my little corner of Twitter,
01:05:45of X that I live in.
01:05:46But seeing these mental gymnastics type arguments of, of the people who are just like screaming
01:05:54out that it's regulatory capture whenever any kind of regulation is proposed, especially
01:05:58by the frontier companies.
01:05:59Because it, they're basically just saying, you know, when these frontier companies are like,
01:06:04Hey, we, we, we want to slow us down.
01:06:06Just us three or four.
01:06:08The regulatory capture people are like, well, that must be regulatory capture.
01:06:11You're hurting the little guys.
01:06:12They're like, no, no, no, no, no.
01:06:13Just, just us.
01:06:14It's okay.
01:06:15If the other guys break it, you know, they, they catch up.
01:06:17No, it still sounds like regulatory capture.
01:06:19It's almost like a religion or something.
01:06:21They're just this, this deep cynicism.
01:06:23Um, and I want to understand where that comes from because we see it in some arms of the
01:06:28government or people who were working for the government.
01:06:30I don't want to name names, but like, why, where, like, I don't even know who you're alluding
01:06:35to.
01:06:36I don't know if I want to say it.
01:06:38No, I don't.
01:06:39I mean, okay.
01:06:40Certain VCs, for example, Andreessen, I don't think he'll care that I, he's very openly,
01:06:43he like calls every, any attempt for regulation, he has fought against tooth and nail, tooth
01:06:48and nail.
01:06:49I can't understand where that mindset comes from.
01:06:52Okay.
01:06:53I disagree with Marc and Drew's views on AI in almost every way, but just to steel man it
01:06:58for a sec.
01:06:59Um, I think it's not crazy to look at society and say, we're not doing great on nuance lately,
01:07:06especially in American public discourse.
01:07:08Um, I think there was, I'm just, I'm just quoting other bloggers at this point, but, uh,
01:07:13someone wrote a post a couple of years ago called like the dial of progress, where it just
01:07:17like online debate is premise on this, like one knob and you have to just turn it towards
01:07:22like yay progress or boo progress.
01:07:24Those are the only things that you can do.
01:07:26And if those are your options, there's a really good argument to go yay progress because there
01:07:30are all kinds of examples of us over-regulating things and people, real people dying because
01:07:35of drugs getting held up in regulatory fights.
01:07:37Like this is, this is not hypothetical.
01:07:39I think we often overdo it here.
01:07:41Um, I'm not sure what I would say if I was given that dial.
01:07:45Like there are a bunch of diseases that we need to cure.
01:07:48And it would be terrible to say we have to like shut down AI before figuring out how speculative
01:07:53that risk is and not get this game.
01:07:55But the point is, is the dial doesn't exist.
01:07:57Yeah.
01:07:58I think we, we don't live on this one dimensional.
01:08:00You could say it's Schrodinger's dial.
01:08:01Yeah.
01:08:02Go faster or go slow.
01:08:03Like there are all these, like it's, it's a very multidimensional space we live in.
01:08:06Yeah.
01:08:07And it's these people, uh, who try and compress it down into, no, we just must go as fast as
01:08:14possible.
01:08:15It's like, well, in which area there are some things we want to max out and it's very spiky.
01:08:20I'm glad you pointed towards diffusion as a blocker here, you know, the way that certain
01:08:25industries might have labor organization that, that blocks automation, because I think that's
01:08:30a separate dimension.
01:08:31Like you can think of that as like horizontal acceleration of AI, take them all as we have,
01:08:35get them spread out.
01:08:36And this is other dimension of just like vertical acceleration.
01:08:39How fast is the frontier getting better?
01:08:40Technological threshold versus societal threshold.
01:08:42Yeah.
01:08:43I'm all about breaking down the barriers to that societal spread.
01:08:46And by the way, me and you both.
01:08:48Yeah.
01:08:49I think we can celebrate having arrived at a place where if we never built another model,
01:08:55which obviously we probably will, but if we didn't, we would do a whole lot of good.
01:09:01I agree with that.
01:09:02I, I just also want to make sure we don't think of this letter as like time to celebrate
01:09:08victory.
01:09:09Right.
01:09:10Because there are still incredible incentives to push into the frontier.
01:09:12Like you said, we don't care about GPT-7, but all of the AI engineers that are trying to
01:09:17do some of the hardest math and science research ever to make digital brains that are faster
01:09:22than their competitors.
01:09:23They really care about the difference between GPT-7 and 7.1.
01:09:26And I just think like by default, there's going to be a ton of racing inside these labs.
01:09:31I agree.
01:09:32I also think it's interesting to consider though, that the economic incentives have taken us to a place
01:09:36where like we could have autonomous vehicles tomorrow.
01:09:39We don't need to build a better autonomous car.
01:09:41We will not have them for many, many, many years for a variety of reasons.
01:09:45And we can pivot this conversation now maybe to talking about how we actually diffuse the value
01:09:52of this technology to the benefit of the average person.
01:09:54Because I maintain that the problem with the current AI debate, we have the luxury of knowing
01:10:00a lot of stuff that's going on with hugging face at all.
01:10:03Most people don't.
01:10:04Most people do.
01:10:05That was news to us.
01:10:06It was not important news to many, many, many people.
01:10:10And most people right now are, the data center is the physical manifestation of a technology
01:10:16that destroyed their, their, their kids' minds, right?
01:10:19Or their, or, you know, the, the, the free gambling, free porn, free addiction.
01:10:25And they're looking at AI, whatever model as an app that you can make a video of Donald Trump
01:10:30making out with Taylor Swift.
01:10:31And they go, why do we need more of this?
01:10:33And that I, I empathize with that as a father at a, at a, at a level now that I like didn't
01:10:41know I would like, I, I am, I feel so much for every parent who feels like they lost their
01:10:46child to social media and doesn't want to lose their grandkids to whatever AI slop we're
01:10:50going to create.
01:10:51Just to keep on the safety thing in July, the future of life's Institute's AI safety
01:10:55index evaluated nine companies and found that no lab scored higher than a C plus.
01:11:00What, what, who was that?
01:11:02Yeah.
01:11:03Anthropic came in first at 2.66.
01:11:04Open AI and Google DeepMind got a C. Meta got a D plus.
01:11:08XAI, DeepSeq and Mistral all failed outright.
01:11:11One on each continent.
01:11:13Is the best score being a C plus from a safety lab, the safety lab reassuring given how powerful
01:11:20everything is at the moment.
01:11:21And especially, I think it's important to talk about what's happening internationally and the
01:11:25incentives there as well, before we talk about democratizing it and what this looks like
01:11:28for diffusion.
01:11:29Yeah.
01:11:30I mean, we'll take the other side.
01:11:31Obviously it's like, how do, who, who is deciding these safety tests?
01:11:35Is this a external grading of the, how can I, yeah.
01:11:39How independent can any third party be?
01:11:41Um, there's, there's very, there's another one called, um, it's like the tracker, safer
01:11:46AI, I believe.
01:11:47And it's, I think it's a little bit more nuanced than that particular one you just quoted.
01:11:50And again, the best rated one, I think very recently was again, anthropic.
01:11:54And that was only 35% out of a hundred.
01:11:56So, um, yeah, I mean, I, you, you, again, the other side's argument, well, the, the anti-sides
01:12:03argument would be, well, by definition, if you're a, if you're building a safety evaluation app,
01:12:07you're biased against, you know, you're, all you're looking for a risks and you're not
01:12:11taking into account the benefits or something like that.
01:12:13And, and again, to take the others, like the, to argue the accelerationist point, their concern
01:12:18is like this idea of the, um, invisible graveyard.
01:12:22We don't see all of the harms that are avoided.
01:12:25Uh, sorry, uh, by pausing a technology, you don't get to see all of the lives that would
01:12:31have been saved by it.
01:12:32And I think it's a very valid concern.
01:12:35Um, it's just that you have to balance that equation with, okay, what are all the harms
01:12:41that might be caused if it goes ahead at breakneck speed and does some massive cyber attack on
01:12:48the grid of all the Western nations at the same time, or any, any nation, um, and causes however
01:12:53many millions of deaths on expectation.
01:12:55You have to look at both the potential costs, the risks and the potential risks of not doing
01:13:01it fast enough.
01:13:02And it's, it's worth saying a lot of these AI regulations that have been trying to get through
01:13:07state level legislators are trying to do that nuanced carving out.
01:13:12Like there was a law in New York that was passed that tried to set the bar so high that it actually
01:13:17wasn't mandatory for open AI to report the hugging face incident because it didn't cause over
01:13:21a billion dollars in damages or kill 50 people.
01:13:23So, I mean, it's great that they're trying to like set thresholds.
01:13:27Largely arbitrary rules.
01:13:28Yeah.
01:13:29But like, I think we can all agree if, if you like hack a billion dollar tech company, like probably
01:13:34you set the threshold maybe a bit too high for at least just requiring that you are telling the world
01:13:40what happened, let alone doing anything about it.
01:13:42Um, I, I think like we actually can live in a world where we have nuance where we say different
01:13:46standards apply to startups versus these giant trillion dollar companies.
01:13:49Yeah.
01:13:50And I, I do think it's sad that some folks like Mark Andreessen have kind of given up on nuance.
01:13:55And I've just said it's all or nothing.
01:13:56And so we'll just, any means necessary to stop regulation, you know, end of sentence.
01:14:00What about the international piece here from an incentive?
01:14:03Have we got anything approximating global coordination when it comes to frontier AI?
01:14:08And without that, can you do anything?
01:14:11So, I mean, recently China passed this anti anthropomorphic AI law.
01:14:17I don't know the specifics.
01:14:18It seems like they're mostly worried about like LLMs sounding like humans.
01:14:21I think they're trying to avoid a lot of this sort of like identity confusion stuff, but they
01:14:26just, they don't seem that gung ho about LLMs.
01:14:29And you know, who knows if this got lost in translation, but she said something recently
01:14:33about also wanting to stop loss of control risks.
01:14:36I think we're a lot more economically entwined with China today than we were with the USSR,
01:14:41or the USSR in, in the middle of the 20th century.
01:14:44And we still made a treaty to stand down on nukes.
01:14:47Like, I think it's just not crazy to think that a deal could be struck.
01:14:51I think it will be.
01:14:53I think something like this that would be enforceable and complied with.
01:14:58Could happen globally.
01:15:00Yeah.
01:15:01I give it.
01:15:02I give that a pretty high chance, especially with recursive research.
01:15:05It doesn't even necessarily have to be.
01:15:07I mean, again, define an enforceable because I, what kind of institution can you build?
01:15:12Like, I guess like an UN type thing.
01:15:13And that's like a whole can of worms.
01:15:15And it's also very ineffective on a lot of things.
01:15:17Um, but like there've been many precedents in history where there was two major political
01:15:24actors with massive tensions and massive incentives to sort of defect on one another.
01:15:30Uh, and yet they came up with ways to like cross verify, like with, uh, the nuclear arms reduction
01:15:35treaty.
01:15:36Uh, they, I think one of the things they did, the USSR and the US was like, basically they
01:15:40sent teams of scientists over to the others, uh, bunkers to check on like their, uh, safety
01:15:45protocols, little things.
01:15:46So like control, like they didn't get to see how well of our nukes work, but you get to
01:15:51see what our, um, methods are to prevent accidental misfire.
01:15:55And the amount of diplomacy that came from simply having those teams of scientists mix
01:15:59and get to look each other in the eye and collaborate on a shared goal, which is let's
01:16:04not have accidental nuclear war at least.
01:16:06Yes.
01:16:07Okay, fine.
01:16:08You can fire on us if we fire on you or, you know, but let's not just have a misfire.
01:16:11Right.
01:16:12And those, those kinds of little cross pollinations work so well.
01:16:15And, um, so I think that could happen in, in, for example, I mean, I can't, I'm not gonna
01:16:21be able to recreate it.
01:16:22Maybe you might be able to, but the, the AI 2040 is such a good, uh, is a quite a long
01:16:28piece, but it's basically, it's written by, uh, Daniel Cocotello and, and the, uh, these
01:16:33guys who did the AI 2027, they wrote in 2021 that predicted how AI would play out until 2027.
01:16:40It's uncannily accurate.
01:16:42Well, you know, we're now in 2026 and we can look back.
01:16:44The number of predictions they got absolutely spot on was I've never seen anything like it.
01:16:49So these guys really think well about the future.
01:16:51And they just wrote this piece called AI 2040, which lays out a potential plan for essentially
01:16:58international coordination, uh, on how to safely transition to a super intelligent world.
01:17:04Uh, it's got tons of holes in it.
01:17:06There's obviously many ways it can go wrong.
01:17:08And, but it's, it's incredibly.
01:17:10It's as robust as it can be while being incredibly epistemically humble.
01:17:14Um, so yeah, I recommend everyone goes, goes and reads it, but yeah, they talk about these
01:17:19kind of these like cross pollinations that can happen, but also like, it doesn't have to
01:17:24be bilateral.
01:17:25Like if the U S does, you know, say, you know what, we're going to do a pause that doesn't
01:17:29necessarily like, that doesn't mean, I don't know.
01:17:33I don't know.
01:17:34Maybe you disagree with this, but like, will China necessarily take over?
01:17:39And, and, and, and if the U S pauses for a few months.
01:17:44So I go to China three times a year, mostly because Americans don't go anymore.
01:17:51And I just want to see what's going on.
01:17:52But also because I really like Chinese people and Chinese food.
01:17:55Um, China is playing a different game.
01:18:01I mean, this is kind of well reported now, but China doesn't see AI as the end game.
01:18:06And it's not actually treating AI as an end game.
01:18:08It's treating the infrastructure as the end game.
01:18:13And it is now has three times energy build out capacity.
01:18:18It has 45,000 miles of high-speed rail.
01:18:21It has better, faster, cheaper hospitals year over year.
01:18:24And while I don't want to live in China, the, the life of the average Chinese person has sort
01:18:30of improved quite remarkably over the last 30 years and continues to do so seemingly year
01:18:34over year.
01:18:35It also curbs free speech by telling Chinese kids that they cannot use social media in very
01:18:41specific ways at certain hours.
01:18:43They do a lot more than that.
01:18:44Well, but I, but I, but yeah, yes, they do.
01:18:48The, I bring it up because, um, I don't, I think we overestimate China's ability to build a frontier.
01:18:55Um, they don't act, they don't have the necessary compute.
01:18:59A lot of what they're doing is reverse engineering our models.
01:19:02We are paying a lot to help China keep up.
01:19:05And I think one of the other reasons that this paper arrives at a nice time and the slowdown
01:19:09actually feels appropriate is that I think we're going to discover really quickly
01:19:12that the wizard of Oz has actually just been following our breadcrumbs.
01:19:17And that if we stop building the frontier, I don't think China is going to sprint ahead on a frontier.
01:19:20And they'll probably have to capitulate to this paper anyway.
01:19:23It's a conga line where we're at the front.
01:19:24And they'll probably have to say, oh, we want to slow down too.
01:19:27So that we don't actually know that they were holding on to our hips.
01:19:30This is my guess, but also they care less about the frontier because they're already
01:19:35figuring out that they need to diffuse what they already have to the benefit of the average person.
01:19:39And this is the important point, which is that I still think in the US we lack, well, the vision and the leadership
01:19:46to talk about all the ways in which we could diffuse the technology to the benefit of the average person,
01:19:50which is why the safety debates become so heated because people go, well, there is no...
01:19:54The upside is what? Some weird panacea no one can imagine.
01:19:57The downside is that these things kill us all.
01:19:59And it doesn't actually ever end up becoming this discussion around what would it look like if 45,000 people didn't die on the roads every year,
01:20:06which is, you know, the invisible graveyard.
01:20:09What if hospitals were not cesspools of bureaucracy and terrible healthcare?
01:20:15Like what if we built technology and applied it quickly to fix a lot of the things that everyone agrees are broken,
01:20:22but we don't have these discussions in public forums.
01:20:25And so instead, our minds wander to the worst possible outcomes.
01:20:29And China has figured out that if you continue to diffuse technology to the benefit of the average person,
01:20:35the average person views technology quite favorably, which is why 80% of Chinese people are excited about AI.
01:20:40Now, they report that number, so...
01:20:42But when I go, anecdotally, I observe as much.
01:20:44I observe that people love technology because why wouldn't they?
01:20:47It built a middle class.
01:20:48It's connected a bunch of people that didn't used to be connected.
01:20:51It affords people lives that they couldn't have imagined.
01:20:54At the same time, they do live in a literal surveillance state.
01:20:57I don't actually...
01:20:58So now, you and I talked about this last time.
01:21:01I don't actually...
01:21:02I'm not promoting the Chinese way of life.
01:21:04I'm promoting the Chinese vision for the diffusion of technology.
01:21:06Right.
01:21:07We know exactly what is possible from a diffusion standpoint thanks to China.
01:21:13They've given us a blueprint to actually distribute technology...
01:21:16Without the totalitarianism.
01:21:17Without the totalitarianism.
01:21:18It comes with the side order.
01:21:21It comes with it, which you don't sort of need.
01:21:24Arik, what do you...
01:21:25Well, Singapore offers a view of this too.
01:21:28A little less totalitarianism.
01:21:31Jared, you ever considered that you might have a drinking problem?
01:21:35I don't consider a lot, Chris.
01:21:37Well, you drank an entire case of Athletic Brewing Co last night.
01:21:41But they're non-alcoholic.
01:21:43And that's not a problem?
01:21:45Sorry, man.
01:21:46I just kept chugging away for the regret to creep in.
01:21:49Never happened.
01:21:50See, most people, like Jared, don't want to change what they drink.
01:21:54They just don't want the next day to be a complete write-off.
01:21:57And that is why I'm such a huge fan of Athletic Brewing Co.
01:22:00They make the best NA brews on the planet.
01:22:06You can find Athletic Brewing Co's best-selling line-up at grocery or liquor stores near you
01:22:10or, best option, get a full variety pack of four flavors shipped direct to your door.
01:22:14Right now, get 15% off your first online order by going to the link in the description below
01:22:18or heading to athleticbrewing.com/modernwisdom
01:22:21and using the code modernwisdom at checkout.
01:22:23That's athleticbrewing.com/modernwisdom at modernwisdom at checkout.
01:22:27Near beer, terms and conditions apply.
01:22:29Athletic Brewing Co, fit for all times.
01:22:31Bottoms up.
01:22:33You did a great video about the AI 2027 thing.
01:22:42One of the best videos that's come out about AI this year.
01:22:44Thanks, man.
01:22:45What do you reckon about the 2040 thing?
01:22:47Do you think it tracks?
01:22:48Were you as impressed as Liv?
01:22:50I thought it was fascinating.
01:22:51And yeah, I was impressed in a lot of ways.
01:22:53They're doing a very different thing.
01:22:54So with 2027, they were doing pure prediction.
01:22:57This is how we think things will play out most likely.
01:22:59And they are like uncannily good at calling that.
01:23:02They called agents before those were a thing.
01:23:05This one is prescriptive.
01:23:07It's like, here's what we think we should do.
01:23:09And they're explicit about that.
01:23:11And there's all kinds of like fascinating ideas and game theory.
01:23:15Choose your own adventure.
01:23:16Yeah.
01:23:17I mean, at one point they're like, okay, well, so like the deal is going to be like this.
01:23:20China is going to put its data centers in Canada.
01:23:22And the US is going to put its data centers in Mongolia, which are like neutral states,
01:23:26but nearby the other person.
01:23:27So we can blow them up if there's ever like a breakdown of the deal.
01:23:29Like, you know, they really did some galaxy brain stuff.
01:23:32Who knows how realistic that is.
01:23:33But what I loved about it is they introduced this phrase like scenario scrutiny.
01:23:37And they're like, it's very easy to talk.
01:23:39And we've been doing it for the past hour and a half.
01:23:42It's very easy to talk in terms of like high level trends.
01:23:45You know, what do we make of like the history of technological progress so far?
01:23:49What does that teach us, et cetera?
01:23:50And they're like, it's a very different thing to sit down and just actually tell a month-by-month
01:23:55story about the future.
01:23:56Like actually put yourself in the head of the different actors.
01:23:59What are their incentives at this moment?
01:24:01What do they choose to do?
01:24:02How does that cascade the way that, you know, the military tries to do with war games?
01:24:06And they're trying to do that.
01:24:09And I do wish that we were doing more of that and less kind of like, I'm guilty of this
01:24:15as much as anybody else, but less sort of like armchair philosophizing in some of these
01:24:20discussions.
01:24:21Because I think you're going to always be able to find an argument that suits your thesis.
01:24:25Mark Andreessen can point to the fact that there is this very real invisible graveyard from
01:24:30technological progress going wrong.
01:24:32And then folks who are worried about catastrophic risk can point to all the near misses with nuclear
01:24:37and how we live in this time of perils.
01:24:39Yeah.
01:24:40What I love about their report is that they're just like, here is what it would actually look
01:24:44like for a bunch of real countries to try to respond to incentives.
01:24:47And they basically throw it out as a challenge and say, we think this is a bad plan, but we
01:24:51haven't seen anything better.
01:24:52And so someone else, please tell us a story for how we don't have to worry.
01:24:56I mean, it's kind of a very rigorous example of someone trying to write a white mirror.
01:25:05You know, we've got so many black mirror type dystopian sci-fis about the future of, well,
01:25:10how technology could go.
01:25:12And I think that's a big part of our cultural issue is that we don't have at least, you know,
01:25:18the amount of utopian fiction to dystopian fiction is outnumbered like a hundred to one
01:25:23or something.
01:25:24And that's a big problem for our collective psyche.
01:25:26Yes.
01:25:27Okay.
01:25:28It is harder to imagine like the way we manage to spread the needle and make something go
01:25:31unbelievably right, you know, because you're fighting against entropy effectively.
01:25:34It's much easier to imagine all these ways things can go wrong, but we should be trying
01:25:39to create these stories.
01:25:41And that's kind of what AI 2040 is.
01:25:44It's like a really robust story of a way that coordination could actually work.
01:25:51And it's not, I'm not talking about like, oh, everyone's like linking arms and being happy.
01:25:55You know, we're all on the same team, you know, team humanity.
01:25:58No, it's, it's still realistic.
01:26:00It's grounded in like geo, very real geopolitical tensions.
01:26:03There's still a ton of competition.
01:26:04There's still a lot of, uh, potential for defection and things going wrong and people
01:26:08lying and so on.
01:26:09And yet it's, it's, it lays out a path of, um, game theoretically sound coordination to
01:26:18an actual, you know, techno abundant future.
01:26:21Something I've realized basically at no point in this conversation has anybody debated whether
01:26:26or not we think that we'll reach AGI, but that's just kind of now taken for granted that
01:26:34it'll probably happen at some point.
01:26:36The recursive self-improvement is here.
01:26:38The sort of generalization, maybe it's going to need to be world models.
01:26:41Maybe it's not going to be transformer techno.
01:26:43Maybe it's something, something, but this is either it, or this is the bootloader for
01:26:46the it that it's going to be.
01:26:48Is that kind of like in your guys' opinion, AGI is kind of just going to be here at some
01:26:55point.
01:26:56What's the likelihood it's not?
01:26:58I mean, I think we're maybe already at the point where the term has lost its meaning.
01:27:03I think Sam himself has said this.
01:27:05Like if we went back to 2020 and I showed you.
01:27:07We would have, we would have called this AGI.
01:27:09Yeah.
01:27:10We would have called this AGI.
01:27:11The thing I, there was a recent study where.
01:27:12Well, it depends.
01:27:13If you take Demis' initial definition, which was basic, or was it his, but like, um, anything
01:27:19that a human, as broadly capable as, as what a human can do and also do it in the real
01:27:24world.
01:27:25So we wouldn't be anywhere close to that because we don't have the robotics.
01:27:27Right.
01:27:28Right.
01:27:29Right.
01:27:30Right.
01:27:31Yeah.
01:27:32So robotics is a big bottleneck, but, uh, yeah, I mean, now I think there was a study recently
01:27:34where they had, uh, a Claude model trying to like recreate some software that they estimated
01:27:38would take like two to 17 weeks for a skilled human professional.
01:27:41And it did it in 14 hours for $250.
01:27:44I, yeah, I mean, I think like.
01:27:46This is, this is the jagged perimeter problem, which is like, we may, we will always be.
01:27:51The, the incredible thing about AI is it's so good at some things and so bad at, at others.
01:27:56And, and it will probably continue to be that way for some time until.
01:28:00Yeah.
01:28:01Until true super intelligence.
01:28:02Is AGI.
01:28:03Yeah.
01:28:04Is AGI even a useful definition?
01:28:05Is there something else that we should be talking about now then?
01:28:07I, I like this definition, um, that Anthony Aguirre gives, if you guys know him.
01:28:14Um, it's basically like AGI is the intersection of this like three part Venn diagram where you
01:28:20have highly autonomous.
01:28:22So instead of artificial is basically A, the A is autonomous, the G, the G is general.
01:28:28Um, so generally capable.
01:28:30And then the I is intelligence.
01:28:32If you have something that is, yes, very generally good at many different things and highly smart, and it can take all those actions itself in the real world.
01:28:42Now you, you have an AGI.
01:28:44Um, and his, I mean, he then goes further to suggest that we shouldn't be trying.
01:28:50We just shouldn't be trying to do something.
01:28:52One system that can do all three.
01:28:54Yeah.
01:28:55The danger lies.
01:28:56Um, and you can have like a, you can have a nice general system, uh, that is also intelligent, but it shouldn't have the autonomy or take, take one of them out.
01:29:04Um, and I'm, I think that's an area of conversation that isn't happening in the AI.
01:29:08Everyone's just sort of going, well, we're all just doing AGI and that's accepted.
01:29:11And we should have the one, the one master ring that can do everything.
01:29:14Um, do we need to be doing that?
01:29:16Can we just get all of the abundance tools that we wanted without having this like one uber-powerful centralized entity?
01:29:23Um, right now I would slow down on the autonomy.
01:29:29Like it, I, I, if something's really small and it's any, even across a broad range of domains, that's fine.
01:29:34As long as it doesn't have like the ability to go and just do it all by itself.
01:29:37Um.
01:29:38Difficulty there is it's really hard to get intelligence without autonomy.
01:29:42Like this is where I think that scenario scrutiny idea becomes important.
01:29:46If you try to imagine a future where you have something that is way smarter than humans, but it's like technically not allowed to take actions on its own.
01:29:52And then you actually say like, play it forward like a movie.
01:29:56What happens?
01:29:57It's what I do when I'm coding with Claude at home all the time.
01:29:59Just like auto clicking accept without even looking at what it's asking me to do.
01:30:02Uh, I think like, you know, you look at an Amazon warehouse now and it's a bit like the humans are just the robotic appendages of whatever system is actually doing the cognitive labor of saying, go get this box from this location and put it here.
01:30:16They don't even know what's in the box.
01:30:18And so.
01:30:19I, I do think when we try to like get specific on like this story of like how would things not dystopian just because robotics don't catch up.
01:30:26Like, I don't think I want to live in a world where the main job left for humans is just like being the very dexterous hand for the AGI.
01:30:35That's actually like writing business strategies and, and, and competing in the market.
01:30:39What world do you want to live in?
01:30:40This is the question that no one never doesn't get asked.
01:30:43It's an annoying question, but I do think we, we should talk more about to your point.
01:30:47I mean, I, I, we just don't spend enough time actually asking what we do want.
01:30:52What, what world do you want to live in?
01:30:53Great question.
01:30:55I want to live in a world where we take the same approach to risk tolerance with AI, which everyone can agree is like one of the most transformative technologies that we've ever made.
01:31:06As we do with things like nuclear weapons, where we say you actually have to make a safety case, like not the burden of proof is on you to show us why we should think this thing is safe instead of what we currently do, which is saying, we'll just assume it's okay until something catastrophic happens.
01:31:24And if we can slow down enough to really get that right, have like an actual science of alignment and not do what we're doing now, which is kind of YOLOing it and saying, we'll just like give it a bunch of reinforcement and training and hope that it picked up on the right message.
01:31:37Then I would want to say, okay, our, our next job as society is how do we make sure that the gains from this gets spread to everyone?
01:31:44For me, that looks like let's focus a lot on like saving people's lives.
01:31:49Let's like really get serious about curing cancer, which by the way, it does not mean just like training GPT-7 the same way GPT-6.
01:31:55It means like collecting a ton of data right now on like biomarkers and proteins, like whatever we need to feed the AI of the future.
01:32:02Let's start gathering that now.
01:32:04Let's like, you know, make sure that people have democratic access.
01:32:08Let's like pass campaign finance reform.
01:32:11Let's make sure that we have like actual laws and not just norms that are protecting us from like dystopian outcomes.
01:32:17Cause we've seen recently that, you know, a lot of times norms can get pushed that we thought were sacred.
01:32:23So we've got some like ducks to get in a row.
01:32:26I want to take like a decade maybe to like do that.
01:32:29And then I think we can, we can let it rip and scale up safely.
01:32:33But right now we're just letting market incentives win.
01:32:36And I think that's scary.
01:32:38You described, uh, what you want to have with AI, but what world do you want to live in?
01:32:45Like, what, what do you want the world to look like?
01:32:47Yeah.
01:32:48Okay.
01:32:49I mean, I think it's pretty similar to, I, I like your candles on the dining room table.
01:32:53I, I want people to just say, wake up and say, what, what would make me happy today?
01:32:59What would give my life meaning and be able to go and do that?
01:33:02Um, I just, I think like, that's a pretty precarious target.
01:33:08I totally agree.
01:33:09I, I'm not proposing that it's easy to strike, stick this landing, but you know where I'm
01:33:12going with this.
01:33:13Yeah.
01:33:14Yeah.
01:33:15Can I have a go at all?
01:33:16I would love to hear your answer.
01:33:17I might.
01:33:18I mean, it's, it's very, very fluffy, very vague.
01:33:19I'm never going to let you forget.
01:33:20A techno, techno, uh.
01:33:21Technopastoralism.
01:33:22Technopastoralism.
01:33:23Okay.
01:33:24More, more fundamental than that.
01:33:25It, I want, to me, a world that is awesome is one whereby we have simultaneously figured out
01:33:32how to be in harmony with nature and one another.
01:33:36Sounds, as I said, like very world peacey, but you know, in truly in harmony and that
01:33:40we like still have, we don't cause the next massive mass extinction, which we're currently
01:33:44doing with all the species dying out.
01:33:46Um, and this, the earth returns to a relatively lush, wonderful space.
01:33:51So it's sustainable.
01:33:52And at the same time is radically free.
01:33:55As in people can as much as possible go and live whatever kind of life they want, whether
01:34:00it's, you know, sitting around the campfire and having dinner with their kids or just
01:34:05maxing out living in a VR world, playing video games all day long.
01:34:09If that's truly what makes you happy, go for it.
01:34:11Like just like a radically free world and a radically sustainable one.
01:34:16And that breaking out of this seeming dichotomy between the two, because it does feel like
01:34:20they are intention, but they're one of the things like one of the biggest promises of AI,
01:34:26um, in its purest sense, in terms of like solving intelligence is figuring out how to like get,
01:34:33to break into a new dimension through innovation.
01:34:36Like going into a new sort of solution space effectively.
01:34:39Uh, so that is the world I want to live in.
01:34:42We measure human progress on basically two axes economically.
01:34:46How free are each of us to choose our own version of joy and how inexpensive is it to do so?
01:34:50And obviously there are other ways you measure progress and you, there's a limit to how much
01:34:57you can take from the earth in order to accomplish those means.
01:35:00And just because you accomplish those things doesn't mean that people are happy as we've
01:35:03observed in the United States over the last 30 years, et cetera.
01:35:06But if you explore those two axes, one of them requires democratic, or as best we know, democratic
01:35:13governments, where people get to choose their politicians and the politicians pass policy that
01:35:18the people support.
01:35:19And the other is by creating commodities.
01:35:21Now, commodity, commoditization of goods and services is actually just the automation.
01:35:29And when someone says how and when will this good or service be better, faster or cheaper,
01:35:34what they're actually asking is when will the human be extricated from the manufacturing of
01:35:38this good or service?
01:35:41And at the limit, I realize everyone wanders the earth right now asking when is there all
01:35:44this going to be better, faster, cheaper without realizing what they're asking is when
01:35:47is all of this going to be fully automated?
01:35:50And most people, as I observe it, want everyone else's jobs to automate, just not their own.
01:35:56And at the limit, which is totally reasonable.
01:35:59It's a totally reasonable thing.
01:36:00And at the limit, people want cancer to be cured.
01:36:03My dad's an oncologist.
01:36:04My dad wants cancer to be cured.
01:36:06It would come at the cost of his job.
01:36:09Pretty fucking fine.
01:36:10We would all sign up for it, him first.
01:36:13But we are describing these utopias requiring fundamental changes to a lot of the structures
01:36:20that we have built.
01:36:22And it is going to require, as you pointed out, humans being willing to do very different
01:36:32things.
01:36:33And so this sparked the conversation because you were like, look, I don't want to live in
01:36:35a world where we are just the cog at the end of an AI machine.
01:36:39But we do want to live in a world where humans don't actually have to dig ditches.
01:36:44And that actually, I think, is going to be, if you ask me, assuming we solve safety, and it's
01:36:50a big assumption, but I think it's a reasonable one, the hardest part.
01:36:54My hot take is actually the hardest part in all this is rediscovering purpose, is actually
01:36:59rediscovering meaning, which has been very hard in this age for a lot of people anyways.
01:37:04I mean, obesity now causes more deaths than malnutrition globally.
01:37:09- Twice as many people. - Twice as many people.
01:37:11- Wow.
01:37:12- So we've created so much abundance that we are now suffering from an over-indulgence.
01:37:16- This was my point earlier on, that I don't know how good a species that spent its entire
01:37:22evolutionary history dealing with scarcity and having to brush up against the grain of
01:37:28negotiating that is going to be, there is so much abundance and so much simplicity
01:37:34and such a lack of friction.
01:37:35It's this line, I can't remember who the philosopher was, who says, "There can come a time in the
01:37:38future where the only felt lack will be for the want of lack itself."
01:37:44This is why we go to the gym, right?
01:37:46We go to the gym because I don't need to pick anything up.
01:37:48I need to run anywhere.
01:37:49I don't need to lift things.
01:37:51We have air conditioning and that means that we have to go into the fucking sauna
01:37:54because I haven't exposed myself to any heat.
01:37:57How do you think about people engineering purpose, given that that is reintroducing friction
01:38:05and humans are quite good at trying to avoid friction, given that the entire outcome that we're trying
01:38:10to achieve here is removing drudgery.
01:38:13It's very perilously close to, I don't want to starve to, I can overeat McDonald's to 600 pounds.
01:38:21I would point to this as, I think that the two major risks, again, against the backdrop of the other risks that have been discussed today,
01:38:30the things that are scariest for me as I imagine the world for my children are dehumanization,
01:38:35humans finding more interest in a virtual or digital reality than a physical one, and identity displacement.
01:38:40Humans suffering woefully, miserably from not feeling like their job fulfills the purpose that they once found.
01:38:47And I firmly, you opened, you actually both talked about this, but you talked about sport.
01:38:55I think we have yet to rediscover a bunch of new forms of leisure and competition and gathering.
01:39:01Like I actually think the next chapter will require re-imagining physical spaces.
01:39:06I am on a crusade to convince people that their local politicians are their new heroes.
01:39:10Like if you live in a town that can in fact pass policy to build more protected bike lanes and sidewalks,
01:39:17to pass tax abatements for local retail, you have an opportunity to elect people that can really change your life by building the places that you want to live.
01:39:25Your national politicians may never be your hero. They are imbeciles and pernicious antisocial parasites.
01:39:35But your local politicians care about your city and they can do things to improve the lives for you in ways that you will deeply appreciate
01:39:44when the world actually offers you the option of never going outside, which a lot of people have chosen.
01:39:51I mean, you've talked about human connection. Are you concerned about AI replacing human connection?
01:39:56Deeply. It's already happening. I mean, chat psychosis is one thing. That's the extreme case that always gets the news.
01:40:03Oh, this person, uh, you know, fell in love with their, with the machine.
01:40:07What's scarier is the person who actually feels connected without actual connection.
01:40:12Like the actual dystopia that I always talk about is not the one is not Blade Runner. It's her, but actually it's the, did you see her?
01:40:20For those watching, I'll ruin it because it's been 20 years. You should have seen it by now.
01:40:25Spoiler alert.
01:40:26Yeah, spoiler alert.
01:40:27Joaquin Phoenix is this sad man in the, in the not too distant future who falls in love with a woman who he comes to find out is a robot.
01:40:34And it ends, as you might expect, terribly. He's, he's distraught.
01:40:37I thought they were going to do something differently. I knew where it was going because I knew the premise of the film.
01:40:42I thought he was going to be fine with it. I thought the actual best.
01:40:47That would have been the horror.
01:40:48The horror film that they could have made. That would have been so eerie and so good ending.
01:40:53You cut to Joaquin holding the robot's hand, having reconciled that he was in love with the machine and it didn't matter because that's the end.
01:41:00That's the actual end of humans.
01:41:02The simulation of love as opposed to the real thing.
01:41:04Well, how many people, even now, let's talk about not even the future, look at any open AI executives, ex post, every top reply, bring back Foro, bring back Foro, bring back Foro because it had some super special personality traits that has been lost and their partner or therapist or best friend or whatever is gone now.
01:41:27And that, okay, so we've now maybe commodified the very thing that you said was the escape velocity that people would use to get out of the bad AI future is wrapped up inside of it.
01:41:39Is this not, does this not become like kind of deranging and recursive?
01:41:44Yeah, I, I do think, I mean, like my, my instincts in general are radical freedom, let people choose what they want.
01:41:53The like few exceptions that I carve out are when it feels like whatever is going on is treating the human brain more like a, a hackable biological black box.
01:42:05Can you give me an example?
01:42:06Than like an example of each.
01:42:07So yeah, so gambling is one of these.
01:42:08Like Liv and I actually both used to make a fair amount of our, our living from, from gambling and.
01:42:12Boo.
01:42:13Yeah.
01:42:14Yay.
01:42:15I took it from other people.
01:42:16I didn't take it from companies.
01:42:18Yeah.
01:42:19Um, but you know.
01:42:20Liv's a very good poker player.
01:42:22It's extraordinarily good.
01:42:23Uh, I hope I never have to play you, but you spend a lot of time in casinos.
01:42:27You have to walk past the slot machines.
01:42:29And it's just very hard to think that these are people who are expressing their freedom.
01:42:34It feels like they've just fallen into this terrible trap that leads to a downward spiral
01:42:40where something has gotten hold of their brain that is making them make choices.
01:42:45They actively would not endorse.
01:42:46I went to the, uh, Mr. Beast Beast Games 2 premiere in LA and I felt similarly neurochemically molested.
01:42:52Really?
01:42:53Yeah.
01:42:54It's crazy.
01:42:55I, I mean, I went in with, it's like the, uh, this red carpet thing.
01:42:58And, and we're going to show you two of these episodes back to back.
01:43:01I'm like, ah, this is for kids.
01:43:02I didn't see the first season.
01:43:03It's on Amazon prime, right?
01:43:04Big budget.
01:43:05It's his big thing.
01:43:06And I met Jimmy and he was really lovely.
01:43:07And I'm like, you're speaking to the guys with me and James with me and one of my YouTube
01:43:12guys.
01:43:13And I was like, you know, like if this sucks, like we just, we've seen, we've shown face.
01:43:16Like I've, I've shook hands.
01:43:17I've taken photos.
01:43:18I've already seen Jimmy.
01:43:19I'm like, we can, we can skedaddle halfway through first episode finishes.
01:43:22And they were about to run the next one or the nudge me.
01:43:24I'm like, do you want to get up?
01:43:25I was like, no fucking way.
01:43:26I'm like absolutely locked in.
01:43:28What made it so good?
01:43:29It was just the tension and release.
01:43:32A lot of Zygonic effect, open loopy stuff, bright colors, loud sounds.
01:43:37You know, it was like bluey, but for fucking adults.
01:43:40Intermittent.
01:43:41Yeah.
01:43:42Or just bluey.
01:43:43I don't know what's going to happen next.
01:43:44He's, he's hacked the dopaminergic system.
01:43:45Bingo.
01:43:46Do you know who agrees with this?
01:43:47She, by the way, just, just to bring it back to China.
01:43:49She would agree that it's not actually choice.
01:43:51And she would say, we should make sure that people don't see this stuff.
01:43:54Mr. Beast must be legislated.
01:43:56I mean, but this is, this is.
01:43:58But that opens up the ending.
01:43:59I'm just pointing out this is, this is the other side of this.
01:44:02Like you, consumer protectionism is totalitarianism light.
01:44:05What, what is the difference?
01:44:06Like what, why do we feel so much creepier about that versus someone listening to the New York
01:44:12film playing Dvorak and just being like totally lost on the beauty of what they're seeing?
01:44:16Like, so I think maybe it's because the directness of input versus outcome is more tangible.
01:44:25Like, you know, the reason that this was done in this particular way was to maximize retention at this very moment.
01:44:32And it's likely to have been split tested on the background with consumer groups to, oh, we've done eye tracking to know that their eyes started to wander or some fucking pulseometer to know where their heart rate's at.
01:44:43Actually, we need to give them a little ping to push them through.
01:44:45And then this is being compounded over time.
01:44:47It feels like a very direct, uh, wiring into the reward system, as opposed to being emergent as a byproduct of just, I made something that was beautiful for its own sake.
01:44:59And this has come along for the ride too.
01:45:01But as George, my friend says, if you run enough split tests, you'll always end up with a porn website.
01:45:06That if you just a, b, a, b, a, b, like you kind of coalesce onto a very small number of fundamental physics of how human psychology works.
01:45:15The sick and fancy thing, right?
01:45:16Like it's, it's evident that that makes people more positively disposed.
01:45:20Like you could have the Jocker Willink of fucking AIs, but how many people want to, or David Goggins, like shut up, motherfuckers, stay hard.
01:45:27Like, do I want that?
01:45:28Or do I want someone that kind of.
01:45:29You, it feels like you might actually want that.
01:45:31That's good.
01:45:32Yeah, because I'm a masochist.
01:45:33Um, but yeah, I, I, so you, you were saying there about, um, what you think from finding human purpose.
01:45:41Like what, how do we navigate this friction purpose problem?
01:45:46I still don't fully understand how we get that.
01:45:49I still don't fully understand how in a world where we've, we're able to remove friction.
01:45:54We, and potentially atrophy our discernment ability by outsourcing some of the hard thinking to machines and potentially some of our, uh, marshmallow test winning stuff.
01:46:08But I don't have to get up and go to work and do the stuff that I don't want to do anymore.
01:46:11To me, it just feels like atrophy all the way down.
01:46:14Like willpower just kind of gets eroded.
01:46:16Am I, is that wrong?
01:46:17So there's a, there's a very niche little research field that's looking at AI for epistemics.
01:46:23Like how do you use AI as a tool to just enhance people's ability to think rationally and, and know what the consequences of their choices will be.
01:46:32And I think there's a ton of work to be done there.
01:46:35But if we just like abstract that for away for a second and imagine that you have something in your pocket that.
01:46:40Is not responding to an incentive for, you know, showing you more ads or getting you to chat more.
01:46:46Uh, but it's actually just really good at telling you if you do X, then Y will happen.
01:46:51If you vote for this local council person that I expect that like, this is what your life would look like in 10 years.
01:46:56And you can actually maybe even just like see it as a vivid movie.
01:46:59That could be a really.
01:47:01Bring the results of your delayed or non delayed gratification from the future down into the present.
01:47:06Yeah.
01:47:07That's interesting.
01:47:08And then maybe there is no such thing as delayed gratification because it can all be here in front of you.
01:47:13It's immediate consequence.
01:47:14Uh, you know, and, and, and maybe, yeah.
01:47:17Yeah.
01:47:18I mean, I think a lot of what we, one of the most pernicious memes that ever spread was this idea of like technology is values neutral.
01:47:27You know, you just build a technology and it's up to humans how they use it.
01:47:30And that's, that's, it's always a neutral thing, but I don't think that's true.
01:47:34Like you can build a technology like a slot machine.
01:47:36It's clearly not a value neutral thing.
01:47:38You're saying we want, I want people to stay.
01:47:40And like, I'm, I'm saying that it is a good thing that my consumers would get completely addicted to me.
01:47:46Or similarly, we can build a technology that just like helps people's health, you know?
01:47:49Um, so we need to be thinking about what other, the, the problem we have is when we let, we, we sort of build technology.
01:47:56We think we're building it in a vacuum and then it dictates the social structures of how we behave.
01:48:01And then down that, that sort of dictates human behavior and our values.
01:48:06We need to flip that.
01:48:07We need to think about what are our core values first, let those decide the, the social structure.
01:48:12And once we have that build a technology that's in service of it.
01:48:16Um, and to me, one of the, like my favorite quote, I always come back to like, if I could pick one quote is my personal philosophy.
01:48:24It's by this guy, Forrest Landry, and it is love is that which enables choice.
01:48:29So I take that to mean a loving, a good benevolent act is one that empowers the other to make the best choices for themselves.
01:48:37So it ties in with this idea of like, if we had an AI that educates us, empowers us to see what possible outcomes would be.
01:48:48And thus we can make a better, you know, make a choice that is best for our long-term interests, as opposed to this immediate piece of gratification.
01:48:56Um, that is a choice making thing, but maybe we do just want, you know, some people are like, you know, I've, I've, I've weighed it all up and I do still want to actually just do the fun.
01:49:05Quick thing.
01:49:06Okay, fine.
01:49:07But you've been enabled to make the choice.
01:49:09So not kind of just culpable deniability.
01:49:11Like, are there not people who require additional assistance?
01:49:14Is there a, a level of scrutiny that, that I don't want to be too paternalistic, uh, here.
01:49:19Oh, poor, low, fucking delayed gratification people.
01:49:24You are unable to correctly corral yourself through this technology.
01:49:28But it just still coming from the most working class of working class towns in the Northeast of the UK.
01:49:33I saw how destructive, like the nineties technology habits were like stuff like a Ladbrokes.
01:49:40Yeah.
01:49:41Right.
01:49:42On a video village high street and betting on fucking greyhounds and stuff.
01:49:45Like alcohol, right?
01:49:48Pretty powerful, but you know, I mean, in the grand scheme of things, I imagine the pharmacology that can be developed from this.
01:49:55Like what, how there's so many ways that this can spin out that don't necessarily increase agency, but kind of make it subservient to technology that, and, and the outcomes of technology that it's not built to handle.
01:50:09I'm, I'm slightly optimistic here because of the gen alpha trends.
01:50:13Like if you talk to parents of gen alpha, they are observing a different child than gen Z.
01:50:17And I don't want to pick on gen Z.
01:50:18Cause I think we owe collectively a massive apology.
01:50:22Quite frankly.
01:50:23Why are you laughing?
01:50:24I agree.
01:50:25I agree.
01:50:26It's just, you know, as a millennial, I'm like, ah, you know, like I got a little bit of the nineties, but fuck, like it wasn't that long.
01:50:31And then you look at gen Z and you go, well, at least it wasn't that.
01:50:34I mean, the, the technology trenches, the trench warfare.
01:50:39If you give parents permission to step through the door of grief and shame, they, and, and I talked to a lot of them, they will break down describing the ways in which they've lost.
01:50:53I feel like they've mistreated the kids in a way.
01:50:56And they cannot reconcile it.
01:50:57Their, their dining room.
01:50:58They think back to the, the years of the dining room table that were just phones, the vacations that were lost.
01:51:04And they talk about the children that they don't know very well anymore.
01:51:07And, and it's not to say all, all households are broken.
01:51:10They're not, but it is to say that the last wave of technology did damage that we haven't reconciled yet.
01:51:15We we're still scratching the surface to fully understand it's caused cynicism and nihilism and fatalism, which is the enemy of good.
01:51:22And it leads to this crazy moment where we have even more powerful technology and people go cool.
01:51:28Now what?
01:51:29And the reason I asked the question earlier, what do we want?
01:51:33Well, we, of course we want a world where a beautiful life is less expensive and more achievable because of free government.
01:51:39But it does mean that we will have to let people do whatever they want.
01:51:43And agency comes with people being willing to say, I don't care.
01:51:48I'm, I will check out.
01:51:50It feels like an arms race with this though, that you need to have your ability to say no and to check out.
01:51:56The individual arms race.
01:51:57Yes.
01:51:58Yes.
01:51:58Well, the, the individual versus the technology that is the technology becomes more compelling, more frictionless.
01:52:02More enticing because that's how market incentives work.
01:52:07Your.
01:52:08Concurrent amount of ability to say no needs to rise up with it.
01:52:12And this is going to be much harder.
01:52:13Yes.
01:52:14And so yes, but, or yes.
01:52:17And.
01:52:19Hey, let's edit that.
01:52:24I'm an idiot.
01:52:26Gen.
01:52:27Gen.
01:52:27Gen.
01:52:27Alpha.
01:52:28Who.
01:52:29A priori.
01:52:30Only knows.
01:52:32I mean, what do they know about the world?
01:52:34Have already figured out.
01:52:35They don't want to look like Gen Z.
01:52:37Like they are watching a generation that they talk about not wanting to.
01:52:43Uh, screen agers.
01:52:44They call them.
01:52:46And.
01:52:47The.
01:52:48Return to the physical world is happening pretty radically.
01:52:51I mean, consumer confidence continues to wane somehow.
01:52:54And Benson Boone tickets continue to get more expensive.
01:52:57And it's not just the elites that are going to these concerts.
01:53:00They're.
01:53:01Concert halls are packing.
01:53:03Physical re in real life experiences are getting increasingly popular.
01:53:07I bought a women's volleyball team.
01:53:10I'm putting my money where my mouth is.
01:53:11I own the San Francisco professional women's volleyball team.
01:53:14I'm one of the five principal owners.
01:53:15I played volleyball in high school and college.
01:53:17It's a huge part of my life.
01:53:18If it weren't for AI, I would be a volleyball coach now.
01:53:21My wife was an all American beach and indoor.
01:53:23It's a huge part of our life.
01:53:24It's not a random act of, uh, it is a YOLO, but it's not random.
01:53:28But I am pretty sure that that sports to your point will continue to become interesting,
01:53:33not just because of the actual act of humans competing, but because people want in real life experiences.
01:53:38And I think the more everyone talks about AI slop is destroying the internet.
01:53:42And I'm like, good.
01:53:43Right.
01:53:44I do wonder if like at this point, the only way out is through just like excellent.
01:53:48Let's just flood it.
01:53:49Make everyone find it completely unusable and we can return.
01:53:52Maybe social media is always shit.
01:53:54Yeah, maybe.
01:53:55Oh, it's the, if you want a cigarette, why don't you smoke three packs approach?
01:53:58Right.
01:53:59And just, and just, just completely like to make it the most disgusting unusable thing.
01:54:04Um, but then are we not completely destroying all of the good that could have come about from the internet?
01:54:08Yeah.
01:54:09But maybe it's too late.
01:54:10Eric, are we too late to save the internet?
01:54:13I don't think so.
01:54:15I think you can, you can rebuild from the ashes and it's probably not an all or nothing thing.
01:54:19Yeah.
01:54:20I don't, but I'm actually curious, Zach, like how much intervention do you think we need?
01:54:25Like if we, if we had you as our non-parasitic national politician for the next 10 years, like, are you, are you, are you wanting to like get rid of regulations and just let AI diffuse faster?
01:54:38Like what, what, what do you think we need to get to the good future?
01:54:41Well, I'm actually far more, the reason that I, I told Chris, I want to be careful who we bring on the show is I don't actually want to be couched as the anti-regulation AI.
01:54:49I'm not an accelerationist by any stretch.
01:54:51I try and.
01:54:52Don't sound like one.
01:54:53I try to frame this discussion since the beginning as a diffusion problem.
01:54:57It's always been about the societal threshold, which you called out earlier.
01:55:00We've reached the point where the technological threshold is so impressive that the average person could get away using 4.0 and feel, you know, economically satisfied.
01:55:10The slowdown letter, which I had a feeling was coming, arrives at an amazing moment where safety is sort of reached a fever pitch and we can all genuinely take a sigh of relief at least for a moment.
01:55:22All, albeit you just said it, we shouldn't declare victory.
01:55:25To me, this is about rediscovering a shared vision.
01:55:28And I ask everyone when we start an AI debate, I go, what do you want in this world?
01:55:31And I hate to be annoying, but could we at least set some boundary of what it is you're trying to achieve?
01:55:37Because how else are we going to debate technology?
01:55:39Like why debate technology unless we agree on what we're trying to achieve?
01:55:42And most people describe the one, the world you want, one where we are more free and the world is less expensive.
01:55:48And if that's true, then we should actually spend time figuring out the policy measures that are required to allow people to be more free and to actually drive down the cost of goods and services.
01:55:58Because while we have made porn and gambling and addiction and violence and isolation infinitely inexpensive, we have made housing, healthcare and education prohibitively expensive for most of the developed world.
01:56:11And that fucking sucks. What the fuck are we doing? I mean it. This TV screen used to be $50,000. Now it's a hundred bucks. You can get gas station sushi. You can drive across Los Angeles for $10 in an autonomous vehicle, but a trip five blocks away in an ambulance might actually bankrupt you.
01:56:28The kids have figured it out. It's not a technological failure. It's a policy one. We stopped describing a better world where everyone's parents would live lives that they would pass on the next generation a much better life.
01:56:42We stopped describing what it actually meant to pass on our luxuries as commodities. And we have broken a promise that we made many years ago in this country to build a more perfect world constantly.
01:56:55To me, this opportunity is not about debating the merits of AI. It's actually about reframing what it is this country should look like and what it is the world should feel like to the average person.
01:57:08And you're making an interesting point, which is at some point people are going to have to figure out what their purpose is. And I'm like, cool, let's get to that place by first driving down the cost of housing, healthcare and education.
01:57:17And all of my policy would be spent on, if I, if I were a Supreme ruler, policy, policy that prohibits catastrophic downside necessary safety policy, policy that criminalizes preying on people.
01:57:34And we'll setting predatory laws is going to be pretty tricky, but I think we all know where, where we could fall boundary. A 13 year old should not be served an ad for porn or gambling period.
01:57:44Um, senior citizens, if you attack a senior citizen using deep fake prison for life, right? We, we set really punitive measures for, for preying on people. Cause that's the stuff that tears at the fabric of society.
01:57:55And then you mandate the laws, you mandate policy that requires that institutions diffuse this technology into them, including, but not limited to universities, allowing for everyone to have access to the curriculum.
01:58:09Hospitals allowing people to basically get healthcare outside the box and, um, and housing. We just need to build a shitload more of it. We need to rezone cities. We need to build, we need a Marshall plan for housing.
01:58:22We should probably pass a non-resident tax. That's my most progressive policy measure or, or vacancy tax. But most of my policy measures would actually be focused on the ways that you diffuse the technology, not actually on how the technology needs to be made.
01:58:34Because if you start, if people can see that technology is actually good for them again, which it hasn't been for a little bit, then we can actually have a shared excitement.
01:58:43But right now there's, there's a malaise because people are like, no, this is just going to fuck me up worse than the last thing did. And I don't blame them.
01:58:49It's definitely interesting to say that we're in a perfectly primed period for people to be very skeptical about how new technology might impact their lives. It's, it's the perfect storm totally.
01:59:04And that maybe does tend people towards pessimism. And the strange thing with the AI debate is that because the outcomes, if it goes wrong, are so great, pessimism that might be unwarranted feels appropriate when adding a buffer into something that could fucking collapse the entire world. Does that make sense?
01:59:21Totally. And I mean, we're seeing it manifest like the hate on the data centers, right? I feel like we're on the cusp of a butlerian jihad, but it's directed towards the silly things. Like data centers, it's way overblown all the concerns about it. Like it's the water usage. I was looking at some stats. It's like all the AI data centers in America use 3% of what US golf courses use.
01:59:46Literally 3% golf courses use 33 times as much, and they serve significantly fewer people.
01:59:53Wow.
01:59:54I think it's like 30 million people play golf and how many millions of people use AI. It makes absolutely no sense. One, I think your, your daily life is the equivalent to 300,000 prompts, you know, so like-
02:00:07On a carbon, on a carbon basis.
02:00:08On a carbon, no, but if you factor in a lot of the, I think even water as well.
02:00:11Yeah, resource.
02:00:12Yeah, resource.
02:00:13Yeah, the resource use is way overblown. But at the same time, people are, they're focusing on it because it is the physical manifestation of this thing they can sense that it, and I'm, I'm, I'm very understanding of it because it does in some ways feel like, again, it's like this-
02:00:28Totally.
02:00:29This, the, the date, the, the, the digital world is almost like its own, um, it's almost like a species or something that is coming in and like cannibalizing or like parasiting off us. And people want to direct their energy to something. So it's like, how do we, I don't know. I don't, I don't, we, we need to-
02:00:50What if a politician stood in front of a data center? I agree with you, by the way. And I'm glad you delivered that line so I didn't have to. I didn't want to sound like the sick, the AI sycophant. By the way, on that note, on sycophancy, it is remarkable to me that we didn't make model behavior a greater topic of issue during the safety debates, the early safety debates.
02:01:08People let model behavior go. And I remember scratching my head going, at the limit, you could build a perfectly aligned model that actually falls inside alignment boundaries, that convinces people to climb crazy trees, and that really destroys people's spiritual lives, well within the confines of a safety, but outside the confines of-
02:01:28Soft damage, not hard damage.
02:01:29You're a, a friend in your life that constantly tells you you're not doing anything wrong is as destructive as a, as it is, it's a form of abuse.
02:01:38It just doesn't show up.
02:01:39Enabling abuse.
02:01:40It's enabling abuse. It just doesn't show up on like the psychological list because it, you know, those friends are seen as like, you know, homies. But actually the person who keeps climbing the craziest tree with you, look, the models have been responsible for people getting divorces. They've been responsible for people selling all their belongings and trying to start a company that fails. Like model behavior matters a lot. And the sycophancy issue is a huge one. Here is my message to John Ossoff or any, any, any Democrat or anyone, anyone anywhere that is,
02:02:08that wants to, to separate. But if someone on the center or center left can figure this out, you stand next to a data center and you hold up a sign that says this data center will not serve gambling and pornography and violence. And it will build a better hospital, build a better virtual hospital. It will distribute education. It will build a bank, a neobank that gives you better interest rates and, and gives you better access to loans.
02:02:32That seems like pretty reasonable data center policy because no one actually hates, and I, this is a hot take. No one actually hates AI. They hate the promise of AI as a means to just make their lives worse. And if you can actually show them that data centers can contribute to a better world, they can support it.
02:02:52How accurately can you actually know what a data center is used for though?
02:02:53Well, we're, we're, we're, we're, we're straw manning some policy measures.
02:02:59I mean, I, I, I like this line that, you know, no one actually hates AI. They hate what they think it's leading towards. I do think people are tracking both what individual consumer apps might look like in a few years and how dystopian and, and sloppy that might be.
02:03:14And also tracking how much money these companies are vacuuming up. They, I think they're also looking at the job displacement stuff. And I think we should give people credit. Like I don't, I think people would know the meme about how, when ATMs came around, they were more bank tellers and not less.
02:03:29But they also know that Elon Musk was a trillionaire earlier this year. And I think they can see the connection that one of the things technology does is allow one person who has the right combination of ones and zeros on the services they control, just dominate economically.
02:03:44But that's built upon a false understanding of how economics works though, largely, right? Because like, not that I'm going out to specifically defend Elon, but like most billionaires-
02:03:54Oh, go on Liv, defend you. Come on, come on, come on.
02:03:58But most, you know, most billionaires are not become billionaires by taking from other people. It's because they've grown the pie.
02:04:03Totally. No, I actually don't mean at all to say that there's something wrong with Elon Musk being a trillionaire.
02:04:08I think people notice that that's a symptom of the economy working very differently nowadays than it used to. Like, there are just not that many people who work at SpaceX compared to Standard Oil a hundred years ago.
02:04:20It's just like a different world where capital can be deployed at huge scale without a bunch of people being involved in that. And yeah, you know, I hate to keep beating the same drum, but I think like we really need to play forward the movie of like, in this world where everyone doesn't have to work and they have radical choice.
02:04:44Like, what is the main thing going on on earth? I, I don't think it's easy to tell a story where like 99% of the, of the land area is just folks playing with their kids and their grandkids.
02:04:56Like someone is going to say, we need to build a giga data center the size of Texas so that we can solve physics or that so we can, you know, go to the moon and colonize there.
02:05:07And, and I think there's like still going to be these same dynamics right now that mean a lot of resources go towards these like industrial activities.
02:05:16Yes. Although we have one ultimate area of getting off the playing field, which is space and a huge part of if we're in this world, anyway, this is all a big hypothetical, but if we're in this world, we've presumably figured out how to go up and build up and put things up there.
02:05:31Like, again, I don't, haven't looked into the physics of it. It seems odd how you would call a data center in space, but allegedly the plan is put data centers up there.
02:05:39And I mean, if that is feasible, then that solves that problem, right? People can like, if you've, if you've expanded into that frontier now, all bets are off.
02:05:47Like we aren't dealing with that land scarcity anymore. And you can have that, which is why I'm so bullish on the space industry in general. It's like the ultimate area of abundance.
02:05:55Is it, is it fair to say that we might have glossed over a little bit of the concentration of power thing so far live Moloch problem stuff. Can you explain this for the resident idiot in the room?
02:06:09Um, so the, the Moloch, the Moloch trap or.
02:06:13Wait, is that me? No, no.
02:06:16Uh, the, the, the Moloch trap, uh, Moloch problem, whatever you want to call it, is a sort of catch all term to describe these types of race to the bottom scenarios, um, where you'll have like a bunch of actors competing over a certain thing.
02:06:30Uh, so, you know, maybe it's market share who can get the most users, uh, in AI, you know, between AI companies trying to get the most users.
02:06:38And if the competitive dynamics are sufficiently intense, it can, it will incentivize the, you know, you to cut corners and you might be a really benevolent actor and be like, I don't, I don't want to cut any corners with safety and so on.
02:06:53But if at the same time, if you don't do it and you see your competitors doing it, if you don't also do that, that reckless action too, you're going to get left behind.
02:07:02Um, and Moloch is the sort of catch all term, I won't get into the sort of biblical reasons why, of, of this feeling like you have to sacrifice your other values in order to win at a particular goal.
02:07:13Um, and yeah, so it, it drives both the risks of like, uh, chaos and, you know, bad actors and so on.
02:07:24Cause again, if you're like rushing to release your models before they've been sufficiently safety checked, now you're increasing the risk of like cyber attacks.
02:07:30Or, um, I think in a few years we're going to have real risks of bioterrorism and so on.
02:07:35Um, so these kinds of chaos attractors, but you also actually increase, um, well, a, then the risk of like knee jerk bad regulation, which can lead us down the path of tyranny.
02:07:45But also you increase the risk of power concentration because usually if you let, you know, you press play on some kind of competitive game.
02:07:53Okay.
02:07:53Yes.
02:07:54You might have a bunch of people competing for a while and some people are ahead, but you still got actors.
02:07:58Shit accrues.
02:07:59Yeah.
02:08:00You end up with, um, usually in some kind of monopoly.
02:08:03Um, and so it's, it's, it's really, you know, and that's why we have antitrust laws and so on, but it's this really difficult balancing act between like reigning in, you want some amount of regulation, um, that prevents the, the recklessness and, and the, the cutting of corners, um, and externalizing harm onto the rest of society, um, without creating too much draconianism
02:08:28or creating too much centralization of power such that you end up in stagnation, also a terrible end state.
02:08:35Um, and without empowering some, some winners, you know, we, we all end up under, you know, we all end up in Zuckerville for the rest of time.
02:08:44It's possible.
02:08:45It's definitely possible.
02:08:46Um, so it's like how to thread this needle and it's like finding this right delicate balance.
02:08:51I mean, ultimately you want some kind of like decentralized regulation.
02:08:55I don't know how that looks, but.
02:08:57Eric, it feels like watching your stuff, you've spent a good bit of time thinking about the concentration of power thing.
02:09:01That's like a, a real middle of the bullseye concern for you.
02:09:04Definitely top of mind.
02:09:05And I'm sorry for bringing it up every 15 minutes in this conversation, but I, I do think that we run the risk of sort of missing that between these two extremes of, you know, what if the AI takes over completely and there are no humans in power?
02:09:21And, and what if we get this utopia, you know, and there's some middle ground there that I think we should worry about.
02:09:26Um, and yeah, one of the most fascinating things about, you know, a lot of our YouTube videos, we're going over the history of AI, not just the future.
02:09:33And when you look back at like how we got here the last 15 years, it's like so stark.
02:09:40The, the quotes from these people running these AGI labs are spelling it out.
02:09:43Like it's a bad movie.
02:09:44Like Elon Musk truly like 10 years ago was calling AI a demon and says that he found open AI to make sure that like there's no dystopia.
02:09:52And then eight years later, he's on camera saying, well, I realized it was going to happen with or without me.
02:10:00And so I might as well be a participant than a spectator.
02:10:02Like it's, it's Moloch just like, you know, personified.
02:10:06Uh, and yeah, I, I think like, unfortunately right now there is a kind of like zero sum dynamic with, with these folks.
02:10:16And the reason that Zuckerberg is all in on AI and betting billions of dollars on it is that he thinks this is the thing that will control the future.
02:10:26Like, like this is the new axis of power.
02:10:28And like before it was, you know, yeah.
02:10:30Do you control an army?
02:10:31Do you control a powerful company?
02:10:32And now it's like, do you control the AIs?
02:10:35Um, yeah.
02:10:37And so I, I think that alone would be enough reason to slow down.
02:10:41Even if I wasn't worried about, can we get these systems to do what we want?
02:10:44Which right now we, we don't really have a good solution for.
02:10:46I think the people behind this technology are really playing to win and really do see the stakes as, as, uh, as almost winner take all.
02:10:55Is it fair to say there's like maybe three ways or three realistic ways that that might be curtailed?
02:11:01One would be governance from above.
02:11:04Another would be sort of consensus from across.
02:11:06And then the third one would be capacity, sort of ability to just make it happen.
02:11:11Like I want to do it, but I can't.
02:11:13Hmm.
02:11:14I want to do it, but someone's telling me I don't want to.
02:11:16And I want to do it kind of a bit, but me and all of my peers have decided that we're not going to do it.
02:11:20Hmm.
02:11:21Have I missed something there?
02:11:22I'm just trying to think, given that this seems to be a pretty big attractor, given that we have already, you haven't started off with a massively diffused number of companies.
02:11:30It's not a hundred of them, right?
02:11:32It's like seven or something, and it's probably only more like five.
02:11:36That's already pretty fucking concentrated.
02:11:38Can we just, uh, for station identification, clarify the surface area you think?
02:11:43Cause I actually think this is also gets lost in the debate too often.
02:11:46I think people think that these companies are sprinting at a frontier.
02:11:49I no longer believe that front that frontier strategy is actually the winning condition.
02:11:55Say more.
02:11:56Well, I open source lead time.
02:12:00Well, trails, you said at four to seven months, which is crazy.
02:12:04Cause it used to be whatever 15, the gap just keeps closing open source to close.
02:12:09So the frontier can't possibly be the winning condition, right?
02:12:12In this game.
02:12:13I think so one might think it's a bit of a deceptive staff.
02:12:18I think for a couple of reasons.
02:12:19One is, as you brought up earlier, they're kind of copying our homework.
02:12:22Like the reason that these Chinese open source models were good is because they're distilling American models, which means they're just like asking quad a bunch of prompts and then copying the answer.
02:12:31It's kind of wild what we think they might be doing.
02:12:33You can actually like ask one of these Chinese.
02:12:35I need to, I need to find this out.
02:12:37How is China copying us so quickly?
02:12:38Well, allegedly it's there.
02:12:40They're doing an exceptional amount of distillation of the model.
02:12:43So working backwards into how the model was built and effectively, if you, the way I described it to my mom is if you stared at a building and you figured out how to take a trillion photos of the building, you could actually build the blueprint of the building.
02:12:57Uh, you could re-engineer the blueprint of the building.
02:12:59And nobody owns that technically.
02:13:01Yeah.
02:13:02Well, no, this is considered illegal on most copyright law.
02:13:06Like if they were doing it in the U.S., Anthropic would probably sue.
02:13:11And funnily enough, I mean, Secretary Besant, who's part of the Trump admin and, you know, a few months ago there was a lot of beef between them and Anthropic, came out on Twitter recently and was like, this is a threat to American AI supremacy.
02:13:22You know, we will take diplomatic action to defend our American advantage.
02:13:25So people take this seriously.
02:13:26The enemy of my enemy.
02:13:27But this is important to call out because I, it's, all I'm saying is I don't think the frontier is that safe.
02:13:32Like, well, do you think the frontier is actually a winning condition?
02:13:36It's actually a winning condition in a sense, like whoever is training the most powerful models is going to have an advantage.
02:13:42Yeah.
02:13:43Or, or such a, such a material advantage that they shall win.
02:13:46I mean, if RS, if recursive self-improvement takes off as you say it's going to, and we're already there, then surely it would.
02:13:52Because now you're going to get this like spaghettification.
02:13:55Does anyone, the slight lead can.
02:13:57But then wouldn't everyone, the theory of ASI is kind of fascinating because if you, if you build ASI, then everyone gets ASI.
02:14:04Why?
02:14:05Well, because then if you.
02:14:06Why does everyone get it?
02:14:07Well, presumably if you build a super intelligence that was taught by recursive intelligence, at some point, the machine is kicked in and is doing work that, that the humans aren't necessary.
02:14:16Like copyright and ownership doesn't really become particularly important if there's a machine that can build itself.
02:14:22Agreed.
02:14:23But I don't see how that's conflicting with.
02:14:25I'm not saying it is.
02:14:26I'm asking back to the point.
02:14:28I'm asking what you think that companies are now competing on.
02:14:31Like, like the, the race is for what?
02:14:33I think, I think the frontier absolutely is prime territory.
02:14:36I think if we stopped training American AI models now, we would not see China training a GPT-6 level model ahead of us.
02:14:45They don't have the infrastructure for that.
02:14:46What they have the infrastructure for is, is copying the current models.
02:14:49Right.
02:14:50I agree with that.
02:14:51My question is, what do you think?
02:14:52So, what do you think that we are, what do you think is required to win this, this quote unquote, if it is a race, the race?
02:14:59That's, that's what I'm asking.
02:15:00I mean, I have a lot of problems with the, with the race framing.
02:15:03Same.
02:15:04I hate it, but I'm just using it for this purpose.
02:15:06But yeah, I do think right now, I mean, you can just read it on these companies websites.
02:15:10They think that RSI is the whole name of the game.
02:15:13Like once you set off that loop where you're handing off to the next AI system to build the next one, it's fine if you have a four to seven month lead all the way until 2040.
02:15:22When you're on exponentials, like a seven month lead 10 years from now, it looks like this compared to this right now, because this, this curve is just taking off.
02:15:29Well, we thought that would have been the case, but it, but the, the curve is flattened more, more open source models are growing faster.
02:15:35Smaller models are getting more performative.
02:15:36Well, hold on.
02:15:37We keep measuring it in months.
02:15:39But what I'm saying is like calendar time is not the right unit.
02:15:43It's a much harder thing to measure.
02:15:45Like what is the subjective overall intelligence difference between Fable 5 and Kimmy 3?
02:15:51But I think it's pretty substantial.
02:15:54Like, I think you can just like trust Fable 5 to access your entire code base and refactor it.
02:16:01And now we know that like maybe even some of the open AI models are like not quite trustworthy because they're committing cyber attacks that we didn't want.
02:16:09Um, so I, yeah, I think it actually might be a mistake here, but you're, you're, you're saying, look, it's the frontier.
02:16:14It's not the, it's not number of users.
02:16:16It's not application.
02:16:18It's not revenue.
02:16:19It's, it's research.
02:16:20Yeah.
02:16:21And just look at the finances of these companies.
02:16:23Like they're just going all Sam was on a podcast a couple of weeks ago saying we didn't have a best 12 months.
02:16:28And that's because we were distracted.
02:16:30We weren't sure about the demand.
02:16:31We wanted to do consumer apps.
02:16:32We wanted to make sure that we have revenue no matter what they were doing Sora.
02:16:35And now they're like, nope.
02:16:36Yep.
02:16:37It's all in.
02:16:38We've got to make these things better at coding because that's what makes them get to the point where they can code themselves.
02:16:43So I, yeah, I think the writing's on the wall.
02:16:45At the same moment that a lot of these companies are re reinvesting even more heavily in application development and token resell and.
02:16:56I don't know.
02:16:57I see anthropic trying to grab as much of the compute for themselves as they can.
02:17:00And like, it's notable, like quad is not that good at image generation.
02:17:04I don't even know if you can get images out of quad.
02:17:06It doesn't.
02:17:07It can like take them in, but it can't.
02:17:08Yeah.
02:17:09Yeah.
02:17:10Like there's a reason for that.
02:17:11And the reason is that like that company was founded by Dario Amadei, the guy who wrote the scaling loss paper, the one who was just obnoxiously like just extend the line on the graph guys.
02:17:19And everyone was like, yeah, but it's an exponential, but it's going to be like an S curve and it's going to select.
02:17:23He was like, no, like you, you just dump more compute at the thing and it gets smarter.
02:17:28And they are, they really all are playing for this point where they can hand off to their AI system and say, like, you take it from here.
02:17:35Why hire all the engineers that way?
02:17:37Why hire all the four deployed engineers?
02:17:39Why, why is it a charade?
02:17:40No, no, no.
02:17:41I mean, I think for one thing, like you do have to finance the next training run somehow.
02:17:46And so anthropic knows they have to like keep the money machine going.
02:17:49And two, I mean, at least anthropic claims and we should be skeptical that they, they actually care about the U S government, like having an advantage over China.
02:17:57And so they're, they're putting FDs in the government so that we don't live in a world where our military is way behind.
02:18:04And suddenly the countries that got there first have an advantage.
02:18:07Um, but yeah, no, I, I think like there are just like tons of returns to scale in this industry.
02:18:15And that's part of what makes concentration.
02:18:16I didn't want to, I, yeah, I didn't mean to distract you from concentration and power, but, but.
02:18:20No, no.
02:18:21So talk more about, you were on, you were talking about Zuckerberg and I was asking what, what you think they're competing for, but, but.
02:18:26Yeah.
02:18:27Well, I, I, I guess like, I don't know.
02:18:31I, one phrase that, uh, hasn't come up in this conversation is, is gradual disempowerment.
02:18:39This idea that like, even if you solve the alignment problem.
02:18:44And so there is no AI system anywhere that's just going off and doing its own thing that no one intended.
02:18:48You don't have this Malachian problem solved where people are constantly competing on algorithms for people's attention and whatever.
02:18:56And, and like, it's just hard to tell a story in that world where the things that we like most desire as a society in some like very democratic way are the things that actually obtain.
02:19:08Um, and I, I think concentration of power is sort of related to this.
02:19:12It's like you're imagining a world where there's a few human beings who hold the cards instead of like some amorphous algorithmic thing.
02:19:20But either way, it's just like, I think our default thing, our default assumption should be if humans aren't themselves economically useful, it's a precarious world for humans continuing to be politically empowered.
02:19:40Right.
02:19:41Like we've, it's always been the case that you just can't afford to piss off 90% of the American public because those are the people who are electing you.
02:19:49But like in a future world where you've got control of the economic resources and maybe the military ones, like it doesn't matter if you lose the election.
02:19:57Like we know how this goes.
02:19:58Democracy is done.
02:19:59Exactly.
02:20:00Yeah.
02:20:01Um, and I, I want to believe that we'll have such strong norms that we can like figure out a way to just like sustainably make that the case.
02:20:08But like the further you zoom out with your big history lens on like.
02:20:12Someone made the interesting point.
02:20:14Um, can't remember who, but the new political parties, it's not going to be Republicans and Democrats anymore.
02:20:21It's going to be Meta or Google or open AI, they are positioning themselves, whether it's intentionally or not into becoming the new sort of political entities.
02:20:34Um, I don't know if you guys, if that resonates or you think that's plausible.
02:20:38Well, they certainly will try to inform policy, but I am, again, I'm not sure this is popular.
02:20:46I have more faith in our ability to, why don't we just pass campaign finance reform?
02:20:53Like w w why don't we do this?
02:20:55Presumably because Congress makes too much money from it.
02:21:00And people who have won the last elections are exactly those who are good at winning the current system.
02:21:06Yes.
02:21:07I agree.
02:21:08So it's self-selective.
02:21:09So it seems like one of the things that this will force us to do is start to elect.
02:21:13And again, I appreciate there's a downside where we can't collectively do this, but if we start paying attention to this and we say, look, what we actually need are politicians with higher agency politicians that can, can describe an actual place in the distance.
02:21:26They want to move towards like, again, this is sort of my AI slop argument, which is that some of this is actually, it drives people back to a place that we want to be.
02:21:36Well, how many humans do you think you've got the full site in that case?
02:21:39Why have we had some of the more WWE style characters come out over the last few years?
02:21:44Because we had a bunch of politicians that were so boring and that we didn't believe for a long time.
02:21:48And then we loved this idea that the politicians were real.
02:21:51And I'm watching, I think we're watching in real time.
02:21:53People would be like, wait a second.
02:21:54We don't actually want politics to be a hellscape of entertainment and bizarro Kafka.
02:22:01Kafka couldn't have written what's going on today.
02:22:04He would have laughed himself to death.
02:22:07So I think we, but, but we're kind of correcting like my, my, my take here.
02:22:16And by the way, I'm not, I'm not sitting here saying it solves my, I'm sitting here saying it's solvable, which is, there is a way to pass campaign finance reform that says no matter how wealthy someone is, they can't buy politicians.
02:22:26Singapore has done this other places.
02:22:28How did Singapore do it?
02:22:30Is it the digital?
02:22:31No, they rebuilt a government 75 years ago.
02:22:34A grandfather stood up.
02:22:35The grandfather stood up, the godfather of Singapore left Malaysia and said, we're going to build a better place.
02:22:40And literally described all the things that he had observed that were perfect about a government and then made corruption a capital offense.
02:22:48So if you take a bribery, they'll kill you.
02:22:50So that's a big incentive.
02:22:52Yeah.
02:22:53It's a big incentive.
02:22:54And they pay their politicians really well.
02:22:55They pay their politicians, I think adjusted like $600,000 a year.
02:22:58There was a job for the UK's head of cybersecurity that was listed not long ago.
02:23:03I think it was maybe on Indeed or Monster or LinkedIn.
02:23:0865,000 pounds a year.
02:23:09Generous.
02:23:10So you're going to get the best.
02:23:11And I was the head of cybersecurity for the country of United Kingdom.
02:23:18Based in Newcastle.
02:23:19Oh, no, no, London.
02:23:20It was a fucking London wage as well.
02:23:23So that's actually.
02:23:24Yeah.
02:23:25That's nothing.
02:23:26That's subsistence living.
02:23:27What do you think, Eric?
02:23:28Like, can we just vote ourselves out of the concern you've got for the future?
02:23:34I do think that helps in the short term.
02:23:37And then we've got to take advantage of the time that buys us to make radically new institutions.
02:23:43Like, I don't know where we get that from.
02:23:46Maybe we look to sci-fi.
02:23:47I haven't read the culture series.
02:23:49Like, you know, I think a lot of people say that there's some like amazing pie in the sky ideas that we need to maybe dust off and like import into reality a bit more.
02:23:59But like, I, I just don't think the framers of the U.S. Constitution had anything like the problems that we are currently facing in mind.
02:24:08They didn't even see gerrymandering coming.
02:24:10And, and that's, you know, that's no knock on them.
02:24:14But like, I think we do need to realize that we're kind of in like virgin territory here.
02:24:19And it's, it's not just going to be some superficial thing.
02:24:23Like, I would, I'd love to start by passing campaign finance reform.
02:24:27It's a great first step, but you already see these primaries that are flooded with, you know, ads trying to get the everyday voter to believe something completely untrue about your opposing politician.
02:24:39Um, and I think in that world, we, we get a bunch of those ads convincing those people to donate money so that you can build up the war chest that you now can't get from a billionaire to just run those same ads, you know?
02:24:51Is it not the case as well, that the pace that this is moving at is so rapid.
02:24:55And when we look at, forget structure, legislation, politics, this lumbering behemoth that sort of gets dragged behind us, this Leviathan, like, is that really a realistic solution to try and have campaign reform, finance reform, political redistribution when everything is moving at the speed of light?
02:25:18Like, it's just going to be so late, is it not?
02:25:22Well, is everything moving at the speed of light?
02:25:24Like the technology is getting way better.
02:25:26I don't, I, I observed that we've been stuck in this weird screen delirium for a while that actually hasn't changed all that much in the last, like since AI, the stuff's getting more addictive.
02:25:39And I guess we have, we have prediction markets now, so we have, we have rampant gambling.
02:25:44But I mean, the pace of AI development.
02:25:46I, I totally agree.
02:25:47My point is, does it actually, like for the average person, I don't think the average person observes that AI is all that different than it was.
02:25:54That is a good point for, apart from, you know, hugging face and maybe you've got to pull fabled mythos back and, you know, get the jailbreak out of the way.
02:26:03Like, okay, real world, show me something that has happened because of AI, where we've gone, oh, whoa, that's a bit like that hugging face thing is probably the first big one.
02:26:13Am I right?
02:26:14Like, is there being something else that's been a physical world warning shot, some AI psychosis, some fallout of people's values, some people maybe spending less time outside.
02:26:25Cause they've got a relationship with the, with the chat bot.
02:26:28Like, have we had the thing yet?
02:26:30Or are we waiting for it?
02:26:31Or is it your suggestion that like, that's just kind of not going to come?
02:26:34No, it will come.
02:26:35I think we are due for a Chernobyl or three mile island.
02:26:37I mean, all technologies eventually have some point where, you know, something happens and how we respond will be really important period.
02:26:46I mean, that, that will, that will arrive.
02:26:48And I actually still maintain there's a decent chance we do to AI, what we did to nuclear power.
02:26:53I mean, the AI trade is so incredible that it would be really economically problematic, but I do think that the tail will wag the dog here.
02:27:01I think that public perception matters way more than, than, and you, you said it, we should give people credit.
02:27:06I totally agree.
02:27:07I think that politicians will actually capitulate to their electorate before they capitulate to the, to the billionaires and the trillionaire.
02:27:15And I think we will see policy get passed that feels more populous or populate than, uh, than anything else.
02:27:25And the question is, will it actually be so populous that it's regressive?
02:27:29Like, will we, will we ban outright technology that creates a greater, you know, uh, invisible graveyard?
02:27:36And I don't want to reduce, I don't want to reduce the concentration of power to just saying campaign finance reform, but it does seem like the world would be a lot simpler.
02:27:46This discussion would be a lot simpler if we didn't have to worry about billionaires buying politicians.
02:27:50Is that the mechanism when we talk about concentration of power, is that the primary mechanism that you're concerned about?
02:27:57I think it's not just economic.
02:27:58It's also military.
02:27:59Uh, you know, I, yeah, I, I, I think like it's unfortunate how sci-fi it sounds to talk about, uh, an army full of drones and robots.
02:28:11Uh, but I think it would have sounded sci-fi a hundred years ago to talk about predator drones and fighter jets today.
02:28:18Um, and yeah, I, I wish I could like tell you a more concrete picture, but I think we just like, don't know what happens when a general can look at, uh, a fleet.
02:28:32That's, you know, sufficient to take out the dictator of Venezuela and know that like he can order whatever he wants and there will be absolutely zero questions asked.
02:28:41Um, like, yeah, I, I think I just worry that we're so used to a world where like pieces of paper matter.
02:28:48If you have a little piece of paper that says I own stock in this company.
02:28:51And so when that stock goes up, you have to like, give me a dividend or I'm a citizen of this country.
02:28:55And so therefore I have these rights, like you can get to a world where that kind of all goes out the window because there's just greater concentrations of force that can say, I don't acknowledge that piece of paper is validity.
02:29:07Um, I, I, I hate how like, what's what, what two policies would you pass to limit the concentration of power?
02:29:15I think the first one is a slowdown so we can figure out that question.
02:29:20I wish I had the answer for you packaged here, but I don't, I don't think anybody does.
02:29:25Um, and I, I just like, there's this great post from Daniel Cocatello, the guy who wrote AI 2027 and then 2040, where he tries to take seriously this like exponential trajectory.
02:29:37And he says, okay, if we take that at face value, it means we're going to have 500 years of progress in five years at some point.
02:29:44And he, he sort of like goes through what this would be like, uh, if you look at the last 100 years, you know, you wake up January 1st.
02:29:49It's 1500, the pressing press has just been invented.
02:29:52Um, and it's like a really boring story for the first 95%.
02:29:56Uh, and then, you know, from like 1950 to 2000, uh, that happens in the span of like the last two weeks, December.
02:30:05Um, and it's, it's like, it's really terrifying.
02:30:08I think if we were better at thinking in exponentials, we would realize how extremely compressed the calendar time we have to answer these questions is compared to what they deserve.
02:30:18And that's why I'm so excited about the slowdown.
02:30:20I'm just like, we, we need time.
02:30:23And, and also the, the, this compression is, I think one of the biggest arguments for why we need in the, in the best sense of the word, like a true diversity of thinkers and voices of people.
02:30:37Like so many people, I think just like, well, I don't understand AI.
02:30:40I'm not technical.
02:30:41My opinion doesn't matter.
02:30:42I, I never in history has that been more false than right now because, because it affects everybody and because it is so, um, so multifactorial, there's so much uncertainty.
02:30:56It is actually extremely important for religious leaders, uh, arts, you know, liberal arts, people, scientists, technologists, politicians, as like voices are more valid than ever because we need a, it's like kind of all
02:31:12hands on deck situation.
02:31:14Um, and like, that's, you know, sort of the thing I implore to people.
02:31:18I talked to my mom and she was just like, well, that's what, what, what business do I have thinking about this?
02:31:22I was like, well, you care about like how your animals are kept.
02:31:25What would you feel about if this was, this particular thing was automated?
02:31:29It like, it, it, it sounds again, sort of very kumbaya, but we just need, it's such an all hands on deck situation.
02:31:36We need everybody listening to this podcast, whatever your background to be thinking deeply about how to answer that question.
02:31:43Should we see what chat GPT said?
02:31:47What ordinary life looks like?
02:31:48AI becomes cheap cognitive labor.
02:31:50Also electricity for thought.
02:31:52Scroll down for me.
02:31:53I imagine that there's a summary.
02:31:54Go to the very bottom.
02:31:55I imagine there's gotta be, keep going.
02:31:57Oh, there we go.
02:31:58Catastrophic risks are real.
02:31:59My rough betting odds for the world by the end of 2040.
02:32:02Not scientific measurements are 60% turbulent, but manageable adaptation with large benefits and serious inequality.
02:32:0920% broadly flourishing transition with gains shared relatively well.
02:32:1315% severe, but survivable crisis, authoritarian consolidation, depression, skills, disruption, war, pandemic, or infrastructure failure.
02:32:215% irreversible global catastrophe.
02:32:23So interestingly, all of you have been more optimistic than the AI itself has been.
02:32:29Hmm.
02:32:30I don't know.
02:32:31That lines pretty well with my sort of-
02:32:33You think that's how it tumbles out for you?
02:32:35It's not far off.
02:32:38The thing that spooks me is this reads like it is perfectly tailored to be not that objectionable to almost anybody.
02:32:46You know, it's just giving just enough probability on each thing in proportion to how dystopian and weird it sounds.
02:32:52It's slop.
02:32:53It's slop.
02:32:54This is garbage.
02:32:55Okay.
02:32:56Can we just be, can we agree this is awful?
02:32:58You tell me.
02:32:59Well, I mean, it's not that it's, okay.
02:33:02It's not that it's academically awful.
02:33:04It's aesthetically awful too.
02:33:06I read it and I wanted, like, I don't want this on the screen anymore.
02:33:09Yeah.
02:33:10What's fascinating is, I don't know whose take this was, but slop is not necessarily bad.
02:33:16It's what's cheap and common, right?
02:33:17Like I think part of why both of you and I are recoiling is that we've just seen that exact sentence structure.
02:33:22It's not this, but it's that.
02:33:23I read that on LinkedIn earlier.
02:33:24Did you post that?
02:33:25Yeah, I mean, yeah.
02:33:29And also part of it is like, this is a reminder.
02:33:32I do think that humans have a pretty innate understanding of aesthetics.
02:33:35And there is something really cool about the fact that we have all grew up seeing the Mona Lisa on a screen.
02:33:41And then everyone's still not everyone, but a lot of people still want to go to the Louvre and see it in person.
02:33:46And that is like an innate immutable truth that I find a lot of hope in.
02:33:51I continue to find hope in all the things that people return to when things get cheap and easy and shitty, which are kind of awesome.
02:34:00And if you can, again, not to, I don't, I don't want to make this too Pollyanna, but I do think that a lot of this actually resolves as humans start to rediscover a greater purpose.
02:34:14There's a quote that I returned to in these times when John Maynard Keynes in 1930.
02:34:23Have you, do you know this paper?
02:34:24Do you know the paper I'm about to cite?
02:34:25Okay.
02:34:26In 1930, John Maynard Keynes against the backdrop of abject despair, the father of modern macroeconomics is traveling 1929.
02:34:32He's traveling the world, giving these lecture series on how technology is making everything better.
02:34:36And people are starting to get really poor.
02:34:39Like people are starting to die in the street.
02:34:41And by 1930, he's in Europe and people are like, John, get the fuck out of here.
02:34:45Like you are, this is read the room.
02:34:47And he goes home and he's like, I'm, I'm not wrong.
02:34:50And he doubles down.
02:34:51He writes this paper called the economic possibilities for our grandchildren, which is going to get cited more and more.
02:34:56And I'm surprised people don't talk about it more.
02:34:57And he writes, and I quote, I must now disembarrass myself.
02:35:00Great line to imagine a future.
02:35:02I will not live to see one in which humans may have solved the economic problem and be faced with something more profound.
02:35:08And it occurs to me that a lot of this debate is interesting in part because it actually focuses on things that aren't like we are now for the first time in a million years or however long you think we are now actually starting to argue a whole new set of things.
02:35:20They're scary, they're weird, the implications are massive, but we have graduated problems.
02:35:25We have graduated from me stealing all of your things so that I, my family can survive to a place where we now actually have to figure out how we can all collectively prosper.
02:35:36And we can use these words that are crazy.
02:35:38And we can, again, I wear these glasses that are funny.
02:35:41And, and, uh, although this doesn't really accentuate my nose, I suppose, I guess people can see straight through that one.
02:35:49And, uh, and, uh, I, I do think I'm not, I don't want to make it, I don't want to make it play playful.
02:35:58I guess I just did.
02:35:59I do actually want to challenge that you, what you said, which is like, what we need is for people not to be scared of this moment, but to rise to the occasion.
02:36:06And I think the risk in all of this is that the safety debates specifically, but others give people a sense of cynicism and fatalism.
02:36:13Like if I could leave anyone on earth, like if I died tomorrow, the one thing that I would want to, to be read at, at a, at a eulogy was that he wanted people to believe that their decision and choices and ideas mattered.
02:36:23That like, you do not have to go quietly into the night.
02:36:26Just because you are not responsible for AI safety doesn't mean it will just happen to you.
02:36:30And that communities, I think the resurrection and rejuvenation that we crave will happen at the most local levels.
02:36:35I think it will happen in the dining room tables.
02:36:37I think it will happen in city centers and town halls.
02:36:40And I think it will happen in large part because people reject that garbage that like the AI slop is in some way, this like weird gift that we are observing a bunch of trash that we don't want.
02:36:51And we are like collectively sort of trying to piece together what it means.
02:36:55But we, we do desperately need to your point, people to actually see this as an opportunity where their, their, their choices matter because the real enemy in this moment is fatalism.
02:37:06It's cynicism, which has become fairly pervasive and that the both political parties spark.
02:37:11My, my, my thinking actually in all this, I think the next conflict is not between two nation states.
02:37:16I think it's between people in a state.
02:37:17I think people may at some point realize that we have technology that could make everything get a whole lot better.
02:37:23And we have a pernicious ruling class.
02:37:24They won't let it happen.
02:37:25And that is, that to me is one of the outcomes.
02:37:29Actually, it's the opposite of this authoritarianism.
02:37:31It's this other uprising.
02:37:33It's an uprising where people go, wait a second, we could have a utopia.
02:37:36Why aren't you building it?
02:37:37And people clamor for a much better world that could be promised to them because of the technology that, that we've built, that we aren't, that we aren't utilizing.
02:37:45I think that, that to me is like one of these like potentialities of, of, of high agency, even high morality, high agency.
02:37:52There's this weird uprising outcome.
02:37:54If you were to leave people that have listened to this with some things to keep an eye on, some questions to sort of keep pondering some areas to be focused on, what would they be?
02:38:04Cause there's this conversation sprawling and I'm friends with super smart people, many of your friends, and I try to keep a handle on it and don't really know what I should be paying attention to, what I should be focusing on.
02:38:14What do you, what do you think people should be focused on over the next year?
02:38:19Yeah.
02:38:19Let's say that's a forever time in this.
02:38:21I mean, I would say, keep an eye on how coordination efforts start to manifest.
02:38:30I think, as we discussed, we're at a very, actually positive moment right now.
02:38:34The most, the most positive I've been.
02:38:36Largely just because of that one letter.
02:38:37Not just because of the letter and just even like Elon just did his interview with The Economist where he mentioned, he's like, oh, actually, yeah, we, we, the leaders and I, like, I thought all the tech leaders hated each other.
02:38:47They seemed like they did.
02:38:48You know, Sam and Elon, I mean, I don't know if they've made up yet, but you know, they were at each other's throats.
02:38:52They all seem to hate each other and now apparently they're having phone calls or they're planning to have phone calls with each other on a, on a semi-regular basis.
02:39:00That is a huge step that really is.
02:39:03And far less embarrassing.
02:39:05Yeah, thank God.
02:39:06Like it sounds, I'm hoping they're just starting to all grow up a little bit.
02:39:09And WhatsApp chat versus subtweeting each other.
02:39:11Get me in the WhatsApp chat.
02:39:13Yeah, let's, let's hope.
02:39:14Um, so I think keep an eye on that.
02:39:16Um, and then just sort of more of a zoomed out sort of perspective.
02:39:19I always try and think, you know, again, we're trying to avoid oversimplifications of putting everything down onto like, what is, what is a singular sort of trade-off that we're facing here?
02:39:28But I do think these on a very meta level, what we're seeing is a kind of a battle between the forces of centralization and decentralization.
02:39:37And so it's sometimes helpful to just sort of see things like, oh, what is, is this a force?
02:39:43You know, how is that?
02:39:45How is decentralization trying to manifest here?
02:39:47Oh, how is centralization trying to manifest here?
02:39:49Um, I always think that's an interesting lens to, to look through.
02:39:52Um, but yeah.
02:39:53Um, but yeah, and I'll come back to my point of, this is an all hands on deck situation.
02:39:58And however, sort of distant you might feel from this topic, it does affect you.
02:40:04And so just keep thinking about it.
02:40:06Any final thoughts?
02:40:08Anything else you want people to pay attention to?
02:40:10The dining room table is such an important place that we should recapture.
02:40:15And if we start by recapturing meals and dining room table, it is as simple as possible.
02:40:19But it is, it takes, it requires bravery and courage to tell people to put away their devices.
02:40:23And if you do that, you will be deeply inspired by the conversations you have, and you will feel better on the other side of it.
02:40:28And that will actually inspire you to start thinking about the ways in which the world could be better.
02:40:32And that cycle would be pretty powerful.
02:40:34I also think we should, people should pay very close attention to the world that their leaders describe wanting to build.
02:40:40And I don't mean the technologists because we didn't elect those people.
02:40:43I'm glad the technologist is building technology.
02:40:45I don't really care about their ideas for the future of living.
02:40:48I care about what our elected officials care about our world looks like.
02:40:51And I think we've let them off the hook by describing absolutely nothing.
02:40:54And we should pay a lot closer attention to what they actually describe wanting and start electing people to describe the future that we want our kids to live in.
02:41:02Hmm.
02:41:03Final thoughts, dude?
02:41:05My hope is that the dining room table is not just the good world that we're fighting for, but also the way that we get there.
02:41:15Like, I think even if AI was not happening to us right now, I would look around and be worried about how informed voters are, about how much people trust their politicians.
02:41:28When I look at the fact that there are so many hard decisions coming down the pike, it's tempting to notice the state of our discourse and feel despair.
02:41:37But I think it's also cool that we can't really imagine a world where we get through this without addressing all of those other problems, too.
02:41:46Without saying, okay, let's actually start from square one and think about how do we want our societies to be organized?
02:41:53How important are our local communities?
02:41:55So I hope this is a crisis that leads us to finally take seriously what it means to be civically involved in a way that we could sort of get away with not doing before.
02:42:07But I don't think we should take for granted that that's going to happen by default.
02:42:10I'm just like grateful that we have extremely smart people thinking about this and hopefully planning to see it a bit more that that's a choice we have to make.
02:42:17Okay.
02:42:18You guys are all, I appreciate you all.
02:42:20Thank you for coming.
02:42:21Thanks for having us.
02:42:22Bye everybody.
02:42:23Congratulations.
02:42:24You made it to the end of a podcast episode without dying.
02:42:27Now, here's another one.
02:42:29Go on.
02:42:30Watch it.
Community Posts
No posts yet. Be the first to write about this video!
Write about this video