Wojciech Zaremba: OpenAI Codex, GPT-3, Robotics, and the Future of AI

Lex Fridman (02:19.880)

life, more kind of valuable, and that means that we should more appreciate it.

Lex Fridman (02:24.840)

Have we always been alone?

Lex Fridman (02:26.000)

So what's your intuition about our galaxy, our universe?

Wojciech Zaremba (02:29.040)

Is it just sprinkled with graveyards of intelligent civilizations, or are

Lex Fridman (02:34.160)

we truly, is life, intelligent life, truly unique?

Wojciech Zaremba (02:37.920)

At the moment, my belief that it is unique, but I would say I could also,

Lex Fridman (02:42.320)

you know, there was like some footage released with UFO objects, which makes

Wojciech Zaremba (02:47.640)

me actually doubt my own belief.

Lex Fridman (02:49.560)

Yes.

Wojciech Zaremba (02:51.040)

Yeah, I can tell you one crazy answer that I have heard.

Lex Fridman (02:53.960)

Yes.

Wojciech Zaremba (02:55.280)

So, apparently, when you look actually at the limits of computation, you

Lex Fridman (03:00.520)

can compute more if the temperature of the universe would drop.

Wojciech Zaremba (03:06.080)

Temperature of the universe would drop down.

Lex Fridman (03:09.720)

So one of the things that aliens might want to do if they are truly optimizing

Wojciech Zaremba (03:15.160)

to maximize amount of compute, which, you know, maybe can lead to, or let's

Lex Fridman (03:18.800)

say simulations or so, it's instead of wasting current entropy of the

Wojciech Zaremba (03:24.120)

universe, because, you know, we, by living, we are actually somewhat

Lex Fridman (03:27.080)

wasting entropy, then you can wait for the universe to cool down such that

Wojciech Zaremba (03:32.080)

you have more computation.

Lex Fridman (03:33.360)

So that's kind of a funny answer.

Wojciech Zaremba (03:34.560)

I'm not sure if I believe in it, but that would be one of the

Lex Fridman (03:37.400)

reasons why you don't see aliens.

Wojciech Zaremba (03:39.760)

It's also possible to some people say that maybe there is not that much

Lex Fridman (03:44.680)

point in actually going to other galaxies if you can go inwards.

Lex Fridman (03:49.400)

So there is no limits of what could be an experience if we could, you

Lex Fridman (03:54.440)

know, connect machines to our brains while there are still some limits

Wojciech Zaremba (03:58.600)

if we want to explore the universe.

Lex Fridman (03:59.880)

Yeah, there could be a lot of ways to go inwards too.

Wojciech Zaremba (04:04.960)

Once you figure out some aspect of physics, we haven't figured out yet.

Lex Fridman (04:08.960)

Maybe you can travel to different dimensions.

Wojciech Zaremba (04:11.000)

I mean, travel in three dimensional space may not be the most fun kind of travel.

Lex Fridman (04:19.080)

There may be like just a huge amount of different ways to travel and it

Wojciech Zaremba (04:22.680)

doesn't require a spaceship going slowly in 3d space to space time.

Lex Fridman (04:28.200)

It also feels, you know, one of the problems is that speed of light

Wojciech Zaremba (04:31.920)

is low and the universe is vast.

Lex Fridman (04:34.760)

And it seems that actually most likely if we want to travel very far, then

Wojciech Zaremba (04:42.360)

we would, instead of actually sending spaceships with humans that weight a

Lex Fridman (04:46.640)

lot, we would send something similar to what Yuri Miller is working on.

Wojciech Zaremba (04:51.000)

These are like a huge sail, which is at first powered or there is a shot of

Lex Fridman (04:56.200)

laser from an air and it can propel it to quarter of speed of light and sail

Wojciech Zaremba (05:02.400)

itself contains a few grams of equipment.

Lex Fridman (05:07.200)

And that might be the way to actually transport matter through universe.

Lex Fridman (05:12.280)

But then when you think what would it mean for humans, it means that we would

Lex Fridman (05:16.160)

need to actually put their 3d printer and, you know, 3d print the human on

Wojciech Zaremba (05:20.080)

other planet, I don't know, play them YouTube or let's say, or like a 3d

Lex Fridman (05:24.400)

print like huge human right away, or maybe a womb or so, um, yeah.

Wojciech Zaremba (05:28.960)

With our current techniques of archeology, if, if, if a civilization

Lex Fridman (05:34.080)

was born and died, uh, long, long enough ago on earth, we wouldn't be able to

Wojciech Zaremba (05:39.560)

tell, and so that makes me really sad.

Lex Fridman (05:43.280)

And so I think about earth in that same way.

Lex Fridman (05:45.520)

How can we leave some remnants if we do destroy ourselves?

Lex Fridman (05:50.040)

How can we leave remnants for aliens in the future to discover?

Wojciech Zaremba (05:53.320)

Like, here's some nice stuff we've done, like Wikipedia and YouTube.

Lex Fridman (05:57.600)

Do we have it like in a satellite orbiting earth with a hard drive?

Lex Fridman (06:02.920)

Like, how, how do we say, how do we back up human civilization?

Lex Fridman (06:07.560)

Uh, the good parts or all of it is good parts so that, uh, it can be

Wojciech Zaremba (06:13.440)

preserved longer than our bodies can.

Lex Fridman (06:15.920)

That's a, that's kind of, um, it's a difficult question.

Wojciech Zaremba (06:19.720)

It also requires the difficult acceptance of the fact that we may die.

Lex Fridman (06:24.040)

And if we die, we may die suddenly as a civilization.

Lex Fridman (06:28.680)

So let's see, I think it kind of depends on the cataclysm.

Lex Fridman (06:32.480)

We have observed in other parts of the universe that birds of gamma rays, uh,

Wojciech Zaremba (06:38.160)

these are, uh, high energy, uh, rays of light that actually can

Lex Fridman (06:42.800)

apparently kill entire galaxy.

Lex Fridman (06:44.440)

So there might be actually nothing, even to, nothing to protect us from it.

Lex Fridman (06:48.960)

I'm also, and I'm looking actually at the past civilizations.

Lex Fridman (06:51.560)

So it's like Aztecs or so they disappear from the surface of the earth.

Lex Fridman (06:56.880)

And one can ask, why is it the case?

Lex Fridman (07:00.240)

And the way I'm thinking about it is, you know, that definitely they had some

Lex Fridman (07:06.520)

problem that they couldn't solve and maybe there was a flood and all of a

Wojciech Zaremba (07:10.720)

sudden they couldn't drink, uh, there was no potable water and they all died.

Lex Fridman (07:15.040)

And, um, I think that, uh, so far the best solution to such a problems is I

Wojciech Zaremba (07:24.200)

guess, technology, so, I mean, if they would know that you can just boil

Lex Fridman (07:27.960)

water and then drink it after, then that would save their civilization.

Lex Fridman (07:31.920)

And even now, when we look actually at the current pandemic, it seems

Lex Fridman (07:36.040)

that there, once again, actually science comes to rest.

Lex Fridman (07:38.680)

And somehow science increases size of the action space.

Lex Fridman (07:42.440)

And I think that's a good thing.

Wojciech Zaremba (07:44.600)

Yeah.

Lex Fridman (07:44.800)

But nature has a vastly larger action space, but still it might be a good thing

Wojciech Zaremba (07:51.440)

for us to keep on increasing action space.

Lex Fridman (07:55.080)

Okay.

Wojciech Zaremba (07:56.200)

Uh, looking at past civilizations.

Lex Fridman (07:58.000)

Yes.

Lex Fridman (07:58.920)

But looking at the destruction of human civilization, perhaps expanding the

Lex Fridman (08:04.560)

action space will add, um, actions that are easily acted upon, easily executed

Lex Fridman (08:12.920)

and as a result, destroy us.

Lex Fridman (08:15.960)

So let's see, I was pondering, uh, why actually even, uh, we have

Wojciech Zaremba (08:21.800)

negative impact on the, uh, globe.

Lex Fridman (08:24.200)

Because, you know, if you ask every single individual, they

Wojciech Zaremba (08:27.680)

would like to have clean air.

Lex Fridman (08:29.720)

They would like healthy planet, but somehow it's not.

Wojciech Zaremba (08:32.320)

It's not the case that as a collective, we are not going in this direction.

Lex Fridman (08:36.840)

I think that there exists very powerful system to describe what we value.

Wojciech Zaremba (08:41.080)

That's capitalism.

Lex Fridman (08:42.000)

It assigns actually monetary values to various activities.

Wojciech Zaremba (08:45.760)

At the moment, the problem in the current system is that there's

Lex Fridman (08:49.000)

some things which we value.

Wojciech Zaremba (08:50.680)

There is no cost assigned to it.

Lex Fridman (08:52.320)

So even though we value clean air, or maybe we also, uh, value, uh,

Wojciech Zaremba (09:00.240)

value lack of destruction on, let's say internet or so at the moment, these

Lex Fridman (09:06.000)

quantities, you know, companies, corporations can pollute them, uh, for free.

Lex Fridman (09:11.680)

So in some sense, I wished or like, and that's, I guess, purpose of politics

Lex Fridman (09:20.040)

to, to align the incentive systems.

Lex Fridman (09:23.000)

And we are kind of maybe even moving in this direction.

Lex Fridman (09:25.680)

The first issue is even to be able to measure the things that we value.

Wojciech Zaremba (09:28.920)

Then we can actually assign the monetary value to them.

Lex Fridman (09:32.720)

Yeah.

Lex Fridman (09:32.840)

And that's, so it's getting the data and also probably through technology,

Lex Fridman (09:38.040)

enabling people to vote and to move money around in a way that is aligned

Wojciech Zaremba (09:44.640)

with their values, and that's very much a technology question.

Lex Fridman (09:48.720)

So like having one president and Congress and voting that happens every four years

Wojciech Zaremba (09:55.880)

or something like that, that's a very outdated idea that could be some

Lex Fridman (09:59.720)

technological improvements to that kind of idea.

Lex Fridman (10:02.080)

So I'm thinking from time to time about these topics, but it's also feels to me

Lex Fridman (10:06.400)

that it's, it's a little bit like, uh, it's hard for me to actually make

Wojciech Zaremba (10:10.240)

correct predictions.

Lex Fridman (10:11.160)

What is the appropriate thing to do?

Wojciech Zaremba (10:13.120)

I extremely trust, uh, Sam Altman, our CEO on these topics here, um, like, uh,

Lex Fridman (10:20.560)

I'm more on the side of being, I guess, naive hippie.

Wojciech Zaremba (10:24.200)

That, uh, yeah, that's your life philosophy.

Lex Fridman (10:29.400)

Um, well, like I think self doubt and, uh, I think hippie implies optimism.

Wojciech Zaremba (10:37.480)

Those, those two things are pretty, pretty good way to operate.

Lex Fridman (10:41.080)

I mean, still, it is hard for me to actually understand how the politics

Wojciech Zaremba (10:46.920)

works or like, uh, how this, like, uh, exactly how the things would play out.

Lex Fridman (10:51.440)

And Sam is, uh, really excellent with it.

Lex Fridman (10:54.120)

What do you think is rarest in the universe?

Lex Fridman (10:56.560)

You said we might be alone.

Wojciech Zaremba (10:58.560)

What's hardest to build is another engineering way to ask that life,

Lex Fridman (11:03.360)

intelligence or consciousness.

Lex Fridman (11:05.680)

So like you said that we might be alone, which is the thing that's hardest to get

Lex Fridman (11:11.000)

to, is it just the origin of life?

Lex Fridman (11:13.680)

Is it the origin of intelligence?

Lex Fridman (11:15.640)

Is it the origin of consciousness?

Wojciech Zaremba (11:17.800)

So, um, let me at first explain to you my kind of mental model, what I think

Lex Fridman (11:23.080)

is needed for life to appear.

Wojciech Zaremba (11:25.560)

Um, so I imagine that at some point there was this primordial, uh, soup of, uh,

Lex Fridman (11:32.840)

amino acids and maybe some proteins in the ocean and, uh, you know, some

Wojciech Zaremba (11:38.240)

proteins were turning into some other proteins through reaction and, uh, you

Lex Fridman (11:42.640)

can also, uh, you know, you can, you know, you can, you know, you can

Wojciech Zaremba (11:46.480)

and, uh, you can almost think about this, uh, cycle of what, uh, turns into what

Lex Fridman (11:52.200)

as there is a graph essentially describing which substance turns into

Wojciech Zaremba (11:55.840)

some other substance and essentially life means that all of a sudden in the graph

Lex Fridman (12:00.480)

has been created that cycle such that the same thing keeps on happening over

Lex Fridman (12:04.640)

and over again, that's what is needed for life to happen.

Lex Fridman (12:07.280)

And in some sense, you can think almost that you have this gigantic graph and it

Wojciech Zaremba (12:12.000)

needs like a sufficient number of edges for the cycle to appear.

Lex Fridman (12:15.400)

Um, then, um, from perspective of intelligence and consciousness, uh, my

Wojciech Zaremba (12:21.760)

current intuition is that they might be quite intertwined.

Lex Fridman (12:26.280)

First of all, it might not be that it's like a binary thing that you

Wojciech Zaremba (12:29.160)

have intelligence or consciousness.

Lex Fridman (12:30.720)

It seems to be, uh, uh, more, uh, continuous component.

Wojciech Zaremba (12:36.800)

Let's see, if we look for instance on the event networks, uh, recognizing

Lex Fridman (12:41.320)

images and people are able to show that the activations of these networks

Wojciech Zaremba (12:46.120)

correlate very strongly, uh, with activations in visual cortex, uh, of

Lex Fridman (12:51.760)

some monkeys, the same seems to be true about language models.

Wojciech Zaremba (12:56.080)

Um, also if you, for instance, um, look, um, if you train agent in, um, 3d

Lex Fridman (13:04.320)

world, um, at first, you know, it, it, it, it barely recognizes what is going

Wojciech Zaremba (13:09.880)

on over the time, it kind of recognizes foreground from a background over the

Lex Fridman (13:14.640)

time, it kind of knows where there is a foot, uh, and it just follows it.

Wojciech Zaremba (13:18.880)

Um, over the time it actually starts having a 3d perception.

Lex Fridman (13:22.800)

So it is possible for instance, to look inside of the head of an agent and ask,

Lex Fridman (13:27.480)

what would it see if it looks to the right?

Lex Fridman (13:29.760)

And the crazy thing is, you know, initially when the agents are barely

Wojciech Zaremba (13:33.600)

trained, that these predictions are pretty bad over the time they become

Lex Fridman (13:37.200)

better and better, you can still see that if you ask what happens when the

Wojciech Zaremba (13:42.840)

head is turned by 360 degrees for some time, they think that the different

Lex Fridman (13:47.400)

thing appears and then at some stage they understand actually that the same

Wojciech Zaremba (13:51.440)

thing supposed to appear.

Lex Fridman (13:52.640)

So they get that understanding of 3d structure.

Wojciech Zaremba (13:55.760)

It's also, you know, very likely that they have inside some level of, of like

Lex Fridman (14:01.960)

a symbolic reasoning, like a particular, these symbols for other agents.

Lex Fridman (14:06.720)

So when you look at DOTA agents, they collaborate together and, uh, and, uh,

Lex Fridman (14:13.800)

no, they, they, they, they have some anticipation of, uh, if, if they would

Wojciech Zaremba (14:17.880)

win battle, they have some, some expectations with respect to other

Lex Fridman (14:21.720)

agents.

Wojciech Zaremba (14:22.360)

I might be, you know, too much anthropomorphizing, um, the, the, the,

Lex Fridman (14:26.160)

how the things look, look, look for me, but then the fact that they have a

Wojciech Zaremba (14:31.400)

symbol for other agents, uh, makes me believe that, uh, at some stage as the,

Lex Fridman (14:37.440)

uh, you know, as they are optimizing for skills, they would have also symbol to

Wojciech Zaremba (14:41.400)

describe themselves.

Lex Fridman (14:43.360)

Uh, this is like a very useful symbol to have.

Lex Fridman (14:46.400)

And this particularity, I would call it like a self consciousness or self

Lex Fridman (14:50.280)

awareness, uh, and, uh, still it might be different from the consciousness.

Lex Fridman (14:55.280)

So I guess the, the way how I'm understanding the word consciousness,

Lex Fridman (14:59.800)

I'd say the experience of drinking a coffee or let's say experience of being

Wojciech Zaremba (15:03.120)

a bat, that's the meaning of the word consciousness.

Lex Fridman (15:06.280)

It doesn't mean to be awake.

Wojciech Zaremba (15:07.760)

Uh, yeah, it feels, it might be also somewhat related to memory and

Lex Fridman (15:13.480)

recurrent connections.

Wojciech Zaremba (15:14.840)

So, um, it's kind of like, if you look at anesthetic drugs, they might be, uh,

Wojciech Zaremba (15:21.480)

uh, like, uh, that they essentially, they, they disturb, uh, uh, brainwaves, uh, such

Wojciech Zaremba (15:30.200)

that, um, maybe memories, not, not form.

Lex Fridman (15:33.960)

And so there's a lessening of consciousness when you do that.

Wojciech Zaremba (15:37.280)

Correct.

Lex Fridman (15:37.840)

And so that's the one way to intuit what is consciousness.

Wojciech Zaremba (15:41.040)

There's also kind of another element here.

Lex Fridman (15:45.360)

It could be that it's, you know, this kind of self awareness

Wojciech Zaremba (15:49.480)

module that you described, plus the actual subjective experience is a

Lex Fridman (15:56.360)

storytelling module that tells us a story about, uh, what we're experiencing.

Wojciech Zaremba (16:05.160)

The crazy thing.

Lex Fridman (16:06.960)

So let's say, I mean, in meditation, they teach people not to speak

Wojciech Zaremba (16:11.200)

story inside of their head.

Lex Fridman (16:12.800)

And there is also some fraction of population who doesn't have actually

Wojciech Zaremba (16:17.280)

a narrator, I know people who don't have a narrator and, you know, they have

Lex Fridman (16:22.120)

to use external people in order to, um, kind of solve tasks that

Wojciech Zaremba (16:27.760)

require internal narrator.

Lex Fridman (16:30.360)

Um, so it seems that it's possible to have the experience without the talk.

Lex Fridman (16:37.440)

What are we talking about when we talk about the internal narrator?

Lex Fridman (16:41.080)

Is that the voice when you're like, yeah, I thought that that's what you are

Wojciech Zaremba (16:44.000)

referring to while I was referring more on the, like, not an actual voice.

Lex Fridman (16:51.120)

I meant like, there's some kind of like subjective experience feels like it's.

Wojciech Zaremba (17:00.400)

It's fundamentally about storytelling to ourselves.

Lex Fridman (17:04.560)

It feels like, like the feeling is a story that is much, uh, much

Wojciech Zaremba (17:13.760)

simpler abstraction than the raw sensory information.

Lex Fridman (17:17.400)

So there feels like it's a very high level of abstraction that, uh, is useful

Wojciech Zaremba (17:23.960)

for me to feel like entity in this world.

Lex Fridman (17:27.280)

M most useful aspect of it is that because I'm conscious, I think there's

Wojciech Zaremba (17:35.920)

an intricate connection to me, not wanting to die.

Lex Fridman (17:39.400)

So like, it's a useful hack to really prioritize not dying, like those

Wojciech Zaremba (17:46.160)

seem to be somehow connected.

Lex Fridman (17:47.560)

So I'm telling the story of like, it's rich.

Wojciech Zaremba (17:50.440)

He feels like something to be me and the fact that me exists in this world.

Lex Fridman (17:55.200)

I want to preserve me.

Lex Fridman (17:56.920)

And so that makes it a useful agent hack.

Lex Fridman (17:59.280)

So I will just refer maybe to that first part, as you said, about that kind

Wojciech Zaremba (18:03.720)

of story of describing who you are.

Lex Fridman (18:05.800)

Um, I was, uh, thinking about that even, so, you know, obviously I'm, I, I like

Wojciech Zaremba (18:13.600)

thinking about consciousness, uh, I like thinking about AI as well, and I'm trying

Lex Fridman (18:18.280)

to see analogies of these things in AI, what would it correspond to?

Wojciech Zaremba (18:22.520)

So, um, um, you know, open AI train, uh, uh, a model called GPT, uh, which, uh,

Wojciech Zaremba (18:34.360)

can generate, uh, pretty, I'm using texts on arbitrary topic and, um, um, and one

Wojciech Zaremba (18:42.280)

way to control GPT is, uh, by putting into prefix at the beginning of the text, some

Wojciech Zaremba (18:49.400)

information, what would be the story about, uh, you can have even chat with, uh, uh,

Wojciech Zaremba (18:55.400)

you know, with GPT by saying that the chat is with Lex or Elon Musk or so, and, uh,

Wojciech Zaremba (19:01.200)

GPT would just pretend to be you or Elon Musk or so, and, uh, uh, it almost feels

Wojciech Zaremba (19:08.720)

that this, uh, story that we give ourselves to describe our life, it's almost like, uh,

Lex Fridman (19:15.360)

things that you put into context of GPT.

Wojciech Zaremba (19:17.360)

Yeah.

Wojciech Zaremba (19:17.680)

The primary, it's the, and so, but the context we provide to GPT is, uh, is multimodal.

Wojciech Zaremba (19:25.520)

It's more so GPT itself is multimodal.

Wojciech Zaremba (19:27.760)

GPT itself, uh, hasn't learned actually from experience of single human, but from the

Wojciech Zaremba (19:33.120)

experience of humanity, it's a chameleon.

Wojciech Zaremba (19:35.520)

You can turn it into anything and in some sense, by providing context, um, it, you

Wojciech Zaremba (19:42.000)

know, behaves as the thing that you wanted it to be.

Lex Fridman (19:45.160)

Um, it's interesting that the, you know, people have a stories of who they are.

Wojciech Zaremba (19:50.520)

And, uh, as you said, these stories, they help them to operate in the world.

Wojciech Zaremba (19:54.400)

Um, but it's also, you know, interesting, I guess, various people find it out through

Wojciech Zaremba (19:59.800)

meditation or so that, uh, there might be some patterns that you have learned when

Wojciech Zaremba (1:00:02.400)

resolve these, uh, inner store stories or like inner traumas, then once there is

Wojciech Zaremba (1:00:08.960)

nothing, uh, left that default, uh, state of human mind is extremely peaceful and

Wojciech Zaremba (1:00:16.040)

happy, extreme, like, uh, some sense it, it feels that the, it feels at least to

Wojciech Zaremba (1:00:24.520)

me that way, how, when I was a child that I can look at any object and it's very

Lex Fridman (1:00:30.400)

beautiful, I have a lot of curiosity about the simple things and that's where

Wojciech Zaremba (1:00:34.680)

the usual meditation takes me.

Lex Fridman (1:00:37.440)

Are you, what are you experiencing?

Wojciech Zaremba (1:00:40.040)

Are you just taking in simple sensory information and they're just enjoying

Lex Fridman (1:00:45.560)

the rawness of that sensory information?

Lex Fridman (1:00:48.120)

So there's no, there's no memories or all that kind of stuff.

Lex Fridman (1:00:52.000)

You're just enjoying being.

Wojciech Zaremba (1:00:54.960)

Yeah, pretty much.

Lex Fridman (1:00:55.920)

I mean, still there is, uh, that it's, it's thoughts are slowing down.

Wojciech Zaremba (1:01:00.880)

Sometimes they pop up, but it's also somehow the extended meditation takes you

Lex Fridman (1:01:06.080)

to the space that they are way more friendly, way more positive.

Wojciech Zaremba (1:01:11.400)

Um, there is also this, uh, this thing that, uh, we've, it almost feels that the.

Lex Fridman (1:01:19.240)

It almost feels that the, we are constantly getting a little bit of a reward

Wojciech Zaremba (1:01:24.240)

function and we are just spreading this reward function on various activities.

Lex Fridman (1:01:28.560)

But if you'll stay still for extended period of time, it kind of accumulates,

Wojciech Zaremba (1:01:33.000)

accumulates, accumulates, and, uh, there is a, there is a sense, there is a sense

Wojciech Zaremba (1:01:38.800)

that some point it passes some threshold and it feels as drop is falling into kind

Wojciech Zaremba (1:01:46.080)

of ocean of love and this, and that's like, uh, this is like a very pleasant.

Lex Fridman (1:01:49.920)

And that's, I'm saying like, uh, that corresponds to the subjective experience.

Wojciech Zaremba (1:01:54.920)

Some people, uh, I guess in spiritual community, they describe it that that's

Lex Fridman (1:02:01.440)

the reality, and I would say, I believe that they're like, uh, all sorts of

Wojciech Zaremba (1:02:04.840)

subjective experience that one can have.

Lex Fridman (1:02:07.320)

And, uh, I believe that for instance, meditation might take you to the

Wojciech Zaremba (1:02:11.720)

subjective experiences with the subject.

Lex Fridman (1:02:13.640)

Vision might take you to the subjective experiences, which are

Wojciech Zaremba (1:02:16.480)

very pleasant, collaborative.

Lex Fridman (1:02:18.080)

And I would like a word to move toward a more collaborative, uh, uh, place.

Wojciech Zaremba (1:02:24.880)

Yeah.

Lex Fridman (1:02:25.240)

I would say that's very pleasant and I enjoy doing stuff like that.

Wojciech Zaremba (1:02:28.440)

I, um, I wonder how that maps to your, uh, mathematical model of love with, uh,

Wojciech Zaremba (1:02:35.040)

the reward function, combining a bunch of things, it seems like our life then is

Wojciech Zaremba (1:02:42.280)

just, we have this reward function and we're accumulating a bunch of stuff

Lex Fridman (1:02:46.120)

in it with weights, it's like, um, like multi objective and what meditation

Wojciech Zaremba (1:02:55.000)

is, is you just remove them, remove them until the weight on one, uh, or

Lex Fridman (1:03:01.040)

just a few is very high and that's where the pleasure comes from.

Wojciech Zaremba (1:03:05.200)

Yeah.

Lex Fridman (1:03:05.480)

So something similar, how I'm thinking about this.

Lex Fridman (1:03:08.200)

So I told you that there is this like, uh, that there is a story of who you are.

Lex Fridman (1:03:14.120)

And I think almost about it as a, you know, text prepended to GPT.

Wojciech Zaremba (1:03:20.400)

Yeah.

Lex Fridman (1:03:21.000)

And, uh, some people refer to it as ego.

Wojciech Zaremba (1:03:24.120)

Okay.

Lex Fridman (1:03:24.600)

There's like a story who, who, who you are.

Wojciech Zaremba (1:03:27.560)

Okay.

Lex Fridman (1:03:28.000)

So ego is the prompt for GPT three or GPT.

Wojciech Zaremba (1:03:31.360)

Yes.

Lex Fridman (1:03:31.600)

Yes.

Lex Fridman (1:03:31.760)

And that's description of you.

Lex Fridman (1:03:32.960)

And then with meditation, you can get to the point that actually you experience

Wojciech Zaremba (1:03:37.080)

things without the prompt and you experience things like as they are, you

Lex Fridman (1:03:42.480)

are not biased over the description, how they supposed to be, uh, that's very

Wojciech Zaremba (1:03:47.040)

pleasant.

Lex Fridman (1:03:47.480)

And then we've respected the reward function.

Wojciech Zaremba (1:03:50.000)

Uh, it's possible to get to the point that the, there is the solution of self.

Lex Fridman (1:03:55.480)

And therefore you can say that the, or you're having a, your, or like a, your

Wojciech Zaremba (1:03:59.480)

brain attempts to simulate the reward function of everyone else or like

Wojciech Zaremba (1:04:03.320)

everything that's that there is this like a love, which feels like a oneness with

Wojciech Zaremba (1:04:07.120)

everything.

Lex Fridman (1:04:08.760)

And that's also, you know, very beautiful, very pleasant.

Wojciech Zaremba (1:04:11.440)

At some point you might have a lot of altruistic thoughts during that moment.

Lex Fridman (1:04:16.120)

And then the self, uh, always comes back.

Lex Fridman (1:04:19.240)

How would you recommend if somebody is interested in meditation, like a big

Lex Fridman (1:04:23.480)

thing to take on as a project, would you recommend a meditation retreat?

Lex Fridman (1:04:27.400)

How many days, what kind of thing would you recommend?

Lex Fridman (1:04:30.160)

I think that actually retreat is the way to go.

Wojciech Zaremba (1:04:32.560)

Um, it almost feels that, uh, um, as I said, like a meditation is a psychedelic,

Lex Fridman (1:04:39.000)

but, uh, when you take it in the small dose, you might barely feel it.

Wojciech Zaremba (1:04:43.280)

Once you get the high dose, actually you're going to feel it.

Lex Fridman (1:04:46.880)

Um, so even cold turkey, if you haven't really seriously meditated for a long

Wojciech Zaremba (1:04:51.800)

period of time, just go to a retreat.

Lex Fridman (1:04:53.920)

Yeah.

Lex Fridman (1:04:54.280)

How many days, how many days?

Lex Fridman (1:04:55.560)

Start weekend one weekend.

Lex Fridman (1:04:57.600)

So like two, three days.

Lex Fridman (1:04:58.800)

And it's like, uh, it's interesting that first or second day, it's hard.

Lex Fridman (1:05:03.520)

And at some point it becomes easy.

Lex Fridman (1:05:06.560)

There's a lot of seconds in a day.

Lex Fridman (1:05:08.520)

How hard is the meditation retreat just sitting there in a chair?

Lex Fridman (1:05:13.040)

So the thing is actually, it literally just depends on your, uh, on the,

Wojciech Zaremba (1:05:20.800)

your own framing, like if you are in the mindset that you are waiting for it to

Lex Fridman (1:05:24.560)

be over, or you are waiting for a Nirvana to happen, you are waiting

Wojciech Zaremba (1:05:28.720)

it will be very unpleasant.

Lex Fridman (1:05:30.680)

And in some sense, even the difficulty, it's not even in the lack of being

Wojciech Zaremba (1:05:36.480)

able to speak with others, like, uh, you're sitting there, your legs

Lex Fridman (1:05:40.360)

will hurt from sitting in terms of like the practical things.

Lex Fridman (1:05:44.480)

Do you experience kind of discomfort, like physical discomfort of just

Wojciech Zaremba (1:05:48.160)

sitting, like your, your butt being numb, your legs being sore, all that kind of

Lex Fridman (1:05:53.720)

stuff?

Lex Fridman (1:05:54.160)

Yes.

Wojciech Zaremba (1:05:54.520)

You experience it.

Lex Fridman (1:05:55.360)

And then the, the, they teach you to observe it rather.

Lex Fridman (1:05:59.320)

And it's like, uh, the crazy thing is you at first might have a feeling

Lex Fridman (1:06:03.280)

toward trying to escape it and that becomes very apparent that that's

Wojciech Zaremba (1:06:07.560)

extremely unpleasant.

Lex Fridman (1:06:09.120)

And then you just, just observe it.

Lex Fridman (1:06:11.840)

And then at some point it just becomes, uh, it just is, it's like, uh, I remember

Lex Fridman (1:06:18.720)

that we've, Ilya told me some time ago that, uh, you know, he takes a cold

Wojciech Zaremba (1:06:22.680)

shower and he's the mindset of taking a cold shower was to embrace suffering.

Lex Fridman (1:06:28.360)

Yeah.

Wojciech Zaremba (1:06:28.960)

Excellent.

Lex Fridman (1:06:29.680)

I do the same.

Lex Fridman (1:06:30.320)

This is your style?

Lex Fridman (1:06:31.240)

Yeah, it's my style.

Wojciech Zaremba (1:06:32.880)

I like this.

Lex Fridman (1:06:34.200)

So my style is actually, I also sometimes take cold showers.

Wojciech Zaremba (1:06:38.960)

It is purely observing how the water goes through my body, like a purely being

Lex Fridman (1:06:43.480)

present, not trying to escape from there.

Wojciech Zaremba (1:06:46.040)

Yeah.

Lex Fridman (1:06:46.800)

And I would say then it actually becomes pleasant.

Wojciech Zaremba (1:06:49.360)

It's not like, ah, well, that that's interesting.

Lex Fridman (1:06:52.200)

Um, I I'm also that mean that's, that's the way to deal with anything really

Wojciech Zaremba (1:06:57.520)

difficult, especially in the physical space is to observe it to say it's pleasant.

Lex Fridman (1:07:04.880)

Hmm.

Wojciech Zaremba (1:07:05.600)

It's a D I would use a different word.

Lex Fridman (1:07:08.480)

You're, um, you're accepting of the full beauty of reality.

Wojciech Zaremba (1:07:14.480)

I would say, cause say pleasant.

Lex Fridman (1:07:16.600)

But yeah, I mean, in some sense it is pleasant.

Wojciech Zaremba (1:07:19.560)

That's the only way to deal with a cold shower is to, to, uh, become an

Lex Fridman (1:07:24.200)

observer and to find joy in it.

Wojciech Zaremba (1:07:28.440)

Um, same with like really difficult, physical, um, exercise or like running

Wojciech Zaremba (1:07:32.920)

for a really long time, endurance events, just anytime you're, any kind of pain.

Wojciech Zaremba (1:07:38.040)

I think the only way to survive it is not to resist it is to observe it.

Lex Fridman (1:07:43.120)

You mentioned, you mentioned, um, you mentioned, um, you mentioned

Wojciech Zaremba (1:07:46.520)

Ilya, Ilya says, it's very, he's our chief scientist, but also

Lex Fridman (1:07:51.920)

he's very close friend of mine.

Wojciech Zaremba (1:07:53.600)

He cofounded open air with you.

Lex Fridman (1:07:56.280)

I've spoken with him a few times.

Wojciech Zaremba (1:07:58.440)

He's brilliant.

Lex Fridman (1:07:59.160)

I really enjoy talking to him.

Wojciech Zaremba (1:08:02.960)

His mind, just like yours works in fascinating ways.

Lex Fridman (1:08:06.960)

Now, both of you are not able to define deep learning simply.

Wojciech Zaremba (1:08:10.000)

Uh, what's it like having him as somebody you have technical discussions with on

Lex Fridman (1:08:15.880)

in the space of machine learning, deep learning, AI, but also life.

Lex Fridman (1:08:21.200)

What's it like when these two, um, agents get into a self play situation in a room?

Lex Fridman (1:08:29.000)

What's it like collaborating with him?

Lex Fridman (1:08:30.840)

So I believe that we have, uh, extreme, uh, respect to each other.

Lex Fridman (1:08:35.320)

So, uh, in, I love Ilya's insight, both like, uh, I guess about

Wojciech Zaremba (1:08:43.720)

consciousness, uh, life AI, but, uh, in terms of the, it's interesting to

Lex Fridman (1:08:49.480)

me, cause you're a brilliant, uh, Thinker in the space of machine

Wojciech Zaremba (1:08:56.080)

learning, like intuition, like digging deep in what works, what doesn't,

Lex Fridman (1:09:01.840)

why it works, why it doesn't, and so is Ilya.

Wojciech Zaremba (1:09:05.200)

I'm wondering if there's interesting deep discussions you've had with him in the

Lex Fridman (1:09:09.600)

past or disagreements that were very productive.

Lex Fridman (1:09:12.280)

So I can say, I also understood over the time, where are my strengths?

Lex Fridman (1:09:18.000)

So obviously we have plenty of AI discussions and, um, um, and do you

Wojciech Zaremba (1:09:24.240)

know, I myself have plenty of ideas, but like I consider Ilya, uh, what

Lex Fridman (1:09:29.440)

of the most prolific AI scientists in the entire world.

Wojciech Zaremba (1:09:33.160)

And, uh, I think that, um, I realized that maybe my super skill, um, is, uh,

Lex Fridman (1:09:40.000)

being able to bring people to collaborate together, that I have some level of

Wojciech Zaremba (1:09:43.800)

empathy that is unique in AI world.

Lex Fridman (1:09:46.760)

And that might come, you know, from either meditation, psychedelics, or

Wojciech Zaremba (1:09:50.800)

let's say I read just hundreds of books on this topic.

Lex Fridman (1:09:53.080)

So, and I also went through a journey of, you know, I developed a

Wojciech Zaremba (1:09:56.920)

lot of, uh, algorithms, so I think that maybe I can, that's my super human skill.

Lex Fridman (1:10:05.320)

Uh, Ilya is, uh, one of the best AI scientists, but then I'm pretty

Wojciech Zaremba (1:10:11.200)

good in assembling teams and I'm also not holding to people.

Lex Fridman (1:10:14.920)

Like I'm growing people and then people become managers at OpenAI.

Wojciech Zaremba (1:10:18.400)

I grew many of them, like a research managers.

Lex Fridman (1:10:20.680)

So you, you find, you find places where you're excellent and he finds like his,

Wojciech Zaremba (1:10:27.240)

his, his deep scientific insights is where he is and you find ways you can,

Lex Fridman (1:10:31.840)

the puzzle pieces fit together.

Wojciech Zaremba (1:10:33.600)

Correct.

Lex Fridman (1:10:33.920)

Like, uh, you know, ultimately, for instance, let's say Ilya, he doesn't

Wojciech Zaremba (1:10:37.680)

manage people, uh, that's not what he likes or so.

Lex Fridman (1:10:42.280)

Um, I like, I like hanging out with people.

Wojciech Zaremba (1:10:45.680)

By default, I'm an extrovert and I care about people.

Lex Fridman (1:10:48.200)

Oh, interesting. Okay. All right. Okay, cool.

Lex Fridman (1:10:50.880)

So that, that fits perfectly together.

Lex Fridman (1:10:52.920)

But I mean, uh, I also just like your intuition about various

Wojciech Zaremba (1:10:56.600)

problems in machine learning.

Lex Fridman (1:10:58.160)

He's definitely one I really enjoy.

Wojciech Zaremba (1:11:01.440)

I remember talking to him about something I was struggling with, which

Lex Fridman (1:11:06.800)

is coming up with a good model for pedestrians, for human beings across

Wojciech Zaremba (1:11:12.920)

the street in the context of autonomous vehicles, and I was like, okay,

Lex Fridman (1:11:16.800)

in the context of autonomous vehicles.

Lex Fridman (1:11:19.840)

And he immediately started to like formulate a framework within which you

Lex Fridman (1:11:24.400)

can evolve a model for pedestrians, like through self play, all that kind of

Wojciech Zaremba (1:11:29.040)

mechanisms, the depth of thought on a particular problem, especially problems

Lex Fridman (1:11:35.040)

he doesn't know anything about is, is fascinating to watch.

Wojciech Zaremba (1:11:38.560)

It makes you realize like, um, yeah, the, the, the limits of the, that the human

Lex Fridman (1:11:46.000)

intellect may be limitless, or it's just impressive to see a descendant of

Wojciech Zaremba (1:11:50.560)

ape come up with clever ideas.

Lex Fridman (1:11:52.640)

Yeah.

Wojciech Zaremba (1:11:53.000)

I mean, so even in the space of deep learning, when you look at various

Lex Fridman (1:11:56.920)

people, there are people now who invented some breakthroughs once, but

Wojciech Zaremba (1:12:03.680)

there are very few people who did it multiple times.

Lex Fridman (1:12:06.280)

And you can think if someone invented it once, that might be just a sheer luck.

Lex Fridman (1:12:11.680)

And if someone invented it multiple times, you know, if a probability of

Lex Fridman (1:12:15.160)

inventing it once is one over a million, then probability of inventing it twice

Wojciech Zaremba (1:12:19.080)

or three times would be one over a million square or, or to the power of

Lex Fridman (1:12:22.200)

three, which, which would be just impossible.

Lex Fridman (1:12:25.040)

So it literally means that it's, it's given that, uh, it's not the luck.

Lex Fridman (1:12:30.680)

Yeah.

Lex Fridman (1:12:30.920)

And Ilya is one of these few people who, uh, uh, who have, uh, a lot of

Lex Fridman (1:12:36.680)

these inventions in his arsenal.

Wojciech Zaremba (1:12:38.640)

It also feels that, um, you know, for instance, if you think about folks

Lex Fridman (1:12:42.800)

like Gauss or Euler, uh, you know, at first they read a lot of books and then

Wojciech Zaremba (1:12:49.760)

they did thinking and then they figure out math and that's how it feels with

Wojciech Zaremba (1:12:55.280)

Ilya, you know, at first he read stuff and then like he spent his thinking cycles.

Lex Fridman (1:13:01.000)

And that's a really good way to put it.

Lex Fridman (1:13:05.680)

When I talk to him, I, I see thinking.

Wojciech Zaremba (1:13:11.320)

He's actually thinking, like, he makes me realize that there's like deep

Lex Fridman (1:13:15.960)

thinking that the human mind can do.

Wojciech Zaremba (1:13:18.280)

Like most of us are not thinking deeply.

Lex Fridman (1:13:21.440)

Uh, like you really have to put in a lot of effort to think deeply.

Wojciech Zaremba (1:13:24.760)

Like I have to really put myself in a place where I think deeply about a

Lex Fridman (1:13:29.040)

problem, it takes a lot of effort.

Wojciech Zaremba (1:13:30.960)

It's like, uh, it's like an airplane taking off or something.

Lex Fridman (1:13:33.680)

You have to achieve deep focus.

Lex Fridman (1:13:35.640)

He he's just, uh, he's what is it?

Lex Fridman (1:13:38.560)

He said, what does it, his brain is like a vertical takeoff in

Wojciech Zaremba (1:13:43.600)

terms of airplane analogy.

Lex Fridman (1:13:45.320)

So it's interesting, but it, I mean, Cal Newport talks about

Wojciech Zaremba (1:13:49.520)

this as ideas of deep work.

Lex Fridman (1:13:51.880)

It's, you know, most of us don't work much at all in terms of like, like deeply

Wojciech Zaremba (1:13:57.400)

think about particular problems, whether it's a math engineering, all that kind

Lex Fridman (1:14:01.400)

of stuff, you want to go to that place often and that's real hard work.

Lex Fridman (1:14:06.480)

And some of us are better than others at that.

Lex Fridman (1:14:08.760)

So I think that the big piece has to do with actually even engineering

Wojciech Zaremba (1:14:13.040)

your environment that says that it's conducive to that.

Lex Fridman (1:14:15.840)

Yeah.

Wojciech Zaremba (1:14:16.040)

So, um, see both Ilya and I, uh, on the frequent basis, we kind of disconnect

Wojciech Zaremba (1:14:22.480)

ourselves from the world in order to be able to do extensive amount of thinking.

Wojciech Zaremba (1:14:26.920)

Yes.

Lex Fridman (1:14:27.480)

So Ilya usually, he just, uh, leaves iPad at hand.

Wojciech Zaremba (1:14:33.400)

He loves his iPad.

Lex Fridman (1:14:34.400)

And, uh, for me, I'm even sometimes, you know, just going for a few days

Wojciech Zaremba (1:14:39.320)

to different location to Airbnb, I'm turning off my phone and there is no

Lex Fridman (1:14:44.520)

access to me and, uh, that's extremely important for me to be able to actually

Wojciech Zaremba (1:14:51.040)

just formulate new thoughts, to do deep work rather than to be reactive.

Lex Fridman (1:14:55.400)

And the, the, the older I am, the more of these random tasks are at hand.

Wojciech Zaremba (1:15:00.440)

Before I go on to that, uh, thread, let me return to our friend, GPT.

Lex Fridman (1:15:06.400)

And let me ask you another ridiculously big question.

Lex Fridman (1:15:09.440)

Can you give an overview of what GPT three is, or like you say in

Lex Fridman (1:15:13.840)

your Twitter bio, GPT N plus one, how it works and why it works.

Wojciech Zaremba (1:15:21.120)

So, um, GPT three is a humongous neural network.

Lex Fridman (1:15:25.640)

Um, let's assume that we know what is neural network, the definition, and it

Wojciech Zaremba (1:15:30.760)

is trained on the entire internet and just to predict next word.

Lex Fridman (1:15:36.000)

So let's say it sees part of the, uh, article and it, the only task that it

Wojciech Zaremba (1:15:41.400)

has at hand, it is to say what would be the next word and what would be the next

Lex Fridman (1:15:45.680)

word and it becomes a really exceptional at the task of figuring out what's the

Lex Fridman (1:15:51.800)

next word. So you might ask, why would, uh, this be an important, uh, task?

Lex Fridman (1:15:57.640)

Why would it be important to predict what's the next word?

Lex Fridman (1:16:01.280)

And it turns out that a lot of problems, uh, can be formulated, uh, as a text

Lex Fridman (1:16:07.920)

completion problem.

Lex Fridman (1:16:08.840)

So GPT is purely, uh, learning to complete the text.

Lex Fridman (1:16:13.120)

And you could imagine, for instance, if you are asking a question, uh, who is

Wojciech Zaremba (1:16:17.240)

the president of the United States, then GPT can give you an answer to it.

Lex Fridman (1:16:22.160)

It turns out that many more things can be formulated this way.

Wojciech Zaremba (1:16:25.720)

You can format text in the way that you have sentence in English.

Wojciech Zaremba (1:16:30.920)

You make it even look like some content of a website, uh, elsewhere, which would

Wojciech Zaremba (1:16:35.600)

be teaching people how to translate things between languages.

Lex Fridman (1:16:38.440)

So it would be EN colon, uh, text in English, FR colon, and then you'll

Wojciech Zaremba (1:16:43.720)

uh, uh, and then you'll ask people and then you ask model to, to continue.

Lex Fridman (1:16:48.560)

And it turns out that the, such a model is predicting translation from English

Wojciech Zaremba (1:16:52.720)

to French.

Wojciech Zaremba (1:16:53.640)

The crazy thing is that this model can be used for way more sophisticated tasks.

Lex Fridman (1:17:00.840)

So you can format text such that it looks like a conversation between two people.

Lex Fridman (1:17:05.640)

And that might be a conversation between you and Elon Musk.

Lex Fridman (1:17:08.920)

And because the model read all the texts about Elon Musk, it will be able to

Lex Fridman (1:17:13.960)

predict Elon Musk words as it would be Elon Musk.

Wojciech Zaremba (1:17:16.480)

It will speak about colonization of Mars, about sustainable future and so on.

Lex Fridman (1:17:22.560)

And it's also possible to, to even give arbitrary personality to the model.

Wojciech Zaremba (1:17:29.200)

You can say, here is a conversation that we've a friendly AI bot.

Lex Fridman (1:17:32.640)

And the model, uh, will complete the text as a friendly AI bot.

Lex Fridman (1:17:37.520)

So, I mean, how do I express how amazing this is?

Lex Fridman (1:17:43.920)

So just to clarify, uh, a conversation, generating a conversation between me and

Wojciech Zaremba (1:17:49.760)

Elon Musk, it wouldn't just generate good examples of what Elon would say.

Lex Fridman (1:17:56.800)

It would get the same results as the conversation between Elon Musk and me.

Wojciech Zaremba (1:18:01.080)

Say it would get the syntax all correct.

Lex Fridman (1:18:04.200)

So like interview style, it would say like Elon call and Lex call, like it,

Wojciech Zaremba (1:18:09.280)

it's not just like, uh, inklings of, um, semantic correctness.

Lex Fridman (1:18:17.720)

It's like the whole thing, grammatical, syntactic, semantic, it's just really,

Wojciech Zaremba (1:18:25.520)

really impressive, uh, generalization.

Lex Fridman (1:18:30.000)

Yeah.

Wojciech Zaremba (1:18:30.280)

I mean, I also want to, you know, provide some caveats so it can generate

Lex Fridman (1:18:34.680)

few paragraphs of coherent text, but as you go to, uh, longer pieces,

Wojciech Zaremba (1:18:38.880)

it, uh, it actually goes off the rails.

Lex Fridman (1:18:41.360)

Okay.

Wojciech Zaremba (1:18:41.480)

If you try to write a book, it won't work out this way.

Lex Fridman (1:18:45.680)

What way does it go off the rails, by the way?

Lex Fridman (1:18:47.840)

Is there interesting ways in which it goes off the rails?

Lex Fridman (1:18:50.560)

Like what falls apart first?

Lex Fridman (1:18:54.040)

So the model is trained on the, all the existing data, uh, that is out there,

Lex Fridman (1:18:58.720)

which means that it is not trained on its own mistakes.

Lex Fridman (1:19:02.040)

So for instance, if it would make a mistake, then, uh, I kept,

Lex Fridman (1:19:06.360)

so to give you, give you an example.

Lex Fridman (1:19:08.160)

So let's say I have a conversation with a model pretending that is Elon Musk.

Lex Fridman (1:19:14.360)

And then I start putting some, uh, I'm start actually making up

Wojciech Zaremba (1:19:19.000)

things which are not factual.

Lex Fridman (1:19:21.360)

Um, I would say like Twitter, but I got you.

Wojciech Zaremba (1:19:25.680)

Sorry.

Lex Fridman (1:19:26.120)

Yeah.

Wojciech Zaremba (1:19:26.440)

Um, like, uh, I don't know.

Wojciech Zaremba (1:19:28.960)

I would say that Elon is my wife and the model will just keep on carrying it on.

Lex Fridman (1:19:35.440)

And as if it's true.

Lex Fridman (1:19:37.120)

Yes.

Lex Fridman (1:19:38.000)

And in some sense, if you would have a normal conversation with Elon,

Lex Fridman (1:19:41.720)

he would be what the fuck.

Wojciech Zaremba (1:19:43.160)

Yeah.

Lex Fridman (1:19:43.760)

There'll be some feedback between, so the model is trained on things

Wojciech Zaremba (1:19:48.480)

that humans have written, but through the generation process, there's

Lex Fridman (1:19:52.280)

no human in the loop feedback.

Wojciech Zaremba (1:19:54.200)

Correct.

Lex Fridman (1:19:55.360)

That's fascinating.

Wojciech Zaremba (1:19:56.240)

Makes sense.

Lex Fridman (1:19:57.000)

So it's magnified.

Wojciech Zaremba (1:19:57.960)

It's like the errors get magnified and magnified and it's also interesting.

Lex Fridman (1:20:04.880)

I mean, first of all, humans have the same problem.

Wojciech Zaremba (1:20:06.760)

It's just that we, uh, we'll make fewer errors and magnify the errors slower.

Lex Fridman (1:20:13.960)

I think that actually what happens with humans is if you have a wrong

Wojciech Zaremba (1:20:17.400)

belief about the world as a kid, then very quickly we'll learn that it's

Lex Fridman (1:20:21.720)

not correct because they are grounded in reality and they are learning

Wojciech Zaremba (1:20:25.320)

from your new experience.

Lex Fridman (1:20:26.400)

Yes.

Lex Fridman (1:20:27.520)

But do you think the model can correct itself too?

Lex Fridman (1:20:30.960)

Won't it through the power of the representation.

Lex Fridman (1:20:34.840)

And so the absence of Elon Musk being your wife information on the

Lex Fridman (1:20:40.560)

internet, won't it correct itself?

Wojciech Zaremba (1:20:43.720)

There won't be examples like that.

Lex Fridman (1:20:45.760)

So the errors will be subtle at first.

Wojciech Zaremba (1:20:48.320)

Subtle at first.

Lex Fridman (1:20:49.200)

And in some sense, you can also say that the data that is not out there is

Wojciech Zaremba (1:20:54.440)

the data, which would represent how the human learns and maybe model would

Lex Fridman (1:21:00.400)

be learned, trained on such a data.

Wojciech Zaremba (1:21:01.800)

Then it would be better off.

Lex Fridman (1:21:03.480)

How intelligent is GPT3 do you think?

Wojciech Zaremba (1:21:06.480)

Like when you think about the nature of intelligence, it

Lex Fridman (1:21:10.080)

seems exceptionally impressive.

Lex Fridman (1:21:14.440)

But then if you think about the big AGI problem, is this

Lex Fridman (1:21:18.040)

footsteps along the way to AGI?

Lex Fridman (1:21:20.120)

So let's see, it seems that intelligence itself is, there are multiple axis of it.

Lex Fridman (1:21:25.920)

And I would expect that the systems that we are building, they might end up being

Wojciech Zaremba (1:21:33.280)

superhuman on some axis and subhuman on some other axis.

Wojciech Zaremba (1:21:37.360)

It would be surprising to me on all axis simultaneously, they would become superhuman.

Wojciech Zaremba (1:21:43.040)

Of course, people ask this question, is GPT a spaceship that would take us to

Lex Fridman (1:21:48.560)

the moon or are we putting a, building a ladder to heaven that we are just

Wojciech Zaremba (1:21:52.360)

building bigger and bigger ladder.

Lex Fridman (1:21:54.520)

And we don't know in some sense, which one of these two.

Lex Fridman (1:21:59.080)

Which one is better?

Lex Fridman (1:22:02.240)

I'm trying to, I like stairway to heaven.

Wojciech Zaremba (1:22:04.120)

It's a good song.

Lex Fridman (1:22:04.840)

So I'm not exactly sure which one is better, but you're saying like the

Wojciech Zaremba (1:22:08.120)

spaceship to the moon is actually effective.

Lex Fridman (1:22:10.680)

Correct.

Lex Fridman (1:22:11.080)

So people who criticize GPT, they say, you guys just building a

Lex Fridman (1:22:17.960)

taller, a ladder, and it will never reach the moon.

Lex Fridman (1:22:22.320)

And at the moment, I would say the way I'm thinking is, is like a scientific question.

Lex Fridman (1:22:28.480)

And I'm also in heart, I'm a builder creator and like, I'm thinking, let's try out, let's

Wojciech Zaremba (1:22:35.040)

see how far it goes.

Lex Fridman (1:22:36.840)

And so far we see constantly that there is a progress.

Wojciech Zaremba (1:22:40.800)

Yeah.

Lex Fridman (1:22:41.320)

So do you think GPT four, GPT five, GPT N plus one will, um, there'll be a phase

Wojciech Zaremba (1:22:52.320)

shift, like a transition to a, to a place where we'll be truly surprised.

Lex Fridman (1:22:56.960)

Then again, like GPT three is already very like truly surprising.

Lex Fridman (1:23:00.880)

The people that criticize GPT three as a stair, as a, what is it?

Lex Fridman (1:23:04.600)

Ladder to heaven.

Wojciech Zaremba (1:23:06.240)

I think too quickly get accustomed to how impressive it is that they're

Wojciech Zaremba (1:23:09.880)

impressive, it is that the prediction of the next word can achieve such depth of

Wojciech Zaremba (1:23:15.080)

semantics, accuracy of syntax, grammar, and semantics.

Lex Fridman (1:23:20.680)

Um, do you, do you think GPT four and five and six will continue to surprise us?

Wojciech Zaremba (1:23:28.120)

I mean, definitely there will be more impressive models that there is a

Lex Fridman (1:23:31.320)

question of course, if there will be a phase shift and, uh, the, also even the

Wojciech Zaremba (1:23:38.560)

way I'm thinking about the, about these models is that when we build these

Wojciech Zaremba (1:23:42.880)

models, you know, we see some level of the capabilities, but we don't even fully

Wojciech Zaremba (1:23:47.560)

understand everything that the model can do.

Lex Fridman (1:23:50.280)

And actually one of the best things to do is to allow other people to probe the

Wojciech Zaremba (1:23:55.880)

model to even see what is possible.

Lex Fridman (1:23:58.880)

Hence the, the using GPT as an API and opening it up to the world.

Wojciech Zaremba (1:24:05.320)

Yeah.

Lex Fridman (1:24:05.600)

I mean, so when I'm thinking from perspective of like, uh, obviously

Wojciech Zaremba (1:24:10.680)

various people are, that have concerns about AGI, including myself.

Lex Fridman (1:24:14.840)

Um, and then when I'm thinking from perspective, what's the strategy even to

Wojciech Zaremba (1:24:18.960)

deploy these things to the world, then the one strategy that I have seen many

Lex Fridman (1:24:23.880)

times working is that iterative deployment that you deploy, um, slightly

Wojciech Zaremba (1:24:29.360)

better versions and you allow other people to criticize you.

Lex Fridman (1:24:32.520)

So you actually, or try it out, you see where are their fundamental issues.

Lex Fridman (1:24:37.200)

And it's almost, you don't want to be in that situation that you are holding

Wojciech Zaremba (1:24:42.320)

into powerful system and there's like a huge overhang, then you deploy it and it

Wojciech Zaremba (1:24:48.320)

might have a random chaotic impact on the world.

Lex Fridman (1:24:50.960)

So you actually want to be in the situation that they are

Wojciech Zaremba (1:24:53.800)

gradually deploying systems.

Lex Fridman (1:24:56.560)

I asked this question of Illya, let me ask you, uh, you this question.

Wojciech Zaremba (1:25:00.680)

I've been reading a lot about Stalin and power.

Lex Fridman (1:25:09.360)

If you're in possession of a system that's like AGI, that's exceptionally

Lex Fridman (1:25:14.480)

powerful, do you think your character and integrity might become corrupted?

Lex Fridman (1:25:21.040)

Like famously power corrupts and absolute power corrupts.

Wojciech Zaremba (1:25:23.920)

Absolutely.

Lex Fridman (1:25:24.440)

So I believe that the, you want at some point to work toward distributing the power.

Wojciech Zaremba (1:25:31.440)

I think that the, you want to be in the situation that actually AGI is not

Wojciech Zaremba (1:25:36.360)

controlled by a small number of people, uh, but, uh, essentially, uh, by a larger

Wojciech Zaremba (1:25:42.680)

collective.

Lex Fridman (1:25:43.560)

So the thing is that requires a George Washington style move in the ascent to

Wojciech Zaremba (1:25:50.360)

power, there's always a moment when somebody gets a lot of power and they

Lex Fridman (1:25:55.280)

have to have the integrity and, uh, the moral compass to give away that power.

Wojciech Zaremba (1:26:01.920)

That humans have been good and bad throughout history at this particular

Lex Fridman (1:26:06.480)

step.

Lex Fridman (1:26:07.400)

And I wonder, I wonder we like blind ourselves in a, for example, between

Lex Fridman (1:26:13.120)

nations, a race, uh, towards, um, they, yeah, AI race between nations, we might

Wojciech Zaremba (1:26:20.440)

blind ourselves and justify to ourselves the development of AI without distributing

Lex Fridman (1:26:25.240)

the power because we want to defend ourselves against China, against Russia,

Wojciech Zaremba (1:26:29.920)

that kind of, that kind of logic.

Lex Fridman (1:26:32.360)

And, um, I wonder how we, um, how we design governance mechanisms that, um,

Wojciech Zaremba (1:26:40.160)

prevent us from becoming power hungry and in the process, destroying ourselves.

Lex Fridman (1:26:46.280)

So let's see, I have been thinking about this topic quite a bit, but I also want

Wojciech Zaremba (1:26:50.600)

to admit that, uh, once again, I actually want to rely way more on Sam Altman on it.

Lex Fridman (1:26:55.840)

He wrote an excellent blog on how even to distribute wealth.

Wojciech Zaremba (1:27:01.280)

Um, and he's proper, he proposed in his blog, uh, to tax, uh, equity of the companies

Lex Fridman (1:27:08.720)

rather than profit and to distribute it.

Lex Fridman (1:27:11.000)

And this is, this is an example of, uh, Washington move.

Wojciech Zaremba (1:27:17.680)

I guess I personally have insane trust in some here already spent plenty of money

Wojciech Zaremba (1:27:24.320)

running, uh, universal basic income, uh, project.

Lex Fridman (1:27:28.360)

That like, uh, gives me, I guess, maybe some level of trust to him, but I also,

Wojciech Zaremba (1:27:34.480)

I guess love him as a friend.

Lex Fridman (1:27:37.720)

Yeah.

Wojciech Zaremba (1:27:38.920)

I wonder because we're sort of summoning a new set of technologies.

Wojciech Zaremba (1:27:44.280)

I wonder if we'll be, um, cognizant, like you're describing the process of open AI,

Lex Fridman (1:27:50.680)

but it could also be at other places like in the U S government, right?

Lex Fridman (1:27:54.360)

Uh, both China and the U S are now full steam ahead on autonomous

Wojciech Zaremba (1:28:00.680)

weapons systems development.

Lex Fridman (1:28:03.200)

And that's really worrying to me because in the framework of something being a

Wojciech Zaremba (1:28:09.680)

national security danger or military danger, you can do a lot of pretty dark

Lex Fridman (1:28:14.880)

things that blind our moral compass.

Lex Fridman (1:28:18.720)

And I think AI will be one of those things, um, in some sense, the, the mission

Lex Fridman (1:28:24.320)

and the work you're doing in open AI is like the counterbalance to that.

Lex Fridman (1:28:28.840)

So you want to have more open AI and less autonomous weapons systems.

Wojciech Zaremba (1:28:33.200)

I, I, I, I like these statements, like to be clear, like this interesting and I'm

Wojciech Zaremba (1:28:37.200)

thinking about it myself, but, uh, this is a place that I, I, I put my trust

Lex Fridman (1:28:43.760)

actually in Sam's hands, because it's extremely hard for me to reason about it.

Wojciech Zaremba (1:28:48.760)

Yeah.

Lex Fridman (1:28:49.200)

I mean, one important statement to make is, um, it's good to think about this.

Wojciech Zaremba (1:28:54.280)

Yeah.

Lex Fridman (1:28:54.640)

No question about it.

Wojciech Zaremba (1:28:55.520)

No question, even like low level quote unquote engineer, like there's such a,

Lex Fridman (1:29:02.680)

um, I remember I, I programmed a car, uh, our RC car, um, and it was, it was

Wojciech Zaremba (1:29:10.080)

programmed a car, uh, our RC car, they went really fast, like 30, 40 miles an hour.

Lex Fridman (1:29:18.480)

And I remember I was like sleep deprived.

Lex Fridman (1:29:21.080)

So I programmed it pretty crappily and it like, uh, the, the, the code froze.

Lex Fridman (1:29:26.440)

So it's doing some basic computer vision and it's going around on track,

Lex Fridman (1:29:30.280)

but it's going full speed.

Wojciech Zaremba (1:29:32.640)

And, uh, there was a bug in the code that, uh, the car just went, it didn't turn.

Wojciech Zaremba (1:29:39.280)

Went straight full speed and smash into the wall.

Lex Fridman (1:29:42.520)

And I remember thinking the seriousness with which you need to approach the

Wojciech Zaremba (1:29:49.480)

design of artificial intelligence systems and the programming of artificial

Lex Fridman (1:29:53.240)

intelligence systems is high because the consequences are high, like that

Wojciech Zaremba (1:29:58.520)

little car smashing into the wall.

Lex Fridman (1:30:00.880)

For some reason, I immediately thought of like an algorithm that controls

Wojciech Zaremba (1:30:04.480)

nuclear weapons, having the same kind of bug.

Lex Fridman (1:30:07.160)

And so like the lowest level engineer and the CEO of a company all need to

Wojciech Zaremba (1:30:11.840)

have the seriousness, uh, in approaching this problem and thinking

Lex Fridman (1:30:15.240)

about the worst case consequences.

Lex Fridman (1:30:17.000)

So I think that is true.

Lex Fridman (1:30:18.800)

I mean, the, what I also recognize in myself and others even asking this

Wojciech Zaremba (1:30:24.840)

question is that it evokes a lot of fear and fear itself ends up being

Lex Fridman (1:30:29.680)

actually quite debilitating.

Wojciech Zaremba (1:30:31.400)

The place where I arrived at the moment might sound cheesy or so, but it's

Wojciech Zaremba (1:30:38.680)

almost to build things out of love rather than fear, like a focus on how, uh, I can,

Wojciech Zaremba (1:30:48.720)

you know, maximize the value, how the systems that I'm building might be, uh,

Lex Fridman (1:30:54.280)

useful.

Wojciech Zaremba (1:30:55.800)

I'm not saying that the fear doesn't exist out there and like it totally

Lex Fridman (1:31:00.400)

makes sense to minimize it, but I don't want to be working because, uh, I'm

Wojciech Zaremba (1:31:04.920)

scared, I want to be working out of passion, out of curiosity, out of the,

Lex Fridman (1:31:10.640)

you know, uh, looking forward for the positive future.

Wojciech Zaremba (1:31:13.840)

With, uh, the definition of love arising from a rigorous practice of empathy.

Lex Fridman (1:31:19.800)

So not just like your own conception of what is good for the world, but

Wojciech Zaremba (1:31:23.600)

always listening to others.

Lex Fridman (1:31:25.160)

Correct.

Wojciech Zaremba (1:31:25.560)

Like the love where I'm considering reward functions of others.

Lex Fridman (1:31:29.160)

Others to limit to infinity is like a sum of like one to N where N is, uh,

Wojciech Zaremba (1:31:35.280)

7 billion or whatever it is.

Lex Fridman (1:31:36.680)

Not, not projecting my reward functions on others.

Wojciech Zaremba (1:31:38.920)

Yeah, exactly.

Lex Fridman (1:31:40.440)

Okay.

Lex Fridman (1:31:41.360)

Can we just take a step back to something else?

Lex Fridman (1:31:43.760)

Super cool, which is, uh, OpenAI Codex.

Lex Fridman (1:31:47.240)

Can you give an overview of what OpenAI Codex and GitHub Copilot is, how it works

Lex Fridman (1:31:53.680)

and why the hell it works so well?

Lex Fridman (1:31:55.280)

So with GPT tree, we noticed that the system, uh, you know, that system train

Wojciech Zaremba (1:32:00.960)

on all the language out there started having some rudimentary coding capabilities.

Lex Fridman (1:32:05.440)

So we're able to ask it, you know, to implement addition function between

Lex Fridman (1:32:10.880)

two numbers and indeed it can write item or JavaScript code for that.

Lex Fridman (1:32:15.320)

And then we thought, uh, we might as well just go full steam ahead and try to

Lex Fridman (1:32:20.520)

create a system that is actually good at what we are doing every day ourselves,

Wojciech Zaremba (1:32:25.800)

which is programming.

Lex Fridman (1:32:27.320)

We optimize models for proficiency in coding.

Wojciech Zaremba (1:32:31.600)

We actually even created models that both have a comprehension of language and code.

Lex Fridman (1:32:38.840)

And Codex is API for these models.

Lex Fridman (1:32:42.600)

So it's first pre trained on language and then codex.

Lex Fridman (1:32:48.840)

Then I don't know if you can say fine tuned because there's a lot of code,

Lex Fridman (1:32:54.600)

but it's language and code.

Lex Fridman (1:32:56.400)

It's language and code.

Wojciech Zaremba (1:32:58.320)

It's also optimized for various things.

Lex Fridman (1:33:00.200)

I can, let's say low latency and so on.

Wojciech Zaremba (1:33:02.600)

Codex is the API, the similar to GPT tree.

Wojciech Zaremba (1:33:06.000)

We expect that there will be proliferation of the potential products that can use

Wojciech Zaremba (1:33:10.560)

coding capabilities and I can, I can speak about it in a second.

Lex Fridman (1:33:14.920)

Copilot is a first product and developed by GitHub.

Lex Fridman (1:33:18.200)

So as we're building, uh, models, we wanted to make sure that these

Wojciech Zaremba (1:33:22.000)

models are useful and we work together with GitHub on building the first product.

Wojciech Zaremba (1:33:27.320)

Copilot is actually, as you code, it suggests you code completions.

Lex Fridman (1:33:32.240)

And we have seen in the past, there are like a various tools that can suggest

Lex Fridman (1:33:36.760)

how to like a few characters of the code or a line of code.

Lex Fridman (1:33:41.000)

Then the thing about Copilot is it can generate 10 lines of code.

Wojciech Zaremba (1:33:44.600)

You, it's often the way how it works is you often write in the comment

Lex Fridman (1:33:49.480)

what you want to happen because people in comments, they describe what happens next.

Wojciech Zaremba (1:33:53.960)

So, um, these days when I code, instead of going to Google to search, uh, for

Lex Fridman (1:34:00.200)

the appropriate code to solve my problem, I say, Oh, for this area, could you

Wojciech Zaremba (1:34:06.200)

smooth it and then, you know, it imports some appropriate libraries and say it

Lex Fridman (1:34:10.520)

uses NumPy convolution or so I, that I was not even aware that exists and

Wojciech Zaremba (1:34:15.000)

it does the appropriate thing.

Lex Fridman (1:34:16.840)

Um, so you, uh, you write a comment, maybe the header of a function

Lex Fridman (1:34:21.440)

and it completes the function.

Lex Fridman (1:34:23.320)

Of course, you don't know what is the space of all the possible small

Wojciech Zaremba (1:34:27.200)

programs that can generate.

Lex Fridman (1:34:28.840)

What are the failure cases?

Lex Fridman (1:34:30.360)

How many edge cases, how many subtle errors there are, how many big errors

Lex Fridman (1:34:34.880)

there are, it's hard to know, but the fact that it works at all in a large

Wojciech Zaremba (1:34:38.840)

number of cases is incredible.

Lex Fridman (1:34:41.000)

It's like, uh, it's a kind of search engine into code that's

Wojciech Zaremba (1:34:45.920)

been written on the internet.

Lex Fridman (1:34:47.720)

Correct.

Lex Fridman (1:34:48.120)

So for instance, when you search things online, then usually you get to the,

Lex Fridman (1:34:53.720)

some particular case, like if you go to stack overflow and people describe

Wojciech Zaremba (1:34:58.920)

that one particular situation, uh, and then they seek for a solution.

Lex Fridman (1:35:03.040)

But in case of a copilot, it's aware of your entire context and in

Wojciech Zaremba (1:35:08.040)

context is, Oh, these are the libraries that they are using.

Lex Fridman (1:35:10.480)

That's the set of the variables that is initialized.

Lex Fridman (1:35:14.120)

And on the spot, it can actually tell you what to do.

Lex Fridman (1:35:17.280)

So the interesting thing is, and we think that the copilot is one

Wojciech Zaremba (1:35:21.280)

possible product using codecs, but there is a place for many more.

Lex Fridman (1:35:25.080)

So internally we tried out, you know, to create other fun products.

Lex Fridman (1:35:29.760)

So it turns out that a lot of tools out there, let's say Google

Lex Fridman (1:35:33.880)

calendar or Microsoft word or so, they all have a internal API

Wojciech Zaremba (1:35:38.480)

to build plugins around them.

Lex Fridman (1:35:41.240)

So there is a way in the sophisticated way to control calendar or Microsoft word.

Wojciech Zaremba (1:35:47.520)

Today, if you want, if you want more complicated behaviors from these

Lex Fridman (1:35:51.160)

programs, you have to add the new button for every behavior.

Lex Fridman (1:35:55.040)

But it is possible to use codecs and tell for instance, to calendar, uh,

Lex Fridman (1:36:00.440)

could you schedule an appointment with Lex next week after 2 PM and it

Wojciech Zaremba (1:36:06.200)

writes corresponding piece of code.

Lex Fridman (1:36:08.920)

And that's the thing that actually you want.

Lex Fridman (1:36:10.800)

So interesting.

Lex Fridman (1:36:11.440)

So you figure out is there's a lot of programs with which

Wojciech Zaremba (1:36:15.000)

you can interact through code.

Lex Fridman (1:36:17.080)

And so there you can generate that code from natural language.

Wojciech Zaremba (1:36:22.480)

That's fascinating.

Lex Fridman (1:36:23.440)

And that's somewhat like also closest to what was the promise of Siri or Alexa.

Lex Fridman (1:36:28.880)

So previously all these behaviors, they were hard coded and it seems

Lex Fridman (1:36:33.680)

that codecs on the fly can pick up the API of let's say, given software.

Lex Fridman (1:36:39.360)

And then it can turn language into use of this API.

Lex Fridman (1:36:42.320)

So without hard coding, you can find, it can translate to machine language.

Wojciech Zaremba (1:36:46.640)

Correct.

Lex Fridman (1:36:47.040)

To, uh, so for example, this would be really exciting for me, like for, um,

Wojciech Zaremba (1:36:51.880)

Adobe products, like Photoshop, uh, which I think action scripted, I think

Lex Fridman (1:36:57.320)

there's a scripting language that communicates with them, same with Premier.

Lex Fridman (1:37:00.440)

And do you could imagine that that allows even to do coding by voice on your phone?

Lex Fridman (1:37:06.480)

So for instance, in the past, okay.

Wojciech Zaremba (1:37:09.000)

As of today, I'm not editing Word documents on my phone because it's

Lex Fridman (1:37:13.760)

just the keyboard is too small.

Lex Fridman (1:37:15.480)

But if I would be able to tell, uh, to my phone, you know, uh, make the

Lex Fridman (1:37:20.520)

header large, then move the paragraphs around and that's actually what I want.

Lex Fridman (1:37:25.040)

So I can tell you one more cool thing, or even how I'm thinking about codecs.

Lex Fridman (1:37:29.720)

So if you look actually at the evolution of, uh, of computers, we started with

Wojciech Zaremba (1:37:36.320)

a very primitive interfaces, which is a punch card and punch card.

Lex Fridman (1:37:40.320)

So Charlie, you make a holes in the, in the plastic card to indicate zeros and ones.

Wojciech Zaremba (1:37:47.040)

And, uh, during that time, there was a small number of specialists

Lex Fridman (1:37:50.720)

who were able to use computers.

Lex Fridman (1:37:52.040)

And by the way, people even suspected that there is no need for many

Lex Fridman (1:37:55.000)

more people to use computers.

Wojciech Zaremba (1:37:56.960)

Um, but then we moved from punch cards to at first assembly and see, and

Lex Fridman (1:38:03.920)

at these programming languages, they were slightly higher level.

Wojciech Zaremba (1:38:07.200)

They allowed many more people to code and they also, uh, led to more

Lex Fridman (1:38:11.920)

of a proliferation of technology.

Wojciech Zaremba (1:38:14.040)

And, uh, you know, further on, there was a jump to say from C++ to Java and Python.

Lex Fridman (1:38:19.960)

And every time it has happened, more people are able to code

Lex Fridman (1:38:23.600)

and we build more technology.

Lex Fridman (1:38:26.200)

And it's even, you know, hard to imagine now, if someone will tell you that you

Wojciech Zaremba (1:38:31.200)

should write code in assembly instead of let's say, Python or Java or JavaScript.

Lex Fridman (1:38:37.160)

And codecs is yet another step toward kind of bringing computers closer to

Wojciech Zaremba (1:38:41.520)

humans such that you communicate with a computer with your own language rather

Lex Fridman (1:38:47.120)

than with a specialized language, and, uh, I think that it will lead to an

Wojciech Zaremba (1:38:52.600)

increase of number of people who can code.

Lex Fridman (1:38:55.280)

Yeah.

Lex Fridman (1:38:55.440)

And then, and the kind of technologies that those people will create is it's

Lex Fridman (1:39:00.160)

innumerable, it could, you know, it could be a huge number of technologies.

Wojciech Zaremba (1:39:03.760)

We're not predicting at all because that's less and less requirement

Wojciech Zaremba (1:39:07.600)

of having a technical mind, a programming mind, you're not opening it to the world

Wojciech Zaremba (1:39:13.480)

of, um, other kinds of minds, creative minds, artistic minds, all that kind of stuff.

Lex Fridman (1:39:19.400)

I would like, for instance, biologists who work on DNA to be able to program

Lex Fridman (1:39:23.800)

and not to need to spend a lot of time learning it.

Lex Fridman (1:39:26.720)

And I, I believe that's a good thing to the world.

Lex Fridman (1:39:29.080)

And I would actually add, I would add, so at the moment I'm a managing codecs

Lex Fridman (1:39:33.800)

team and also language team, and I believe that there is like a plenty

Wojciech Zaremba (1:39:37.800)

of brilliant people out there and they should have a lot of experience.

Lex Fridman (1:39:41.640)

There and they should apply.

Wojciech Zaremba (1:39:44.360)

Oh, okay.

Lex Fridman (1:39:45.080)

Yeah.

Wojciech Zaremba (1:39:45.320)

Awesome.

Lex Fridman (1:39:45.880)

So what's the language and the codecs is, so those are kind of,

Wojciech Zaremba (1:39:48.960)

they're overlapping teams.

Wojciech Zaremba (1:39:50.760)

It's like GPT, the raw language, and then the codecs is like applied to programming.

Wojciech Zaremba (1:39:57.120)

Correct.

Lex Fridman (1:39:57.480)

And they are quite intertwined.

Wojciech Zaremba (1:40:00.000)

There are many more things involved making this, uh, models,

Lex Fridman (1:40:03.960)

uh, extremely efficient and deployable.

Wojciech Zaremba (1:40:06.480)

Okay.

Lex Fridman (1:40:06.600)

For instance, there are people who are working to, you know, make our data

Wojciech Zaremba (1:40:10.800)

centers, uh, amazing, or there are people who work on putting these

Wojciech Zaremba (1:40:14.960)

models into production or, uh, or even pushing it at the very limit of the scale.

Lex Fridman (1:40:21.640)

So all aspects from, from the infrastructure to the actual machine.

Lex Fridman (1:40:25.240)

So I'm just saying there are multiple teams while the, and the team working

Wojciech Zaremba (1:40:29.640)

on codecs and language, uh, I guess I'm, I'm directly managing them.

Lex Fridman (1:40:33.560)

I would like, I would love to hire more interested in machine learning.

Wojciech Zaremba (1:40:37.560)

This is probably one of the most exciting problems and like systems

Lex Fridman (1:40:41.960)

to be working on is it's actually, it's, it's, it's pretty cool.

Wojciech Zaremba (1:40:45.560)

Like what, what, uh, the program synthesis, like generating a

Lex Fridman (1:40:48.760)

programs is very interesting, very interesting problem that has echoes

Wojciech Zaremba (1:40:53.480)

of reasoning and intelligence in it.

Lex Fridman (1:40:57.080)

It's and I think there's a lot of fundamental questions that you might

Wojciech Zaremba (1:41:00.520)

be able to sneak, uh, sneak up to by generating programs.

Lex Fridman (1:41:05.480)

Yeah, that one more exciting thing about the programs is that, so I said

Wojciech Zaremba (1:41:09.600)

that the, um, you know, the, in case of language, that one of the travels

Lex Fridman (1:41:13.720)

is even evaluating language.

Lex Fridman (1:41:15.200)

So when the things are made up, you, you need somehow either a human to,

Wojciech Zaremba (1:41:20.840)

to say that this doesn't make sense or so in case of program, there is one extra

Wojciech Zaremba (1:41:25.360)

lever that we can actually execute programs and see what they evaluate to.

Lex Fridman (1:41:29.400)

So that process might be somewhat, uh, more automated in, in order to improve

Wojciech Zaremba (1:41:35.800)

the, uh, qualities of generations.

Lex Fridman (1:41:38.440)

Oh, that's fascinating.

Lex Fridman (1:41:39.160)

So like the, wow, that's really interesting.

Lex Fridman (1:41:42.120)

So, so for the language, the, you know, the simulation to actually

Wojciech Zaremba (1:41:45.680)

execute it as a human mind.

Lex Fridman (1:41:47.440)

Yeah.

Wojciech Zaremba (1:41:48.280)

For programs, there is a, there is a computer on which you can evaluate it.

Lex Fridman (1:41:53.760)

Wow.

Wojciech Zaremba (1:41:54.960)

That's a brilliant little insight.

Wojciech Zaremba (1:41:58.400)

Insight that the thing compiles and runs that's first and second, you can evaluate

Wojciech Zaremba (1:42:04.880)

on a, like do automated unit testing and in some sense, it seems to me that we'll

Lex Fridman (1:42:11.000)

be able to make a tremendous progress.

Wojciech Zaremba (1:42:12.920)

You know, we are in the paradigm that there is way more data.

Lex Fridman (1:42:17.320)

There is like a transcription of millions of, uh, of, uh, software engineers.

Wojciech Zaremba (1:42:23.520)

Yeah.

Lex Fridman (1:42:24.320)

Yeah.

Wojciech Zaremba (1:42:24.820)

So, uh, I mean, you just mean, cause I was going to ask you about reliability.

Lex Fridman (1:42:29.300)

The thing about programs is you don't know if they're going to, like a program

Wojciech Zaremba (1:42:35.260)

that's controlling a nuclear power plant has to be very reliable.

Lex Fridman (1:42:39.140)

So I wouldn't start with controlling nuclear power plant maybe one day,

Lex Fridman (1:42:43.140)

but that's not actually, that's not on the current roadmap.

Lex Fridman (1:42:46.420)

That's not the step one.

Lex Fridman (1:42:48.540)

And you know, it's the Russian thing.

Lex Fridman (1:42:50.460)

You just want to go to the most powerful, destructive, most powerful

Wojciech Zaremba (1:42:53.500)

the most powerful, destructive thing right away run by JavaScript.

Lex Fridman (1:42:57.660)

But I got you.

Lex Fridman (1:42:58.300)

So this is a lower impact, but nevertheless, when you make me

Lex Fridman (1:43:01.020)

realize it is possible to achieve some levels of reliability by doing testing.

Lex Fridman (1:43:06.620)

And you could, you could imagine that, you know, maybe there are ways for

Lex Fridman (1:43:09.820)

model to write event code for testing itself and so on, and there exists

Wojciech Zaremba (1:43:15.340)

a ways to create the feedback loops that the model could keep on improving.

Lex Fridman (1:43:19.260)

Yeah. By writing programs that generate tests for the instance, for instance.

Lex Fridman (1:43:26.940)

And that's how we get consciousness, because it's metacompression.

Lex Fridman (1:43:30.660)

That's what you're going to write.

Wojciech Zaremba (1:43:31.540)

That's the comment.

Lex Fridman (1:43:32.460)

That's the prompt that generates consciousness.

Wojciech Zaremba (1:43:34.900)

Compressor of compressors.

Lex Fridman (1:43:36.780)

You just write that.

Lex Fridman (1:43:38.500)

Do you think the code that generates consciousness will be simple?

Lex Fridman (1:43:42.300)

So let's see.

Wojciech Zaremba (1:43:44.140)

I mean, ultimately, the core idea behind will be simple,

Lex Fridman (1:43:48.060)

but there will be also decent amount of engineering involved.

Wojciech Zaremba (1:43:53.380)

Like in some sense, it seems that, you know, spreading these models

Lex Fridman (1:43:58.580)

on many machines, it's not that trivial.

Wojciech Zaremba (1:44:01.860)

Yeah.

Lex Fridman (1:44:02.260)

And we find all sorts of innovations that make our models more efficient.

Wojciech Zaremba (1:44:08.460)

I believe that first models that I guess are conscious or like a truly intelligent,

Lex Fridman (1:44:14.460)

they will have all sorts of tricks, but then again, there's a Richard Sutton

Wojciech Zaremba (1:44:21.620)

argument that maybe the tricks are temporary things that they might be

Lex Fridman (1:44:25.780)

temporary things and in some sense, it's also even important to, to know

Wojciech Zaremba (1:44:32.300)

that even the cost of a trick.

Lex Fridman (1:44:33.780)

So sometimes people are eager to put the trick while forgetting that

Wojciech Zaremba (1:44:38.220)

there is a cost of maintenance or like a long term cost, long term cost

Wojciech Zaremba (1:44:43.300)

or maintenance, or maybe even flexibility of code to actually implement new ideas.

Lex Fridman (1:44:48.980)

So even if you have something that gives you 2x, but it requires, you know,

Lex Fridman (1:44:53.100)

1000 lines of code, I'm not sure if it's actually worth it.

Lex Fridman (1:44:56.300)

So in some sense, you know, if it's five lines of code and 2x, I would take it.

Lex Fridman (1:45:02.060)

And we see many of this, but also, you know, that requires some level of,

Wojciech Zaremba (1:45:07.620)

I guess, lack of attachment to code that we are willing to remove it.

Lex Fridman (1:45:12.540)

Yeah.

Lex Fridman (1:45:14.620)

So you led the OpenAI robotics team.

Lex Fridman (1:45:17.580)

Can you give an overview of the cool things you were able to

Lex Fridman (1:45:20.460)

accomplish, what are you most proud of?

Lex Fridman (1:45:22.780)

So when we started robotics, we knew that actually reinforcement learning works

Lex Fridman (1:45:26.060)

and it is possible to solve fairly complicated problems.

Wojciech Zaremba (1:45:29.940)

Like for instance, AlphaGo is an evidence that it is possible to build superhuman

Wojciech Zaremba (1:45:36.020)

Go players, DOTA2 is an evidence that it's possible to build superhuman agents

Lex Fridman (1:45:44.060)

playing DOTA, so I asked myself a question, you know, what about robots out there?

Lex Fridman (1:45:48.820)

Could we train machines to solve arbitrary tasks in the physical world?

Lex Fridman (1:45:53.820)

Our approach was, I guess, let's pick a complicated problem that if we would

Wojciech Zaremba (1:45:59.620)

solve it, that means that we made some significant progress in the domain.

Lex Fridman (1:46:04.260)

And if can progress the domain, and then we went after the problem.

Lex Fridman (1:46:08.220)

So we noticed that actually the robots out there, they are kind of at the moment

Lex Fridman (1:46:13.780)

optimized per task, so you can have a robot that it's like, if you have a robot

Wojciech Zaremba (1:46:18.420)

opening a bottle, it's very likely that the end factor is that bottle opener.

Lex Fridman (1:46:24.060)

And the, and in some sense, that's a hack to be able to solve a task,

Wojciech Zaremba (1:46:27.780)

which makes any task easier and ask myself, so what would be a robot that

Lex Fridman (1:46:33.180)

can actually solve many tasks?

Lex Fridman (1:46:35.300)

And we conclude that human hands have such a quality that indeed they are, you

Lex Fridman (1:46:42.900)

know, you have five kind of tiny arms attached individually.

Wojciech Zaremba (1:46:48.060)

They can manipulate pretty broad spectrum of objects.

Lex Fridman (1:46:51.860)

So we went after a single hand, like trying to solve Rubik's cube single handed.

Wojciech Zaremba (1:46:57.420)

We picked this task because we thought that there is no way to hard code it.

Lex Fridman (1:47:01.740)

And it's also, we picked the robot on which it would be hard to hard code it.

Lex Fridman (1:47:05.700)

And we went after the solution such that it could generalize to other problems.

Lex Fridman (1:47:11.180)

And just to clarify, it's one robotic hand solving the Rubik's cube.

Wojciech Zaremba (1:47:16.300)

The hard part isn't the solution to the Rubik's cube is the manipulation of the,

Lex Fridman (1:47:21.180)

of like having it not fall out of the hand, having it use the, uh, five baby

Wojciech Zaremba (1:47:27.100)

arms to, uh, what is it like rotate different parts of the Rubik's cube to

Lex Fridman (1:47:32.020)

achieve the solution.

Wojciech Zaremba (1:47:33.140)

Correct.

Lex Fridman (1:47:33.940)

Yeah.

Lex Fridman (1:47:34.660)

So what, uh, what was the hardest part about that?

Lex Fridman (1:47:38.380)

What was the approach taken there?

Lex Fridman (1:47:40.180)

What are you most proud of?

Lex Fridman (1:47:41.460)

Obviously we have like a strong belief in reinforcement learning.

Wojciech Zaremba (1:47:44.980)

And, uh, you know, one path it is to do reinforcement learning, the real

Lex Fridman (1:47:49.660)

world other path is to, uh, uh, that simulation in some sense, the tricky

Wojciech Zaremba (1:47:55.860)

part about the real world is at the moment, our models, they require a lot

Lex Fridman (1:47:59.620)

of data and there is essentially no data.

Wojciech Zaremba (1:48:02.220)

And, uh, I did, we decided to go through the path of the simulation.

Lex Fridman (1:48:07.060)

And in simulation, you can have infinite amount of data.

Wojciech Zaremba (1:48:09.780)

The tricky part is the fidelity of the simulation.

Lex Fridman (1:48:12.740)

And also can you in simulation represent everything that you represent

Wojciech Zaremba (1:48:16.780)

otherwise in the real world.

Lex Fridman (1:48:18.940)

And, you know, it turned out that, uh, that, you know, because there is

Wojciech Zaremba (1:48:22.900)

lack of fidelity, it is possible to what we, what we arrived at is training

Lex Fridman (1:48:29.820)

a model that doesn't solve one simulation, but it actually solves the

Wojciech Zaremba (1:48:34.180)

entire range of simulations, which, uh, uh, in terms of like, uh, what's

Lex Fridman (1:48:39.260)

the, exactly the friction of the cube or the weight or so, and the single AI

Wojciech Zaremba (1:48:45.260)

that can solve all of them ends up working well with the reality.

Lex Fridman (1:48:49.220)

How do you generate the different simulations?

Wojciech Zaremba (1:48:51.300)

So, uh, you know, there's plenty of parameters out there.

Lex Fridman (1:48:54.260)

We just pick them randomly.

Wojciech Zaremba (1:48:55.820)

And, uh, and in simulation model just goes for thousands of years and keeps

Lex Fridman (1:49:01.740)

on solving Rubik's cube in each of them.

Lex Fridman (1:49:03.780)

And the thing is that neural network that we used, it has a memory.

Lex Fridman (1:49:09.260)

And as it presses, for instance, the side of the, of the cube, it can sense,

Wojciech Zaremba (1:49:15.620)

oh, that's actually, this side was, uh, difficult to press.

Wojciech Zaremba (1:49:19.620)

I should press it stronger and throughout this process kind of, uh, learn it's even

Lex Fridman (1:49:24.540)

how to, uh, how to solve this particular instance of the Rubik's cube, like even

Wojciech Zaremba (1:49:29.060)

mass, it's kind of like, uh, you know, sometimes when you go to a gym and after,

Wojciech Zaremba (1:49:34.660)

um, after bench press, you try to leave the class and you kind of forgot, uh, and,

Lex Fridman (1:49:44.060)

and your head goes like up right away because kind of you got used to maybe

Wojciech Zaremba (1:49:48.900)

different weight and it takes a second to adjust and this kind of, of a memory,

Lex Fridman (1:49:54.940)

the model gained through the process of interacting with the cube in the

Wojciech Zaremba (1:49:58.180)

simulation, I appreciate you speaking to the audience with the bench press,

Lex Fridman (1:50:02.660)

all the bros in the audience, probably working out right now.

Wojciech Zaremba (1:50:05.780)

There's probably somebody listening to this actually doing bench press.

Lex Fridman (1:50:09.300)

Um, so maybe, uh, put the bar down and pick up the water bottle and you'll

Wojciech Zaremba (1:50:13.900)

know exactly what, uh, what Jack is talking about.

Lex Fridman (1:50:17.060)

Okay.

Wojciech Zaremba (1:50:17.540)

Okay.

Lex Fridman (1:50:18.500)

So what, uh, what was the hardest part of getting the whole thing to work?

Lex Fridman (1:50:24.780)

So the hardest part is at the moment when it comes to, uh, physical work, when it

Lex Fridman (1:50:31.660)

comes to robots, uh, they require maintenance, it's hard to replicate a

Wojciech Zaremba (1:50:36.740)

million times it's, uh, it's also, it's hard to replay things exactly.

Lex Fridman (1:50:41.620)

I remember this situation that one guy at our company, he had like a model that

Wojciech Zaremba (1:50:48.460)

performs way better than other models in solving Rubik's cube.

Lex Fridman (1:50:52.580)

And, uh, you know, we kind of didn't know what's going on, why it's that.

Wojciech Zaremba (1:50:58.420)

And, uh, it turned out, uh, that, you know, he was running it from his laptop

Lex Fridman (1:51:04.420)

that had better CPU or better, maybe local GPU as well.

Wojciech Zaremba (1:51:09.540)

And, uh, because of that, there was less of a latency and the model was the same.

Lex Fridman (1:51:14.780)

And that actually made solving Rubik's cube more reliable.

Lex Fridman (1:51:18.820)

So in some sense, there might be some subtle bugs like that when it comes

Lex Fridman (1:51:22.300)

to running things in the real world.

Wojciech Zaremba (1:51:24.700)

Even hinting on that, you could imagine that the initial models you would like

Lex Fridman (1:51:29.420)

to have models, which are insanely huge neural networks, and you would like to

Wojciech Zaremba (1:51:34.140)

give them even more time for thinking.

Lex Fridman (1:51:36.460)

And when you have these real time systems, uh, then you might be constrained

Wojciech Zaremba (1:51:41.980)

actually by the amount of latency.

Wojciech Zaremba (1:51:44.660)

And, uh, ultimately I would like to build a system that it is worth for you to wait

Wojciech Zaremba (1:51:50.940)

five minutes because it gives you the answer that you're willing to wait for

Lex Fridman (1:51:55.220)

five minutes.

Lex Fridman (1:51:56.260)

So latency is a very unpleasant constraint under which to operate.

Lex Fridman (1:51:59.820)

Correct.

Lex Fridman (1:52:00.620)

And also there is actually one more thing, which is tricky about robots.

Lex Fridman (1:52:04.260)

Uh, there is actually, uh, no, uh, not much data.

Lex Fridman (1:52:08.060)

So the data that I'm speaking about would be a data of, uh, first person

Lex Fridman (1:52:13.380)

experience from the robot and like a gigabytes of data like that, if we would

Wojciech Zaremba (1:52:17.660)

have gigabytes of data like that, of robots solving various problems, it would

Lex Fridman (1:52:21.900)

be very easy to make a progress on robotics.

Lex Fridman (1:52:24.420)

And you can see that in case of text or code, there is a lot of data, like a

Lex Fridman (1:52:28.660)

first person perspective, they don't writing code.

Wojciech Zaremba (1:52:31.980)

Yeah. So you had this, you mentioned this really interesting idea that if you were

Lex Fridman (1:52:37.740)

to build like a successful robotics company, so open as mission is much

Wojciech Zaremba (1:52:42.100)

bigger than robotics, this is one of the, one of the things you've worked on, but

Lex Fridman (1:52:46.500)

if it was a robotics company, they, you wouldn't so quickly dismiss supervised

Wojciech Zaremba (1:52:51.260)

learning, uh, correct that you would build a robot that, uh, was perhaps what

Wojciech Zaremba (1:52:58.300)

like, um, an empty shell, like dumb, and they would operate under teleoperation.

Lex Fridman (1:53:04.660)

So you would invest, that's just one way to do it, invest in human supervision,

Lex Fridman (1:53:09.700)

like direct human control of the robots as it's learning and over time, add

Wojciech Zaremba (1:53:14.740)

more and more automation.

Lex Fridman (1:53:16.380)

That's correct.

Lex Fridman (1:53:16.860)

So let's say that's how I would build a robotics company today.

Lex Fridman (1:53:20.780)

If I would be building a robotics company, which is, you know, spend 10

Wojciech Zaremba (1:53:23.620)

million dollars or so recording human trajectories, controlling a robot.

Lex Fridman (1:53:29.100)

After you find a thing that the robot should be doing, that there's a market

Wojciech Zaremba (1:53:34.860)

fit for, like you can make a lot of money with that product.

Lex Fridman (1:53:37.380)

Correct.

Wojciech Zaremba (1:53:37.700)

Correct.

Lex Fridman (1:53:38.100)

Yeah.

Wojciech Zaremba (1:53:38.500)

Uh, so I would record data and then I would essentially train supervised

Lex Fridman (1:53:43.500)

learning model on it.

Wojciech Zaremba (1:53:45.020)

That might be the path today.

Lex Fridman (1:53:47.220)

Long term.

Wojciech Zaremba (1:53:47.860)

I think that actually what is needed is to have a robot that can

Lex Fridman (1:53:52.340)

train powerful models over video.

Wojciech Zaremba (1:53:55.580)

So, um, you have seen maybe a models that can generate images like Dali and people

Lex Fridman (1:54:02.740)

are looking into models, generating videos, they're like, uh, bodies,

Wojciech Zaremba (1:54:06.300)

algorithmic questions, even how to do it.

Lex Fridman (1:54:08.500)

And it's unclear if there is enough compute for this purpose, but, uh, I, I

Wojciech Zaremba (1:54:13.220)

suspect that the models that which would have a level of understanding of video,

Lex Fridman (1:54:19.300)

same as GPT has a level of understanding of text, could be used, uh, to train

Wojciech Zaremba (1:54:25.620)

robots to solve tasks.

Lex Fridman (1:54:26.580)

They would have a lot of common sense.

Wojciech Zaremba (1:54:29.780)

If one day, I'm pretty sure one day there will be a robotics company by robotics

Lex Fridman (1:54:36.420)

company, I mean, the primary source of income is, is from robots that is worth

Wojciech Zaremba (1:54:42.740)

over $1 trillion.

Lex Fridman (1:54:44.740)

What do you think that company will do?

Wojciech Zaremba (1:54:49.940)

I think self driving cars.

Lex Fridman (1:54:51.620)

No, it's interesting.

Wojciech Zaremba (1:54:53.260)

Cause my mind went to personal robotics, robots in the home.

Lex Fridman (1:54:57.220)

It seems like there's much more market opportunity there.

Wojciech Zaremba (1:55:00.300)

I think it's very difficult to achieve.

Wojciech Zaremba (1:55:04.420)

I mean, this, this, this might speak to something important, which is I understand

Wojciech Zaremba (1:55:09.460)

self driving much better than understand robotics in the home.

Lex Fridman (1:55:12.180)

So I understand how difficult it is to actually solve self driving to a, to a

Wojciech Zaremba (1:55:17.500)

level, not just the actual computer vision and the control problem and just the

Lex Fridman (1:55:22.060)

basic problem of self driving, but creating a product that would undeniably

Wojciech Zaremba (1:55:28.100)

be, um, that will cost less money.

Lex Fridman (1:55:31.300)

Like it will save you a lot of money, like orders of magnitude, less money

Wojciech Zaremba (1:55:34.220)

that could replace Uber drivers, for example.

Lex Fridman (1:55:36.780)

So car sharing that's autonomous, that creates a similar or better

Wojciech Zaremba (1:55:41.380)

experience in terms of how quickly you get from A to B or just whatever, the

Lex Fridman (1:55:46.220)

pleasantness of the experience, the efficiency of the experience, the value

Wojciech Zaremba (1:55:50.260)

of the experience, and at the same time, the car itself costs cheaper.

Lex Fridman (1:55:55.300)

I think that's very difficult to achieve.

Wojciech Zaremba (1:55:57.340)

I think there's a lot more, um, low hanging fruit in the home.

Lex Fridman (1:56:03.780)

That, that, that could be, I also want to give you a perspective on like how

Wojciech Zaremba (1:56:08.340)

challenging it would be at home or like it maybe kind of depends on that exact

Lex Fridman (1:56:12.900)

problem that you'd be solving.

Wojciech Zaremba (1:56:14.100)

Like if we're speaking about these robotic arms and hands, these things,

Lex Fridman (1:56:20.220)

they cost tens of thousands of dollars or maybe a hundred K and, um, you know,

Wojciech Zaremba (1:56:27.580)

maybe, obviously, maybe there would be economy of scale.

Lex Fridman (1:56:30.260)

These things would be cheaper, but actually for any household to buy it,

Wojciech Zaremba (1:56:34.540)

the price would have to go down to maybe a thousand bucks.

Lex Fridman (1:56:37.340)

Yeah.

Wojciech Zaremba (1:56:38.340)

I personally think that, uh, so self driving car, it provides a clear service.

Lex Fridman (1:56:44.500)

I don't think robots in the home, there'll be a trillion dollar company

Wojciech Zaremba (1:56:48.180)

will just be all about service, meaning it will not necessarily be about like

Lex Fridman (1:56:53.260)

a robotic arm that's helps you.

Wojciech Zaremba (1:56:56.100)

I don't know, open a bottle or wash the dishes or, uh, any of that kind of stuff.

Lex Fridman (1:57:02.580)

It has to be able to take care of that whole, the therapist thing.

Wojciech Zaremba (1:57:05.940)

You mentioned, I think that's, um, of course there's a line between what

Lex Fridman (1:57:10.700)

is a robot and what is not like, does it really need a body?

Lex Fridman (1:57:14.460)

But you know, some, um, uh, AI system with some embodiment, I think.

Lex Fridman (1:57:20.340)

So the tricky part when you think actually what's the difficult part is,

Wojciech Zaremba (1:57:24.260)

um, when the robot has like, when there is a diversity of the environment

Lex Fridman (1:57:29.940)

with which the robot has to interact, that becomes hard.

Wojciech Zaremba (1:57:31.980)

So, you know, on the one spectrum, you have, uh, industrial robots as they

Lex Fridman (1:57:36.740)

are doing over and over the same thing, it is possible to some extent to

Wojciech Zaremba (1:57:40.900)

prescribe the movements and we've very small amount of intelligence, the, the

Lex Fridman (1:57:46.300)

movement can be repeated millions of times.

Wojciech Zaremba (1:57:48.100)

Um, the, it, there are also, you know, various pieces of industrial robots

Lex Fridman (1:57:52.700)

where it becomes harder and harder.

Wojciech Zaremba (1:57:54.500)

I can, for instance, in case of Tesla, it might be a matter of putting a, a

Lex Fridman (1:57:59.460)

rack inside of a car and, you know, because the rack kind of moves around,

Wojciech Zaremba (1:58:03.860)

it's, uh, it's not that easy.

Lex Fridman (1:58:05.580)

It's not exactly the same every time.

Wojciech Zaremba (1:58:08.100)

That's not being the case that you need actually humans to do it.

Lex Fridman (1:58:11.500)

Uh, while, you know, welding cars together, it's a very repetitive process.

Wojciech Zaremba (1:58:16.100)

Um, then in case of self driving itself, uh, that difficulty has to do with the

Lex Fridman (1:58:23.460)

diversity of the environment, but still the car itself, um, the problem

Wojciech Zaremba (1:58:27.860)

that they are solving is you try to avoid even interacting with things.

Lex Fridman (1:58:32.540)

You are not touching anything around because touching itself is hard.

Lex Fridman (1:58:36.140)

And then if you would have in the home, uh, robot that, you know, has to

Wojciech Zaremba (1:58:40.580)

touch things and like if these things, they change the shape, if there is a huge

Wojciech Zaremba (1:58:44.140)

variety of things to be touched, then that's difficult.

Lex Fridman (1:58:46.860)

If you are speaking about the robot, which there is, you know, head that

Wojciech Zaremba (1:58:50.300)

is smiling in some way with cameras that either doesn't, you know, touch things.

Lex Fridman (1:58:54.660)

That's relatively simple.

Wojciech Zaremba (1:58:55.900)

Okay. So to both agree and to push back.

Lex Fridman (1:59:00.060)

So you're referring to touch, like soft robotics, like the actual touch, but.

Wojciech Zaremba (1:59:08.060)

I would argue that you could formulate just basic interaction between, um, like

Wojciech Zaremba (1:59:13.900)

non contact interaction is also a kind of touch and that might be very difficult

Wojciech Zaremba (1:59:18.660)

to solve that's the basic, this not disagreement, but that's the basic open

Lex Fridman (1:59:22.620)

question to me with self driving cars and this agreement with Elon, which

Wojciech Zaremba (1:59:27.540)

is how much interaction is required to solve self driving cars.

Lex Fridman (1:59:31.260)

How much touch is required?

Wojciech Zaremba (1:59:33.180)

You said that in your intuition, touch is not required.

Lex Fridman (1:59:37.380)

And my intuition to create a product that's compelling to use, you're going

Wojciech Zaremba (1:59:41.820)

to have to, uh, interact with pedestrians, not just avoid pedestrians,

Lex Fridman (1:59:46.740)

but interact with them when we drive around.

Wojciech Zaremba (1:59:49.980)

In major cities, we're constantly threatening everybody's life with

Lex Fridman (1:59:54.100)

our movements, um, and that's how they respect us.

Wojciech Zaremba (1:59:57.740)

There's a game to ready going out with pedestrians and I'm afraid you can't

Lex Fridman (20:05.400)

you were a kid that actually are not serving you anymore.

Lex Fridman (20:08.600)

And you also might be thinking that that's who you are and that's actually just a story.

Lex Fridman (20:13.320)

Mm hmm.

Wojciech Zaremba (20:15.040)

Yeah.

Lex Fridman (20:15.240)

So it's a useful hack, but sometimes it gets us into trouble.

Wojciech Zaremba (20:18.240)

It's a local optima.

Lex Fridman (20:19.360)

It's a local optima.

Wojciech Zaremba (20:20.200)

You wrote that Stephen Hawking, he tweeted, Stephen Hawking asked what

Lex Fridman (20:24.880)

breathes fire into equations, which meant what makes given mathematical

Wojciech Zaremba (20:29.440)

equations realize the physics of a universe.

Lex Fridman (20:33.120)

Similarly, I wonder what breathes fire into computation.

Lex Fridman (20:37.520)

What makes given computation conscious?

Lex Fridman (20:40.600)

Okay.

Lex Fridman (20:41.240)

So how do we engineer consciousness?

Lex Fridman (20:44.400)

How do you breathe fire and magic?

Lex Fridman (20:47.280)

How do you breathe fire and magic into the machine?

Lex Fridman (20:51.800)

So, um, it seems clear to me that not every computation is conscious.

Wojciech Zaremba (20:57.280)

I mean, you can, let's say, just keep on multiplying one matrix over and over

Lex Fridman (21:01.520)

again and might be gigantic matrix.

Wojciech Zaremba (21:03.920)

You can put a lot of computation.

Lex Fridman (21:05.480)

I don't think it would be conscious.

Lex Fridman (21:07.080)

So in some sense, the question is, uh, what are the computations which could be

Lex Fridman (21:13.000)

conscious, uh, I mean, so, so one assumption is that it has to do purely

Wojciech Zaremba (21:18.280)

with computation that you can abstract away matter and other possibilities

Lex Fridman (21:22.160)

that it's very important was the realization of computation that it has

Wojciech Zaremba (21:25.400)

to do with some, uh, uh, force fields or so, and they bring consciousness.

Lex Fridman (21:30.520)

At the moment, my intuition is that it can be fully abstracted away.

Lex Fridman (21:33.680)

So in case of computation, you can ask yourself, what are the mathematical

Lex Fridman (21:38.280)

objects or so that could bring such a properties?

Lex Fridman (21:41.440)

So for instance, if we think about the models, uh, AI models, the, what they

Lex Fridman (21:49.000)

truly try to do, uh, or like a models like GPT is, uh, uh, you know, they try

Wojciech Zaremba (21:57.000)

to predict, uh, next word or so.

Lex Fridman (22:00.480)

And this turns out to be equivalent to, uh, compressing, uh, text.

Wojciech Zaremba (22:05.920)

Um, and, uh, because in some sense, compression means that, uh, you learn

Wojciech Zaremba (22:11.120)

the model of reality and you have just to, uh, remember where are your mistakes.

Wojciech Zaremba (22:16.320)

The better you are in predicting the, and, and, and in some sense, when we

Lex Fridman (22:20.640)

look at our experience, also, when you look, for instance, at the car driving,

Wojciech Zaremba (22:24.080)

you know, in which direction it will go, you are good like in prediction.

Lex Fridman (22:27.720)

And, um, you know, it might be the case that the consciousness is intertwined

Wojciech Zaremba (22:32.880)

with, uh, compression, it might be also the case that self consciousness, uh,

Lex Fridman (22:38.400)

has to do with compress or trying to compress itself.

Wojciech Zaremba (22:41.280)

So, um, okay.

Lex Fridman (22:43.600)

I was just wondering, what are the objects in, you know, mathematics or

Wojciech Zaremba (22:47.640)

computer science, which are mysterious that could, uh, that, that, that could

Lex Fridman (22:52.360)

have to do with consciousness.

Lex Fridman (22:53.520)

And then I thought, um, you know, you, you see in mathematics, there is

Lex Fridman (22:59.680)

something called Gadel theorem, uh, which means, okay, you have, if you have

Wojciech Zaremba (23:03.720)

sufficiently complicated mathematical system, it is possible to point the

Lex Fridman (23:08.440)

mathematical system back on itself.

Wojciech Zaremba (23:10.800)

In computer science, there is, uh, something called helping problem.

Lex Fridman (23:14.280)

It's, it's somewhat similar construction.

Lex Fridman (23:16.800)

So I thought that, you know, if we believe that, uh, that, uh, that under

Lex Fridman (23:22.960)

assumption that consciousness has to do with, uh, with compression, uh, then

Wojciech Zaremba (23:28.320)

you could imagine that the, that the, as you keep on compressing things, then

Lex Fridman (23:32.760)

at some point, it actually makes sense for the compressor to compress itself.

Wojciech Zaremba (23:36.720)

Metacompression consciousness is metacompression.

Lex Fridman (23:40.760)

That's a, that's an I, an, an, an idea.

Lex Fridman (23:44.360)

And in some sense, you know, the crazy, thank you.

Lex Fridman (23:47.280)

So, uh, but do you think if we think of a Turing machine, a universal

Lex Fridman (23:52.280)

Turing machine, can that achieve consciousness?

Lex Fridman (23:55.880)

So is there some thing beyond our traditional definition

Lex Fridman (24:00.240)

of computation that's required?

Lex Fridman (24:02.200)

So it's a specific computation.

Lex Fridman (24:03.920)

And I said, this computation has to do with compression and, uh, the compression

Lex Fridman (24:08.760)

itself, maybe other way of putting it is like, uh, you are internally creating

Wojciech Zaremba (24:13.040)

the model of reality in order, like, uh, it's like a, you try inside to simplify

Lex Fridman (24:18.040)

reality in order to predict what's going to happen.

Wojciech Zaremba (24:20.200)

And, um, that also feels somewhat similar to how I think actually about my own

Lex Fridman (24:25.200)

conscious experience, though clearly I don't have access to reality.

Wojciech Zaremba (24:29.040)

The only access to reality is through, you know, cable going to my brain and my

Wojciech Zaremba (24:33.240)

brain is creating a simulation of reality and I have access to the simulation of

Wojciech Zaremba (24:37.320)

reality.

Lex Fridman (24:38.400)

Are you by any chance, uh, aware of, uh, the Hutter prize, Marcus Hutter?

Wojciech Zaremba (24:44.200)

He, uh, he made this prize for compression.

Lex Fridman (24:48.160)

Uh, Wikipedia pages, and, uh, there's a few qualities to it.

Wojciech Zaremba (24:53.560)

One, I think has to be perfect compression, which makes, I think that

Lex Fridman (24:57.640)

little cork makes it much less, um, applicable to the general task of

Wojciech Zaremba (25:03.520)

intelligence, because it feels like intelligence is always going to be messy.

Lex Fridman (25:07.720)

Uh, like perfect compression is feels like it's not the right goal, but

Wojciech Zaremba (25:14.280)

it's nevertheless a very interesting goal.

Lex Fridman (25:19.280)

So for him, intelligence equals compression.

Lex Fridman (25:22.680)

And so the smaller you make the file, given a large Wikipedia page, the

Lex Fridman (25:29.240)

more intelligent the system has to be.

Wojciech Zaremba (25:31.200)

Yeah, that makes sense.

Lex Fridman (25:31.920)

So you can make perfect compression if you store errors.

Lex Fridman (25:34.920)

And I think that actually what he meant is you have algorithm plus errors.

Lex Fridman (25:37.960)

Uh, by the way, Hutter, Hutter is, uh, he was a PhD advisor of Sean

Wojciech Zaremba (25:44.720)

Leck, who is a DeepMind, uh, uh, DeepMind cofounder.

Lex Fridman (25:48.600)

Yeah.

Wojciech Zaremba (25:49.080)

Yeah.

Lex Fridman (25:49.360)

So there's an interesting, uh, and now he's a DeepMind, there's an

Wojciech Zaremba (25:53.600)

interesting, uh, network of people.

Lex Fridman (25:55.720)

And he's one of the people that I think seriously took on the task of

Lex Fridman (26:02.680)

what would an AGI system look like?

Lex Fridman (26:04.960)

Uh, I think for a longest time, the question of AGI was not taken

Wojciech Zaremba (26:12.680)

seriously or rather rigorously.

Lex Fridman (26:15.640)

And he did just that, like mathematically speaking, what

Wojciech Zaremba (26:19.800)

would the model look like if you remove the constraints of it, having to be,

Lex Fridman (26:23.440)

uh, um, having to have a reasonable amount of memory, reasonable amount

Wojciech Zaremba (26:31.880)

of, uh, running time, complexity, uh, computation time, what would it look

Lex Fridman (26:36.400)

like and essentially it's, it's a half math, half philosophical discussion

Wojciech Zaremba (26:41.760)

of, uh, how would it like a reinforcement learning type of

Lex Fridman (26:45.240)

framework look like for an AGI?

Wojciech Zaremba (26:47.520)

Yeah.

Lex Fridman (26:47.800)

So he developed the framework even to describe what's optimal with

Wojciech Zaremba (26:51.640)

respect to reinforcement learning.

Lex Fridman (26:53.240)

Like there is a theoretical framework, which is, as you said, under assumption,

Wojciech Zaremba (26:57.040)

there is infinite amount of memory and compute.

Lex Fridman (26:59.000)

Um, there was actually one person before his name is Solomonov, who

Wojciech Zaremba (27:03.560)

there extended, uh, Solomonov work to reinforcement learning, but there

Lex Fridman (27:07.840)

exists the, uh, theoretical algorithm, which is optimal algorithm to build

Wojciech Zaremba (27:13.840)

intelligence and I can actually explain you the algorithm.

Lex Fridman (27:16.560)

Yes.

Wojciech Zaremba (27:18.080)

Let's go.

Lex Fridman (27:18.960)

Let's go.

Lex Fridman (27:19.880)

So the task itself, can I just pause how absurd it is for brain in a

Lex Fridman (27:26.680)

skull, trying to explain the algorithm for intelligence, just go ahead.

Wojciech Zaremba (27:31.120)

It is pretty crazy.

Lex Fridman (27:32.160)

It is pretty crazy that, you know, the brain itself is actually so

Wojciech Zaremba (27:34.640)

small and it can ponder, uh, how to design algorithms that optimally

Lex Fridman (27:40.960)

solve the problem of intelligence.

Wojciech Zaremba (27:42.560)

Okay.

Lex Fridman (27:43.440)

All right.

Lex Fridman (27:43.640)

So what's the algorithm?

Lex Fridman (27:44.920)

So let's see.

Lex Fridman (27:46.120)

So first of all, the task itself is, uh, described as, uh, you have infinite

Lex Fridman (27:51.560)

sequence of zeros and ones.

Wojciech Zaremba (27:53.560)

Okay.

Lex Fridman (27:53.840)

Okay. You read, uh, N bits and they are about to predict N plus one bit.

Lex Fridman (27:59.120)

So that's the task.

Lex Fridman (28:00.160)

And you could imagine that every task could be casted as such a task.

Lex Fridman (28:04.440)

So if for instance, you have images and labels, you can just turn every image

Lex Fridman (28:08.800)

into a sequence of zeros and ones, then label, you concatenate labels and

Wojciech Zaremba (28:12.960)

you, and that that's actually the, the, and you could, you could start by

Lex Fridman (28:16.680)

having training data first, and then afterwards you have test data.

Lex Fridman (28:20.480)

So theoretically any problem could be casted as a problem of predicting

Lex Fridman (28:25.480)

zeros and ones on this, uh, infinite tape.

Wojciech Zaremba (28:28.320)

So, um, so let's say you read already N bits and you want to predict N plus

Lex Fridman (28:35.240)

one bit, and I will ask you to write every possible program that generates

Wojciech Zaremba (28:42.160)

these N bits.

Lex Fridman (28:43.560)

Okay.

Wojciech Zaremba (28:43.760)

So, um, and you can have, you, you choose programming language.

Lex Fridman (28:47.880)

It can be Python or C plus plus.

Lex Fridman (28:49.720)

And the difference between programming languages, uh, might be, there is

Lex Fridman (28:53.480)

a difference by constant asymptotically, your predictions will be equivalent.

Lex Fridman (28:59.160)

So you read N bits, you enumerate all the programs that produce

Lex Fridman (29:04.080)

these N bits in their output.

Lex Fridman (29:06.680)

And then in order to predict N plus one bit, you actually weight the programs

Lex Fridman (29:13.480)

according to their length.

Lex Fridman (29:15.440)

And there is like a, some specific formula, how you weight them.

Lex Fridman (29:18.480)

And then the N plus, uh, one bit prediction is the prediction, uh, from each

Wojciech Zaremba (29:24.120)

of these program, according to that weight.

Lex Fridman (29:27.040)

Like statistically, you pick, so the smaller the program, the more likely

Wojciech Zaremba (29:31.880)

you, you are to pick the, its output.

Lex Fridman (29:35.480)

So, uh, that's, that algorithm is grounded in the hope or the intuition

Wojciech Zaremba (29:42.280)

that the simple answer is the right one.

Lex Fridman (29:44.600)

It's a formalization of it.

Wojciech Zaremba (29:46.000)

Um, it also means like, if you would ask the question after how many years

Lex Fridman (29:52.600)

would, you know, sun explode, uh, you can say, hmm, it's more likely

Wojciech Zaremba (29:58.080)

the answer is due to some power because they're shorter program.

Lex Fridman (2:00:02.940)

just formulate autonomous driving as a collision avoidance problem.

Lex Fridman (2:00:08.820)

So I think it goes beyond like a collision avoidance is the

Lex Fridman (2:00:12.380)

first order approximation.

Wojciech Zaremba (2:00:14.180)

Uh, but then at least in case of Tesla, you can't just

Lex Fridman (2:00:18.420)

at least in case of Tesla, they are gathering data from people driving their

Wojciech Zaremba (2:00:22.500)

cars and I believe that's an example of supervised data that they can train

Lex Fridman (2:00:27.220)

their models, uh, on, and they are doing it, uh, which, you know, can give

Wojciech Zaremba (2:00:32.900)

a model dislike, uh, another level of, uh, of, uh, behavior that is needed

Lex Fridman (2:00:38.900)

to actually interact with the real world.

Wojciech Zaremba (2:00:41.140)

Yeah.

Lex Fridman (2:00:41.340)

It's interesting how much data is required to achieve that.

Wojciech Zaremba (2:00:45.340)

Um, w what do you think of the whole Tesla autopilot approach, the computer

Lex Fridman (2:00:49.380)

vision based approach with multiple cameras and there's a data engine.

Wojciech Zaremba (2:00:53.380)

It's a multitask, multiheaded neural network, and it's this fascinating

Lex Fridman (2:00:57.820)

process of, uh, similar to what you're talking about with the robotics

Wojciech Zaremba (2:01:02.780)

approach, uh, which is, you know, you deploy in your own network and

Lex Fridman (2:01:06.540)

then there's humans that use it and then it runs into trouble in a bunch

Wojciech Zaremba (2:01:10.940)

of places and that stuff is sent back.

Lex Fridman (2:01:12.780)

So like the deployment discovers a bunch of edge cases and those edge

Wojciech Zaremba (2:01:17.740)

cases are sent back for supervised annotation, thereby improving the

Lex Fridman (2:01:22.140)

neural network and that's deployed again.

Wojciech Zaremba (2:01:24.540)

It goes over and over until the network becomes really good at the task of

Lex Fridman (2:01:29.340)

driving becomes safer and safer.

Lex Fridman (2:01:31.580)

What do you think of that kind of approach to robotics?

Lex Fridman (2:01:34.700)

I believe that's the way to go.

Lex Fridman (2:01:36.100)

So in some sense, even when I was speaking about, you know, collecting

Lex Fridman (2:01:39.660)

trajectories from humans, that's like a first step and then you deploy

Wojciech Zaremba (2:01:43.180)

the system and then you have humans revising the, all the issues.

Lex Fridman (2:01:46.620)

And in some sense, like at this approach converges to system that doesn't make

Wojciech Zaremba (2:01:51.580)

mistakes because for the cases where there are mistakes, you got their

Lex Fridman (2:01:54.700)

data, how to fix them and the system will keep on improving.

Lex Fridman (2:01:58.220)

So there's a very, to me, difficult question of how hard that, you know,

Lex Fridman (2:02:02.460)

how long that converging takes, how hard it is.

Wojciech Zaremba (2:02:04.940)

The other aspect of autonomous vehicles, this probably applies to certain

Lex Fridman (2:02:09.180)

robotics applications is society, right?

Wojciech Zaremba (2:02:12.700)

They put as, as the quality of the system converges.

Lex Fridman (2:02:18.220)

So one, there's a human factors perspective of psychology of humans being

Wojciech Zaremba (2:02:21.820)

able to supervise those even with teleoperation, those robots.

Lex Fridman (2:02:25.740)

And the other is society willing to accept robots.

Wojciech Zaremba (2:02:29.100)

Currently society is much harsher on self driving cars than it is on human

Lex Fridman (2:02:32.540)

driven cars in terms of the expectation of safety.

Lex Fridman (2:02:35.660)

So the bar is set much higher than for humans.

Lex Fridman (2:02:39.100)

And so if there's a death in an autonomous vehicle, that's seen as a much more,

Wojciech Zaremba (2:02:47.180)

much more dramatic than a death in the human driven vehicle.

Lex Fridman (2:02:50.940)

Part of the success of deployment of robots is figuring out how to make robots

Wojciech Zaremba (2:02:55.260)

part of society, both on the, just the human side, on the media side, on the

Lex Fridman (2:03:01.100)

media journalist side, and also on the policy government side.

Lex Fridman (2:03:04.780)

And that seems to be, maybe you can put that into the objective function to

Lex Fridman (2:03:08.620)

optimize, but that is, that is definitely a tricky one.

Lex Fridman (2:03:14.860)

And I wonder if that is actually the trickiest part for self driving cars or

Lex Fridman (2:03:18.460)

any system that's safety critical.

Wojciech Zaremba (2:03:21.340)

It's not the algorithm, it's the society accepting it.

Wojciech Zaremba (2:03:24.460)

Yeah, I would say, I believe that the part of the process of deployment is actually

Wojciech Zaremba (2:03:31.020)

showing people that the given things can be trusted and, you know, trust is also

Lex Fridman (2:03:36.860)

like a glass that is actually really easy to crack it and damage it.

Lex Fridman (2:03:43.100)

And I think that's actually very common with, with innovation, that there's

Lex Fridman (2:03:52.300)

some resistance toward it and it's just the natural progression.

Lex Fridman (2:03:56.620)

So in some sense, people will have to keep on proving that indeed these

Lex Fridman (2:04:00.140)

systems are worth being used.

Lex Fridman (2:04:02.780)

And I would say, I also found out that often the best way to convince people

Lex Fridman (2:04:09.420)

is by letting them experience it.

Wojciech Zaremba (2:04:11.660)

Yeah, absolutely.

Lex Fridman (2:04:12.540)

That's the case with Tesla autopilot, for example, that's the case with, yeah,

Wojciech Zaremba (2:04:17.180)

with basically robots in general.

Lex Fridman (2:04:18.940)

It's kind of funny to hear people talk about robots.

Wojciech Zaremba (2:04:22.220)

Like there's a lot of fear, even with like legged robots, but when they

Lex Fridman (2:04:27.420)

actually interact with them, there's joy.

Wojciech Zaremba (2:04:31.420)

I love interacting with them.

Lex Fridman (2:04:32.780)

And the same with the car, with a robot, if it starts being useful, I think

Wojciech Zaremba (2:04:38.860)

people immediately understand.

Lex Fridman (2:04:40.460)

And if the product is designed well, they fall in love.

Wojciech Zaremba (2:04:43.340)

You're right.

Lex Fridman (2:04:44.300)

It's actually even similar when I'm thinking about the car.

Wojciech Zaremba (2:04:46.940)

It's actually even similar when I'm thinking about Copilot, the GitHub Copilot.

Lex Fridman (2:04:51.260)

There was a spectrum of responses that people had.

Lex Fridman (2:04:54.460)

And ultimately the important piece was to let people try it out.

Lex Fridman (2:05:00.140)

And then many people just loved it.

Wojciech Zaremba (2:05:02.620)

Especially like programmers.

Lex Fridman (2:05:05.020)

Yeah, programmers, but like some of them, you know, they came with a fear.

Wojciech Zaremba (2:05:08.300)

Yeah.

Lex Fridman (2:05:08.860)

But then you try it out and you think, actually, that's cool.

Wojciech Zaremba (2:05:11.820)

And, you know, you can try to resist the same way as, you know, you could

Lex Fridman (2:05:15.180)

resist moving from punch cards to, let's say, C++ or so.

Lex Fridman (2:05:20.860)

And it's a little bit futile.

Lex Fridman (2:05:23.980)

So we talked about generation of program, generation of language, even

Wojciech Zaremba (2:05:30.540)

self supervised learning in the visual space for robotics and then

Lex Fridman (2:05:33.820)

reinforcement learning.

Lex Fridman (2:05:35.100)

What do you, in like this whole beautiful spectrum of AI, do you think is a

Lex Fridman (2:05:40.700)

good benchmark, a good test to strive for to achieve intelligence?

Wojciech Zaremba (2:05:47.740)

That's a strong test of intelligence.

Lex Fridman (2:05:49.820)

You know, it started with Alan Turing and the Turing test.

Wojciech Zaremba (2:05:53.260)

Maybe you think natural language conversation is a good test.

Lex Fridman (2:05:57.100)

So, you know, it would be nice if, for instance, machine would be able to

Wojciech Zaremba (2:06:01.340)

solve Riemann hypothesis in math.

Lex Fridman (2:06:04.540)

That would be, I think that would be very impressive.

Lex Fridman (2:06:07.420)

So theorem proving, is that to you, proving theorems is a good, oh, oh,

Lex Fridman (2:06:12.940)

like one thing that the machine did, you would say, damn.

Wojciech Zaremba (2:06:16.460)

Exactly.

Lex Fridman (2:06:18.460)

Okay.

Wojciech Zaremba (2:06:19.420)

That would be quite, quite impressive.

Lex Fridman (2:06:22.460)

I mean, the tricky part about the benchmarks is, you know, as we are

Wojciech Zaremba (2:06:26.940)

getting closer with them, we have to invent new benchmarks.

Lex Fridman (2:06:29.340)

There is actually no ultimate benchmark out there.

Wojciech Zaremba (2:06:31.660)

Yeah.

Lex Fridman (2:06:31.820)

See, my thought with the Riemann hypothesis would be the moment the

Wojciech Zaremba (2:06:36.140)

machine proves it, we would say, okay, well then the problem was easy.

Lex Fridman (2:06:40.860)

That's what happens.

Lex Fridman (2:06:42.060)

And I mean, in some sense, that's actually what happens over the years

Lex Fridman (2:06:46.140)

in AI that like, we get used to things very quickly.

Wojciech Zaremba (2:06:50.380)

You know something, I talked to Rodney Brooks.

Lex Fridman (2:06:52.300)

I don't know if you know who that is.

Wojciech Zaremba (2:06:54.380)

He called AlphaZero homework problem.

Lex Fridman (2:06:57.020)

Cause he was saying like, there's nothing special about it.

Wojciech Zaremba (2:06:59.740)

It's not a big leap.

Lex Fridman (2:07:00.780)

And I didn't, well, he's coming from one of the aspects that we referred

Wojciech Zaremba (2:07:05.260)

to is he was part of the founding of iRobot, which deployed now tens

Lex Fridman (2:07:10.140)

of millions of robot in the home.

Lex Fridman (2:07:11.900)

So if you see robots that are actually in the homes of people as the

Lex Fridman (2:07:18.540)

legitimate instantiation of artificial intelligence, then yes, maybe an AI

Wojciech Zaremba (2:07:23.340)

that plays a silly game like go and chess is not a real accomplishment,

Lex Fridman (2:07:26.460)

but to me it's a fundamental leap.

Lex Fridman (2:07:29.180)

But I think we as humans then say, okay, well then that that game of

Lex Fridman (2:07:33.740)

chess or go wasn't that difficult compared to the thing that's currently

Wojciech Zaremba (2:07:37.660)

unsolved.

Lex Fridman (2:07:38.220)

So my intuition is that from perspective of the evolution of these AI

Wojciech Zaremba (2:07:44.940)

systems will at first seen the tremendous progress in digital space.

Lex Fridman (2:07:49.820)

And the, you know, the main thing about digital space is also that you

Wojciech Zaremba (2:07:52.700)

can, everything is that there is a lot of recorded data.

Lex Fridman (2:07:56.300)

Plus you can very rapidly deploy things to billions of people.

Wojciech Zaremba (2:07:59.900)

While in case of a physical space, the deployment part takes multiple

Lex Fridman (2:08:05.260)

years.

Wojciech Zaremba (2:08:05.500)

You have to manufacture things and, you know, delivering it to actual

Lex Fridman (2:08:10.300)

people, it's very hard.

Lex Fridman (2:08:13.580)

So I'm expecting that the first and that prices in digital space of

Lex Fridman (2:08:19.980)

goods, they would go, you know, down to the, let's say marginal costs

Wojciech Zaremba (2:08:24.220)

are two zero.

Lex Fridman (2:08:25.020)

And also the question is how much of our life will be in digital because

Wojciech Zaremba (2:08:28.780)

it seems like we're heading towards more and more of our lives being in

Lex Fridman (2:08:31.980)

the digital space.

Lex Fridman (2:08:33.260)

So like innovation in the physical space might become less and less

Lex Fridman (2:08:37.100)

significant.

Wojciech Zaremba (2:08:38.060)

Like why do you need to drive anywhere if most of your life is spent in

Lex Fridman (2:08:42.700)

virtual reality?

Wojciech Zaremba (2:08:44.060)

I still would like, you know, to at least at the moment, my impression

Lex Fridman (2:08:47.980)

is that I would like to have a physical contact with other people.

Lex Fridman (2:08:51.020)

And that's very important to me.

Lex Fridman (2:08:52.940)

We don't have a way to replicate it in the computer.

Wojciech Zaremba (2:08:55.180)

It might be the case that over the time it will change.

Lex Fridman (2:08:57.260)

Like in 10 years from now, why not have like an arbitrary infinite number

Lex Fridman (2:09:02.380)

of people you can interact with?

Lex Fridman (2:09:04.060)

Some of them are real, some are not with arbitrary characteristics that

Wojciech Zaremba (2:09:09.740)

you can define based on your own preferences.

Lex Fridman (2:09:12.700)

I think that's maybe where we are heading and maybe I'm resisting the

Wojciech Zaremba (2:09:15.900)

future.

Lex Fridman (2:09:16.460)

Yeah, I'm telling you, if I got to choose, if I could live in Elder

Wojciech Zaremba (2:09:25.100)

Scrolls Skyrim versus the real world, I'm not so sure I would stay with

Lex Fridman (2:09:29.820)

the real world.

Wojciech Zaremba (2:09:31.420)

Yeah, I mean, the question is, so will VR be sufficient to get us there

Lex Fridman (2:09:35.900)

or do you need to, you know, plug electrodes in the brain?

Lex Fridman (2:09:40.140)

And it would be nice if these electrodes wouldn't be invasive.

Lex Fridman (2:09:45.020)

Or at least like provably non destructive.

Lex Fridman (2:09:49.020)

But in the digital space, do you think we'll be able to solve the

Lex Fridman (2:09:53.420)

Turing test, the spirit of the Turing test, which is, do you think we'll

Wojciech Zaremba (2:09:57.020)

be able to achieve compelling natural language conversation between

Lex Fridman (2:10:02.380)

people, like have friends that are AI systems on the internet?

Wojciech Zaremba (2:10:07.100)

I totally think it's doable.

Lex Fridman (2:10:08.780)

Do you think the current approach of GPT will take us there?

Lex Fridman (2:10:12.460)

So there is, you know, the part of at first learning all the content

Lex Fridman (2:10:16.700)

out there and I think that Steel System should keep on learning as

Wojciech Zaremba (2:10:20.060)

it speaks with you.

Lex Fridman (2:10:21.260)

Yeah.

Wojciech Zaremba (2:10:21.500)

Yeah, and I think that should work.

Lex Fridman (2:10:23.900)

The question is how exactly to do it.

Wojciech Zaremba (2:10:25.660)

And, you know, obviously we have people at OpenAI asking these

Lex Fridman (2:10:29.740)

questions and kind of at first pre training on all existing content

Wojciech Zaremba (2:10:35.100)

is like a backbone and is a decent backbone.

Lex Fridman (2:10:39.340)

Do you think AI needs a body connecting to our robotics question to

Wojciech Zaremba (2:10:45.820)

truly connect with humans or can most of the connection be in the

Lex Fridman (2:10:49.100)

digital space?

Lex Fridman (2:10:49.820)

So let's see, we know that there are people who met each other online

Lex Fridman (2:10:55.260)

and they fell in love.

Wojciech Zaremba (2:10:57.740)

Yeah.

Lex Fridman (2:10:58.620)

So it seems that it's conceivable to establish connection, which is

Wojciech Zaremba (2:11:03.740)

purely through internet.

Lex Fridman (2:11:07.340)

Of course, it might be more compelling the more modalities you add.

Lex Fridman (2:11:12.140)

So it would be like you're proposing like a Tinder, but for AI, you

Lex Fridman (2:11:16.620)

like swipe right and left and half the systems are AI and the other is

Wojciech Zaremba (2:11:21.100)

humans and you don't know which is which.

Lex Fridman (2:11:24.380)

That would be our formulation of Turing test.

Wojciech Zaremba (2:11:27.980)

The moment AI is able to achieve more swipe right or left, whatever,

Lex Fridman (2:11:33.260)

the moment it's able to be more attractive than other humans, it

Wojciech Zaremba (2:11:36.940)

passes the Turing test.

Lex Fridman (2:11:38.060)

Then you would pass the Turing test in attractiveness.

Wojciech Zaremba (2:11:40.620)

That's right.

Lex Fridman (2:11:41.100)

Well, no, like attractiveness just to clarify.

Wojciech Zaremba (2:11:42.940)

There will be conversation.

Lex Fridman (2:11:44.060)

Not just visual.

Wojciech Zaremba (2:11:44.780)

Right, right.

Lex Fridman (2:11:45.260)

It's also attractiveness with wit and humor and whatever makes

Wojciech Zaremba (2:11:51.660)

conversation is pleasant for humans.

Lex Fridman (2:11:56.060)

Okay.

Wojciech Zaremba (2:11:56.700)

All right.

Lex Fridman (2:11:58.780)

So you're saying it's possible to achieve in the digital space.

Wojciech Zaremba (2:12:02.620)

In some sense, I would almost ask that question.

Lex Fridman (2:12:05.180)

Why wouldn't that be possible?

Wojciech Zaremba (2:12:07.980)

Well, I have this argument with my dad all the time.

Lex Fridman (2:12:11.180)

He thinks that touch and smell are really important.

Lex Fridman (2:12:13.820)

So they can be very important.

Lex Fridman (2:12:16.700)

And I'm saying the initial systems, they won't have it.

Wojciech Zaremba (2:12:20.380)

Still, there are people being born without these senses and I believe

Lex Fridman (2:12:28.380)

that they can still fall in love and have meaningful life.

Wojciech Zaremba (2:12:32.140)

Yeah.

Lex Fridman (2:12:32.460)

I wonder if it's possible to go close to all the way by just training

Wojciech Zaremba (2:12:37.500)

on transcripts of conversations.

Lex Fridman (2:12:40.620)

I wonder how far that takes us.

Lex Fridman (2:12:42.220)

So I think that actually still you want images like I would like.

Lex Fridman (2:12:45.980)

So I don't have kids, but like I could imagine having AI Tutor.

Wojciech Zaremba (2:12:50.620)

It has to see, you know, kids drawing some pictures on the paper.

Lex Fridman (2:12:56.300)

And also facial expressions, all that kind of stuff.

Wojciech Zaremba (2:12:58.460)

We use dogs and humans use their eyes to communicate with each other.

Lex Fridman (2:13:04.060)

I think that's a really powerful mechanism of communication.

Wojciech Zaremba (2:13:07.500)

Body language too, that words are much lower bandwidth.

Lex Fridman (2:13:12.540)

And for body language, we still, you know, we kind of have a system

Wojciech Zaremba (2:13:15.340)

that displays an image of its or facial expression on the computer.

Lex Fridman (2:13:19.980)

Doesn't have to move, you know, mechanical pieces or so.

Lex Fridman (2:13:23.420)

So I think that, you know, that there is like kind of a progression.

Lex Fridman (2:13:27.420)

You can imagine that text might be the simplest to tackle.

Lex Fridman (2:13:31.660)

But this is not a complete human experience at all.

Lex Fridman (2:13:36.700)

You expand it to, let's say images, both for input and output.

Lex Fridman (2:13:41.260)

And what you describe is actually the final, I guess, frontier.

Lex Fridman (2:13:45.900)

What makes us human, the fact that we can touch each other or smell or so.

Lex Fridman (2:13:50.060)

And it's the hardest from perspective of data and deployment.

Lex Fridman (2:13:54.140)

And I believe that these things might happen gradually.

Lex Fridman (2:13:59.660)

Are you excited by that possibility?

Lex Fridman (2:14:01.340)

This particular application of human to AI friendship and interaction?

Lex Fridman (2:14:07.820)

So let's see.

Lex Fridman (2:14:09.660)

Like would you, do you look forward to a world?

Wojciech Zaremba (2:14:12.380)

You said you're living with a few folks and you're very close friends with them.

Lex Fridman (2:14:16.060)

Do you look forward to a day where one or two of those friends are AI systems?

Lex Fridman (2:14:19.580)

So if the system would be truly wishing me well, rather than being in the situation

Lex Fridman (2:14:25.180)

that it optimizes for my time to interact with the system.

Wojciech Zaremba (2:14:28.460)

The line between those is, it's a gray area.

Lex Fridman (2:14:33.500)

I think that's the distinction between love and possession.

Lex Fridman (2:14:39.340)

And these things, they might be often correlated for humans, but you might find that there are

Lex Fridman (2:14:46.620)

some friends with whom you haven't spoke for months.

Wojciech Zaremba (2:14:49.660)

Yeah.

Lex Fridman (2:14:50.060)

And then you pick up the phone, it's as the time hasn't passed.

Wojciech Zaremba (2:14:54.620)

They are not holding to you.

Lex Fridman (2:14:55.820)

And I will, I wouldn't like to have AI system that, you know, it's trying to convince me

Wojciech Zaremba (2:15:02.300)

to spend time with it.

Wojciech Zaremba (2:15:03.420)

I would like the system to optimize for what I care about and help me in achieving my own goals.

Lex Fridman (2:15:12.300)

But there's some, I mean, I don't know, there's some manipulation, there's some possessiveness,

Wojciech Zaremba (2:15:17.900)

there's some insecurities, this fragility, all those things are necessary to form a close

Wojciech Zaremba (2:15:23.340)

friendship over time, to go through some dark shit together, some bliss and happiness together.

Lex Fridman (2:15:29.740)

I feel like there's a lot of greedy self centered behavior within that process.

Wojciech Zaremba (2:15:35.020)

My intuition, but I might be wrong, is that human computer interaction doesn't have to

Lex Fridman (2:15:41.340)

go through a computer being greedy, possessive, and so on.

Wojciech Zaremba (2:15:46.140)

It is possible to train systems, maybe, that they actually

Wojciech Zaremba (2:15:50.700)

they are, I guess, prompted or fine tuned or so to truly optimize for what you care about.

Lex Fridman (2:15:57.020)

And you could imagine that, you know, the way how the process would look like is at

Wojciech Zaremba (2:16:01.980)

some point, we as humans, we look at the transcript of the conversation or like an entire

Wojciech Zaremba (2:16:08.860)

interaction and we say, actually here, there was more loving way to go about it.

Lex Fridman (2:16:14.700)

And we supervise system toward being more loving, or maybe we train the system such

Wojciech Zaremba (2:16:20.540)

that it has a reward function toward being more loving.

Lex Fridman (2:16:23.180)

Yeah.

Wojciech Zaremba (2:16:23.740)

Or maybe the possibility of the system being an asshole and manipulative and possessive

Lex Fridman (2:16:29.820)

every once in a while is a feature, not a bug.

Wojciech Zaremba (2:16:33.580)

Because some of the happiness that we experience when two souls meet each other, when two humans

Lex Fridman (2:16:40.860)

meet each other, is a kind of break from the assholes in the world.

Lex Fridman (2:16:45.420)

And so you need assholes in AI as well, because, like, it'll be like a breath of fresh air

Wojciech Zaremba (2:16:52.060)

to discover an AI that the three previous AIs you had are too friendly or no, or cruel

Wojciech Zaremba (2:17:00.540)

or whatever.

Lex Fridman (2:17:01.340)

It's like some kind of mix.

Lex Fridman (2:17:03.020)

And then this one is just right, but you need to experience the full spectrum.

Lex Fridman (2:17:07.420)

Like, I think you need to be able to engineer assholes.

Lex Fridman (2:17:11.500)

So let's see.

Wojciech Zaremba (2:17:14.380)

Because there's some level to us being appreciated to appreciate the human experience.

Wojciech Zaremba (2:17:21.180)

We need the dark and the light.

Lex Fridman (2:17:24.300)

So that kind of reminds me.

Wojciech Zaremba (2:17:27.100)

I met a while ago at the meditation retreat, one woman, and she told me, you know,

Lex Fridman (2:17:35.820)

beautiful, beautiful woman, and she had a she had a crutch.

Wojciech Zaremba (2:17:41.260)

Okay.

Lex Fridman (2:17:41.980)

She had the trouble walking on one leg.

Wojciech Zaremba (2:17:44.940)

I asked her what has happened.

Lex Fridman (2:17:47.340)

And she said that five years ago she was in Maui, Hawaii, and she was eating a salad and

Wojciech Zaremba (2:17:55.820)

some snail fell into the salad.

Lex Fridman (2:17:57.980)

And apparently there are neurotoxic snails over there.

Lex Fridman (2:18:02.380)

And she got into coma for a year.

Lex Fridman (2:18:04.380)

Okay.

Lex Fridman (2:18:05.740)

And apparently there is, you know, high chance of even just dying.

Lex Fridman (2:18:09.660)

But she was in the coma.

Wojciech Zaremba (2:18:10.860)

At some point, she regained partially consciousness.

Lex Fridman (2:18:14.860)

She was able to hear people in the room.

Wojciech Zaremba (2:18:18.380)

People behave as she wouldn't be there.

Wojciech Zaremba (2:18:21.100)

You know, at some point she started being able to speak, but she was mumbling like a

Wojciech Zaremba (2:18:25.900)

barely able to express herself.

Lex Fridman (2:18:28.460)

Then at some point she got into wheelchair.

Wojciech Zaremba (2:18:30.700)

Then at some point she actually noticed that she can move her toe and then she knew that

Lex Fridman (2:18:38.140)

she will be able to walk.

Lex Fridman (2:18:40.220)

And then, you know, that's where she was five years after.

Lex Fridman (2:18:42.620)

And she said that since then she appreciates the fact that she can move her toe.

Lex Fridman (2:18:48.460)

And I was thinking, hmm, do I need to go through such experience to appreciate that I have

Lex Fridman (2:18:53.580)

I can move my toe?

Wojciech Zaremba (2:18:55.020)

Wow, that's a really good story and really deep example.

Lex Fridman (2:18:58.300)

Yeah.

Lex Fridman (2:18:58.780)

And in some sense, it might be the case that we don't see light if we haven't went through

Lex Fridman (2:19:05.420)

the darkness.

Lex Fridman (2:19:06.380)

But I wouldn't say that we should.

Wojciech Zaremba (2:19:08.780)

We shouldn't assume that that's the case, which it may be able to engineer shortcuts.

Wojciech Zaremba (2:19:14.460)

Yeah.

Wojciech Zaremba (2:19:15.180)

Ilya had this, you know, belief that maybe one has to go for a week or six months to

Wojciech Zaremba (2:19:22.220)

do some challenging camp to just experience, you know, a lot of difficulties and then comes

Lex Fridman (2:19:29.660)

back and actually everything is bright, everything is beautiful.

Wojciech Zaremba (2:19:33.500)

I'm with Ilya on this.

Lex Fridman (2:19:34.460)

It must be a Russian thing.

Lex Fridman (2:19:35.500)

Where are you from originally?

Lex Fridman (2:19:36.940)

I'm Polish.

Wojciech Zaremba (2:19:37.900)

Polish.

Lex Fridman (2:19:39.740)

Okay.

Wojciech Zaremba (2:19:41.500)

I'm tempted to say that explains a lot.

Lex Fridman (2:19:43.500)

But yeah, there's something about the Russian, the necessity of suffering.

Wojciech Zaremba (2:19:47.820)

I believe suffering or rather struggle is necessary.

Lex Fridman (2:19:52.700)

I believe that struggle is necessary.

Wojciech Zaremba (2:19:54.300)

I mean, in some sense, you even look at the story of any superhero in the movie.

Lex Fridman (2:20:00.380)

It's not that it was like everything goes easy, easy, easy, easy.

Wojciech Zaremba (2:20:03.340)

I like how that's your ground truth is the story of superheroes.

Lex Fridman (2:20:07.820)

Okay.

Wojciech Zaremba (2:20:09.260)

You mentioned that you used to do research at night and go to bed at like 6 a.m.

Lex Fridman (2:20:13.420)

or 7 a.m.

Wojciech Zaremba (2:20:14.140)

I still do that often.

Lex Fridman (2:20:18.860)

What sleep schedules have you tried to make for a productive and happy life?

Wojciech Zaremba (2:20:23.180)

Like, is there is there some interesting wild sleeping patterns that you engaged that you

Lex Fridman (2:20:29.500)

found that works really well for you?

Wojciech Zaremba (2:20:31.420)

I tried at some point decreasing number of hours of sleep like a gradually like a half

Lex Fridman (2:20:37.180)

an hour every few days to this.

Wojciech Zaremba (2:20:39.100)

You know, I was hoping to just save time.

Lex Fridman (2:20:41.980)

That clearly didn't work for me.

Wojciech Zaremba (2:20:43.500)

Like at some point, there's like a phase shift and I felt tired all the time.

Lex Fridman (2:20:50.380)

You know, there was a time that I used to work during the nights.

Wojciech Zaremba (2:20:53.980)

The nice thing about the nights is that no one disturbs you.

Lex Fridman (2:20:57.740)

And even I remember when I was meeting for the first time with Greg Brockman, his

Wojciech Zaremba (2:21:04.620)

CTO and chairman of OpenAI, our meeting was scheduled to 5 p.m.

Lex Fridman (2:21:09.660)

And I overstepped for the meeting.

Wojciech Zaremba (2:21:11.740)

Over slept for the meeting at 5 p.m.

Lex Fridman (2:21:14.060)

Yeah, now you sound like me.

Wojciech Zaremba (2:21:15.740)

That's hilarious.

Lex Fridman (2:21:16.540)

OK, yeah.

Lex Fridman (2:21:17.660)

And at the moment, in some sense, my sleeping schedule also has to do with the fact that

Lex Fridman (2:21:23.820)

I'm interacting with people.

Wojciech Zaremba (2:21:26.780)

I sleep without an alarm.

Wojciech Zaremba (2:21:28.620)

So, yeah, the the team thing you mentioned, the extrovert thing, because most humans operate

Wojciech Zaremba (2:21:35.900)

during a certain set of hours, you're forced to then operate at the same set of hours.

Lex Fridman (2:21:42.220)

But I'm not quite there yet.

Wojciech Zaremba (2:21:46.460)

I found a lot of joy, just like you said, working through the night because it's quiet

Lex Fridman (2:21:51.900)

because the world doesn't disturb you.

Lex Fridman (2:21:53.660)

And there's some aspect counter to everything you're saying.

Wojciech Zaremba (2:21:57.580)

There's some joyful aspect to sleeping through the mess of the day because people are having

Wojciech Zaremba (2:22:03.660)

people are having meetings and sending emails and there's drama meetings.

Lex Fridman (2:22:08.060)

I can sleep through all the meetings.

Wojciech Zaremba (2:22:09.980)

You know, I have meetings every day and they prevent me from having sufficient amount of

Lex Fridman (2:22:14.140)

time for focused work.

Lex Fridman (2:22:16.780)

And then I modified my calendar and I said that I'm out of office Wednesday, Thursday

Lex Fridman (2:22:23.980)

and Friday every day and I'm having meetings only Monday and Tuesday.

Lex Fridman (2:22:27.500)

And that busty positively influenced my mood that I have literally like at three days for

Lex Fridman (2:22:33.420)

fully focused work.

Wojciech Zaremba (2:22:34.380)

Yeah.

Lex Fridman (2:22:35.580)

So there's better solutions to this problem than staying awake all night.

Wojciech Zaremba (2:22:39.980)

OK, you've been part of development of some of the greatest ideas in artificial intelligence.

Lex Fridman (2:22:45.420)

What would you say is your process for developing good novel ideas?

Wojciech Zaremba (2:22:49.820)

You have to be aware that clearly there are many other brilliant people around.

Lex Fridman (2:22:53.820)

So you have to ask yourself a question, why the given idea, let's say, wasn't tried by

Wojciech Zaremba (2:23:02.780)

someone else and in some sense, it has to do with, you know, kind of simple.

Lex Fridman (2:23:10.140)

It might sound simple, but like a thinking outside of the box.

Lex Fridman (2:23:12.940)

And what do I mean here?

Wojciech Zaremba (2:23:14.780)

So, for instance, for a while, people in academia, they assumed that you have a feeling that

Wojciech Zaremba (2:23:23.260)

you have a fixed data set and then you optimize the algorithms in order to get the best performance.

Lex Fridman (2:23:31.500)

And that was so in great assumption that no one thought about training models on

Wojciech Zaremba (2:23:39.580)

anti internet or like that.

Lex Fridman (2:23:42.700)

Maybe some people thought about it, but it felt to many as unfair.

Lex Fridman (2:23:48.540)

And in some sense, that's almost like a it's not my idea or so, but that's an example of

Lex Fridman (2:23:53.180)

breaking at the typical assumption.

Lex Fridman (2:23:55.740)

So you want to be in the paradigm that you're breaking at the typical assumption.

Lex Fridman (2:24:00.540)

In the context of the community, getting to pick your data set is cheating.

Wojciech Zaremba (2:24:06.540)

Correct.

Lex Fridman (2:24:07.020)

And in some sense, so that was that was assumption that many people had out there.

Lex Fridman (2:24:11.260)

And then if you free yourself from assumptions, then they are likely to achieve something

Lex Fridman (2:24:19.020)

that others cannot do.

Lex Fridman (2:24:20.380)

And in some sense, if you are trying to do exactly the same things as others, it's very

Lex Fridman (2:24:24.940)

likely that you're going to have the same results.

Wojciech Zaremba (2:24:26.940)

Yeah, I but there's also that kind of tension, which is asking yourself the question, why

Lex Fridman (2:24:34.220)

haven't others done this?

Wojciech Zaremba (2:24:35.660)

Because, I mean, I get a lot of good ideas, but I think probably most of them suck when

Lex Fridman (2:24:44.620)

they meet reality.

Lex Fridman (2:24:45.900)

So so actually, I think the other big piece is getting into habit of generating ideas,

Wojciech Zaremba (2:24:53.500)

training your brain towards generating ideas and not even suspending judgment of the ideas.

Lex Fridman (2:25:00.860)

So in some sense, I noticed myself that even if I'm in the process of generating ideas,

Wojciech Zaremba (2:25:06.380)

if I tell myself, oh, that was a bad idea, then that actually interrupts the process

Lex Fridman (2:25:12.860)

and I cannot generate more ideas because I'm actually focused on the negative part, why

Lex Fridman (2:25:17.180)

it won't work.

Wojciech Zaremba (2:25:17.980)

Yes.

Lex Fridman (2:25:19.020)

But I created also environment in the way that it's very easy for me to store new ideas.

Wojciech Zaremba (2:25:25.020)

So, for instance, next to my bed, I have a voice recorder and it happens to me often

Lex Fridman (2:25:31.900)

like I wake up during the night and I have some idea.

Wojciech Zaremba (2:25:35.020)

In the past, I was writing them down on my phone, but that means, you know, turning on

Wojciech Zaremba (2:25:40.380)

the screen and that wakes me up or like pulling a paper, which requires, you know, turning

Wojciech Zaremba (2:25:45.500)

on the light.

Lex Fridman (2:25:47.500)

These days, I just start recording it.

Lex Fridman (2:25:49.660)

What do you think, I don't know if you know who Jim Keller is.

Lex Fridman (2:25:55.740)

I know Jim Keller.

Wojciech Zaremba (2:25:57.740)

He's a big proponent of thinking harder on a problem right before sleep so that he can

Wojciech Zaremba (2:26:03.580)

sleep through it and solve it in his sleep or like come up with radical stuff in his

Wojciech Zaremba (2:26:08.700)

sleep that's trying to get me to do this.

Lex Fridman (2:26:11.180)

So it happened from my experience perspective, it happened to me many times during the high

Wojciech Zaremba (2:26:19.020)

school days when I was doing mathematics that I had a solution to my problem as I woke up.

Wojciech Zaremba (2:26:27.260)

At the moment, regarding thinking hard about the given problem is I'm trying to actually

Wojciech Zaremba (2:26:33.420)

devote substantial amount of time to think about important problems, not just before

Lex Fridman (2:26:37.500)

the sleep.

Wojciech Zaremba (2:26:39.020)

I'm organizing amount of the huge chunks of time such that I'm not constantly working

Wojciech Zaremba (2:26:44.060)

on the urgent problems, but I actually have time to think about the important one.

Lex Fridman (2:26:48.220)

So you do it naturally.

Lex Fridman (2:26:49.740)

But his idea is that you kind of prime your brain to make sure that that's the focus.

Wojciech Zaremba (2:26:56.060)

Oftentimes people have other worries in their life that's not fundamentally deep problems

Wojciech Zaremba (2:27:00.700)

like I don't know, just stupid drama in your life and even at work, all that kind of stuff.

Wojciech Zaremba (2:27:06.860)

He wants to kind of pick the most important problem that you're thinking about and go

Lex Fridman (2:27:12.620)

to bed on that.

Wojciech Zaremba (2:27:13.820)

I think that's wise.

Wojciech Zaremba (2:27:14.940)

I mean, the other thing that comes to my mind is also I feel the most fresh in the morning.

Lex Fridman (2:27:20.380)

So during the morning, I try to work on the most important things rather than just being

Lex Fridman (2:27:25.900)

pulled by urgent things or checking email or so.

Lex Fridman (2:27:29.740)

What do you do with the...

Wojciech Zaremba (2:27:30.620)

Because I've been doing the voice recorder thing too, but I end up recording so many

Wojciech Zaremba (2:27:35.020)

messages it's hard to organize.

Lex Fridman (2:27:37.260)

I have the same problem.

Wojciech Zaremba (2:27:38.540)

Now I have heard that Google Pixel is really good in transcribing text and I might get

Lex Fridman (2:27:44.380)

a Google Pixel just for the sake of transcribing text.

Wojciech Zaremba (2:27:47.020)

Yeah, people listening to this, if you have a good voice recorder suggestion that transcribe,

Lex Fridman (2:27:50.940)

please let me know.

Wojciech Zaremba (2:27:52.780)

Some of it has to do with OpenAI codecs too.

Lex Fridman (2:27:57.900)

Like some of it is simply like the friction.

Wojciech Zaremba (2:28:01.900)

I need apps that remove that friction between voice and the organization of the resulting

Lex Fridman (2:28:08.940)

transcripts and all that kind of stuff.

Lex Fridman (2:28:11.980)

But yes, you're right.

Wojciech Zaremba (2:28:12.940)

Absolutely, like during, for me it's walking, sleep too, but walking and running, especially

Wojciech Zaremba (2:28:20.460)

running, get a lot of thoughts during running and there's no good mechanism for recording

Lex Fridman (2:28:25.500)

thoughts.

Lex Fridman (2:28:25.980)

So one more thing that I do, I have a separate phone which has no apps.

Lex Fridman (2:28:33.660)

Maybe it has like audible or let's say Kindle.

Wojciech Zaremba (2:28:37.180)

No one has this phone number, this kind of my meditation phone.

Lex Fridman (2:28:40.060)

Yeah.

Lex Fridman (2:28:40.620)

And I try to expand the amount of time that that's the phone that I'm having.

Wojciech Zaremba (2:28:47.180)

It has also Google Maps if I need to go somewhere and I also use this phone to write down ideas.

Wojciech Zaremba (2:28:52.860)

Ah, that's a really good idea.

Lex Fridman (2:28:55.660)

That's a really good idea.

Wojciech Zaremba (2:28:57.020)

Often actually what I end up doing is even sending a message from that phone to the other

Lex Fridman (2:29:01.740)

phone.

Lex Fridman (2:29:02.380)

So that's actually my way of recording messages or I just put them into notes.

Lex Fridman (2:29:06.780)

I love it.

Lex Fridman (2:29:07.340)

What advice would you give to a young person, high school, college, about how to be successful?

Wojciech Zaremba (2:29:15.660)

You've done a lot of incredible things in the past decade, so maybe, maybe have some.

Wojciech Zaremba (2:29:20.940)

There's something, there might be something.

Lex Fridman (2:29:22.540)

There might be something.

Wojciech Zaremba (2:29:25.020)

I mean, might sound like a simplistic or so, but I would say literally just follow your

Lex Fridman (2:29:33.020)

passion, double down on it.

Lex Fridman (2:29:34.140)

And if you don't know what's your passion, just figure out what could be a, what could

Lex Fridman (2:29:38.460)

be a passion.

Lex Fridman (2:29:39.100)

So that might be an exploration.

Lex Fridman (2:29:41.900)

When I was in elementary school was math and chemistry.

Lex Fridman (2:29:46.300)

And I remember for some time I gave up on math because my school teacher, she told me

Lex Fridman (2:29:52.300)

that I'm dumb.

Lex Fridman (2:29:54.940)

And I guess maybe an advice would be just ignore people if they tell you that you're

Lex Fridman (2:30:00.140)

dumb.

Wojciech Zaremba (2:30:00.860)

You're dumb.

Lex Fridman (2:30:01.420)

You're dumb. You mentioned something offline about chemistry and explosives.

Lex Fridman (2:30:08.540)

What was that about?

Lex Fridman (2:30:09.660)

So let's see.

Lex Fridman (2:30:11.900)

So a story goes like that.

Lex Fridman (2:30:16.860)

I got into chemistry.

Wojciech Zaremba (2:30:18.300)

Maybe I was like a second grade of my elementary school, third grade.

Lex Fridman (2:30:23.500)

I started going to chemistry classes.

Wojciech Zaremba (2:30:27.740)

I really love building stuff.

Lex Fridman (2:30:30.060)

And I did all the experiments that they describe in the book, like, you know, how to create

Wojciech Zaremba (2:30:35.740)

oxygen with vinegar and baking soda or so.

Lex Fridman (2:30:39.660)

Okay.

Lex Fridman (2:30:40.780)

So I did all the experiments and at some point I was, you know, so what's next?

Lex Fridman (2:30:45.740)

What can I do?

Lex Fridman (2:30:47.260)

And explosives, they also, it's like a, you have a clear reward signal, you know, if the

Lex Fridman (2:30:53.180)

thing worked or not.

Lex Fridman (2:30:54.140)

So I remember at first I got interested in producing hydrogen.

Lex Fridman (2:31:00.780)

That was kind of funny experiment from school.

Wojciech Zaremba (2:31:03.260)

You can just burn it.

Lex Fridman (2:31:04.380)

And then I moved to nitroglycerin.

Lex Fridman (2:31:07.420)

So that's also relatively easy to synthesize.

Lex Fridman (2:31:11.260)

I started producing essentially dynamite and detonating it with a friend.

Wojciech Zaremba (2:31:16.540)

I remember there was a, you know, there was at first like maybe two attempts that I went

Lex Fridman (2:31:20.860)

with a friend to detonate what we built and it didn't work out.

Lex Fridman (2:31:25.020)

And like a third time he was like, ah, it won't work.

Lex Fridman (2:31:27.660)

Like, let's don't waste time.

Wojciech Zaremba (2:31:30.220)

And, you know, we were, I was carrying this, this, you know, that tube with dynamite, I

Wojciech Zaremba (2:31:38.700)

don't know, pound or so, dynamite in my backpack, we're like riding on the bike to the edges

Wojciech Zaremba (2:31:45.260)

of the city.

Lex Fridman (2:31:45.820)

Yeah, and attempt number three, this was be attempt number three.

Wojciech Zaremba (2:31:51.340)

Attempt number three.

Lex Fridman (2:31:52.860)

And now we dig a hole to put it inside.

Wojciech Zaremba (2:31:57.420)

It actually had the, you know, electrical detonator.

Lex Fridman (2:32:02.220)

We draw a cable behind the tree.

Wojciech Zaremba (2:32:05.660)

I even, I never had, I haven't ever seen like a explosion before.

Lex Fridman (2:32:10.140)

So I thought that there would be a lot of, you know, a lot of, you know, a lot of, you

Wojciech Zaremba (2:32:15.580)

know, there will be a lot of sound.

Wojciech Zaremba (2:32:17.980)

But, you know, we're like laying down and I'm holding the cable and the battery.

Wojciech Zaremba (2:32:22.380)

At some point, you know, we kind of like a three to one and I just connected it and it

Lex Fridman (2:32:28.380)

felt like the ground shake.

Wojciech Zaremba (2:32:30.300)

It was like more like a sound.

Lex Fridman (2:32:32.860)

And then the soil started kind of lifting up and started falling on us.

Wojciech Zaremba (2:32:37.740)

Yeah.

Lex Fridman (2:32:38.380)

Wow.

Lex Fridman (2:32:39.180)

And then, you know, the friend said, let's make sure the next time we have helmets.

Lex Fridman (2:32:45.580)

But also, you know, I'm happy that nothing happened to me.

Wojciech Zaremba (2:32:48.940)

It could have been the case that I lost the limbo or so.

Wojciech Zaremba (2:32:52.300)

Yeah, but that's childhood of an engineering mind with a strong reward signal of an

Wojciech Zaremba (2:33:01.900)

explosion.

Lex Fridman (2:33:03.660)

I love it.

Wojciech Zaremba (2:33:04.140)

My there's some aspect of chemists that the chemists I know, like my dad with plasma

Lex Fridman (2:33:10.140)

chemistry, plasma physics, he was very much into explosives, too.

Wojciech Zaremba (2:33:13.740)

It's a worrying quality of people that work in chemistry that they love.

Lex Fridman (2:33:18.300)

I think it is that exactly is the strong signal that the thing worked.

Wojciech Zaremba (2:33:23.500)

There is no doubt.

Lex Fridman (2:33:24.620)

There's no doubt.

Wojciech Zaremba (2:33:25.660)

There's some magic.

Lex Fridman (2:33:26.860)

It's almost like a reminder that physics works, that chemistry works.

Wojciech Zaremba (2:33:31.420)

It's cool.

Lex Fridman (2:33:32.220)

It's almost like a little glimpse at nature that you yourself engineer.

Wojciech Zaremba (2:33:36.540)

I that's why I really like artificial intelligence, especially robotics, is you create a little

Wojciech Zaremba (2:33:43.420)

piece of nature and in some sense, even for me with explosives, the motivation was creation

Wojciech Zaremba (2:33:49.020)

rather than destruction.

Lex Fridman (2:33:50.060)

Yes, exactly.

Wojciech Zaremba (2:33:51.740)

In terms of advice, I forgot to ask about just machine learning and deep learning for

Wojciech Zaremba (2:33:57.180)

people who are specifically interested in machine learning, how would you recommend

Lex Fridman (2:34:01.980)

they get into the field?

Lex Fridman (2:34:03.580)

So I would say re implement everything and also there is plenty of courses.

Lex Fridman (2:34:08.620)

So like from scratch?

Lex Fridman (2:34:10.380)

So on different levels of abstraction in some sense, but I would say re implement something

Wojciech Zaremba (2:34:14.780)

from scratch, re implement something from a paper, re implement something, you know,

Lex Fridman (2:34:19.100)

from podcasts that you have heard about.

Wojciech Zaremba (2:34:21.420)

I would say that's a powerful way to understand things.

Lex Fridman (2:34:23.820)

So it's often the case that you read the description and you think you understand, but you truly

Wojciech Zaremba (2:34:30.220)

understand once you build it, then you actually know what really matter in the description.

Lex Fridman (2:34:36.220)

Is there a particular topics that you find people just fall in love with?

Wojciech Zaremba (2:34:41.020)

I've seen.

Wojciech Zaremba (2:34:44.220)

I tend to really enjoy reinforcement learning because it's much more, it's much easier

Wojciech Zaremba (2:34:51.500)

to get to a point where you feel like you created something special, like fun games

Lex Fridman (2:34:57.260)

kind of things that are rewarding.

Wojciech Zaremba (2:34:58.620)

It's rewarding.

Lex Fridman (2:34:59.100)

Yeah.

Wojciech Zaremba (2:35:01.100)

As opposed to like re implementing from scratch, more like supervised learning kind of things.

Lex Fridman (2:35:07.740)

It's yeah.

Wojciech Zaremba (2:35:08.940)

So, you know, if someone would optimize for things to be rewarding, then it feels that

Lex Fridman (2:35:15.260)

the things that are somewhat generative, they have such a property.

Lex Fridman (2:35:18.460)

So you have, for instance, adversarial networks, or do you have just even generative language

Lex Fridman (2:35:23.580)

models?

Lex Fridman (2:35:24.700)

And you can even see, internally, we have seen this thing with our releases.

Lex Fridman (2:35:30.780)

So we have, we released recently two models.

Wojciech Zaremba (2:35:33.820)

There is one model called Dali that generates images, and there is other model called Clip

Wojciech Zaremba (2:35:39.340)

that actually you provide various possibilities, what could be the answer to what is on the

Wojciech Zaremba (2:35:45.500)

picture, and it can tell you which one is the most likely.

Lex Fridman (2:35:48.700)

And in some sense, in case of the first one, Dali, it is very easy for you to understand

Wojciech Zaremba (2:35:56.220)

that actually there is magic going on.

Lex Fridman (2:35:59.740)

And in the case of the second one, even though it is insanely powerful, and you know, people

Wojciech Zaremba (2:36:04.860)

from a vision community, they, as they started probing it inside, they actually understood

Lex Fridman (2:36:12.540)

how far it goes.

Lex Fridman (2:36:13.740)

How far it goes, it's difficult for a person at first to see how well it works.

Lex Fridman (2:36:21.500)

And that's the same, as you said, that in case of supervised learning models, you might

Wojciech Zaremba (2:36:25.260)

not kind of see, or it's not that easy for you to understand the strength.

Lex Fridman (2:36:31.180)

Even though you don't believe in magic, to see the magic.

Wojciech Zaremba (2:36:33.820)

To see the magic, yeah.

Lex Fridman (2:36:35.020)

It's a generative.

Wojciech Zaremba (2:36:36.220)

That's really brilliant.

Lex Fridman (2:36:37.340)

So anything that's generative, because then you are at the core of the creation.

Wojciech Zaremba (2:36:42.860)

You get to experience creation without much effort.

Lex Fridman (2:36:46.620)

Unless you have to do it from scratch, but.

Lex Fridman (2:36:48.540)

And it feels that, you know, humans are wired.

Lex Fridman (2:36:51.900)

There is some level of reward for creating stuff.

Wojciech Zaremba (2:36:54.700)

Yeah.

Lex Fridman (2:36:56.380)

Of course, different people have a different weight on this reward.

Wojciech Zaremba (2:36:59.100)

Yeah.

Lex Fridman (2:37:00.460)

In the big objective function.

Wojciech Zaremba (2:37:01.740)

In the big objective function of a person.

Lex Fridman (2:37:03.980)

Of a person.

Wojciech Zaremba (2:37:05.420)

You wrote that beautiful is what you intensely pay attention to.

Lex Fridman (2:37:10.860)

Even a cockroach is beautiful.

Lex Fridman (2:37:12.380)

If you look very closely, can you expand on this?

Lex Fridman (2:37:16.300)

What is beauty?

Lex Fridman (2:37:18.620)

So what I'm, I wrote here actually corresponds to my subjective experience that I had through

Lex Fridman (2:37:26.060)

extended periods of meditation.

Wojciech Zaremba (2:37:28.540)

It's, it's pretty crazy that at some point the meditation gets you to the place that

Lex Fridman (2:37:34.380)

you have really increased focus, increased attention.

Wojciech Zaremba (2:37:39.820)

Increased attention.

Lex Fridman (2:37:40.940)

And then you look at the very simple objects that were all the time around you can look

Wojciech Zaremba (2:37:45.580)

at the table or on the pen or at the nature.

Lex Fridman (2:37:49.260)

And you notice more and more details and it becomes very pleasant to look at it.

Lex Fridman (2:37:56.780)

And it, once again, it kind of reminds me of my childhood.

Lex Fridman (2:38:01.260)

Like I just pure joy of being.

Wojciech Zaremba (2:38:03.900)

It's also, I have seen even the reverse effect that by default, regardless of what we possess,

Lex Fridman (2:38:11.580)

we very quickly get used to it.

Lex Fridman (2:38:14.300)

And you know, you can have a very beautiful house and if you don't put sufficient effort,

Lex Fridman (2:38:21.500)

you're just going to get used to it and it doesn't bring any more joy,

Wojciech Zaremba (2:38:25.500)

regardless of what you have.

Lex Fridman (2:38:27.180)

Yeah.

Wojciech Zaremba (2:38:27.680)

Well, I actually, I find that material possessions get in the way of that experience of pure

Lex Fridman (2:38:36.960)

joy.

Lex Fridman (2:38:38.720)

So I've always, I've been very fortunate to just find joy in simple things.

Wojciech Zaremba (2:38:45.360)

Just, just like you're saying, just like, I don't know, objects in my life, just stupid

Wojciech Zaremba (2:38:50.800)

objects like this cup, like thing, you know, just objects sounds okay.

Wojciech Zaremba (2:38:55.440)

I'm not being eloquent, but literally objects in the world, they're just full of joy.

Wojciech Zaremba (2:39:00.880)

Cause it's like, I can't believe when I can't believe that I'm fortunate enough to be alive

Lex Fridman (2:39:07.360)

to experience these objects.

Lex Fridman (2:39:09.680)

And then two, I can't believe humans are clever enough to have built these objects.

Lex Fridman (2:39:15.120)

The hierarchy of pleasure that that provides is infinite.

Wojciech Zaremba (2:39:19.520)

I mean, even if you look at the cup of water, so, you know, you see first like a level of

Wojciech Zaremba (2:39:24.000)

like a reflection of light, but then you think, you know, man, there's like a trillions upon

Wojciech Zaremba (2:39:28.320)

of trillions of particles bouncing against each other.

Wojciech Zaremba (2:39:32.000)

There is also the tension on the surface that, you know, if the back, back could like a stand

Wojciech Zaremba (2:39:38.560)

on it and move around.

Lex Fridman (2:39:40.000)

And you think it also has this like a magical property that as you decrease temperature,

Wojciech Zaremba (2:39:45.440)

it actually expands in volume, which allows for the, you know, legs to freeze on the,

Wojciech Zaremba (2:39:51.680)

on the surface and at the bottom to have actually not freeze, which allows for life like a crazy.

Wojciech Zaremba (2:39:58.080)

Yeah.

Wojciech Zaremba (2:39:58.560)

You look in detail at some object and you think actually, you know, this table, that

Wojciech Zaremba (2:40:03.520)

was just a figment of someone's imagination at some point.

Lex Fridman (2:40:06.400)

And then there was like a thousands of people involved to actually manufacture it and put

Wojciech Zaremba (2:40:10.560)

it here.

Lex Fridman (2:40:11.120)

And by default, no one cares.

Lex Fridman (2:40:15.280)

And then you can start thinking about evolution, how it all started from single cell organisms

Lex Fridman (2:40:19.360)

that led to this table.

Lex Fridman (2:40:21.280)

And these thoughts, they give me life appreciation and even lack of thoughts, just the pure raw

Lex Fridman (2:40:27.360)

signal also gives the life appreciation.

Wojciech Zaremba (2:40:29.920)

See, the thing is, and then that's coupled for me with the sadness that the whole ride

Wojciech Zaremba (2:40:37.360)

ends and perhaps is deeply coupled in that the fact that this experience, this moment

Wojciech Zaremba (2:40:43.440)

ends, gives it, gives it an intensity that I'm not sure I would otherwise have.

Lex Fridman (2:40:50.160)

So in that same way, I tried to meditate on my own death.

Wojciech Zaremba (2:40:53.600)

Often.

Lex Fridman (2:40:54.880)

Do you think about your mortality?

Lex Fridman (2:40:58.160)

Are you afraid of death?

Lex Fridman (2:41:01.840)

So fear of death is like one of the most fundamental fears that each of us has.

Wojciech Zaremba (2:41:07.840)

We might be not even aware of it.

Wojciech Zaremba (2:41:09.680)

It requires to look inside, to even recognize that it's out there and there is still, let's

Wojciech Zaremba (2:41:15.440)

say, this property of nature that if things would last forever, then they would be also

Lex Fridman (2:41:22.960)

boring to us.

Wojciech Zaremba (2:41:24.880)

The fact that the things change in some way gives any meaning to them.

Wojciech Zaremba (2:41:29.520)

I also, you know, found out that it seems to be very healing to people to have these

Wojciech Zaremba (2:41:40.800)

short experiences, like, I guess, psychedelic experiences in which they experience death

Wojciech Zaremba (2:41:49.440)

of self in which they let go of this fear and then maybe can even increase the intensity

Wojciech Zaremba (2:41:58.160)

can even increase the appreciation of the moment.

Wojciech Zaremba (2:42:01.520)

It seems that many people, they can easily comprehend the fact that the money is finite

Wojciech Zaremba (2:42:12.160)

while they don't see that time is finite.

Lex Fridman (2:42:15.680)

I have this like a discussion with Ilya from time to time.

Wojciech Zaremba (2:42:18.640)

He's like, you know, man, like the life will pass very fast.

Lex Fridman (2:42:23.520)

At some point I will be 40, 50, 60, 70 and then it's over.

Wojciech Zaremba (2:42:26.640)

This is true, which also makes me believe that, you know, that every single moment it

Lex Fridman (2:42:33.120)

is so unique that should be appreciated.

Lex Fridman (2:42:37.600)

And this also makes me think that I should be acting on my life because otherwise it

Lex Fridman (2:42:44.560)

will pass.

Wojciech Zaremba (2:42:46.240)

I also like this framework of thinking from Jeff Bezos on regret minimization that like

Wojciech Zaremba (2:42:53.280)

I would like if I will be at that deathbed to look back on my life and not regret that

Wojciech Zaremba (2:43:01.520)

I haven't done something.

Lex Fridman (2:43:03.280)

It's usually you might regret that you haven't tried.

Wojciech Zaremba (2:43:07.680)

I'm fine with failing.

Lex Fridman (2:43:10.640)

I haven't tried.

Lex Fridman (2:43:13.120)

What's the Nietzsche eternal occurrence?

Wojciech Zaremba (2:43:15.360)

Try to live a life that if you had to live it infinitely many times, that would be the

Wojciech Zaremba (2:43:20.480)

you'd be okay with that kind of life.

Lex Fridman (2:43:24.640)

So try to live it optimally.

Wojciech Zaremba (2:43:27.120)

I can say that it's almost like I'm.

Lex Fridman (2:43:33.280)

I'm available to me where I am in my life.

Wojciech Zaremba (2:43:36.640)

I'm extremely grateful for actually people whom I met.

Lex Fridman (2:43:40.480)

I would say I think that I'm decently smart and so on.

Lex Fridman (2:43:44.320)

But I think that actually to a great extent where I am has to do with the people who I

Lex Fridman (2:43:50.160)

met.

Lex Fridman (2:43:52.320)

Would you be okay if after this conversation you died?

Lex Fridman (2:43:56.320)

So if I'm dead, then it kind of I don't have a choice anymore.

Lex Fridman (2:44:01.600)

So in some sense, there's like plenty of things that I would like to try out in my life.

Lex Fridman (2:44:07.040)

I feel that I'm gradually going one by one and I'm just doing them.

Wojciech Zaremba (2:44:10.480)

I think that the list will be always infinite.

Lex Fridman (2:44:13.120)

Yeah, so might as well go today.

Wojciech Zaremba (2:44:16.800)

Yeah, I mean, to be clear, I'm not looking forward to die.

Lex Fridman (2:44:20.800)

I would say if there is no choice, I would accept it.

Lex Fridman (2:44:24.320)

But like in some sense, I'm if there would be a choice, if there would be a possibility

Lex Fridman (2:44:30.480)

to leave, I would fight for leaving.

Wojciech Zaremba (2:44:33.680)

I find it's more.

Wojciech Zaremba (2:44:37.120)

I find it's more honest and real to think about, you know, dying today at the end of

Wojciech Zaremba (2:44:44.560)

the day.

Wojciech Zaremba (2:44:46.080)

That seems to me, at least to my brain, more honest slap in the face as opposed to I still

Wojciech Zaremba (2:44:52.960)

have 10 years like today, then I'm much more about appreciating the cup and the table and

Lex Fridman (2:44:59.520)

so on and less about like silly worldly accomplishments and all those kinds of things.

Lex Fridman (2:45:04.960)

But we have in the company a person who say at some point found out that they have cancer

Lex Fridman (2:45:11.760)

and that also gives, you know, huge perspective with respect to what matters now.

Wojciech Zaremba (2:45:16.000)

Yeah.

Wojciech Zaremba (2:45:16.560)

And, you know, often people in situations like that, they conclude that actually what

Wojciech Zaremba (2:45:20.320)

matters is human connection.

Lex Fridman (2:45:22.720)

And love and that's people conclude also if you have kids, kids as family.

Wojciech Zaremba (2:45:28.720)

You, I think, tweeted, we don't assign the minus infinity reward to our death.

Lex Fridman (2:45:35.440)

Such a reward would prevent us from taking any risk.

Wojciech Zaremba (2:45:38.640)

We wouldn't be able to cross the road in fear of being hit by a car.

Lex Fridman (2:45:42.480)

So in the objective function, you mentioned fear of death might be fundamental to the

Wojciech Zaremba (2:45:46.400)

human condition.

Lex Fridman (2:45:48.400)

So, as I said, let's assume that they're like a reward functions in our brain.

Lex Fridman (2:45:52.640)

And the interesting thing is even realization, how different reward functions can play with

Lex Fridman (2:46:01.840)

your behavior.

Wojciech Zaremba (2:46:03.440)

As a matter of fact, I wouldn't say that you should assign infinite negative reward to

Lex Fridman (2:46:09.280)

anything because that messes up the math.

Wojciech Zaremba (2:46:12.400)

The math doesn't work out.

Lex Fridman (2:46:13.600)

It doesn't work out.

Lex Fridman (2:46:14.320)

And as you said, even, you know, government or some insurance companies, you said they

Lex Fridman (2:46:19.440)

assign $9 million to human life.

Lex Fridman (2:46:22.720)

And I'm just saying it with respect to, that might be a hard statement to ourselves, but

Lex Fridman (2:46:29.600)

in some sense that there is a finite value of our own life.

Wojciech Zaremba (2:46:34.640)

I'm trying to put it from perspective of being less, of being more egoless and realizing

Lex Fridman (2:46:43.440)

fragility of my own life.

Lex Fridman (2:46:44.800)

And in some sense, the fear of death might prevent you from acting because anything can

Lex Fridman (2:46:53.760)

cause death.

Wojciech Zaremba (2:46:56.080)

Yeah.

Lex Fridman (2:46:56.560)

And I'm sure actually, if you were to put death in the objective function, there's probably

Lex Fridman (2:47:00.800)

so many aspects to death and fear of death and realization of death and mortality.

Wojciech Zaremba (2:47:06.960)

There's just whole components of finiteness of not just your life, but every experience

Lex Fridman (2:47:13.600)

and so on that you're going to have to formalize mathematically.

Lex Fridman (2:47:18.320)

And also, you know, that might lead to you spending a lot of compute cycles on this like

Wojciech Zaremba (2:47:27.040)

a deliberating this terrible future instead of experiencing now.

Lex Fridman (2:47:32.480)

And then in some sense, it's also kind of unpleasant simulation to run in your head.

Wojciech Zaremba (2:47:36.480)

Yeah.

Lex Fridman (2:47:39.040)

Do you think there's an objective function that describes the entirety of human life?

Lex Fridman (2:47:45.920)

So, you know, usually the way you ask that is what is the meaning of life?

Lex Fridman (2:47:50.560)

Is there a universal objective functions that captures the why of life?

Wojciech Zaremba (2:47:55.760)

So, yeah, I mean, I suspect that they will ask this question, but it's also a question

Lex Fridman (2:48:03.440)

that I ask myself many, many times.

Wojciech Zaremba (2:48:06.320)

See, I can tell you a framework that I have these days to think about this question.

Lex Fridman (2:48:10.320)

So I think that fundamentally, meaning of life has to do with some of our reward actions

Wojciech Zaremba (2:48:16.480)

that we have in brain and they might have to do with, let's say, for instance, curiosity

Lex Fridman (2:48:21.680)

or human connection, which might mean understanding others.

Wojciech Zaremba (2:48:27.760)

It's also possible for a person to slightly modify their reward function.

Wojciech Zaremba (2:48:32.080)

Usually they mostly stay fixed, but it's possible to modify reward function and you can pretty

Wojciech Zaremba (2:48:37.280)

much choose.

Lex Fridman (2:48:38.080)

So in some sense, the reward functions, optimizing reward functions, they will give you a life

Wojciech Zaremba (2:48:42.480)

satisfaction.

Lex Fridman (2:48:44.000)

Is there some randomness in the function?

Wojciech Zaremba (2:48:45.920)

I think when you are born, there is some randomness.

Wojciech Zaremba (2:48:48.000)

You can see that some people, for instance, they care more about building stuff.

Wojciech Zaremba (2:48:53.840)

Some people care more about caring for others.

Lex Fridman (2:48:56.880)

Some people, there are all sorts of default reward functions.

Lex Fridman (2:49:00.880)

And then in some sense, you can ask yourself, what is the satisfying way for you to go after

Lex Fridman (2:49:08.400)

this reward function?

Lex Fridman (2:49:09.680)

And you just go after this reward function.

Lex Fridman (2:49:11.280)

And, you know, some people also ask, are you satisfied with your life?

Lex Fridman (2:49:15.120)

And, you know, some people also ask, are these reward functions real?

Wojciech Zaremba (2:49:19.840)

I almost think about it as, let's say, if you would have to discover mathematics, in

Wojciech Zaremba (2:49:27.680)

mathematics, you are likely to run into various objects like complex numbers or differentiation,

Lex Fridman (2:49:34.640)

some other objects.

Lex Fridman (2:49:35.680)

And these are very natural objects that arise.

Lex Fridman (2:49:38.320)

And similarly, the reward functions that we are having in our brain, they are somewhat

Wojciech Zaremba (2:49:42.480)

very natural, that, you know, there is a reward function for understanding, like a comprehension,

Lex Fridman (2:49:52.080)

curiosity, and so on.

Lex Fridman (2:49:53.280)

So in some sense, they are in the same way natural as their natural objects in mathematics.

Lex Fridman (2:49:59.040)

Interesting.

Lex Fridman (2:49:59.680)

So, you know, there's the old sort of debate, is mathematics invented or discovered?

Lex Fridman (2:50:05.600)

You're saying reward functions are discovered.

Lex Fridman (2:50:07.840)

So nature.

Lex Fridman (2:50:08.880)

So nature provided some, you can still, let's say, expand it throughout the life.

Wojciech Zaremba (2:50:12.960)

Some of the reward functions, they might be futile.

Wojciech Zaremba (2:50:15.360)

Like, for instance, there might be a reward function, maximize amount of wealth.

Wojciech Zaremba (2:50:20.320)

Yeah.

Lex Fridman (2:50:20.800)

And this is more like a learned reward function.

Lex Fridman (2:50:25.520)

But we know also that some reward functions, if you optimize them, you won't be quite satisfied.

Wojciech Zaremba (2:50:32.240)

Well, I don't know which part of your reward function resulted in you coming today, but

Wojciech Zaremba (2:50:37.040)

I am deeply appreciative that you did spend your valuable time with me.

Lex Fridman (2:50:40.960)

Wojtek is really fun talking to you.

Wojciech Zaremba (2:50:43.920)

You're brilliant.

Lex Fridman (2:50:45.200)

You're a good human being.

Lex Fridman (2:50:46.320)

And it's an honor to meet you and an honor to talk to you.

Lex Fridman (2:50:48.880)

Thanks for talking today, brother.

Wojciech Zaremba (2:50:50.720)

Thank you, Lex a lot.

Lex Fridman (2:50:51.600)

I appreciated your questions, curiosity.

Wojciech Zaremba (2:50:54.240)

I had a lot of time being here.

Lex Fridman (2:50:57.120)

Thanks for listening to this conversation with Wojtek Zaremba.

Wojciech Zaremba (2:51:00.480)

To support this podcast, please check out our sponsors in the description.

Lex Fridman (2:51:04.480)

And now, let me leave you with some words from Arthur C. Clarke, who is the author of

Wojciech Zaremba (2:51:10.000)

2001 A Space Odyssey.

Wojciech Zaremba (2:51:12.800)

It may be that our role on this planet is not to worship God, but to create him.

Wojciech Zaremba (2:51:18.800)

Thank you for listening, and I hope to see you next time.

Lex Fridman (30:02.160)

Yeah.

Wojciech Zaremba (30:02.920)

Um, then other, well, I don't have a good intuition about, uh, how different

Lex Fridman (30:08.240)

the space of short programs are from the space of large programs.

Lex Fridman (30:12.120)

Like, what is the universe where short programs, uh, like run things?

Lex Fridman (30:18.520)

Uh, so, so I said, the things have to agree with N bits.

Lex Fridman (30:22.160)

So even if you have, you, you need to start, okay.

Lex Fridman (30:25.640)

If, if you have very short program and they're like a steel, some has, if, if

Wojciech Zaremba (30:29.920)

it's not perfectly prediction of N bits, you have to start errors.

Lex Fridman (30:33.760)

What are the errors?

Lex Fridman (30:34.520)

And that gives you the full program that agrees on N bits.

Lex Fridman (30:38.200)

Oh, so you don't agree with the N bits.

Lex Fridman (30:40.160)

And you store, that's like a longer, a longer program, slightly longer program

Lex Fridman (30:45.000)

because it can take these extra bits of errors.

Wojciech Zaremba (30:47.440)

That's fascinating.

Lex Fridman (30:48.480)

What's what's your intuition about the, the programs that are able to do cool

Wojciech Zaremba (30:55.920)

stuff like intelligence and consciousness, are they, uh, perfectly like, is, is it,

Lex Fridman (31:02.560)

uh, is there if then statements in them?

Lex Fridman (31:05.680)

So like, is there a lot of a good, uh, if then statements in them?

Lex Fridman (31:08.920)

So like, is there a lot of exceptions that they're storing?

Wojciech Zaremba (31:11.480)

So, um, you could imagine if there would be tremendous amount of if statements,

Lex Fridman (31:16.280)

then they wouldn't be that short.

Wojciech Zaremba (31:17.720)

In case of neural networks, you could imagine that, um, what happens is, uh,

Lex Fridman (31:24.280)

they, uh, when you start with an initialized neural network, uh, it stores

Wojciech Zaremba (31:29.840)

internally many possibilities, how the, uh, how the problem can be solved.

Lex Fridman (31:34.840)

And SGD is kind of magnifying some, some, uh, some, uh, paths, which are slightly

Wojciech Zaremba (31:42.720)