Dmitri Dolgov: Waymo and the Future of Self-Driving Cars

Lex Fridman (01:59.900)

If you enjoy this thing, subscribe on YouTube, review it with five stars

Lex Fridman (02:03.220)

and up a podcast, follow on Spotify, support on Patreon, or connect with

Lex Fridman (02:07.380)

me on Twitter at Lex Friedman.

Lex Fridman (02:10.140)

And now here's my conversation with Dimitri Dolgov.

Lex Fridman (02:14.940)

When did you first fall in love with MIT?

Dmitri Dolgov (02:17.500)

When did you first fall in love with robotics or even computer

Lex Fridman (02:20.860)

science more in general?

Dmitri Dolgov (02:22.300)

Computer science first at a fairly young age, then robotics happened much later.

Lex Fridman (02:28.340)

I think my first interesting introduction to computers was in the late 80s when

Dmitri Dolgov (02:39.020)

we got our first computer, I think it was an IBM, I think IBM AT.

Lex Fridman (02:44.580)

Those things that had like a turbo button in the front, the radio

Dmitri Dolgov (02:48.020)

precedent, you know, make, make the thing goes faster.

Lex Fridman (02:50.500)

Did that already have floppy disks?

Dmitri Dolgov (02:52.420)

Yeah.

Lex Fridman (02:52.780)

Yeah.

Dmitri Dolgov (02:52.980)

Yeah.

Lex Fridman (02:53.140)

Yeah.

Dmitri Dolgov (02:53.340)

Like the, the 5.4 inch ones.

Lex Fridman (02:57.140)

I think there was a bigger inch.

Lex Fridman (02:58.780)

So good.

Lex Fridman (02:59.220)

When something then five inches and three inches.

Dmitri Dolgov (03:02.060)

Yeah, I think that was the five.

Dmitri Dolgov (03:03.100)

I don't, I maybe that was before that was the giant plates and it didn't get that.

Lex Fridman (03:07.180)

But it was definitely not the, not the three inch ones.

Lex Fridman (03:09.300)

Anyway, so that, that, you know, we got that computer, I spent the first few

Dmitri Dolgov (03:15.660)

months just playing video games as you would expect, I got bored of that.

Lex Fridman (03:20.900)

So I started messing around and trying to figure out how to, you know, make

Dmitri Dolgov (03:25.740)

the thing do other stuff, got into exploring programming and a couple of

Lex Fridman (03:33.340)

years later, it got to a point where, I actually wrote a game, a lot of games

Lex Fridman (03:39.620)

and a game developer, a Japanese game developer actually offered to buy it

Lex Fridman (03:43.660)

for me for a few hundred bucks.

Lex Fridman (03:45.100)

But you know, for, for a kid in Russia, that's a big deal.

Lex Fridman (03:48.740)

That's a big deal.

Dmitri Dolgov (03:49.180)

Yeah.

Lex Fridman (03:49.780)

I did not take the deal.

Dmitri Dolgov (03:51.140)

Wow.

Lex Fridman (03:51.700)

Integrity.

Dmitri Dolgov (03:52.500)

Yeah.

Lex Fridman (03:53.020)

I, I instead, yes, that was not the most acute financial move that I made in my

Dmitri Dolgov (03:58.860)

life, you know, looking back at it now, I, I instead put it, well, you know, I had

Lex Fridman (04:02.380)

a reason I put it online, it was, what'd you call it back in the days?

Lex Fridman (04:07.180)

It was a freeware thing, right?

Lex Fridman (04:08.460)

It was not open source, but you could upload the binaries, you would put the

Dmitri Dolgov (04:11.260)

game online and the idea was that, you know, people like it and then they, you

Lex Fridman (04:14.420)

know, contribute on the send you a little donations, right?

Lex Fridman (04:16.540)

So I did my quick math of like, you know, of course, you know, thousands and

Dmitri Dolgov (04:20.020)

millions of people are going to play my game, send me a couple of bucks a piece,

Dmitri Dolgov (04:22.620)

you know, should definitely do that.

Lex Fridman (04:24.500)

As I said, not, not the best.

Dmitri Dolgov (04:26.580)

You're already playing with business models at that young age.

Lex Fridman (04:29.300)

Remember what language it was?

Lex Fridman (04:30.500)

What programming, it was a Pascal, which what Pascal, Pascal, and that

Lex Fridman (04:35.020)

a graphical component, so it's not text based.

Dmitri Dolgov (04:37.180)

Yeah.

Lex Fridman (04:37.460)

Yeah.

Dmitri Dolgov (04:37.660)

It was, uh, like, uh, I think there are 300, 320 by 200, uh, whatever it was.

Lex Fridman (04:43.180)

I think that kind of the earlier, that's the resolution, right?

Lex Fridman (04:46.420)

And I actually think the reason why this company wanted to buy it is not like the

Lex Fridman (04:49.420)

fancy graphics or the implementation.

Dmitri Dolgov (04:51.340)

That was maybe the idea, uh, of my actual game, the idea of the game.

Lex Fridman (04:57.140)

Well, one of the things I, it's so funny.

Dmitri Dolgov (04:59.020)

I'm used to play this game called golden X and the simplicity of the graphics and

Lex Fridman (05:05.860)

something about the simplicity of the music, like it's still haunts me.

Dmitri Dolgov (05:10.740)

I don't know if that's a childhood thing.

Dmitri Dolgov (05:12.060)

I don't know if that's the same thing for call of duty these days for young kids,

Lex Fridman (05:15.340)

but I still think that the simple one of the games are simple.

Lex Fridman (05:21.260)

That simple purity makes for like allows your imagination to take over and

Dmitri Dolgov (05:28.260)

thereby creating a more magical experience.

Lex Fridman (05:30.780)

Like now with better and better graphics, it feels like your

Dmitri Dolgov (05:34.100)

imagination doesn't get to, uh, create worlds, which is kind of interesting.

Lex Fridman (05:38.980)

Um, it could be just an old man on a porch, like way waving at kids

Dmitri Dolgov (05:43.020)

these days that have no respect.

Lex Fridman (05:44.780)

But I still think that graphics almost get in the way of the experience.

Dmitri Dolgov (05:49.860)

I don't know.

Lex Fridman (05:50.740)

Flip a bird.

Dmitri Dolgov (05:51.860)

Yeah, I don't know if the imagination is closed.

Lex Fridman (05:57.300)

I don't yet, but that that's more about games that op like that's more

Dmitri Dolgov (06:01.540)

like Tetris world where they optimally masterfully, like create a fun, short

Lex Fridman (06:09.580)

term dopamine experience versus I'm more referring to like role playing

Dmitri Dolgov (06:14.540)

games where there's like a story you can live in it for months or years.

Lex Fridman (06:18.860)

Um, like, uh, there's an elder scroll series, which is probably my favorite

Dmitri Dolgov (06:23.900)

set of games that was a magical experience.

Lex Fridman (06:26.540)

And that the graphics are terrible.

Dmitri Dolgov (06:28.460)

The characters were all randomly generated, but they're, I don't know.

Lex Fridman (06:31.500)

That's it pulls you in.

Dmitri Dolgov (06:33.700)

There's a story.

Lex Fridman (06:34.700)

It's like an interactive version of an elder scrolls Tolkien world.

Lex Fridman (06:40.580)

And you get to live in it.

Lex Fridman (06:42.140)

I don't know.

Dmitri Dolgov (06:43.460)

I miss it.

Lex Fridman (06:44.580)

It's one of the things that suck about being an adult is there's no, you have

Dmitri Dolgov (06:49.540)

to live in the real world as opposed to the elder scrolls world, you know, whatever

Lex Fridman (06:54.060)

brings you joy, right?

Lex Fridman (06:54.940)

Minecraft, right?

Lex Fridman (06:55.940)

Minecraft is a great example.

Dmitri Dolgov (06:56.780)

You create, like it's not the fancy graphics, but it's the creation of your own worlds.

Lex Fridman (07:01.420)

Yeah, that one is crazy.

Dmitri Dolgov (07:02.700)

You know, one of the pitches for being a parent that people tell me is that you

Lex Fridman (07:06.700)

can like use the excuse of parenting to, to go back into the video game world.

Lex Fridman (07:12.180)

And like, like that's like, you know, father, son, father, daughter time, but

Lex Fridman (07:17.940)

really you just get to play video games with your kids.

Lex Fridman (07:19.820)

So anyway, at that time, did you have any ridiculously ambitious dreams of where as

Lex Fridman (07:27.260)

a creator, you might go as an engineer?

Dmitri Dolgov (07:29.780)

Did you, what, what did you think of yourself as, as an engineer, as a tinker,

Lex Fridman (07:33.860)

or did you want to be like an astronaut or something like that?

Dmitri Dolgov (07:37.220)

You know, I'm tempted to make something up about, you know, robots, uh, engineering

Lex Fridman (07:42.220)

or, you know, mysteries of the universe, but that's not the actual memory that

Dmitri Dolgov (07:45.940)

pops into my mind when you, when you asked me about childhood dreams.

Lex Fridman (07:48.780)

So I'll actually share the, the, the real thing, uh, when I was maybe four or five

Dmitri Dolgov (07:55.860)

years old, I, you know, as we all do, I thought about, you know, what I wanted

Lex Fridman (08:00.340)

to do when I grow up and I had this dream of being a traffic control cop.

Dmitri Dolgov (08:08.660)

Uh, you know, they don't have those today's I think, but you know, back in

Lex Fridman (08:11.380)

the eighties and in Russia, uh, you probably are familiar with that Lex.

Dmitri Dolgov (08:15.300)

They had these, uh, you know, police officers that would stand in the middle

Lex Fridman (08:19.300)

of intersection all day and they would have their like stripe back, black and

Dmitri Dolgov (08:21.940)

white batons that they would use to control the flow of traffic and, you

Lex Fridman (08:26.060)

know, for whatever reasons, I was strangely infatuated with this whole

Dmitri Dolgov (08:29.940)

process and like that, that was my dream.

Lex Fridman (08:32.220)

Uh, that's what I wanted to do when I grew up and, you know, my parents, uh,

Dmitri Dolgov (08:37.100)

both physics profs, by the way, I think were, you know, a little concerned, uh,

Lex Fridman (08:41.860)

with that level of ambition coming from their child.

Dmitri Dolgov (08:44.220)

Uh, uh, you know, that age.

Lex Fridman (08:46.740)

Well, that it's an interesting, I don't know if you can relate,

Lex Fridman (08:50.060)

but I very much love that idea.

Lex Fridman (08:52.300)

I have a OCD nature that I think lends itself very close to the engineering

Dmitri Dolgov (08:57.980)

mindset, which is you want to kind of optimize, you know, solve a problem by

Lex Fridman (09:05.580)

create, creating an automated solution, like a, like a set of rules, that set

Dmitri Dolgov (09:11.220)

of rules you can follow and then thereby make it ultra efficient.

Lex Fridman (09:14.820)

I don't know if that's, it was of that nature.

Dmitri Dolgov (09:17.340)

I certainly have that.

Lex Fridman (09:18.580)

There's like fact, like SimCity and factory building games, all those

Dmitri Dolgov (09:22.420)

kinds of things kind of speak to that engineering mindset, or

Lex Fridman (09:26.020)

did you just like the uniform?

Dmitri Dolgov (09:27.500)

I think it was more of the latter.

Lex Fridman (09:28.900)

I think it was the uniform and the, you know, the, the stripe baton that

Dmitri Dolgov (09:33.140)

made cars go in the right directions.

Lex Fridman (09:36.820)

But I guess, you know, I, it is, I did end up, uh, I guess, uh,

Dmitri Dolgov (09:40.980)

you know, working on the transportation industry one way or another uniform.

Lex Fridman (09:44.860)

No, but that's right.

Dmitri Dolgov (09:46.900)

Maybe, maybe, maybe it was my, you know, deep inner infatuation with the,

Lex Fridman (09:51.620)

you know, traffic control batons that led to this career.

Dmitri Dolgov (09:55.460)

Okay.

Lex Fridman (09:55.740)

What, uh, when did you, when was the leap from programming to robotics?

Dmitri Dolgov (10:00.940)

That happened later.

Lex Fridman (10:01.620)

That was after grad school, uh, after, and I actually, the most self driving

Dmitri Dolgov (10:05.340)

cars was I think my first real hands on introduction to robotics.

Lex Fridman (10:10.020)

But I never really had that much hands on experience in school and training.

Dmitri Dolgov (10:14.580)

I, you know, worked on applied math and physics.

Lex Fridman (10:17.340)

Then in college, I did more half, uh, abstract computer science.

Lex Fridman (10:23.540)

And it was after grad school that I really got involved in robotics, which

Lex Fridman (10:28.380)

was actually self driving cars.

Dmitri Dolgov (10:29.980)

And, you know, that was a big flip.

Lex Fridman (10:32.500)

What, uh, what grad school?

Lex Fridman (10:34.140)

So I went to grad school in Michigan, and then I did a postdoc at Stanford,

Lex Fridman (10:37.020)

uh, which is, that was the postdoc where I got to play with self driving cars.

Dmitri Dolgov (10:42.260)

Yeah.

Lex Fridman (10:42.540)

So we'll return there.

Dmitri Dolgov (10:43.740)

Let's go back to, uh, to Moscow.

Lex Fridman (10:46.020)

So, uh, you know, for episode 100, I talked to my dad and also I

Dmitri Dolgov (10:50.260)

grew up with my dad, I guess.

Lex Fridman (10:53.860)

Uh, so I had to put up with them for many years and, uh, he, he went to the

Dmitri Dolgov (11:01.260)

FISTIEG or MIPT, it's weird to say in English, cause I've heard all this

Lex Fridman (11:06.260)

in Russian, Moscow Institute of Physics and Technology.

Lex Fridman (11:09.900)

And to me, that was like, I met some super interesting, as a child, I met

Lex Fridman (11:15.500)

some super interesting characters.

Dmitri Dolgov (11:17.740)

It felt to me like the greatest university in the world, the most elite

Lex Fridman (11:21.460)

university in the world, and just the people that I met that came out of there

Dmitri Dolgov (11:26.340)

were like, not only brilliant, but also special humans.

Lex Fridman (11:32.300)

It seems like that place really tested the soul, uh, both like in terms

Dmitri Dolgov (11:37.900)

of technically and like spiritually.

Lex Fridman (11:40.620)

So that could be just the romanticization of that place.

Dmitri Dolgov (11:43.660)

I'm not sure, but so maybe you can speak to it, but is it correct to

Lex Fridman (11:47.660)

say that you spent some time at FISTIEG?

Dmitri Dolgov (11:50.060)

Yeah, that's right.

Lex Fridman (11:50.740)

Six years.

Dmitri Dolgov (11:51.460)

Uh, I got my bachelor's and master's in physics and math there.

Lex Fridman (11:55.260)

And it's actually interesting cause my, my dad, and actually both my parents,

Dmitri Dolgov (11:59.220)

uh, went there and I think all the stories that I heard, uh, like, just

Lex Fridman (12:03.740)

like you, Alex, uh, growing up about the place and, you know, how interesting

Lex Fridman (12:07.700)

and special and magical it was, I think that was a significant, maybe the

Lex Fridman (12:11.820)

main reason, uh, I wanted to go there, uh, for college, uh, enough so that

Dmitri Dolgov (12:16.460)

I actually went back to Russia from the U S I graduated high school in the U S.

Lex Fridman (12:21.820)

Um, and you went back there.

Dmitri Dolgov (12:23.780)

I went back there.

Lex Fridman (12:24.300)

Yeah, that's exactly the reaction most of my peers in college had.

Dmitri Dolgov (12:28.220)

But, you know, perhaps a little bit stronger that like, you know, point

Lex Fridman (12:32.300)

me out as this crazy kid, were your parents supportive of that?

Dmitri Dolgov (12:34.780)

Yeah.

Lex Fridman (12:35.540)

Yeah.

Dmitri Dolgov (12:35.740)

My games, your previous question, they, uh, they supported me and, you know,

Lex Fridman (12:38.780)

letting me kind of pursue my passions and the things that I was interested in.

Dmitri Dolgov (12:43.380)

That's a bold move.

Lex Fridman (12:44.140)

Wow.

Lex Fridman (12:44.380)

What was it like there?

Lex Fridman (12:45.340)

It was interesting, you know, definitely fairly hardcore on the fundamentals

Dmitri Dolgov (12:49.220)

of, you know, math and physics and, uh, you know, lots of good memories,

Lex Fridman (12:53.300)

uh, from, you know, from those times.

Dmitri Dolgov (12:55.460)

So, okay.

Lex Fridman (12:56.180)

So Stanford.

Lex Fridman (12:57.140)

How'd you get into autonomous vehicles?

Lex Fridman (12:59.180)

I had the great fortune, uh, and great honor to join Stanford's

Dmitri Dolgov (13:06.060)

DARPA urban challenge team.

Lex Fridman (13:07.460)

And, uh, 2006 there, this was a third in the sequence of the DARPA challenges.

Dmitri Dolgov (13:12.300)

There were two grand challenges prior to that.

Lex Fridman (13:14.940)

And then in 2007, they held the DARPA urban challenge.

Dmitri Dolgov (13:19.380)

So, you know, I was doing my, my postdoc I had, I joined the team and, uh, worked

Lex Fridman (13:25.900)

on motion planning, uh, for, you know, that, that competition.

Dmitri Dolgov (13:30.220)

So, okay.

Lex Fridman (13:30.700)

So for people who might not know, I know from, from certain autonomous vehicles is

Dmitri Dolgov (13:35.740)

a funny world in a certain circle of people, everybody knows everything.

Lex Fridman (13:39.860)

And then the certain circle, uh, nobody knows anything in terms of general public.

Lex Fridman (13:45.020)

So it's interesting.

Lex Fridman (13:46.020)

It's, it's a good question of what to talk about, but I do think that the urban

Dmitri Dolgov (13:50.420)

challenge is worth revisiting. It's a fun little challenge.

Lex Fridman (13:56.100)

One that, first of all, like sparked so much, so many incredible minds to focus

Dmitri Dolgov (14:03.020)

on one of the hardest problems of our time in artificial intelligence.

Lex Fridman (14:06.740)

So that's, that's a success from a perspective of a single little challenge.

Lex Fridman (14:11.140)

But can you talk about like, what did the challenge involve?

Lex Fridman (14:14.220)

So were there pedestrians, were there other cars, what was the goal?

Lex Fridman (14:18.660)

Uh, who was on the team?

Lex Fridman (14:20.540)

How long did it take any fun, fun sort of specs?

Dmitri Dolgov (14:25.460)

Sure, sure, sure.

Lex Fridman (14:26.220)

So the way the challenge was constructed and just a little bit of backgrounding,

Dmitri Dolgov (14:29.900)

as I mentioned, this was the third, uh, competition in that series.

Lex Fridman (14:34.220)

The first year we're at the grand challenge called the grand challenge.

Dmitri Dolgov (14:36.940)

The goal there was to just drive in a completely static environment.

Lex Fridman (14:40.260)

You know, you had to drive in a desert, uh, that was very successful.

Lex Fridman (14:45.780)

So then DARPA followed with what they called the urban challenge, where the

Lex Fridman (14:49.980)

goal was to have, you know, build vehicles that could operate in more dynamic

Dmitri Dolgov (14:54.780)

environments and, you know, share them with other vehicles.

Lex Fridman (14:56.780)

There were no pedestrians there, but what DARPA did is they took over

Dmitri Dolgov (15:00.460)

an abandoned air force base.

Lex Fridman (15:02.460)

Uh, and it was kind of like a little fake city that they built out there.

Lex Fridman (15:06.460)

And they had a bunch of, uh, robots, uh, you know, cars, uh, that were

Lex Fridman (15:11.500)

autonomous, uh, in there all at the same time.

Dmitri Dolgov (15:13.740)

Uh, mixed in with other vehicles driven by professional, uh, drivers and each

Lex Fridman (15:19.420)

car, uh, had a mission and so there's a crude map that they received, uh,

Dmitri Dolgov (15:24.980)

beginning and they had a mission and go here and then there and over here.

Lex Fridman (15:28.300)

Um, and they kind of all were sharing this environment at the same time.

Dmitri Dolgov (15:32.980)

They had to interact with each other.

Lex Fridman (15:34.300)

They had to interact with the human drivers.

Dmitri Dolgov (15:36.060)

There's this very first, very rudimentary, um, version of, uh,

Lex Fridman (15:42.340)

self driving car that, you know, could operate, uh, and, uh, in a, in an

Dmitri Dolgov (15:47.300)

environment, you know, shared with other dynamic actors that, as you said,

Dmitri Dolgov (15:50.860)

you know, really, you know, many ways, you know, kickstarted this whole industry.

Dmitri Dolgov (15:55.220)

Okay.

Lex Fridman (15:55.420)

So who was on the team and how'd you do?

Dmitri Dolgov (15:58.420)

I forget.

Lex Fridman (15:59.780)

Uh, I came in second.

Dmitri Dolgov (16:02.460)

Uh, perhaps that was my contribution to the team.

Lex Fridman (16:05.140)

I think the Stanford team came in first in the DARPA challenge.

Dmitri Dolgov (16:07.700)

Uh, but then I joined the team and, you know, you were the one with the

Lex Fridman (16:10.300)

bug in the code, I mean, do you have sort of memories of some

Dmitri Dolgov (16:13.860)

particularly challenging things or, you know, one of the cool things,

Dmitri Dolgov (16:18.900)

it's not, you know, this isn't a product, this isn't the thing that, uh, you know,

Dmitri Dolgov (16:24.220)

it there's, you have a little bit more freedom to experiment so you can take

Lex Fridman (16:27.220)

risks and there's, uh, so you can make mistakes.

Lex Fridman (16:30.460)

Uh, so is there interesting mistakes?

Lex Fridman (16:33.100)

Is there interesting challenges that stand out to you as some, like, taught

Lex Fridman (16:36.060)

you, um, a good technical lesson or a good philosophical lesson from that time?

Lex Fridman (16:42.540)

Yeah.

Dmitri Dolgov (16:43.020)

Uh, you know, definitely, definitely a very memorable time, not really

Lex Fridman (16:46.260)

challenged, but like one of the most vivid memories that I have from the time.

Lex Fridman (16:52.100)

And I think that was actually one of the days that really got me hooked, uh, on

Lex Fridman (16:58.660)

this whole field was, uh, the first time I got to run my software and I got to

Dmitri Dolgov (17:05.660)

software on the car and, uh, I was working on a part of our planning algorithm,

Lex Fridman (17:11.580)

uh, that had to navigate in parking lots.

Lex Fridman (17:13.860)

So it was something that, you know, called free space emotion planning.

Lex Fridman (17:16.780)

So the very first version of that, uh, was, you know, we tried on the car, it

Dmitri Dolgov (17:20.940)

was on Stanford's campus, uh, in the middle of the night and you had this

Lex Fridman (17:24.420)

little course constructed with cones, uh, in the middle of a parking lot.

Lex Fridman (17:28.380)

So we're there in like 3 am, you know, by the time we got the code to, you

Lex Fridman (17:31.700)

know, uh, uh, you know, compile and turn over, uh, and, you know, it drove, I

Dmitri Dolgov (17:36.300)

could actually did something quite reasonable and, you know, it was of

Lex Fridman (17:39.980)

course very buggy at the time and had all kinds of problems, but it was pretty

Dmitri Dolgov (17:46.700)

darn magical.

Lex Fridman (17:48.180)

I remember going back and, you know, later at night and trying to fall

Dmitri Dolgov (17:52.300)

asleep and just, you know, being unable to fall asleep for the rest of the

Lex Fridman (17:55.220)

night, uh, just my mind was blown.

Dmitri Dolgov (17:57.900)

Just like, and that, that, that's what I've been doing ever since for more

Lex Fridman (18:02.340)

than a decade, uh, in terms of challenges and, uh, you know, interesting

Dmitri Dolgov (18:06.460)

memories, like on the day of the competition, uh, it was pretty nerve

Lex Fridman (18:09.780)

wrecking.

Dmitri Dolgov (18:10.300)

Uh, I remember standing there with Mike Montemarillo, who was, uh, the

Lex Fridman (18:13.780)

software lead and wrote most of the code.

Dmitri Dolgov (18:15.780)

I think I did one little part of the planner, Mike, you know, incredibly

Lex Fridman (18:19.420)

that, you know, pretty much the rest of it, uh, with, with, you know, a bunch

Dmitri Dolgov (18:22.820)

of other incredible people, but I remember standing on the day of the

Lex Fridman (18:25.660)

competition, uh, you know, watching the car, you know, with Mike and cars

Lex Fridman (18:29.860)

are completely empty, right?

Lex Fridman (18:32.180)

They're all there lined up in the beginning of the race and then, you

Dmitri Dolgov (18:35.300)

know, DARPA sends them, you know, on their mission one by one.

Lex Fridman (18:38.500)

So then leave and Mike, you just, they had these sirens, they all had

Lex Fridman (18:42.180)

their different silence silence, right?

Lex Fridman (18:43.580)

Each siren had its own personality, if you will.

Dmitri Dolgov (18:46.260)

So, you know, off they go and you don't see them.

Lex Fridman (18:48.460)

You just kind of, and then every once in a while they come a little bit

Dmitri Dolgov (18:50.740)

closer to where the audience is and you can kind of hear, you know, the

Lex Fridman (18:55.060)

sound of your car and then, you know, it seems to be moving along.

Lex Fridman (18:57.380)

So that, you know, gives you hope.

Lex Fridman (18:58.700)

And then, you know, it goes away and you can't hear it for too long.

Lex Fridman (19:01.500)

You start getting anxious, right?

Lex Fridman (19:02.420)

So it's a little bit like, you know, sending your kids to college and like,

Dmitri Dolgov (19:04.140)

you know, kind of you invested in them.

Lex Fridman (19:05.700)

You hope you, you, you, you, you, you, you build it properly, but like,

Dmitri Dolgov (19:09.100)

it's still, uh, anxiety inducing.

Lex Fridman (19:11.700)

Uh, so that was, uh, an incredibly, uh, fun, uh, few days in terms of, you

Dmitri Dolgov (19:16.860)

know, bugs, as you mentioned, you know, one that that was my bug that caused

Lex Fridman (19:20.740)

us the loss of the first place, uh, is still a debate that, you know,

Dmitri Dolgov (19:24.540)

occasionally have with people on the CMU team, CMU came first, I should

Lex Fridman (19:27.820)

mention, uh, that you haven't heard of them, but yeah, it's something, you

Dmitri Dolgov (19:32.380)

know, it's a small school, but it's, it's, it's, you know, really a glitch

Lex Fridman (19:35.340)

that, you know, they happen to succeed at something robotics related.

Dmitri Dolgov (19:38.140)

Very scenic though.

Lex Fridman (19:39.060)

So most people go there for the scenery.

Dmitri Dolgov (19:41.460)

Um, yeah, it's a beautiful campus.

Lex Fridman (19:45.340)

I'm like, unlike Stanford.

Lex Fridman (19:46.780)

So for people, yeah, that's true.

Lex Fridman (19:48.420)

Unlike Stanford, for people who don't know, CMU is one of the great robotics

Lex Fridman (19:51.540)

and sort of artificial intelligence universities in the world, CMU, Carnegie

Lex Fridman (19:55.300)

Mellon university, okay, sorry, go ahead.

Dmitri Dolgov (19:58.380)

Good, good PSA.

Lex Fridman (19:59.420)

So in the part that I contributed to, which was navigating parking lots and

Dmitri Dolgov (1:00:01.260)

you know, we have our user experience research team,

Lex Fridman (1:00:04.220)

tracking over time, that's things about longitudinal is

Dmitri Dolgov (1:00:07.020)

cool. That's that's exactly right. And you know, that's

Lex Fridman (1:00:09.260)

another really valuable feedback, source of feedback.

Lex Fridman (1:00:12.700)

And we're just covering a tremendous amount, right?

Lex Fridman (1:00:16.380)

People go grocery shopping, and they like want to load, you

Dmitri Dolgov (1:00:19.420)

know, 20 bags of groceries in our cars and like that, that's

Lex Fridman (1:00:22.140)

one workflow that you maybe don't think about, you know,

Dmitri Dolgov (1:00:26.700)

getting just right when you're building the driverless

Lex Fridman (1:00:29.500)

product. I have people like, you know, who bike as part of

Dmitri Dolgov (1:00:34.940)

their trip. So they, you know, bike somewhere, then they get

Lex Fridman (1:00:37.100)

on our cars, they take apart their bike, they load into our

Dmitri Dolgov (1:00:39.660)

vehicle, then go and that's, you know, how they, you know,

Lex Fridman (1:00:42.140)

where we want to pull over and how that, you know, get in and

Dmitri Dolgov (1:00:45.340)

get out process works, provides very useful feedback in terms

Lex Fridman (1:00:51.020)

of what makes a good pickup and drop off location, we get

Dmitri Dolgov (1:00:55.420)

really valuable feedback. And in fact, we had to do some really

Lex Fridman (1:01:00.780)

interesting work with high definition maps, and thinking

Dmitri Dolgov (1:01:05.180)

about walking directions. And if you imagine you're in a store,

Lex Fridman (1:01:08.700)

right in some giant space, and then you know, you want to be

Dmitri Dolgov (1:01:11.020)

picked up somewhere, like if you just drop a pin at a current

Lex Fridman (1:01:14.380)

location, which is maybe in the middle of a shopping mall, like

Dmitri Dolgov (1:01:16.780)

what's the best location for the car to come pick you up? And

Lex Fridman (1:01:20.140)

you can have simple heuristics where you're just going to take

Dmitri Dolgov (1:01:22.220)

your you know, you clean in distance and find the nearest

Lex Fridman (1:01:25.500)

spot where the car can pull over that's closest to you. But

Dmitri Dolgov (1:01:28.300)

oftentimes, that's not the most convenient one. You know, I have

Lex Fridman (1:01:30.220)

many anecdotes where that heuristic breaks in horrible

Dmitri Dolgov (1:01:32.860)

ways. One example that I often mentioned is somebody wanted to

Lex Fridman (1:01:38.220)

be, you know, dropped off in Phoenix. And you know, we got

Dmitri Dolgov (1:01:44.300)

car picked location that was close, the closest to there,

Lex Fridman (1:01:49.180)

you know, where the pin was dropped on the map in terms of,

Dmitri Dolgov (1:01:51.820)

you know, latitude and longitude. But it happened to be

Lex Fridman (1:01:55.180)

on the other side of a parking lot that had this row of

Dmitri Dolgov (1:01:58.460)

cacti. And the poor person had to like walk all around the

Lex Fridman (1:02:01.500)

parking lot to get to where they wanted to be in 110 degree

Dmitri Dolgov (1:02:04.300)

heat. So that, you know, that was about so then, you know, we

Lex Fridman (1:02:06.620)

took all take all of these, all that feedback from our users

Lex Fridman (1:02:10.060)

and incorporate it into our system and improve it. Yeah, I

Lex Fridman (1:02:14.060)

feel like that's like requires AGI to solve the problem of

Dmitri Dolgov (1:02:17.900)

like, when you're, which is a very common case, when you're in

Lex Fridman (1:02:21.260)

a big space of some kind, like apartment building, it doesn't

Dmitri Dolgov (1:02:24.700)

matter, it's some large space. And then you call the, like a

Lex Fridman (1:02:29.180)

Waymo from there, right? Like, whatever, it doesn't matter,

Dmitri Dolgov (1:02:32.780)

ride share vehicle. And like, where's the pin supposed to

Lex Fridman (1:02:37.580)

drop? I feel like that's, you don't think, I think that

Dmitri Dolgov (1:02:41.580)

requires AGI. I'm gonna, in order to solve. Okay, the

Lex Fridman (1:02:45.660)

alternative, which I think the Google search engine is taught

Dmitri Dolgov (1:02:50.700)

is like, there's something really valuable about the

Lex Fridman (1:02:55.420)

perhaps slightly dumb answer, but a really powerful one,

Dmitri Dolgov (1:02:58.620)

which is like, what was done in the past by others? Like, what

Lex Fridman (1:03:02.780)

was the choice made by others? That seems to be like in terms

Dmitri Dolgov (1:03:06.380)

of Google search, when you have like billions of searches, you

Lex Fridman (1:03:09.900)

could, you could see which, like when they recommend what you

Dmitri Dolgov (1:03:13.820)

might possibly mean, they suggest based on not some machine

Lex Fridman (1:03:17.660)

learning thing, which they also do, but like, on what was

Dmitri Dolgov (1:03:20.860)

successful for others in the past and finding a thing that

Lex Fridman (1:03:23.580)

they were happy with. Is that integrated at all? Waymo, like

Dmitri Dolgov (1:03:27.820)

what, what pickups worked for others? It is. I think you're

Lex Fridman (1:03:31.740)

exactly right. So there's a real, it's an interesting

Dmitri Dolgov (1:03:34.220)

problem. Naive solutions have interesting failure modes. So

Lex Fridman (1:03:43.580)

there's definitely lots of things that can be done to

Dmitri Dolgov (1:03:48.780)

improve. And both learning from, you know, what works, but

Lex Fridman (1:03:54.940)

doesn't work in actual heal from getting richer data and

Dmitri Dolgov (1:03:57.980)

getting more information about the environment and richer

Lex Fridman (1:04:01.500)

maps. But you're absolutely right, that there's something

Dmitri Dolgov (1:04:04.060)

like there's some properties of solutions that in terms of the

Lex Fridman (1:04:07.580)

effect that they have on users so much, much, much better than

Dmitri Dolgov (1:04:10.140)

others, right? And predictability and

Lex Fridman (1:04:11.900)

understandability is important. So you can have maybe

Dmitri Dolgov (1:04:14.460)

something that is not quite as optimal, but is very natural

Lex Fridman (1:04:17.260)

and predictable to the user and kind of works the same way all

Dmitri Dolgov (1:04:21.580)

the time. And that matters, that matters a lot for the user

Lex Fridman (1:04:25.260)

experience. And but you know, to get to the basics, the pretty

Dmitri Dolgov (1:04:30.300)

fundamental property is that the car actually arrives where you

Lex Fridman (1:04:35.420)

told it to, right? Like, you can always, you know, change it,

Dmitri Dolgov (1:04:37.180)

see it on the map, and you can move it around if you don't

Lex Fridman (1:04:39.100)

like it. And but like, that property that the car actually

Dmitri Dolgov (1:04:42.620)

shows up reliably is critical, which, you know, where compared

Lex Fridman (1:04:47.740)

to some of the human driven analogs, I think, you know, you

Dmitri Dolgov (1:04:52.780)

can have more predictability. It's actually the fact, if I

Lex Fridman (1:04:56.460)

have a little bit of a detour here, I think the fact that

Dmitri Dolgov (1:05:00.140)

it's, you know, your phone and the cars, two computers talking

Lex Fridman (1:05:03.100)

to each other, can lead to some really interesting things we

Dmitri Dolgov (1:05:06.140)

can do in terms of the user interfaces, both in terms of

Lex Fridman (1:05:09.740)

function, like the car actually shows up exactly where you told

Dmitri Dolgov (1:05:13.340)

it, you want it to be, but also some, you know, really

Lex Fridman (1:05:16.140)

interesting things on the user interface, like as the car is

Dmitri Dolgov (1:05:18.380)

driving, as you call it, and it's on the way to come pick

Lex Fridman (1:05:21.180)

you up. And of course, you get the position of the car and the

Dmitri Dolgov (1:05:23.580)

route on the map. But and they actually follow that route, of

Lex Fridman (1:05:26.860)

course. But it can also share some really interesting

Dmitri Dolgov (1:05:29.580)

information about what it's doing. So, you know, our cars, as

Lex Fridman (1:05:34.140)

they are coming to pick you up, if it's come, if a car is

Dmitri Dolgov (1:05:36.940)

coming up to a stop sign, it will actually show you that

Lex Fridman (1:05:39.180)

like, it's there sitting, because it's at a stop sign or

Dmitri Dolgov (1:05:41.340)

a traffic light will show you that it's got, you know,

Lex Fridman (1:05:42.860)

sitting at a red light. So, you know, they're like little

Dmitri Dolgov (1:05:44.700)

things, right? But I find those little touches really

Lex Fridman (1:05:51.340)

interesting, really magical. And it's just, you know, little

Dmitri Dolgov (1:05:54.620)

things like that, that you can do to kind of delight your

Lex Fridman (1:05:57.180)

users. You know, this makes me think of, there's some products

Dmitri Dolgov (1:06:02.940)

that I just love. Like, there's a there's a company called

Lex Fridman (1:06:07.340)

Rev, Rev.com, where I like for this podcast, for example, I

Dmitri Dolgov (1:06:13.500)

can drag and drop a video. And then they do all the

Lex Fridman (1:06:17.900)

captioning. It's humans doing the captioning, but they

Dmitri Dolgov (1:06:21.340)

connect, they automate everything of connecting you to

Lex Fridman (1:06:24.780)

the humans, and they do the captioning and transcription.

Dmitri Dolgov (1:06:27.180)

It's all effortless. And it like, I remember when I first

Lex Fridman (1:06:29.980)

started using them, I was like, life's good. Like, because it

Dmitri Dolgov (1:06:35.500)

was so painful to figure that out earlier. The same thing

Lex Fridman (1:06:39.020)

with something called iZotope RX, this company I use for

Dmitri Dolgov (1:06:43.260)

cleaning up audio, like the sound cleanup they do. It's

Lex Fridman (1:06:46.380)

like drag and drop, and it just cleans everything up very

Dmitri Dolgov (1:06:49.580)

nicely. Another experience like that I had with Amazon

Lex Fridman (1:06:52.940)

OneClick purchase, first time. I mean, other places do that

Dmitri Dolgov (1:06:57.180)

now, but just the effortlessness of purchasing,

Lex Fridman (1:07:00.140)

making it frictionless. It kind of communicates to me, like,

Dmitri Dolgov (1:07:04.380)

I'm a fan of design. I'm a fan of products that you can just

Lex Fridman (1:07:08.700)

create a really pleasant experience. The simplicity of

Dmitri Dolgov (1:07:12.540)

it, the elegance just makes you fall in love with it. So on

Lex Fridman (1:07:16.380)

the, do you think about this kind of stuff? I mean, it's

Dmitri Dolgov (1:07:19.820)

exactly what we've been talking about. It's like the little

Lex Fridman (1:07:22.540)

details that somehow make you fall in love with the product.

Dmitri Dolgov (1:07:25.500)

Is that, we went from like urban challenge days, where

Lex Fridman (1:07:30.860)

love was not part of the conversation, probably. And to

Dmitri Dolgov (1:07:34.540)

this point where there's a, where there's human beings and

Lex Fridman (1:07:39.180)

you want them to fall in love with the experience. Is that

Dmitri Dolgov (1:07:42.780)

something you're trying to optimize for? Try to think

Lex Fridman (1:07:45.020)

about, like, how do you create an experience that people love?

Dmitri Dolgov (1:07:48.700)

Absolutely. I think that's the vision is removing any friction

Lex Fridman (1:07:55.100)

or complexity from getting our users, our writers to where

Dmitri Dolgov (1:08:02.300)

they want to go. Making that as simple as possible. And then,

Lex Fridman (1:08:06.780)

you know, beyond that, just transportation, making things

Lex Fridman (1:08:10.620)

and goods get to their destination as seamlessly as

Lex Fridman (1:08:13.580)

possible. I talked about a drag and drop experience where I

Dmitri Dolgov (1:08:17.020)

kind of express your intent and then it just magically happens.

Lex Fridman (1:08:20.460)

And for our writers, that's what we're trying to get to is

Dmitri Dolgov (1:08:23.100)

you download an app and you click and car shows up. It's

Lex Fridman (1:08:28.380)

the same car. It's very predictable. It's a safe and

Dmitri Dolgov (1:08:33.580)

high quality experience. And then it gets you in a very

Lex Fridman (1:08:37.500)

reliable, very convenient, frictionless way to where you

Dmitri Dolgov (1:08:43.900)

want to be. And along the journey, I think we also want to

Lex Fridman (1:08:47.900)

do little things to delight our users. Like the ride sharing

Dmitri Dolgov (1:08:52.940)

companies, because they don't control the experience, I

Lex Fridman (1:08:56.620)

think they can't make people fall in love necessarily with

Dmitri Dolgov (1:09:00.300)

the experience. Or maybe they, they haven't put in the effort,

Lex Fridman (1:09:04.140)

but I think if I were to speak to the ride sharing experience

Dmitri Dolgov (1:09:08.060)

I currently have, it's just very, it's just very

Lex Fridman (1:09:11.340)

convenient, but there's a lot of room for like falling in love

Dmitri Dolgov (1:09:16.540)

with it. Like we can speak to sort of car companies, car

Lex Fridman (1:09:20.140)

companies do this. Well, you can fall in love with a car,

Dmitri Dolgov (1:09:22.380)

right? And be like a loyal car person, like whatever. Like I

Lex Fridman (1:09:26.620)

like badass hot rods, I guess, 69 Corvette. And at this point,

Dmitri Dolgov (1:09:31.260)

you know, you can't really, cars are so, owning a car is so

Lex Fridman (1:09:35.580)

20th century, man. But is there something about the Waymo

Dmitri Dolgov (1:09:41.020)

experience where you hope that people will fall in love with

Lex Fridman (1:09:43.660)

it? Is that part of it? Or is it part of, is it just about

Dmitri Dolgov (1:09:48.780)

making a convenient ride, not ride sharing, I don't know what

Lex Fridman (1:09:52.380)

the right term is, but just a convenient A to B autonomous

Dmitri Dolgov (1:09:56.060)

transport or like, do you want them to fall in love with

Lex Fridman (1:10:02.460)

Waymo? To maybe elaborate a little bit. I mean, almost like

Dmitri Dolgov (1:10:06.140)

from a business perspective, I'm curious, like how, do you

Lex Fridman (1:10:11.820)

want to be in the background invisible or do you want to be

Dmitri Dolgov (1:10:15.260)

like a source of joy that's in very much in the foreground? I

Lex Fridman (1:10:20.060)

want to provide the best, most enjoyable transportation

Dmitri Dolgov (1:10:24.540)

solution. And that means building it, building our

Lex Fridman (1:10:31.260)

product and building our service in a way that people do.

Dmitri Dolgov (1:10:34.300)

Kind of use in a very seamless, frictionless way in their

Lex Fridman (1:10:41.900)

day to day lives. And I think that does mean, you know, in

Dmitri Dolgov (1:10:45.580)

some way falling in love in that product, right, just kind of

Lex Fridman (1:10:48.300)

becomes part of your routine. It comes down my mind to safety,

Dmitri Dolgov (1:10:54.700)

predictability of the experience, and privacy aspects

Lex Fridman (1:11:02.060)

of it, right? Our cars, you get the same car, you get very

Dmitri Dolgov (1:11:07.100)

predictable behavior. And you get a lot of different

Lex Fridman (1:11:11.340)

things. And that is important. And if you're going to use it

Dmitri Dolgov (1:11:14.940)

in your daily life, privacy, and when you're in a car, you

Lex Fridman (1:11:18.700)

can do other things. You're spending a bunch, just another

Dmitri Dolgov (1:11:21.020)

space where you're spending a significant part of your life.

Lex Fridman (1:11:24.380)

And so not having to share it with other people who you don't

Dmitri Dolgov (1:11:27.820)

want to share it with, I think is a very nice property. Maybe

Lex Fridman (1:11:32.380)

you want to take a phone call or do something else in the

Dmitri Dolgov (1:11:34.540)

vehicle. And, you know, safety on the quality of the driving,

Lex Fridman (1:11:40.620)

as well as the physical safety of not having to share that

Dmitri Dolgov (1:11:45.660)

ride is important to a lot of people. What about the idea

Lex Fridman (1:11:52.300)

that when there's somebody like a human driving, and they do

Dmitri Dolgov (1:11:56.940)

a rolling stop on a stop sign, like sometimes like, you know,

Lex Fridman (1:12:01.180)

you get an Uber or Lyft or whatever, like human driver,

Dmitri Dolgov (1:12:04.220)

and, you know, they can be a little bit aggressive as

Lex Fridman (1:12:07.980)

drivers. It feels like there's not all aggression is bad. Now

Dmitri Dolgov (1:12:14.540)

that may be a wrong, again, 20th century conception of

Lex Fridman (1:12:17.500)

driving. Maybe it's possible to create a driving experience.

Dmitri Dolgov (1:12:21.100)

Like if you're in the back, busy doing something, maybe

Lex Fridman (1:12:24.940)

aggression is not a good thing. It's a very different kind of

Dmitri Dolgov (1:12:27.740)

experience perhaps. But it feels like in order to navigate

Lex Fridman (1:12:32.540)

this world, you need to, how do I phrase this? You need to kind

Dmitri Dolgov (1:12:38.540)

of bend the rules a little bit, or at least test the rules. I

Lex Fridman (1:12:42.140)

don't know what language politicians use to discuss this,

Lex Fridman (1:12:44.540)

but whatever language they use, you like flirt with the rules.

Lex Fridman (1:12:48.700)

I don't know. But like you sort of have a bit of an aggressive

Dmitri Dolgov (1:12:55.580)

way of driving that asserts your presence in this world,

Lex Fridman (1:13:00.460)

thereby making other vehicles and people respect your

Dmitri Dolgov (1:13:03.660)

presence and thereby allowing you to sort of navigate

Lex Fridman (1:13:06.780)

through intersections in a timely fashion. I don't know if

Dmitri Dolgov (1:13:10.060)

any of that made sense, but like, how does that fit into the

Lex Fridman (1:13:14.300)

experience of driving autonomously? Is that?

Dmitri Dolgov (1:13:18.620)

It's a lot of thoughts. This is you're hitting on a very

Lex Fridman (1:13:20.460)

important point of a number of behavioral components and, you

Dmitri Dolgov (1:13:27.500)

know, parameters that make your driving feel assertive and

Lex Fridman (1:13:34.380)

natural and comfortable and predictable. Our cars will

Dmitri Dolgov (1:13:37.260)

follow rules, right? They will do the safest thing possible in

Lex Fridman (1:13:39.740)

all situations. Let me be clear on that. But if you think of

Dmitri Dolgov (1:13:43.580)

really, really good drivers, just think about

Lex Fridman (1:13:47.660)

professional lemon drivers, right? They will follow the

Dmitri Dolgov (1:13:49.740)

rules. They're very, very smooth, and yet they're very

Lex Fridman (1:13:53.900)

efficient. But they're assertive. They're comfortable

Dmitri Dolgov (1:13:58.140)

for the people in the vehicle. They're predictable for the

Lex Fridman (1:14:02.140)

other people outside the vehicle that they share the

Dmitri Dolgov (1:14:03.820)

environment with. And that's the kind of driver that we want

Lex Fridman (1:14:06.540)

to build. And you think if maybe there's a sport analogy

Dmitri Dolgov (1:14:11.100)

there, right? You can do in very many sports, the true

Lex Fridman (1:14:17.740)

professionals are very efficient in their movements,

Lex Fridman (1:14:20.620)

right? They don't do like, you know, hectic flailing, right?

Lex Fridman (1:14:25.100)

They're, you know, smooth and precise, right? And they get

Dmitri Dolgov (1:14:29.020)

the best results. So that's the kind of driver that we want to

Lex Fridman (1:14:30.860)

build. In terms of, you know, aggressiveness. Yeah, you can

Dmitri Dolgov (1:14:33.100)

like, you know, roll through the stop signs. You can do crazy

Lex Fridman (1:14:35.740)

lane changes. It typically doesn't get you to your

Dmitri Dolgov (1:14:38.060)

destination faster. Typically not the safest or most

Lex Fridman (1:14:40.700)

predictable, very most comfortable thing to do. But

Dmitri Dolgov (1:14:45.820)

there is a way to do both. And that's what we're

Lex Fridman (1:14:49.660)

doing. We're trying to build the driver that is safe,

Dmitri Dolgov (1:14:53.820)

comfortable, smooth, and predictable. Yeah, that's a

Lex Fridman (1:14:58.140)

really interesting distinction. I think in the early days of

Dmitri Dolgov (1:15:00.380)

autonomous vehicles, the vehicles felt cautious as

Lex Fridman (1:15:03.660)

opposed to efficient. And I'm still probably, but when I

Dmitri Dolgov (1:15:08.620)

rode in the Waymo, I mean, there was, it was, it was quite

Lex Fridman (1:15:13.500)

assertive. It moved pretty quickly. Like, yeah, then he's

Dmitri Dolgov (1:15:19.740)

one of the surprising feelings was that it actually, it went

Lex Fridman (1:15:22.940)

fast. And it didn't feel like, awkwardly cautious than

Dmitri Dolgov (1:15:28.300)

autonomous vehicle. Like, like, so I've also programmed

Lex Fridman (1:15:31.900)

autonomous vehicles and everything I've ever built was

Dmitri Dolgov (1:15:34.860)

felt awkwardly, either overly aggressive. Okay, especially

Lex Fridman (1:15:39.260)

when it was my code, or like, awkwardly cautious is the way

Dmitri Dolgov (1:15:44.860)

I would put it. And Waymo's vehicle felt like, assertive

Lex Fridman (1:15:53.180)

and I think efficient is like the right terminology here.

Dmitri Dolgov (1:15:57.180)

It wasn't, and I also like the professional limo driver,

Lex Fridman (1:16:01.340)

because we often think like, you know, an Uber driver or a

Dmitri Dolgov (1:16:06.060)

bus driver or a taxi. This is the funny thing is people

Lex Fridman (1:16:09.820)

think they track taxi drivers are professionals. They, I

Dmitri Dolgov (1:16:14.940)

mean, it's, it's like, that that's like saying, I'm a

Lex Fridman (1:16:18.460)

professional walker, just because I've been walking all

Dmitri Dolgov (1:16:20.780)

my life. I think there's an art to it, right? And if you take

Lex Fridman (1:16:25.580)

it seriously as an art form, then there's a certain way that

Dmitri Dolgov (1:16:30.700)

mastery looks like. It's interesting to think about what

Lex Fridman (1:16:33.900)

does mastery look like in driving? And perhaps what we

Dmitri Dolgov (1:16:39.180)

associate with like aggressiveness is unnecessary,

Lex Fridman (1:16:43.020)

like, it's not part of the experience of driving. It's

Dmitri Dolgov (1:16:46.940)

like, unnecessary fluff, that efficiency, you can be,

Lex Fridman (1:16:54.860)

you can create a good driving experience within the rules.

Dmitri Dolgov (1:17:00.380)

That's, I mean, you're the first person to tell me this.

Lex Fridman (1:17:03.100)

So it's, it's kind of interesting. I need to think

Dmitri Dolgov (1:17:04.940)

about this, but that's exactly what it felt like with Waymo.

Lex Fridman (1:17:07.900)

I kind of had this intuition. Maybe it's the Russian thing.

Dmitri Dolgov (1:17:10.060)

I don't know that you have to break the rules in life to get

Lex Fridman (1:17:13.740)

anywhere, but maybe, maybe it's possible that that's not the

Dmitri Dolgov (1:17:19.020)

case in driving. I have to think about that, but it

Lex Fridman (1:17:23.500)

certainly felt that way on the streets of Phoenix when I was

Dmitri Dolgov (1:17:25.980)

there in Waymo, that, that, that that was a very pleasant

Lex Fridman (1:17:29.340)

experience and it wasn't frustrating in that like, come

Dmitri Dolgov (1:17:32.460)

on, move already kind of feeling. It wasn't, that wasn't

Lex Fridman (1:17:35.260)

there. Yeah. I mean, that's what, that's what we're going

Dmitri Dolgov (1:17:37.900)

after. I don't think you have to pick one. I think truly good

Lex Fridman (1:17:41.420)

driving. It gives you both efficiency, a certainness, but

Dmitri Dolgov (1:17:45.020)

also comfort and predictability and safety. And, you know, it's,

Lex Fridman (1:17:49.900)

that's what fundamental improvements in the core

Dmitri Dolgov (1:17:54.940)

capabilities truly unlock. And you can kind of think of it as,

Lex Fridman (1:17:59.260)

you know, a precision and recall trade off. You have certain

Dmitri Dolgov (1:18:01.980)

capabilities of your model. And then it's very easy when, you

Lex Fridman (1:18:04.460)

know, you have some curve of precision and recall, you can

Dmitri Dolgov (1:18:06.540)

move things around and can choose your operating point and

Lex Fridman (1:18:08.540)

your training of precision versus recall, false positives

Dmitri Dolgov (1:18:10.700)

versus false negatives. Right. But then, and you know, you can

Lex Fridman (1:18:14.220)

tune things on that curve and be kind of more cautious or more

Dmitri Dolgov (1:18:16.940)

aggressive, but then aggressive is bad or, you know, cautious is

Lex Fridman (1:18:19.340)

bad, but true capabilities come from actually moving the whole

Dmitri Dolgov (1:18:22.540)

curve up. And then you are kind of on a very different plane of

Lex Fridman (1:18:28.540)

those trade offs. And that, that's what we're trying to do

Dmitri Dolgov (1:18:31.340)

here is to move the whole curve up. Before I forget, let's talk

Lex Fridman (1:18:34.700)

about trucks a little bit. So I also got a chance to check out

Dmitri Dolgov (1:18:39.420)

some of the Waymo trucks. I'm not sure if we want to go too

Lex Fridman (1:18:44.300)

much into that space, but it's a fascinating one. So maybe we

Dmitri Dolgov (1:18:47.180)

can mention at least briefly, you know, Waymo is also now

Lex Fridman (1:18:51.020)

doing autonomous trucking and how different like

Dmitri Dolgov (1:18:56.540)

philosophically and technically is that whole space of

Lex Fridman (1:18:58.780)

problems. It's one of our two big products and you know,

Dmitri Dolgov (1:19:06.060)

commercial applications of our driver, right? Right. Hailing

Lex Fridman (1:19:09.020)

and deliveries. You know, we have Waymo One and Waymo Via

Dmitri Dolgov (1:19:12.700)

moving people and moving goods. You know, trucking is an

Lex Fridman (1:19:16.220)

example of moving goods. We've been working on trucking since

Dmitri Dolgov (1:19:21.580)

2017. It is a very interesting space. And your question of

Lex Fridman (1:19:31.340)

how different is it? It has this really nice property that

Dmitri Dolgov (1:19:35.020)

the first order challenges, like the science, the hard

Lex Fridman (1:19:38.780)

engineering, whether it's, you know, hardware or, you know,

Dmitri Dolgov (1:19:42.140)

onboard software or off board software, all of the, you know,

Lex Fridman (1:19:45.420)

systems that you build for, you know, training your ML models

Dmitri Dolgov (1:19:48.780)

for, you know, evaluating your time system. Like those

Lex Fridman (1:19:51.820)

fundamentals carry over. Like the true challenges of, you

Dmitri Dolgov (1:19:56.460)

know, driving perception, semantic understanding,

Lex Fridman (1:20:00.620)

prediction, decision making, planning, evaluation, the

Dmitri Dolgov (1:20:04.860)

simulator, ML infrastructure, those carry over. Like the data

Lex Fridman (1:20:08.780)

and the application and kind of the domains might be

Dmitri Dolgov (1:20:12.380)

different, but the most difficult problems, all of that

Lex Fridman (1:20:16.060)

carries over between the domains. So that's very nice.

Lex Fridman (1:20:19.420)

So that's how we approach it. We're kind of build investing

Lex Fridman (1:20:22.300)

in the core, the technical core. And then there's

Dmitri Dolgov (1:20:26.220)

specialization of that core technology to different

Lex Fridman (1:20:30.620)

product lines, to different commercial applications. So on

Dmitri Dolgov (1:20:34.540)

just to tease it apart a little bit on trucks. So starting with

Lex Fridman (1:20:38.140)

the hardware, the configuration of the sensors is different.

Dmitri Dolgov (1:20:42.140)

They're different physically, geometrically, you know, different

Lex Fridman (1:20:46.300)

vehicles. So for example, we have two of our main laser on

Dmitri Dolgov (1:20:50.860)

the trucks on both sides so that we have, you know, not have the

Lex Fridman (1:20:54.380)

blind spots. Whereas on the JLR eye pace, we have, you know, one

Dmitri Dolgov (1:20:59.100)

of it sitting at the very top, but the actual sensors are

Lex Fridman (1:21:02.940)

almost the same. Now we're largely the same. So all of the

Dmitri Dolgov (1:21:06.700)

investment that over the years we've put into building our

Lex Fridman (1:21:11.180)

custom lighters, custom radars, pulling the whole system

Dmitri Dolgov (1:21:13.580)

together, that carries over very nicely. Then, you know, on the

Lex Fridman (1:21:16.540)

perception side, the like the fundamental challenges of

Dmitri Dolgov (1:21:20.780)

seeing, understanding the world, whether it's, you know, object

Lex Fridman (1:21:22.940)

detection, classification, you know, tracking, semantic

Dmitri Dolgov (1:21:25.740)

understanding, all that carries over. You know, yes, there's

Lex Fridman (1:21:28.300)

some specialization when you're driving on freeways, you know,

Dmitri Dolgov (1:21:31.100)

range becomes more important. The domain is a little bit

Lex Fridman (1:21:33.820)

different. But again, the fundamentals carry over very,

Dmitri Dolgov (1:21:36.860)

very nicely. Same, and you guess you get into prediction or

Lex Fridman (1:21:41.100)

decision making, right, the fundamentals of what it takes to

Dmitri Dolgov (1:21:45.260)

predict what other people are going to do to find the long

Lex Fridman (1:21:49.580)

tail to improve your system in that long tail of behavior

Dmitri Dolgov (1:21:53.420)

prediction and response that carries over right and so on and

Lex Fridman (1:21:56.060)

so on. So I mean, that's pretty exciting. By the way, does

Dmitri Dolgov (1:22:00.060)

Waymo via include using the smaller vehicles for

Lex Fridman (1:22:05.100)

transportation of goods? That's an interesting distinction. So

Dmitri Dolgov (1:22:07.580)

I would say there's three interesting modes of operation.

Lex Fridman (1:22:13.020)

So one is moving humans, one is moving goods, and one is like

Dmitri Dolgov (1:22:16.860)

moving nothing, zero occupancy, meaning like you're going to

Lex Fridman (1:22:21.740)

the destination, your empty vehicle. I mean, it's the third

Dmitri Dolgov (1:22:27.580)

is the less of it. If that's the entirety of it, it's the less,

Lex Fridman (1:22:29.820)

you know, exciting from the commercial perspective.

Dmitri Dolgov (1:22:31.660)

Well, I mean, in terms of like, if you think about what's

Lex Fridman (1:22:38.140)

inside a vehicle as it's moving, because it does, you

Dmitri Dolgov (1:22:42.060)

know, some significant fraction of the vehicle's movement has

Lex Fridman (1:22:45.580)

to be empty. I mean, it's kind of fascinating. Maybe just on

Dmitri Dolgov (1:22:50.700)

that small point, is there different control and like

Lex Fridman (1:22:57.340)

policies that are applied for zero occupancy vehicle? So

Dmitri Dolgov (1:23:01.500)

vehicle with nothing in it, or is it just move as if there is

Lex Fridman (1:23:04.940)

a person inside? What was with some subtle differences?

Dmitri Dolgov (1:23:09.500)

As a first order approximation, there are no differences. And

Lex Fridman (1:23:13.100)

if you think about, you know, safety and comfort and quality

Dmitri Dolgov (1:23:17.740)

of driving, only part of it has to do with the people or the

Lex Fridman (1:23:26.540)

goods inside of the vehicle. But you don't want to be, you

Dmitri Dolgov (1:23:29.340)

know, you want to drive smoothly, as we discussed, not

Lex Fridman (1:23:31.820)

for the purely for the benefit of whatever you have inside the

Dmitri Dolgov (1:23:34.780)

car, right? It's also for the benefit of the people outside

Lex Fridman (1:23:38.540)

kind of fitting naturally and predictably into that whole

Dmitri Dolgov (1:23:41.660)

environment, right? So, you know, yes, there are some

Lex Fridman (1:23:43.820)

second order things you can do, you can change your route, and

Dmitri Dolgov (1:23:47.180)

you optimize maybe kind of your fleet, things at the fleet

Lex Fridman (1:23:50.860)

scale. And you would take into account whether some of your

Dmitri Dolgov (1:23:54.300)

you know, some of your cars are actually, you know, serving a

Lex Fridman (1:23:58.780)

useful trip, whether with people or with goods, whereas, you

Dmitri Dolgov (1:24:01.180)

know, other cars are, you know, driving completely empty to that

Lex Fridman (1:24:05.180)

next valuable trip that they're going to provide. But that those

Dmitri Dolgov (1:24:09.500)

are mostly second order effects. Okay, cool. So Phoenix

Lex Fridman (1:24:14.380)

is, is an incredible place. And what you've announced in

Dmitri Dolgov (1:24:18.780)

Phoenix is, it's kind of amazing. But, you know, that's

Lex Fridman (1:24:23.340)

just like one city. How do you take over the world? I mean,

Dmitri Dolgov (1:24:30.220)

I'm asking for a friend. One step at a time.

Lex Fridman (1:24:35.980)

Is that a cartoon pinky in the brain? Yeah. Okay. But, you

Dmitri Dolgov (1:24:40.460)

know, gradually is a true answer. So I think the heart of

Lex Fridman (1:24:44.540)

your question is, can you ask a better question than I asked?

Dmitri Dolgov (1:24:48.860)

You're asking a great question. Answer that one. I'm just

Lex Fridman (1:24:52.940)

gonna, you know, phrase it in the terms that I want to

Dmitri Dolgov (1:24:56.300)

answer. Exactly right. Brilliant. Please. You know,

Lex Fridman (1:25:01.660)

where are we today? And, you know, what happens next? And

Lex Fridman (1:25:04.940)

what does it take to go beyond Phoenix? And what does it

Lex Fridman (1:25:08.220)

take to get this technology to more places and more people

Dmitri Dolgov (1:25:13.660)

around the world, right? So our next big area of focus is

Lex Fridman (1:25:23.100)

exactly that. Larger scale commercialization and just,

Dmitri Dolgov (1:25:26.700)

you know, scaling up. If I think about, you know, the

Lex Fridman (1:25:35.340)

main, and, you know, Phoenix gives us that platform and

Dmitri Dolgov (1:25:39.100)

gives us that foundation of upon which we can build. And

Lex Fridman (1:25:44.940)

it's, there are few really challenging aspects of this

Dmitri Dolgov (1:25:51.580)

whole problem that you have to pull together in order to build

Lex Fridman (1:25:56.460)

the technology in order to deploy it into the field to go

Dmitri Dolgov (1:26:03.900)

from a driverless car to a fleet of cars that are providing a

Lex Fridman (1:26:09.820)

service, and then all the way to commercialization. So, and

Dmitri Dolgov (1:26:14.140)

then, you know, this is what we have in Phoenix. We've taken

Lex Fridman (1:26:15.980)

the technology from a proof point to an actual deployment

Lex Fridman (1:26:21.100)

and have taken our driver from, you know, one car to a fleet

Lex Fridman (1:26:25.980)

that can provide a service. Beyond that, if I think about

Lex Fridman (1:26:29.980)

what it will take to scale up and, you know, deploy in, you

Lex Fridman (1:26:35.820)

know, more places with more customers, I tend to think about

Dmitri Dolgov (1:26:41.740)

three main dimensions, three main axes of scale. One is the

Lex Fridman (1:26:48.380)

core technology, you know, the hardware and software core

Dmitri Dolgov (1:26:51.660)

capabilities of our driver. The second dimension is

Lex Fridman (1:26:56.540)

evaluation and deployment. And the third one is the, you know,

Dmitri Dolgov (1:27:01.900)

product, commercial, and operational excellence. So you

Lex Fridman (1:27:06.060)

can talk a bit about where we are along, you know, each one of

Dmitri Dolgov (1:27:09.660)

those three dimensions about where we are today and, you

Lex Fridman (1:27:11.900)

know, what has, what will happen next. On, you know, the core

Dmitri Dolgov (1:27:16.780)

technology, you know, the hardware and software, you

Lex Fridman (1:27:19.580)

know, together comprise a driver, we, you know, obviously

Dmitri Dolgov (1:27:25.420)

have that foundation that is providing fully driverless

Lex Fridman (1:27:30.460)

trips to our customers as we speak, in fact. And we've

Dmitri Dolgov (1:27:34.780)

learned a tremendous amount from that. So now what we're

Lex Fridman (1:27:39.500)

doing is we are incorporating all those lessons into some

Dmitri Dolgov (1:27:44.380)

pretty fundamental improvements in our core technology, both on

Lex Fridman (1:27:47.180)

the hardware side and on the software side to build a more

Dmitri Dolgov (1:27:51.660)

general, more robust solution that then will enable us to

Lex Fridman (1:27:54.860)

massively scale beyond Phoenix. So on the hardware side, all of

Dmitri Dolgov (1:28:00.460)

those lessons are now incorporated into this fifth

Lex Fridman (1:28:05.180)

generation hardware platform that is, you know, being

Dmitri Dolgov (1:28:09.500)

deployed right now. And that's the platform, the fourth

Lex Fridman (1:28:13.180)

generation, the thing that we have right now driving in

Dmitri Dolgov (1:28:14.860)

Phoenix, it's good enough to operate fully driverlessly,

Lex Fridman (1:28:18.700)

you know, night and day, you know, various speeds and

Dmitri Dolgov (1:28:21.500)

various conditions, but the fifth generation is the platform

Lex Fridman (1:28:25.020)

upon which we want to go to massive scale. We, in turn,

Dmitri Dolgov (1:28:30.140)

we've really made qualitative improvements in terms of the

Lex Fridman (1:28:32.620)

capability of the system, the simplicity of the architecture,

Dmitri Dolgov (1:28:35.980)

the reliability of the redundancy. It is designed to be

Lex Fridman (1:28:39.900)

manufacturable at very large scale and, you know, provides

Dmitri Dolgov (1:28:42.300)

the right unit economics. So that's the next big step for us

Lex Fridman (1:28:46.380)

on the hardware side. That's already there for scale,

Dmitri Dolgov (1:28:49.580)

the version five. That's right. And is that coincidence or

Lex Fridman (1:28:53.500)

should we look into a conspiracy theory that it's the

Dmitri Dolgov (1:28:55.580)

same version as the pixel phone? Is that what's the

Lex Fridman (1:28:59.660)

hardware? They neither confirm nor deny. All right, cool. So,

Lex Fridman (1:29:04.220)

sorry. So that's the, okay, that's that axis. What else?

Lex Fridman (1:29:08.140)

So similarly, you know, hardware is a very discreet

Dmitri Dolgov (1:29:11.100)

jump, but, you know, similar to how we're making that change

Lex Fridman (1:29:14.940)

from the fourth generation hardware to the fifth, we're

Dmitri Dolgov (1:29:16.940)

making similar improvements on the software side to make it

Lex Fridman (1:29:19.420)

more, you know, robust and more general and allow us to kind of

Dmitri Dolgov (1:29:22.300)

quickly scale beyond Phoenix. So that's the first dimension of

Lex Fridman (1:29:25.740)

core technology. The second dimension is evaluation and

Dmitri Dolgov (1:29:27.980)

deployment. How do you measure your system? How do you

Lex Fridman (1:29:34.300)

evaluate it? How do you build a release and deployment process

Dmitri Dolgov (1:29:37.500)

where, you know, with confidence, you can, you know,

Lex Fridman (1:29:40.780)

regularly release new versions of your driver into a fleet?

Lex Fridman (1:29:45.420)

How do you get good at it so that it is not, you know, a

Lex Fridman (1:29:49.180)

huge tax on your researchers and engineers that, you know, so

Dmitri Dolgov (1:29:52.540)

you can, how do you build all these, you know, processes, the

Lex Fridman (1:29:55.740)

frameworks, the simulation, the evaluation, the data science,

Dmitri Dolgov (1:29:58.620)

the validation, so that, you know, people can focus on

Lex Fridman (1:30:01.340)

improving the system and kind of the releases just go out the

Dmitri Dolgov (1:30:04.380)

door and get deployed across the fleet. So we've gotten really

Lex Fridman (1:30:07.340)

good at that in Phoenix. That's been a tremendously difficult

Dmitri Dolgov (1:30:11.820)

problem, but that's what we have in Phoenix right now that gives

Lex Fridman (1:30:15.180)

us that foundation. And now we're working on kind of

Dmitri Dolgov (1:30:17.660)

incorporating all the lessons that we've learned to make it

Lex Fridman (1:30:20.220)

more efficient, to go to new places, you know, and scale up

Lex Fridman (1:30:22.860)

and just kind of, you know, stamp things out. So that's that

Lex Fridman (1:30:25.660)

second dimension of evaluation and deployment. And the third

Dmitri Dolgov (1:30:28.700)

dimension is product, commercial, and operational

Lex Fridman (1:30:33.340)

excellence, right? And again, Phoenix there is providing an

Dmitri Dolgov (1:30:38.140)

incredibly valuable platform. You know, that's why we're doing

Lex Fridman (1:30:40.940)

things end to end in Phoenix. We're learning, as you know, we

Dmitri Dolgov (1:30:43.660)

discussed a little earlier today, tremendous amount of

Lex Fridman (1:30:47.900)

really valuable lessons from our users getting really

Dmitri Dolgov (1:30:50.460)

incredible feedback. And we'll continue to iterate on that and

Lex Fridman (1:30:54.860)

incorporate all those lessons into making our product, you

Dmitri Dolgov (1:30:59.420)

know, even better and more convenient for our users.

Lex Fridman (1:31:01.660)

So you're converting this whole process in Phoenix into

Dmitri Dolgov (1:31:06.620)

something that could be copy and pasted elsewhere. So like,

Lex Fridman (1:31:11.260)

perhaps you didn't think of it that way when you were doing

Dmitri Dolgov (1:31:13.180)

the experimentation in Phoenix, but so how long did you

Lex Fridman (1:31:17.660)

basically, and you can correct me, but you've, I mean, it's

Dmitri Dolgov (1:31:22.140)

still early days, but you've taken the full journey in

Lex Fridman (1:31:24.700)

Phoenix, right? As you were saying of like what it takes to

Dmitri Dolgov (1:31:29.180)

basically automate. I mean, it's not the entirety of Phoenix,

Lex Fridman (1:31:31.900)

right? But I imagine it can encompass the entirety of

Dmitri Dolgov (1:31:36.300)

Phoenix. That's some near term date, but that's not even

Lex Fridman (1:31:41.340)

perhaps important. Like as long as it's a large enough

Dmitri Dolgov (1:31:43.740)

geographic area. So what, how copy pastable is that process

Lex Fridman (1:31:51.580)

currently and how like, you know, like when you copy and

Dmitri Dolgov (1:31:58.300)

paste in Google docs, I think now in, or in word, you can

Lex Fridman (1:32:05.260)

like apply source formatting or apply destination formatting.

Lex Fridman (1:32:09.340)

So how, when you copy and paste the Phoenix into like, say

Lex Fridman (1:32:14.620)

Boston, how do you apply the destination formatting? Like

Lex Fridman (1:32:20.060)

how much of the core of the entire process of bringing an

Lex Fridman (1:32:25.980)

actual public transportation, autonomous transportation

Dmitri Dolgov (1:32:30.460)

service to a city is there in Phoenix that you understand

Lex Fridman (1:32:35.340)

enough to copy and paste into Boston or wherever? So we're

Dmitri Dolgov (1:32:39.660)

not quite there yet. We're not at a point where we're kind of

Lex Fridman (1:32:41.980)

massively copy and pasting all over the place. But Phoenix,

Lex Fridman (1:32:47.100)

what we did in Phoenix, and we very intentionally have chosen

Lex Fridman (1:32:50.940)

Phoenix as our first full deployment area, you know,

Dmitri Dolgov (1:32:56.620)

exactly for that reason to kind of tease the problem apart,

Lex Fridman (1:32:59.580)

look at each dimension and focus on the fundamentals of

Dmitri Dolgov (1:33:03.180)

complexity and de risking those dimensions, and then bringing

Lex Fridman (1:33:06.460)

the entire thing together to get all the way and force

Dmitri Dolgov (1:33:09.340)

ourselves to learn all those hard lessons on technology,

Lex Fridman (1:33:12.460)

hardware and software, on the evaluation deployment, on

Dmitri Dolgov (1:33:15.740)

operating a service, operating a business using actually

Lex Fridman (1:33:20.060)

serving our customers all the way so that we're fully

Dmitri Dolgov (1:33:22.860)

informed about the most difficult, most important

Lex Fridman (1:33:27.580)

challenges to get us to that next step of massive copy and

Dmitri Dolgov (1:33:31.180)

pasting as you said. And that's what we're doing right now.

Lex Fridman (1:33:38.860)

We're incorporating all those things that we learned into

Dmitri Dolgov (1:33:41.740)

that next system that then will allow us to kind of copy and

Lex Fridman (1:33:44.860)

paste all over the place and to massively scale to, you know,

Dmitri Dolgov (1:33:47.500)

more users and more locations. I mean, you know, just talk a

Lex Fridman (1:33:50.300)

little bit about, you know, what does that mean along those

Dmitri Dolgov (1:33:52.380)

different dimensions? So on the hardware side, for example,

Lex Fridman (1:33:55.020)

again, it's that switch from the fourth to the fifth

Dmitri Dolgov (1:33:57.980)

generation. And the fifth generation is designed to kind

Lex Fridman (1:34:00.380)

of have that property. Can you say what other cities you're

Dmitri Dolgov (1:34:04.540)

thinking about? Like, I'm thinking about, sorry, we're in

Lex Fridman (1:34:09.020)

San Francisco now. I thought I want to move to San Francisco,

Lex Fridman (1:34:12.380)

but I'm thinking about moving to Austin. I don't know why

Lex Fridman (1:34:16.540)

people are not being very nice about San Francisco currently,

Lex Fridman (1:34:19.580)

but maybe it's a small, maybe it's in vogue right now.

Lex Fridman (1:34:23.340)

But Austin seems, I visited there and it was, I was in a

Dmitri Dolgov (1:34:28.060)

Walmart. It's funny, these moments like turn your life.

Lex Fridman (1:34:32.860)

There's this very nice woman with kind eyes, just like stopped

Lex Fridman (1:34:38.860)

and said, he looks so handsome in that tie, honey, to me. This

Lex Fridman (1:34:44.460)

has never happened to me in my life, but just the sweetness of

Dmitri Dolgov (1:34:47.260)

this woman is something I've never experienced, certainly on

Lex Fridman (1:34:49.980)

the streets of Boston, but even in San Francisco where people

Dmitri Dolgov (1:34:53.100)

wouldn't, that's just not how they speak or think. I don't

Lex Fridman (1:34:57.100)

know. There's a warmth to Austin that love. And since

Dmitri Dolgov (1:35:00.700)

Waymo does have a little bit of a history there, is that a

Lex Fridman (1:35:04.060)

possibility? Is this your version of asking the question

Dmitri Dolgov (1:35:07.980)

of like, you know, Dimitri, I know you can't share your

Lex Fridman (1:35:09.980)

commercial and deployment roadmap, but I'm thinking about

Dmitri Dolgov (1:35:12.780)

moving to San Francisco, Austin, like, you know, blink twice if

Lex Fridman (1:35:16.300)

you think I should move to it. That's true. That's true. You

Dmitri Dolgov (1:35:19.900)

got me. You know, we've been testing all over the place. I

Lex Fridman (1:35:23.900)

think we've been testing more than 25 cities. We drive

Dmitri Dolgov (1:35:26.860)

in San Francisco. We drive in, you know, Michigan for snow.

Lex Fridman (1:35:31.740)

We are doing significant amount of testing in the Bay Area,

Dmitri Dolgov (1:35:34.220)

including San Francisco, which is not like, because we're

Lex Fridman (1:35:37.340)

talking about the very different thing, which is like a

Dmitri Dolgov (1:35:40.060)

full on large geographic area, public service. You can't share

Lex Fridman (1:35:46.380)

and you, okay. What about Moscow? When is that happening?

Dmitri Dolgov (1:35:54.140)

Take on Yandex. I'm not paying attention to those folks.

Lex Fridman (1:35:58.700)

They're doing, you know, there's a lot of fun. I mean,

Dmitri Dolgov (1:36:02.380)

maybe as a way of a question, you didn't speak to sort of like

Lex Fridman (1:36:10.540)

policy or like, is there tricky things with government and so

Dmitri Dolgov (1:36:15.020)

on? Like, is there other friction that you've

Lex Fridman (1:36:20.860)

encountered except sort of technological friction of

Dmitri Dolgov (1:36:25.260)

solving this very difficult problem? Is there other stuff

Lex Fridman (1:36:28.540)

that you have to overcome when deploying a public service in

Dmitri Dolgov (1:36:33.340)

a city? That's interesting. It's very important. So we

Lex Fridman (1:36:38.860)

put significant effort in creating those partnerships and

Dmitri Dolgov (1:36:44.540)

you know, those relationships with governments at all levels,

Lex Fridman (1:36:48.380)

local governments, municipalities, state level,

Dmitri Dolgov (1:36:50.860)

federal level. We've been engaged in very deep

Lex Fridman (1:36:53.900)

conversations from the earliest days of our projects.

Dmitri Dolgov (1:36:57.020)

Whenever at all of these levels, whenever we go

Lex Fridman (1:37:01.020)

to test or operate in a new area, we always lead

Dmitri Dolgov (1:37:07.500)

with a conversation with the local officials.

Lex Fridman (1:37:10.860)

But the result of that investment is that no,

Dmitri Dolgov (1:37:13.740)

it's not challenges we have to overcome, but it is very

Lex Fridman (1:37:16.780)

important that we continue to have this conversation.

Dmitri Dolgov (1:37:19.980)

Oh, yeah. I love politicians too. Okay, so Mr. Elon Musk said that

Lex Fridman (1:37:27.340)

LiDAR is a crutch. What are your thoughts?

Dmitri Dolgov (1:37:32.940)

I wouldn't characterize it exactly that way. I know I think LiDAR is

Lex Fridman (1:37:36.540)

very important. It is a key sensor that we use just like

Dmitri Dolgov (1:37:42.540)

other modalities, right? As we discussed, our cars use cameras, LiDAR

Lex Fridman (1:37:46.700)

and radars. They are all very important. They are

Dmitri Dolgov (1:37:52.700)

at the kind of the physical level. They are very different. They have very

Lex Fridman (1:37:57.900)

different, you know, physical characteristics.

Dmitri Dolgov (1:38:00.300)

Cameras are passive. LiDARs and radars are active.

Lex Fridman (1:38:03.100)

Use different wavelengths. So that means they complement each other

Dmitri Dolgov (1:38:07.420)

very nicely and together combined, they can be used to

Lex Fridman (1:38:14.700)

build a much safer and much more capable system.

Dmitri Dolgov (1:38:20.620)

So, you know, to me it's more of a question,

Lex Fridman (1:38:25.020)

you know, why the heck would you handicap yourself and not use one

Dmitri Dolgov (1:38:28.700)

or more of those sensing modalities when they, you know, undoubtedly just make your

Lex Fridman (1:38:32.860)

system more capable and safer. Now,

Dmitri Dolgov (1:38:39.100)

it, you know, what might make sense for one product or

Lex Fridman (1:38:45.180)

one business might not make sense for another one.

Lex Fridman (1:38:48.380)

So if you're talking about driver assist technologies, you make certain design

Lex Fridman (1:38:51.980)

decisions and you make certain trade offs and make different ones if you are

Dmitri Dolgov (1:38:55.260)

building a driver that you deploy in fully driverless

Lex Fridman (1:38:59.820)

vehicles. And, you know, in LiDAR specifically, when this question comes up,

Dmitri Dolgov (1:39:04.940)

I, you know, typically the criticisms that I hear or, you know, the

Lex Fridman (1:39:11.820)

counterpoints is that cost and aesthetics.

Lex Fridman (1:39:16.060)

And I don't find either of those, honestly, very compelling.

Lex Fridman (1:39:20.460)

So on the cost side, there's nothing fundamentally prohibitive

Dmitri Dolgov (1:39:24.380)

about, you know, the cost of LiDARs. You know, radars used to be very expensive

Lex Fridman (1:39:28.620)

before people started, you know, before people made certain advances in

Dmitri Dolgov (1:39:32.140)

technology and, you know, started to manufacture them at massive scale and

Lex Fridman (1:39:35.980)

deploy them in vehicles, right? You know, similar with LiDARs. And this is

Dmitri Dolgov (1:39:39.740)

where the LiDARs that we have on our cars, especially the fifth generation,

Lex Fridman (1:39:43.260)

you know, we've been able to make some pretty qualitative discontinuous

Dmitri Dolgov (1:39:48.220)

jumps in terms of the fundamental technology that allow us to

Lex Fridman (1:39:51.580)

manufacture those things at very significant scale and at a fraction

Dmitri Dolgov (1:39:56.380)

of the cost of both our previous generation

Lex Fridman (1:40:00.300)

as well as a fraction of the cost of, you know, what might be available

Dmitri Dolgov (1:40:03.980)

on the market, you know, off the shelf right now. And, you know, that improvement

Lex Fridman (1:40:07.100)

will continue. So I think, you know, cost is not a

Dmitri Dolgov (1:40:10.700)

real issue. Second one is, you know, aesthetics.

Lex Fridman (1:40:14.300)

You know, I don't think that's, you know, a real issue either.

Dmitri Dolgov (1:40:18.060)

Beauty is in the eye of the beholder. Yeah. You can make LiDAR sexy again.

Lex Fridman (1:40:22.860)

I think you're exactly right. I think it is sexy. Like, honestly, I think form

Dmitri Dolgov (1:40:25.740)

all of function. Well, okay. You know, I was actually, somebody brought this up to

Lex Fridman (1:40:30.060)

me. I mean, all forms of LiDAR, even

Dmitri Dolgov (1:40:34.940)

like the ones that are like big, you can make

Lex Fridman (1:40:37.580)

look, I mean, you can make look beautiful.

Dmitri Dolgov (1:40:40.700)

There's no sense in which you can't integrate it into design.

Lex Fridman (1:40:44.060)

Like, there's all kinds of awesome designs. I don't think

Dmitri Dolgov (1:40:47.820)

small and humble is beautiful. It could be

Lex Fridman (1:40:51.260)

like, you know, brutalism or like, it could be

Dmitri Dolgov (1:40:55.580)

like harsh corners. I mean, like I said, like hot rods. Like, I don't like, I don't

Lex Fridman (1:40:59.340)

necessarily like, like, oh man, I'm going to start so much

Dmitri Dolgov (1:41:02.700)

controversy with this. I don't like Porsches. Okay.

Lex Fridman (1:41:07.420)

The Porsche 911, like everyone says it's the most beautiful.

Dmitri Dolgov (1:41:10.700)

No, no. It's like, it's like a baby car. It doesn't make any sense.

Lex Fridman (1:41:15.340)

But everyone, it's beauty is in the eye of the beholder. You're already looking at

Dmitri Dolgov (1:41:18.940)

me like, what is this kid talking about? I'm happy to talk about. You're digging your

Lex Fridman (1:41:24.060)

own hole. The form and function and my take on the

Dmitri Dolgov (1:41:27.980)

beauty of the hardware that we put on our vehicles,

Lex Fridman (1:41:30.940)

you know, I will not comment on your Porsche monologue.

Dmitri Dolgov (1:41:34.700)

Okay. All right. So, but aesthetics, fine. But there's an underlying, like,

Lex Fridman (1:41:39.340)

philosophical question behind the kind of lighter question is

Dmitri Dolgov (1:41:43.900)

like, how much of the problem can be solved

Lex Fridman (1:41:48.060)

with computer vision, with machine learning?

Lex Fridman (1:41:51.660)

So I think without sort of disagreements and so on,

Lex Fridman (1:41:58.460)

it's nice to put it on the spectrum because Waymo is doing a lot of machine

Dmitri Dolgov (1:42:03.340)

learning as well. It's interesting to think how much of

Lex Fridman (1:42:06.460)

driving, if we look at five years, 10 years, 50 years down the road,

Lex Fridman (1:42:11.260)

what can be learned in almost more and more and more

Lex Fridman (1:42:15.340)

end to end way. If we look at what Tesla is doing

Dmitri Dolgov (1:42:19.820)

with, as a machine learning problem, they're doing a multitask learning

Lex Fridman (1:42:24.300)

thing where it's just, they break up driving into a bunch of learning tasks

Lex Fridman (1:42:27.820)

and they have one single neural network and they're just collecting huge amounts

Lex Fridman (1:42:30.540)

of data that's training that. I've recently hung out with George

Dmitri Dolgov (1:42:33.340)

Hotz. I don't know if you know George.

Lex Fridman (1:42:37.820)

I love him so much. He's just an entertaining human being.

Dmitri Dolgov (1:42:41.820)

We were off mic talking about Hunter S. Thompson. He's the Hunter S. Thompson

Lex Fridman (1:42:45.340)

of autonomous driving. Okay. So he, I didn't realize this with comma

Dmitri Dolgov (1:42:49.420)

AI, but they're like really trying to end to end.

Lex Fridman (1:42:53.180)

They're the machine, like looking at the machine learning problem, they're

Dmitri Dolgov (1:42:58.460)

really not doing multitask learning, but it's

Lex Fridman (1:43:01.580)

computing the drivable area as a machine learning task

Lex Fridman (1:43:05.980)

and hoping that like down the line, this level two system, this driver

Lex Fridman (1:43:11.500)

assistance will eventually lead to

Dmitri Dolgov (1:43:15.340)

allowing you to have a fully autonomous vehicle. Okay. There's an underlying

Lex Fridman (1:43:19.260)

deep philosophical question there, technical question

Dmitri Dolgov (1:43:22.540)

of how much of driving can be learned. So LiDAR is an effective tool today

Lex Fridman (1:43:29.420)

for actually deploying a successful service in Phoenix, right? That's safe,

Dmitri Dolgov (1:43:33.820)

that's reliable, et cetera, et cetera. But the question,

Lex Fridman (1:43:39.260)

and I'm not saying you can't do machine learning on LiDAR, but the question is

Dmitri Dolgov (1:43:43.100)

that like how much of driving can be learned eventually.

Lex Fridman (1:43:47.340)

Can we do fully autonomous? That's learned.

Dmitri Dolgov (1:43:49.980)

Yeah. You know, learning is all over the place

Lex Fridman (1:43:53.340)

and plays a key role in every part of our system.

Dmitri Dolgov (1:43:56.620)

As you said, I would, you know, decouple the sensing modalities

Lex Fridman (1:44:01.180)

from the, you know, ML and the software parts of it.

Dmitri Dolgov (1:44:05.180)

LiDAR, radar, cameras, like it's all machine learning.

Lex Fridman (1:44:09.740)

All of the object detection classification, of course, like that's

Dmitri Dolgov (1:44:12.220)

what, you know, these modern deep nets and count nets are very

Lex Fridman (1:44:15.100)

good at. You feed them raw data, massive amounts of raw data,

Lex Fridman (1:44:19.820)

and that's actually what our custom build LiDARs and radars are really good

Lex Fridman (1:44:23.900)

at. And radars, they don't just give you point

Dmitri Dolgov (1:44:25.500)

estimates of, you know, objects in space, they give you raw,

Lex Fridman (1:44:28.060)

like, physical observations. And then you take all of that raw information,

Dmitri Dolgov (1:44:31.660)

you know, there's colors of the pixels, whether it's, you know, LiDARs returns

Lex Fridman (1:44:34.780)

and some auxiliary information. It's not just distance,

Dmitri Dolgov (1:44:36.780)

right? And, you know, angle and distance is much richer information that you get

Lex Fridman (1:44:39.500)

from those returns, plus really rich information from the

Dmitri Dolgov (1:44:42.460)

radars. You fuse it all together and you feed it into those massive

Lex Fridman (1:44:45.820)

ML models that then, you know, lead to the best results in terms of, you

Dmitri Dolgov (1:44:51.340)

know, object detection, classification, state estimation.

Lex Fridman (1:44:55.820)

So there's a side to interop, but there is a fusion. I mean, that's something

Dmitri Dolgov (1:44:59.020)

that people didn't do for a very long time,

Lex Fridman (1:45:01.020)

which is like at the sensor fusion level, I guess,

Dmitri Dolgov (1:45:04.540)

like early on fusing the information together, whether

Lex Fridman (1:45:07.660)

so that the the sensory information that the vehicle receives from the different

Dmitri Dolgov (1:45:11.980)

modalities or even from different cameras is

Lex Fridman (1:45:15.180)

combined before it is fed into the machine learning models.

Dmitri Dolgov (1:45:19.020)

Yeah, so I think this is one of the trends you're seeing more of that you

Lex Fridman (1:45:21.660)

mentioned end to end. There's different interpretation of end to end. There is

Dmitri Dolgov (1:45:24.780)

kind of the purest interpretation of I'm going to

Lex Fridman (1:45:27.980)

like have one model that goes from raw sensor data to like,

Dmitri Dolgov (1:45:32.300)

you know, steering torque and, you know, gas breaks. That, you know,

Lex Fridman (1:45:35.100)

that's too much. I don't think that's the right way to do it.

Dmitri Dolgov (1:45:37.500)

There's more, you know, smaller versions of end to end

Lex Fridman (1:45:40.620)

where you're kind of doing more end to end learning or core training or

Dmitri Dolgov (1:45:45.500)

depropagation of kind of signals back and forth across

Lex Fridman (1:45:48.700)

the different stages of your system. There's, you know, really good ways it

Dmitri Dolgov (1:45:51.900)

gets into some fairly complex design choices where on one

Lex Fridman (1:45:55.180)

hand you want modularity and decomposability,

Dmitri Dolgov (1:45:57.980)

decomposability of your system. But on the other hand,

Lex Fridman (1:46:01.580)

you don't want to create interfaces that are too narrow or too brittle

Dmitri Dolgov (1:46:05.100)

to engineered where you're giving up on the generality of the solution or you're

Lex Fridman (1:46:08.380)

unable to properly propagate signal, you know, reach signal forward and losses

Dmitri Dolgov (1:46:12.940)

and, you know, back so you can optimize the whole system jointly.

Lex Fridman (1:46:17.500)

So I would decouple and I guess what you're seeing in terms of the fusion

Dmitri Dolgov (1:46:21.180)

of the sensing data from different modalities as well as kind of fusion

Lex Fridman (1:46:25.580)

at in the temporal level going more from, you know, frame by frame

Dmitri Dolgov (1:46:30.060)

where, you know, you would have one net that would do frame by frame detection

Lex Fridman (1:46:32.780)

and camera and then, you know, something that does frame by frame and

Dmitri Dolgov (1:46:35.500)

lighter and then radar and then you fuse it, you know, in a weaker engineered way

Lex Fridman (1:46:39.260)

later. Like the field over the last, you know,

Dmitri Dolgov (1:46:41.260)

decade has been evolving in more kind of joint fusion, more end to end models that

Lex Fridman (1:46:45.260)

are, you know, solving some of these tasks, you know, jointly and there's

Dmitri Dolgov (1:46:48.060)

tremendous power in that and, you know, that's the

Lex Fridman (1:46:50.860)

progression that kind of our technology, our stack has been on as well.

Dmitri Dolgov (1:46:54.700)

Now to your, you know, that so I would decouple the kind of sensing and how

Lex Fridman (1:46:57.980)

that information is fused from the role of ML and the entire stack.

Dmitri Dolgov (1:47:01.340)

And, you know, I guess it's, there's trade offs and, you know, modularity and

Lex Fridman (1:47:06.460)

how do you inject inductive bias into your system?

Dmitri Dolgov (1:47:11.260)

All right, this is, there's tremendous power

Lex Fridman (1:47:15.180)

in being able to do that. So, you know, we have, there's no

Dmitri Dolgov (1:47:19.660)

part of our system that is not heavily, that does not heavily, you know, leverage

Lex Fridman (1:47:25.180)

data driven development or state of the art ML.

Lex Fridman (1:47:29.580)

But there's mapping, there's a simulator, there's perception, you know, object

Lex Fridman (1:47:33.580)

level, you know, perception, whether it's

Dmitri Dolgov (1:47:34.940)

semantic understanding, prediction, decision making, you know, so forth and

Lex Fridman (1:47:38.220)

so on.

Dmitri Dolgov (1:47:42.060)

It's, you know, of course, object detection and classification, like you're

Lex Fridman (1:47:45.100)

finding pedestrians and cars and cyclists and, you know, cones and signs

Lex Fridman (1:47:48.460)

and vegetation and being very good at estimating

Lex Fridman (1:47:51.740)

kind of detection, classification, and state estimation. There's just stable

Dmitri Dolgov (1:47:54.460)

stakes, like that's step zero of this whole stack. You can be

Lex Fridman (1:47:57.900)

incredibly good at that, whether you use cameras or light as a

Dmitri Dolgov (1:48:00.700)

radar, but that's just, you know, that's stable stakes, that's just step zero.

Lex Fridman (1:48:03.660)

Beyond that, you get into the really interesting challenges of semantic

Dmitri Dolgov (1:48:06.380)

understanding at the perception level, you get into scene level reasoning, you

Lex Fridman (1:48:10.140)

get into very deep problems that have to do with prediction and joint

Dmitri Dolgov (1:48:13.900)

prediction and interaction, so the interaction

Lex Fridman (1:48:16.140)

between all the actors in the environment, pedestrians, cyclists, other

Dmitri Dolgov (1:48:19.260)

cars, and you get into decision making, right? So, how do you build a lot of

Lex Fridman (1:48:22.300)

systems? So, we leverage ML very heavily in all of

Dmitri Dolgov (1:48:26.300)

these components. I do believe that the best results you

Lex Fridman (1:48:30.140)

achieve by kind of using a hybrid approach and

Dmitri Dolgov (1:48:33.340)

having different types of ML, having

Lex Fridman (1:48:38.140)

different models with different degrees of inductive bias

Dmitri Dolgov (1:48:41.580)

that you can have, and combining kind of model,

Lex Fridman (1:48:45.260)

you know, free approaches with some model based approaches and some

Dmitri Dolgov (1:48:49.180)

rule based, physics based systems. So, you know, one example I can give

Lex Fridman (1:48:54.380)

you is traffic lights. There's a problem of the detection of

Dmitri Dolgov (1:48:58.940)

traffic light state, and obviously that's a great problem for, you know, computer

Lex Fridman (1:49:02.700)

vision confidence, or, you know, that's their bread and

Dmitri Dolgov (1:49:05.260)

butter, right? That's how you build that. But then the

Lex Fridman (1:49:08.220)

interpretation of, you know, of a traffic light, that you're

Dmitri Dolgov (1:49:11.740)

gonna need to learn that, right? You don't need to build some,

Lex Fridman (1:49:15.820)

you know, complex ML model that, you know, infers

Dmitri Dolgov (1:49:18.940)

with some, you know, precision and recall that red means stop.

Lex Fridman (1:49:22.540)

Like, it's a very clear engineered signal

Dmitri Dolgov (1:49:25.500)

with very clear semantics, right? So you want to induce that bias, like how you

Lex Fridman (1:49:29.500)

induce that bias, and that whether, you know, it's a

Dmitri Dolgov (1:49:31.740)

constraint or a cost, you know, function in your stack, but like

Lex Fridman (1:49:36.460)

it is important to be able to inject that, like, clear semantic

Dmitri Dolgov (1:49:40.860)

signal into your stack. And, you know, that's what we do.

Lex Fridman (1:49:44.220)

And, but then the question of, like, and that's when you

Dmitri Dolgov (1:49:47.340)

apply it to yourself, when you are making decisions whether you want to stop

Lex Fridman (1:49:50.860)

for a red light, you know, or not.

Lex Fridman (1:49:54.540)

But if you think about how other people treat traffic lights,

Lex Fridman (1:49:57.820)

we're back to the ML version of that. You know they're supposed to stop

Dmitri Dolgov (1:50:01.260)

for a red light, but that doesn't mean they will.

Lex Fridman (1:50:02.860)

So then you're back in the, like, very heavy

Dmitri Dolgov (1:50:07.820)

ML domain where you're picking up on, like, very subtle cues about,

Lex Fridman (1:50:11.420)

you know, they have to do with the behavior of objects, pedestrians, cyclists,

Dmitri Dolgov (1:50:15.260)

cars, and the whole, you know, entire configuration of the scene

Lex Fridman (1:50:19.420)

that allow you to make accurate predictions on whether they will, in

Dmitri Dolgov (1:50:22.220)

fact, stop or run a red light. So it sounds like already for Waymo,

Lex Fridman (1:50:27.020)

like, machine learning is a huge part of the stack.

Lex Fridman (1:50:29.820)

So it's a huge part of, like, not just, so obviously the first, the level

Lex Fridman (1:50:36.300)

zero, or whatever you said, which is, like,

Dmitri Dolgov (1:50:38.860)

just the object detection of things that, you know, with no other machine learning

Lex Fridman (1:50:42.380)

can do, but also starting to do prediction behavior and so on to

Dmitri Dolgov (1:50:46.380)

model the, what other, what the other parties in the

Lex Fridman (1:50:49.660)

scene, entities in the scene are going to do.

Lex Fridman (1:50:51.580)

So machine learning is more and more playing a role in that

Lex Fridman (1:50:55.260)

as well. Of course. Oh, absolutely. I think we've been

Dmitri Dolgov (1:50:59.020)

going back to the, you know, earliest days, like, you know, DARPA,

Lex Fridman (1:51:02.060)

the DARPA Grand Challenge, our team was leveraging, you know, machine

Dmitri Dolgov (1:51:05.820)

learning. It was, like, pre, you know, ImageNet, and it was a very

Lex Fridman (1:51:08.540)

different type of ML, but, and I think actually it was before

Dmitri Dolgov (1:51:11.660)

my time, but the Stanford team during the Grand Challenge had a very

Lex Fridman (1:51:15.340)

interesting machine learned system that would, you know, use

Dmitri Dolgov (1:51:18.940)

LiDAR and camera. We've been driving in the

Lex Fridman (1:51:21.340)

desert, and it, we had built the model where it would kind of

Dmitri Dolgov (1:51:26.940)

extend the range of free space reasoning. We get a

Lex Fridman (1:51:29.900)

clear signal from LiDAR, and then it had a model that said, hey, like,

Dmitri Dolgov (1:51:33.020)

this stuff on camera kind of sort of looks like this stuff in LiDAR, and I

Lex Fridman (1:51:35.900)

know this stuff that I'm seeing in LiDAR, I'm very confident it's free space,

Lex Fridman (1:51:38.860)

so let me extend that free space zone into the camera range that would allow

Lex Fridman (1:51:43.420)

the vehicle to drive faster. And then we've been building on top of

Dmitri Dolgov (1:51:45.980)

that and kind of staying and pushing the state of the art in ML,

Lex Fridman (1:51:48.860)

in all kinds of different ML over the years. And in fact,

Dmitri Dolgov (1:51:52.620)

from the early days, I think, you know, 2010 is probably the year

Lex Fridman (1:51:56.940)

where Google, maybe 2011 probably, got pretty heavily involved in

Dmitri Dolgov (1:52:03.500)

machine learning, kind of deep nuts, and at that time it was probably the only

Lex Fridman (1:52:07.660)

company that was very heavily investing in kind of state of the art ML and

Dmitri Dolgov (1:52:11.980)

self driving cars. And they go hand in hand.

Lex Fridman (1:52:16.220)

And we've been on that journey ever since. We're doing, pushing

Dmitri Dolgov (1:52:19.980)

a lot of these areas in terms of research at Waymo, and we

Lex Fridman (1:52:24.060)

collaborate very heavily with the researchers in

Dmitri Dolgov (1:52:26.620)

Alphabet, and all kinds of ML, supervised ML,

Lex Fridman (1:52:30.060)

unsupervised ML, published some

Dmitri Dolgov (1:52:34.380)

interesting research papers in the space,

Lex Fridman (1:52:37.900)

especially recently. It's just a super active learning as well.

Dmitri Dolgov (1:52:41.180)

Yeah, so super, super active. Of course, there's, you know, kind of the more

Lex Fridman (1:52:45.260)

mature stuff, like, you know, ConvNets for, you know, object detection.

Lex Fridman (1:52:48.940)

But there's some really interesting, really active work that's happening

Lex Fridman (1:52:52.860)

in kind of more, you know, in bigger models and, you know,

Dmitri Dolgov (1:52:58.300)

models that have more structure to them,

Dmitri Dolgov (1:53:02.540)

you know, not just, you know, large bitmaps and reason about temporal sequences.

Lex Fridman (1:53:06.860)

And some of the interesting breakthroughs that you've, you know, we've seen

Lex Fridman (1:53:10.700)

in language models, right? You know, transformers,

Dmitri Dolgov (1:53:14.140)

you know, GPT3 inference. There's some really interesting applications of some

Lex Fridman (1:53:19.100)

of the core breakthroughs to those problems

Dmitri Dolgov (1:53:21.260)

of, you know, behavior prediction, as well as, you know, decision making and

Lex Fridman (1:53:24.540)

planning, right? You can think about it, kind of the the behavior,

Dmitri Dolgov (1:53:27.900)

how, you know, the path, the trajectories, the how people drive.

Lex Fridman (1:53:31.500)

They have kind of a share, a lot of the fundamental structure,

Dmitri Dolgov (1:53:34.620)

you know, this problem. There's, you know, sequential,

Lex Fridman (1:53:38.220)

you know, nature. There's a lot of structure in this representation.

Dmitri Dolgov (1:53:41.900)

There is a strong locality, kind of like in sentences, you know, words that follow

Lex Fridman (1:53:45.900)

each other. They're strongly connected, but there's

Dmitri Dolgov (1:53:48.140)

also kind of larger context that doesn't have that locality, and you also see that

Lex Fridman (1:53:51.580)

in driving, right? What, you know, is happening in the scene

Dmitri Dolgov (1:53:53.740)

as a whole has very strong implications on,

Lex Fridman (1:53:57.020)

you know, the kind of the next step in that sequence where

Dmitri Dolgov (1:54:00.940)

whether you're, you know, predicting what other people are going to do, whether

Lex Fridman (1:54:03.980)

you're making your own decisions, or whether in the simulator you're

Dmitri Dolgov (1:54:07.020)

building generative models of, you know, humans walking, cyclists

Lex Fridman (1:54:10.620)

riding, and other cars driving. That's all really fascinating, like how

Dmitri Dolgov (1:54:14.220)

it's fascinating to think that transformer models and all this,

Lex Fridman (1:54:17.340)

all the breakthroughs in language and NLP that might be applicable to like

Dmitri Dolgov (1:54:21.900)

driving at the higher level, at the behavioral level, that's kind of

Lex Fridman (1:54:24.620)

fascinating. Let me ask about pesky little creatures

Dmitri Dolgov (1:54:27.900)

called pedestrians and cyclists. They seem, so humans are a problem. If we

Lex Fridman (1:54:32.620)

can get rid of them, I would. But unfortunately, they're all sort of

Dmitri Dolgov (1:54:36.940)

a source of joy and love and beauty, so let's keep them around.

Lex Fridman (1:54:39.980)

They're also our customers. For your perspective, yes, yes,

Dmitri Dolgov (1:54:43.340)

for sure. They're a source of money, very good.

Lex Fridman (1:54:46.620)

But I don't even know where I was going. Oh yes,

Dmitri Dolgov (1:54:52.300)

pedestrians and cyclists, you know,

Lex Fridman (1:54:57.260)

they're a fascinating injection into the system of

Dmitri Dolgov (1:55:00.620)

uncertainty of like a game theoretic dance of what to do. And also

Lex Fridman (1:55:09.020)

they have perceptions of their own, and they can tweet

Dmitri Dolgov (1:55:13.420)

about your product, so you don't want to run them over

Lex Fridman (1:55:17.500)

from that perspective. I mean, I don't know, I'm joking a lot, but

Dmitri Dolgov (1:55:21.580)

I think in seriousness, like, you know, pedestrians are a complicated

Lex Fridman (1:55:27.340)

computer vision problem, a complicated behavioral problem. Is there something

Dmitri Dolgov (1:55:31.340)

interesting you could say about what you've learned

Lex Fridman (1:55:34.140)

from a machine learning perspective, from also an autonomous vehicle,

Lex Fridman (1:55:38.380)

and a product perspective about just interacting with the humans in this

Lex Fridman (1:55:42.140)

world? Yeah, just to state on record, we care

Dmitri Dolgov (1:55:45.180)

deeply about the safety of pedestrians, you know, even the ones that don't have

Lex Fridman (1:55:48.380)

Twitter accounts. Thank you. All right, cool.

Dmitri Dolgov (1:55:52.940)

Not me. But yes, I'm glad, I'm glad somebody does.

Lex Fridman (1:55:57.500)

Okay. But you know, in all seriousness, safety

Dmitri Dolgov (1:56:01.340)

of vulnerable road users, pedestrians or cyclists, is one of our

Lex Fridman (1:56:07.260)

highest priorities. We do a tremendous amount of testing

Lex Fridman (1:56:12.220)

and validation, and put a very significant emphasis

Lex Fridman (1:56:16.220)

on, you know, the capabilities of our systems that have to do with safety

Dmitri Dolgov (1:56:20.540)

around those unprotected vulnerable road users.

Dmitri Dolgov (1:56:23.820)

You know, cars, just, you know, discussed earlier in Phoenix, we have completely

Dmitri Dolgov (1:56:27.660)

empty cars, completely driverless cars, you know, driving in this very large area,

Lex Fridman (1:56:31.740)

and you know, some people use them to, you know, go to school, so they'll drive

Dmitri Dolgov (1:56:35.260)

through school zones, right? So, kids are kind of the very special

Lex Fridman (1:56:39.660)

class of those vulnerable user road users, right? You want to be,

Dmitri Dolgov (1:56:42.220)

you know, super, super safe, and super, super cautious around those. So, we take

Lex Fridman (1:56:45.980)

it very, very, very seriously. And you know, what does it take to

Dmitri Dolgov (1:56:50.460)

be good at it? You know,

Lex Fridman (1:56:55.180)

an incredible amount of performance across your whole stack. You know,

Dmitri Dolgov (1:57:02.060)

starts with hardware, and again, you want to use all

Lex Fridman (1:57:05.820)

sensing modalities available to you. Imagine driving on a residential road

Dmitri Dolgov (1:57:09.500)

at night, and kind of making a turn, and you don't have, you know, headlights

Lex Fridman (1:57:13.100)

covering some part of the space, and like, you know, a kid might

Dmitri Dolgov (1:57:16.220)

run out. And you know, lighters are amazing at that. They

Dmitri Dolgov (1:57:20.620)

see just as well in complete darkness as they do during the day, right? So, just

Dmitri Dolgov (1:57:24.300)

again, it gives you that extra,

Dmitri Dolgov (1:57:27.900)

you know, margin in terms of, you know, capability, and performance, and safety,

Lex Fridman (1:57:32.540)

and quality. And in fact, we oftentimes, in these

Lex Fridman (1:57:35.420)

kinds of situations, we have our system detect something,

Dmitri Dolgov (1:57:38.460)

in some cases even earlier than our trained operators in the car might do,

Lex Fridman (1:57:42.140)

right? Especially, you know, in conditions like, you know, very dark nights.

Dmitri Dolgov (1:57:46.620)

So, starts with sensing, then, you know, perception

Lex Fridman (1:57:50.380)

has to be incredibly good. And you have to be very, very good

Dmitri Dolgov (1:57:54.300)

at kind of detecting pedestrians in all kinds of situations, and all kinds

Lex Fridman (1:58:00.780)

of environments, including, you know, people in weird poses,

Dmitri Dolgov (1:58:03.580)

people kind of running around, and you know, being partially occluded.

Lex Fridman (1:58:09.900)

So, you know, that's step number one, right?

Dmitri Dolgov (1:58:13.180)

Then, you have to have in very high accuracy,

Lex Fridman (1:58:17.580)

and very low latency, in terms of your reactions

Dmitri Dolgov (1:58:21.180)

to, you know, what, you know, these actors might do, right? And we've put a

Lex Fridman (1:58:27.500)

tremendous amount of engineering, and tremendous amount of validation, in to

Dmitri Dolgov (1:58:30.780)

make sure our system performs properly. And, you know, oftentimes, it

Lex Fridman (1:58:35.020)

does require a very strong reaction to do the safe thing. And, you know, we

Dmitri Dolgov (1:58:38.140)

actually see a lot of cases like that. That's the long tail of really rare,

Lex Fridman (1:58:41.820)

you know, really, you know, crazy events that contribute to the safety

Dmitri Dolgov (1:58:48.620)

around pedestrians. Like, one example that comes to mind, that we actually

Lex Fridman (1:58:52.300)

happened in Phoenix, where we were driving

Dmitri Dolgov (1:58:56.940)

along, and I think it was a 45 mile per hour road, so you have pretty high speed

Lex Fridman (1:59:00.540)

traffic, and there was a sidewalk next to it, and

Dmitri Dolgov (1:59:03.420)

there was a cyclist on the sidewalk. And as we were in the right lane,

Lex Fridman (1:59:09.100)

right next to the side, so it was a multi lane road, so as we got close

Dmitri Dolgov (1:59:13.260)

to the cyclist on the sidewalk, it was a woman, you know, she tripped and fell.

Lex Fridman (1:59:17.180)

Just, you know, fell right into the path of our vehicle, right?

Lex Fridman (1:59:20.540)

And our, you know, car, you know, this was actually with a

Lex Fridman (1:59:25.820)

test driver, our test drivers, did exactly the right thing.

Dmitri Dolgov (1:59:29.820)

They kind of reacted, and came to stop. It requires both very strong steering,

Lex Fridman (1:59:33.100)

and, you know, strong application of the brake. And then we simulated what our

Dmitri Dolgov (1:59:37.020)

system would have done in that situation, and it did, you know,

Lex Fridman (1:59:39.260)

exactly the same thing. And that speaks to, you know, all of

Dmitri Dolgov (1:59:43.180)

those components of really good state estimation and

Lex Fridman (1:59:46.620)

tracking. And, like, imagine, you know, a person

Dmitri Dolgov (1:59:49.020)

on a bike, and they're falling over, and they're doing that right in front of you,

Lex Fridman (1:59:52.140)

right? So you have to be really, like, things are changing. The appearance of

Dmitri Dolgov (1:59:54.300)

that whole thing is changing, right? And a person goes one way, they're falling on

Lex Fridman (1:59:57.820)

the road, they're, you know, being flat on the ground in front of

Dmitri Dolgov (20:06.380)

the way that part of the mission work is, uh, you in a parking lot, you

Lex Fridman (20:12.180)

would get from DARPA an outline of the map.

Dmitri Dolgov (20:15.700)

You basically get this, you know, giant polygon that defined the

Lex Fridman (20:18.540)

perimeter of the parking lot, uh, and there would be an entrance and, you

Dmitri Dolgov (20:21.700)

know, so maybe multiple entrances or access to it, and then you would get a

Lex Fridman (20:25.300)

goal, uh, within that open space, uh, X, Y, you know, heading where the car had

Dmitri Dolgov (20:32.180)

to park and had no information about the optical, so obstacles that the car might

Lex Fridman (20:36.380)

encounter there.

Lex Fridman (20:36.860)

So it had to navigate a kind of completely free space, uh, from the

Lex Fridman (20:40.740)

entrance to the parking lot into that parking space.

Lex Fridman (20:43.740)

And then, uh, once parked there, it had to, uh, exit the parking lot, you know,

Lex Fridman (20:50.100)

while of course, I'm counting and reasoning about all the obstacles that

Dmitri Dolgov (20:53.060)

it encounters in real time.

Lex Fridman (20:54.860)

So, uh, Our interpretation, or at least my interpretation of the rules was that

Dmitri Dolgov (21:00.940)

you had to reverse out of the parking spot.

Lex Fridman (21:03.420)

And that's what our cars did.

Dmitri Dolgov (21:04.900)

Even if there's no obstacle in front, that's not what CMU's car did.

Lex Fridman (21:08.620)

And it just kind of drove right through.

Lex Fridman (21:10.620)

So there's still a debate.

Lex Fridman (21:12.260)

And of course, you know, as you stop and then reverse out and go out the

Dmitri Dolgov (21:14.860)

different way that costs you some time.

Lex Fridman (21:16.580)

And so there's still a debate whether, you know, it was my poor implementation

Dmitri Dolgov (21:20.300)

that cost us extra time or whether it was, you know, CMU, uh, violating an

Lex Fridman (21:26.100)

important rule of the competition.

Dmitri Dolgov (21:27.380)

And, you know, I have my own, uh, opinion here in terms of other bugs.

Lex Fridman (21:30.700)

And like, uh, I, I have to apologize to Mike Montemarila, uh, for sharing this

Dmitri Dolgov (21:34.380)

on air, but it is actually, uh, one of the more memorable ones.

Lex Fridman (21:38.180)

Uh, and it's something that's kind of become a bit of, uh, a metaphor and

Dmitri Dolgov (21:42.940)

a label in the industry, uh, since then, I think, you know, at least in some

Lex Fridman (21:46.100)

circles, it's called the victory circle or victory lap.

Dmitri Dolgov (21:49.020)

Um, and, uh, uh, our cars did that.

Lex Fridman (21:53.060)

So in one of the missions in the urban challenge, in one of the courses, uh,

Dmitri Dolgov (21:57.580)

there was this big oval, right by the start and finish of the race.

Lex Fridman (22:02.020)

So the ARPA had a lot of the missions would finish kind of in that same location.

Dmitri Dolgov (22:05.620)

Uh, and it was pretty cool because you could see the cars come by, you know,

Lex Fridman (22:08.620)

kind of finished that part leg of the trip, that leg of the mission, and then,

Dmitri Dolgov (22:11.780)

you know, go on and finish the rest of it.

Lex Fridman (22:15.220)

Uh, and other vehicles would, you know, come hit their waypoint, uh, and, you

Dmitri Dolgov (22:22.260)

know, exit the oval and off they would go.

Lex Fridman (22:24.340)

Our car on the hand, which hit the checkpoint, and then it would do an extra

Dmitri Dolgov (22:28.060)

lap around the oval and only then, you know, uh, leave and go on its merry way.

Lex Fridman (22:31.620)

So over the course of the full day, it accumulated, uh, uh,

Dmitri Dolgov (22:34.620)

some extra time and the problem was that we had a bug where it wouldn't, you know,

Lex Fridman (22:38.100)

start reasoning about the next waypoint and plan a route to get to that next

Dmitri Dolgov (22:41.380)

point until it hit a previous one.

Lex Fridman (22:42.820)

And in that particular case, by the time you hit the, that, that one, it was too

Dmitri Dolgov (22:46.180)

late for us to consider the next one and kind of make a lane change.

Lex Fridman (22:49.140)

So at every time we would do like an extra lap.

Dmitri Dolgov (22:50.940)

So, you know, and that's the Stanford victory lap.

Lex Fridman (22:55.060)

The victory lap.

Dmitri Dolgov (22:57.100)

Oh, that's there's, I feel like there's something philosophically

Lex Fridman (22:59.580)

profound in there somehow, but, uh, I mean, ultimately everybody is

Dmitri Dolgov (23:03.620)

a winner in that kind of competition.

Lex Fridman (23:06.140)

And it led to sort of famously to the creation of, um, Google self driving

Dmitri Dolgov (23:13.100)

car project and now Waymo.

Lex Fridman (23:15.740)

So can we, uh, give an overview of how is Waymo born?

Lex Fridman (23:20.340)

How's the Google self driving car project born?

Lex Fridman (23:23.180)

What's the, what is the mission?

Lex Fridman (23:24.780)

What is the hope?

Lex Fridman (23:26.300)

What is it is the engineering kind of, uh, set of milestones that

Dmitri Dolgov (23:32.460)

it seeks to accomplish, there's a lot of questions in there.

Lex Fridman (23:35.780)

Uh, yeah, uh, I don't know, kind of the DARPA urban challenge and the DARPA

Lex Fridman (23:40.060)

and previous DARPA grand challenges, uh, kind of led, I think to a very large

Lex Fridman (23:44.420)

degree to that next step and then, you know, Larry and Sergey, um, uh, Larry

Dmitri Dolgov (23:48.100)

Page and Sergey Brin, uh, uh, Google founders course, uh, I saw that

Lex Fridman (23:52.180)

competition and believed in the technology.

Dmitri Dolgov (23:54.940)

So, you know, the Google self driving car project was born, you know, at that time.

Lex Fridman (23:59.900)

And we started in 2009, it was a pretty small group of us, about a dozen people,

Dmitri Dolgov (24:04.820)

uh, who came together, uh, to, to work on this project at Google.

Lex Fridman (24:09.620)

At that time we saw an incredible early result in the DARPA urban challenge.

Dmitri Dolgov (24:18.140)

I think we're all incredibly excited, uh, about where we got to and we believed

Lex Fridman (24:23.980)

in the future of the technology, but we still had a very, you know,

Dmitri Dolgov (24:27.500)

very, you know, rudimentary understanding of the problem space.

Lex Fridman (24:31.660)

So the first goal of this project in 2009 was to really better

Dmitri Dolgov (24:37.660)

understand what we're up against.

Dmitri Dolgov (24:39.620)

Uh, and, you know, with that goal in mind, when we started the project, we created a

Dmitri Dolgov (24:44.340)

few milestones for ourselves, uh, that.

Lex Fridman (24:48.300)

Maximized learnings.

Dmitri Dolgov (24:49.700)

Well, the two milestones were, you know, uh, one was to drive a hundred thousand

Lex Fridman (24:54.300)

miles in autonomous mode, which was at that time, you know, orders of magnitude

Dmitri Dolgov (24:57.940)

that, uh, more than anybody has ever done.

Lex Fridman (25:01.100)

And the second milestone was to drive 10 routes, uh, each one was a hundred miles

Dmitri Dolgov (25:07.060)

long, uh, and there were specifically chosen to become extra spicy and extra

Lex Fridman (25:12.700)

complicated and sample the full complexity of the, that, that, uh, domain.

Dmitri Dolgov (25:18.460)

Um, uh, and you had to drive each one from beginning to end with no intervention,

Lex Fridman (25:24.140)

no human intervention.

Lex Fridman (25:24.900)

So you would get to the beginning of the course, uh, you would press the button

Lex Fridman (25:28.460)

that would engage in autonomy and you had to go for a hundred miles, you know,

Dmitri Dolgov (25:32.900)

beginning to end, uh, with no interventions.

Lex Fridman (25:35.220)

Um, and it sampled again, the full complexity of driving conditions.

Dmitri Dolgov (25:40.460)

Some, uh, were on freeways.

Lex Fridman (25:42.940)

We had one route that went all through all the freeways and all

Dmitri Dolgov (25:45.180)

the bridges in the Bay area.

Lex Fridman (25:46.820)

You know, we had, uh, some that went around Lake Tahoe and kind of mountains,

Dmitri Dolgov (25:50.540)

uh, roads.

Lex Fridman (25:52.060)

We had some that drove through dense urban, um, environments like in downtown

Dmitri Dolgov (25:56.900)

Palo Alto and through San Francisco.

Lex Fridman (25:59.460)

So it was incredibly, uh, interesting, uh, to work on.

Lex Fridman (26:04.900)

And it, uh, it took us just under two years, uh, about a year and a half,

Lex Fridman (26:10.940)

a little bit more to finish both of these milestones.

Lex Fridman (26:14.180)

And in that process, uh, you know, it was an incredible amount of fun,

Lex Fridman (26:20.100)

probably the most fun I had in my professional career.

Lex Fridman (26:22.780)

And you're just learning so much.

Lex Fridman (26:24.700)

You are, you know, the goal here is to learn and prototype.

Lex Fridman (26:26.820)

You're not yet starting to build a production system, right?

Lex Fridman (26:29.180)

So you just, you were, you know, this is when you're kind of working 24 seven

Lex Fridman (26:33.380)

and you're hacking things together.

Lex Fridman (26:34.740)

And you also don't know how hard this is.

Dmitri Dolgov (26:37.620)

I mean, that's the point.

Lex Fridman (26:38.820)

Like, so, I mean, that's an ambitious, if I put myself in that mindset, even

Dmitri Dolgov (26:42.780)

still, that's a really ambitious set of goals.

Lex Fridman (26:46.660)

Like just those two picking, picking 10 different, difficult, spicy challenges.

Lex Fridman (26:56.580)

And then having zero interventions.

Lex Fridman (26:59.460)

So like not saying gradually we're going to like, you know, over a period of 10

Dmitri Dolgov (27:05.940)

years, we're going to have a bunch of routes and gradually reduce the number

Lex Fridman (27:09.580)

of interventions, you know, that literally says like, by as soon as

Dmitri Dolgov (27:13.980)

possible, we want to have zero and on hard roads.

Lex Fridman (27:17.980)

So like, to me, if I was facing that, it's unclear that whether that takes

Dmitri Dolgov (27:23.180)

two years or whether that takes 20 years.

Lex Fridman (27:26.420)

I mean, it took us under two.

Dmitri Dolgov (27:27.820)

I guess that that speaks to a really big difference between doing something

Lex Fridman (27:32.980)

once and having a prototype where you are going after, you know, learning

Dmitri Dolgov (27:37.820)

about the problem versus how you go about engineering a product that, you

Lex Fridman (27:42.780)

know, where you look at, you know, you do properly do evaluation, you look

Dmitri Dolgov (27:47.180)

at metrics, you drive down and you're confident that you can do that.

Lex Fridman (27:50.380)

And I guess that's the, you know, why it took a dozen people, you know, 16

Dmitri Dolgov (27:55.820)

months or a little bit more than that back in 2009 and 2010 with the

Lex Fridman (28:00.780)

technology of, you know, the more than a decade ago that amount of time to

Dmitri Dolgov (28:05.420)

achieve that milestone of, you know, 10 routes, a hundred miles each and no

Lex Fridman (28:10.220)

interventions, and, you know, it took us a little bit longer to get to, you

Dmitri Dolgov (28:17.340)

know, a full driverless product that customers use.

Lex Fridman (28:20.380)

That's another really important moment.

Dmitri Dolgov (28:21.740)

Is there some memories of technical lessons or just one, like, what did you

Lex Fridman (28:29.580)

learn about the problem of driving from that experience?

Dmitri Dolgov (28:32.220)

I mean, we can, we can now talk about like what you learned from modern day

Lex Fridman (28:36.540)

Waymo, but I feel like you may have learned some profound things in those

Dmitri Dolgov (28:41.420)

early days, even more so because it feels like what Waymo is now is to trying

Lex Fridman (28:47.580)

to, you know, how to do scale, how to make sure you create a product, how to

Dmitri Dolgov (28:51.020)

make sure it's like safety and all those things, which is all fascinating

Lex Fridman (28:54.140)

challenges, but like you were facing the more fundamental philosophical

Dmitri Dolgov (28:59.500)

problem of driving in those early days.

Lex Fridman (29:02.540)

Like what the hell is driving as an autonomous, or maybe I'm again

Dmitri Dolgov (29:07.820)

romanticizing it, but is it, is there, is there some valuable lessons you

Lex Fridman (29:14.540)

picked up over there at those two years?

Dmitri Dolgov (29:18.060)

A ton.

Lex Fridman (29:19.020)

The most important one is probably that we believe that it's doable and we've

Dmitri Dolgov (29:25.500)

gotten far enough into the problem that, you know, we had a, I think only a

Lex Fridman (29:31.500)

glimpse of the true complexity of the, that the domain, you know, it's a

Dmitri Dolgov (29:37.020)

little bit like, you know, climbing a mountain where you kind of, you know,

Lex Fridman (29:39.260)

see the next peak and you think that's kind of the summit, but then you get

Dmitri Dolgov (29:42.140)

to that and you kind of see that, that this is just the start of the journey.

Lex Fridman (29:46.140)

But we've tried, we've sampled enough of the problem space and we've made

Dmitri Dolgov (29:50.620)

enough rapid success, even, you know, with technology of 2009, 2010, that

Lex Fridman (29:57.020)

it gave us confidence to then, you know, pursue this as a real product.

Dmitri Dolgov (2:00:00.380)

you. You know, the bike goes flying the other direction.

Lex Fridman (2:00:03.340)

Like, the two objects that used to be one, they're now, you know,

Dmitri Dolgov (2:00:06.060)

are splitting apart, and the car has to, like, detect all of that.

Dmitri Dolgov (2:00:09.020)

Like, milliseconds matter, and it doesn't, you know, it's not good enough to just

Dmitri Dolgov (2:00:12.620)

brake. You have to, like, steer and brake, and there's traffic around you.

Lex Fridman (2:00:15.660)

So, like, it all has to come together, and it was really great

Dmitri Dolgov (2:00:19.180)

to see in this case, and other cases like that, that we're actually seeing in the

Lex Fridman (2:00:22.060)

wild, that our system is, you know, performing

Dmitri Dolgov (2:00:25.100)

exactly the way that we would have liked, and is able to,

Lex Fridman (2:00:28.620)

you know, avoid collisions like this.

Dmitri Dolgov (2:00:30.620)

That's such an exciting space for robotics.

Lex Fridman (2:00:32.780)

Like, in that split second to make decisions of life and death.

Dmitri Dolgov (2:00:37.500)

I don't know. The stakes are high, in a sense, but it's also beautiful

Lex Fridman (2:00:41.580)

that for somebody who loves artificial intelligence, the possibility that an AI

Dmitri Dolgov (2:00:47.020)

system might be able to save a human life.

Lex Fridman (2:00:49.980)

That's kind of exciting as a problem, like, to wake up.

Dmitri Dolgov (2:00:53.740)

It's terrifying, probably, for an engineer to wake up,

Lex Fridman (2:00:57.420)

and to think about, but it's also exciting because it's, like,

Dmitri Dolgov (2:01:01.020)

it's in your hands. Let me try to ask a question that's often brought up about

Lex Fridman (2:01:05.420)

autonomous vehicles, and it might be fun to see if you have

Dmitri Dolgov (2:01:09.420)

anything interesting to say, which is about the trolley problem.

Lex Fridman (2:01:14.620)

So, a trolley problem is an interesting philosophical construct

Dmitri Dolgov (2:01:19.260)

that highlights, and there's many others like it,

Lex Fridman (2:01:23.260)

of the difficult ethical decisions that we humans have before us in this

Dmitri Dolgov (2:01:29.900)

complicated world. So, specifically is the choice

Lex Fridman (2:01:34.060)

between if you are forced to choose to kill

Dmitri Dolgov (2:01:39.020)

a group X of people versus a group Y of people, like

Lex Fridman (2:01:42.700)

one person. If you did nothing, you would kill one person, but if

Dmitri Dolgov (2:01:48.220)

you would kill five people, and if you decide to swerve out of the way, you

Lex Fridman (2:01:51.340)

would only kill one person. Do you do nothing, or you choose to do

Dmitri Dolgov (2:01:55.180)

something? You can construct all kinds of, sort of,

Lex Fridman (2:01:58.060)

ethical experiments of this kind that, I think, at least on a positive note,

Dmitri Dolgov (2:02:05.500)

inspire you to think about, like, introspect

Lex Fridman (2:02:09.660)

what are the physics of our morality, and there's usually not

Dmitri Dolgov (2:02:16.220)

good answers there. I think people love it because it's just an exciting

Lex Fridman (2:02:20.700)

thing to think about. I think people who build autonomous

Dmitri Dolgov (2:02:24.060)

vehicles usually roll their eyes, because this is not,

Lex Fridman (2:02:30.060)

this one as constructed, this, like, literally never comes up

Dmitri Dolgov (2:02:34.060)

in reality. You never have to choose between killing

Lex Fridman (2:02:38.300)

one or, like, one of two groups of people,

Lex Fridman (2:02:41.660)

but I wonder if you can speak to, is there some something interesting

Lex Fridman (2:02:48.780)

to you as an engineer of autonomous vehicles that's within the trolley

Dmitri Dolgov (2:02:52.620)

problem, or maybe more generally, are there

Lex Fridman (2:02:55.740)

difficult ethical decisions that you find

Dmitri Dolgov (2:02:58.940)

that an algorithm must make? On the specific version of the trolley problem,

Lex Fridman (2:03:03.340)

which one would you do, if you're driving? The question itself

Dmitri Dolgov (2:03:07.900)

is a profound question, because we humans ourselves

Lex Fridman (2:03:11.340)

cannot answer, and that's the very point. I would kill both.

Dmitri Dolgov (2:03:18.700)

Yeah, humans, I think you're exactly right in that, you know, humans are not

Dmitri Dolgov (2:03:21.340)

particularly good. I think they're kind of phrased as, like, what would a computer do,

Dmitri Dolgov (2:03:24.460)

but, like, humans, you know, are not very good, and actually oftentimes

Lex Fridman (2:03:28.540)

I think that, you know, freezing and kind of not doing anything, because,

Dmitri Dolgov (2:03:32.620)

like, you've taken a few extra milliseconds to just process, and then

Lex Fridman (2:03:35.500)

you end up, like, doing the worst of the possible outcomes, right? So,

Dmitri Dolgov (2:03:38.700)

I do think that, as you've pointed out, it can be

Lex Fridman (2:03:42.220)

a bit of a distraction, and it can be a bit of a kind of red herring. I think

Dmitri Dolgov (2:03:45.660)

it's an interesting, you know, discussion

Lex Fridman (2:03:47.820)

in the realm of philosophy, right? But in terms of

Dmitri Dolgov (2:03:51.580)

what, you know, how that affects the actual

Lex Fridman (2:03:54.780)

engineering and deployment of self driving vehicles,

Dmitri Dolgov (2:03:57.660)

it's not how you go about building a system, right? We've talked

Lex Fridman (2:04:02.780)

about how you engineer a system, how you, you know, go about evaluating

Dmitri Dolgov (2:04:06.460)

the different components and, you know, the safety of the entire thing.

Lex Fridman (2:04:09.820)

How do you kind of inject the, you know, various

Dmitri Dolgov (2:04:13.740)

model based, safety based arguments, and, like, yes, you reason at parts of the

Lex Fridman (2:04:17.580)

system, you know, you reason about the

Lex Fridman (2:04:20.540)

probability of a collision, the severity of that collision, right?

Lex Fridman (2:04:24.220)

And that is incorporated, and there's, you know, you have to properly reason

Dmitri Dolgov (2:04:27.180)

about the uncertainty that flows through the system, right? So,

Lex Fridman (2:04:29.500)

you know, those, you know, factors definitely play a role in how

Dmitri Dolgov (2:04:34.540)

the cars then behave, but they tend to be more

Dmitri Dolgov (2:04:36.700)

of, like, the emergent behavior. And what you see, like, you're absolutely right

Dmitri Dolgov (2:04:39.740)

that these, you know, clear theoretical problems that they, you

Lex Fridman (2:04:43.740)

know, you don't encounter that in the system, and really kind of being

Dmitri Dolgov (2:04:46.940)

back to our previous discussion of, like, what, you know, what, you

Lex Fridman (2:04:49.980)

know, which one do you choose? Well, you know, oftentimes, like,

Dmitri Dolgov (2:04:53.900)

you made a mistake earlier. Like, you shouldn't be in that situation

Lex Fridman (2:04:57.420)

in the first place, right? And in reality, the system comes up.

Dmitri Dolgov (2:05:00.620)

If you build a very good, safe, and capable driver,

Lex Fridman (2:05:03.740)

you have enough, you know, clues in the environment that you

Dmitri Dolgov (2:05:08.380)

drive defensively, so you don't put yourself in that situation, right? And

Lex Fridman (2:05:11.340)

again, you know, it has, you know, this, if you go back to that analogy of, you

Dmitri Dolgov (2:05:14.060)

know, precision and recoil, like, okay, you can make a, you know, very hard trade

Lex Fridman (2:05:16.860)

off, but like, neither answer is really good.

Lex Fridman (2:05:19.500)

But what instead you focus on is kind of moving

Lex Fridman (2:05:22.460)

the whole curve up, and then you focus on building the right capability on the

Dmitri Dolgov (2:05:26.140)

right defensive driving, so that, you know, you don't put yourself in the

Lex Fridman (2:05:28.620)

situation like this. I don't know if you have a good answer

Dmitri Dolgov (2:05:32.380)

for this, but people love it when I ask this question

Lex Fridman (2:05:35.420)

about books. Are there books in your life that you've enjoyed,

Dmitri Dolgov (2:05:42.460)

philosophical, fiction, technical, that had a big impact on you as an engineer or

Lex Fridman (2:05:47.100)

as a human being? You know, everything from science fiction

Dmitri Dolgov (2:05:50.300)

to a favorite textbook. Is there three books that stand out that

Lex Fridman (2:05:53.500)

you can think of? Three books. So I would, you know, that

Dmitri Dolgov (2:05:57.340)

impacted me, I would say,

Lex Fridman (2:06:02.860)

and this one is, you probably know it well,

Lex Fridman (2:06:06.380)

but not generally well known, I think, in the U.S., or kind of

Lex Fridman (2:06:11.420)

internationally, The Master and Margarita. It's one of, actually, my

Dmitri Dolgov (2:06:16.620)

favorite books. It is, you know, by

Lex Fridman (2:06:20.860)

Russian, it's a novel by Russian author Mikhail Bulgakov, and it's just, it's a

Dmitri Dolgov (2:06:26.300)

great book. It's one of those books that you can, like,

Lex Fridman (2:06:28.220)

reread your entire life, and it's very accessible. You can read it as a kid,

Dmitri Dolgov (2:06:32.300)

and, like, it's, you know, the plot is interesting. It's, you know, the

Lex Fridman (2:06:35.900)

devil, you know, visiting the Soviet Union,

Dmitri Dolgov (2:06:38.140)

and, you know, but it, like, you read it, reread it

Lex Fridman (2:06:41.980)

at different stages of your life, and you enjoy it for

Dmitri Dolgov (2:06:46.060)

different, very different reasons, and you keep finding, like, deeper and deeper

Lex Fridman (2:06:49.580)

meaning, and, you know, kind of affected, you know,

Dmitri Dolgov (2:06:52.220)

had a, definitely had an, like, imprint on me, you know, mostly from the,

Dmitri Dolgov (2:06:57.580)

probably kind of the cultural, stylistic aspect. Like, it makes you think one of

Dmitri Dolgov (2:07:00.940)

those books that, you know, is good and makes you think, but also has,

Lex Fridman (2:07:04.300)

like, this really, you know, silly, quirky, dark sense of, you know,

Dmitri Dolgov (2:07:07.740)

humor. It captures the Russian soul more than

Lex Fridman (2:07:10.140)

many, perhaps, many other books. On that, like, slight note,

Dmitri Dolgov (2:07:13.020)

just out of curiosity, one of the saddest things is I've read that book

Lex Fridman (2:07:17.180)

in English. Did you, by chance, read it in English or in Russian?

Dmitri Dolgov (2:07:22.460)

In Russian, only in Russian, and I actually, that is a question I had,

Lex Fridman (2:07:26.060)

kind of posed to myself every once in a while, like, I wonder how well it

Dmitri Dolgov (2:07:30.780)

translates, if it translates at all, and there's the

Lex Fridman (2:07:33.420)

language aspect of it, and then there's the cultural aspect, so

Dmitri Dolgov (2:07:35.980)

I, actually, I'm not sure if, you know, either of those would

Dmitri Dolgov (2:07:39.260)

work well in English. Now, I forget their names, but, so, when the COVID lifts a

Dmitri Dolgov (2:07:43.740)

little bit, I'm traveling to Paris for several reasons. One is just, I've

Lex Fridman (2:07:48.780)

never been to Paris, I want to go to Paris, but

Dmitri Dolgov (2:07:50.700)

there's the most famous translators of Dostoevsky, Tolstoy, of most of

Lex Fridman (2:07:57.020)

Russian literature live there. There's a couple, they're famous,

Dmitri Dolgov (2:08:00.540)

a man and a woman, and I'm going to, sort of, have a series of conversations with

Lex Fridman (2:08:03.980)

them, and in preparation for that, I'm starting

Dmitri Dolgov (2:08:06.780)

to read Dostoevsky in Russian, so I'm really embarrassed to say that I read

Lex Fridman (2:08:10.380)

this, everything I've read in Russian literature of, like,

Dmitri Dolgov (2:08:13.820)

serious depth has been in English, even though

Lex Fridman (2:08:18.540)

I can also read, I mean, obviously, in Russian, but

Dmitri Dolgov (2:08:21.820)

for some reason, it seemed,

Lex Fridman (2:08:26.940)

in the optimization of life, it seemed the improper decision to do, to read in

Dmitri Dolgov (2:08:31.420)

Russian, like, you know, like, I don't need to,

Lex Fridman (2:08:35.020)

I need to think in English, not in Russian, but now I'm changing my mind on

Dmitri Dolgov (2:08:38.700)

that, and so, the question of how well I translate, it's a

Lex Fridman (2:08:41.340)

really fun to method one, like, even with Dostoevsky.

Dmitri Dolgov (2:08:43.900)

So, from what I understand, Dostoevsky translates easier,

Lex Fridman (2:08:47.340)

others don't as much. Obviously, the poetry doesn't translate as well,

Dmitri Dolgov (2:08:52.380)

I'm also the music big fan of Vladimir Vosotsky,

Lex Fridman (2:08:57.740)

he doesn't obviously translate well, people have tried,

Lex Fridman (2:09:02.700)

but mastermind, I don't know, I don't know about that one, I just know in

Lex Fridman (2:09:06.300)

English, you know, as fun as hell in English, so, so, but

Dmitri Dolgov (2:09:10.140)

it's a curious question, and I want to study it rigorously from both the

Lex Fridman (2:09:13.340)

machine learning aspect, and also because I want to do a

Dmitri Dolgov (2:09:16.940)

couple of interviews in Russia, that

Lex Fridman (2:09:21.980)

I'm still unsure of how to properly conduct an interview

Dmitri Dolgov (2:09:27.100)

across a language barrier, it's a fascinating question

Lex Fridman (2:09:30.380)

that ultimately communicates to an American audience. There's a few

Dmitri Dolgov (2:09:34.060)

Russian people that I think are truly special human beings,

Lex Fridman (2:09:39.260)

and I feel, like, I sometimes encounter this with some

Dmitri Dolgov (2:09:44.780)

incredible scientists, and maybe you encounter this

Lex Fridman (2:09:48.300)

as well at some point in your life, that it feels like because of the language

Dmitri Dolgov (2:09:52.780)

barrier, their ideas are lost to history. It's a sad thing, I think about, like,

Lex Fridman (2:09:57.660)

Chinese scientists, or even authors that, like,

Dmitri Dolgov (2:10:01.500)

that we don't, in an English speaking world, don't get to appreciate

Lex Fridman (2:10:05.820)

some, like, the depth of the culture because it's lost in translation,

Lex Fridman (2:10:09.180)

and I feel like I would love to show that to the world,

Lex Fridman (2:10:13.260)

like, I'm just some idiot, but because I have this,

Dmitri Dolgov (2:10:16.940)

like, at least some semblance of skill in speaking Russian,

Lex Fridman (2:10:20.860)

I feel like, and I know how to record stuff on a video camera,

Dmitri Dolgov (2:10:25.020)

I feel like I want to catch, like, Grigori Perlman, who's a mathematician, I'm not

Lex Fridman (2:10:28.700)

sure if you're familiar with him, I want to talk to him, like, he's a

Dmitri Dolgov (2:10:31.740)

fascinating mind, and to bring him to a wider audience in English speaking

Lex Fridman (2:10:35.980)

will be fascinating, but that requires to be rigorous about this question

Dmitri Dolgov (2:10:40.060)

of how well Bulgakov translates. I mean, I know it's a silly

Lex Fridman (2:10:46.380)

concept, but it's a fundamental one, because how do you translate, and

Dmitri Dolgov (2:10:50.940)

that's the thing that Google Translate is also facing

Lex Fridman (2:10:54.940)

as a more machine learning problem, but I wonder as a more

Dmitri Dolgov (2:10:59.020)

bigger problem for AI, how do we capture the magic

Lex Fridman (2:11:03.020)

that's there in the language? I think that's a really interesting,

Dmitri Dolgov (2:11:08.860)

really challenging problem. If you do read it, Master and Margarita

Lex Fridman (2:11:12.540)

in English, sorry, in Russian, I'd be curious

Dmitri Dolgov (2:11:16.700)

to get your opinion, and I think part of it is language, but part of it's just,

Lex Fridman (2:11:20.620)

you know, centuries of culture, that, you know, the cultures are

Dmitri Dolgov (2:11:23.260)

different, so it's hard to connect that.

Lex Fridman (2:11:28.060)

Okay, so that was my first one, right? You had two more. The second one I

Dmitri Dolgov (2:11:31.420)

would probably pick is the science fiction by the

Lex Fridman (2:11:35.660)

Strogatsky brothers. You know, it's up there with, you know,

Dmitri Dolgov (2:11:38.460)

Isaac Asimov and, you know, Ray Bradbury and, you know, company. The

Lex Fridman (2:11:43.340)

Strogatsky brothers kind of appealed more to me. I think it made more of an

Dmitri Dolgov (2:11:47.740)

impression on me growing up. I apologize if I'm

Lex Fridman (2:11:53.500)

showing my complete ignorance. I'm so weak on sci fi. What did

Dmitri Dolgov (2:11:57.100)

they write? Oh, Roadside Picnic,

Lex Fridman (2:12:04.060)

Heart to Be a God,

Dmitri Dolgov (2:12:07.580)

Beetle in an Ant Hill, Monday Starts on Saturday. Like, it's

Lex Fridman (2:12:14.700)

not just science fiction. It also has very interesting, you know,

Dmitri Dolgov (2:12:17.500)

interpersonal and societal questions, and some of the

Lex Fridman (2:12:21.580)

language is just completely hilarious.

Dmitri Dolgov (2:12:27.820)

That's the one. Oh, interesting. Monday Starts on Saturday. So,

Lex Fridman (2:12:31.500)

I need to read. Okay, oh boy. You put that in the category of science fiction?

Dmitri Dolgov (2:12:36.300)

That one is, I mean, this was more of a silly,

Lex Fridman (2:12:39.900)

you know, humorous work. I mean, there is kind of...

Dmitri Dolgov (2:12:43.260)

It's profound too, right? Science fiction, right? It's about, you know, this

Lex Fridman (2:12:46.380)

research institute, and it has deep parallels to

Dmitri Dolgov (2:12:50.620)

serious research, but the setting, of course,

Lex Fridman (2:12:53.660)

is that they're working on, you know, magic, right? And there's a

Lex Fridman (2:12:56.380)

lot of stuff. And that's their style, right?

Lex Fridman (2:13:00.300)

And, you know, other books are very different, right? You know,

Dmitri Dolgov (2:13:03.260)

Heart to Be a God, right? It's about kind of this higher society being injected

Lex Fridman (2:13:07.100)

into this primitive world, and how they operate there,

Lex Fridman (2:13:09.660)

and some of the very deep ethical questions there,

Lex Fridman (2:13:13.420)

right? And, like, they've got this full spectrum. Some is, you know, more about

Dmitri Dolgov (2:13:16.540)

kind of more adventure style. But, like, I enjoy all of

Lex Fridman (2:13:19.580)

their books. There's just, you know, probably a couple.

Dmitri Dolgov (2:13:21.820)

Actually, one I think that they consider their most important work.

Lex Fridman (2:13:24.780)

I think it's The Snail on a Hill. I'm not exactly sure how it

Dmitri Dolgov (2:13:29.660)

translates. I tried reading a couple times. I still don't get it.

Lex Fridman (2:13:32.620)

But everything else I fully enjoyed. And, like, for one of my birthdays as a kid, I

Dmitri Dolgov (2:13:36.540)

got, like, their entire collection, like, occupied a giant shelf in my room, and

Lex Fridman (2:13:40.060)

then, like, over the holidays, I just, like,

Dmitri Dolgov (2:13:42.220)

you know, my parents couldn't drag me out of the room, and I read the whole thing

Lex Fridman (2:13:44.700)

cover to cover. And I really enjoyed it.

Lex Fridman (2:13:49.500)

And that's one more. For the third one, you know, maybe a little bit

Lex Fridman (2:13:52.540)

darker, but, you know, comes to mind is Orwell's

Dmitri Dolgov (2:13:56.700)

1984. And, you know, you asked what made an

Lex Fridman (2:14:01.180)

impression on me and the books that people should read. That one, I think,

Dmitri Dolgov (2:14:03.900)

falls in the category of both. You know, definitely it's one of those

Lex Fridman (2:14:06.860)

books that you read, and you just kind of, you know, put it

Dmitri Dolgov (2:14:11.100)

down and you stare in space for a while. You know, that kind of work. I think

Lex Fridman (2:14:16.460)

there's, you know, lessons there. People should

Dmitri Dolgov (2:14:19.980)

not ignore. And, you know, nowadays, with, like,

Lex Fridman (2:14:24.220)

everything that's happening in the world, I,

Dmitri Dolgov (2:14:26.060)

like, can't help it, but, you know, have my mind jump to some,

Lex Fridman (2:14:29.420)

you know, parallels with what Orwell described. And, like, there's this whole,

Dmitri Dolgov (2:14:34.220)

you know, concept of double think and ignoring logic and, you know, holding

Lex Fridman (2:14:38.460)

completely contradictory opinions in your mind and not have that not bother

Dmitri Dolgov (2:14:41.820)

you and, you know, sticking to the party line

Lex Fridman (2:14:44.140)

at all costs. Like, you know, there's something there.

Dmitri Dolgov (2:14:48.220)

If anything, 2020 has taught me, and I'm a huge fan of Animal Farm, which is a

Lex Fridman (2:14:52.940)

kind of friendly, as a friend of 1984 by Orwell.

Dmitri Dolgov (2:14:57.900)

It's kind of another thought experiment of how our society

Lex Fridman (2:15:03.660)

may go in directions that we wouldn't like it to go.

Lex Fridman (2:15:07.340)

But if anything that's been kind of heartbreaking to an

Lex Fridman (2:15:14.300)

optimist about 2020 is that

Dmitri Dolgov (2:15:18.940)

that society is kind of fragile. Like, we have this,

Lex Fridman (2:15:22.140)

this is a special little experiment we have going on.

Lex Fridman (2:15:25.900)

And not, it's not unbreakable. Like, we should be careful to, like, preserve

Lex Fridman (2:15:32.300)

whatever the special thing we have going on. I mean, I think 1984

Lex Fridman (2:15:36.380)

and these books, The Brave New World, they're

Lex Fridman (2:15:39.820)

helpful in thinking, like, stuff can go wrong

Dmitri Dolgov (2:15:43.660)

in nonobvious ways. And it's, like, it's up to us to preserve it.

Lex Fridman (2:15:48.380)

And it's, like, it's a responsibility. It's been weighing heavy on me because, like,

Dmitri Dolgov (2:15:51.980)

for some reason, like, more than my mom follows me on Twitter and I

Lex Fridman (2:15:57.580)

feel like I have, like, now somehow a

Dmitri Dolgov (2:15:59.980)

responsibility to

Lex Fridman (2:16:03.100)

do this world. And it dawned on me that, like,

Dmitri Dolgov (2:16:07.980)

me and millions of others are, like, the little ants

Lex Fridman (2:16:12.300)

that maintain this little colony, right? So we have a responsibility not to

Dmitri Dolgov (2:16:17.020)

be, I don't know what the right analogy is, but

Lex Fridman (2:16:20.060)

I'll put a flamethrower to the place. We want to

Dmitri Dolgov (2:16:23.420)

not do that. And there's interesting complicated ways of doing that as 1984

Lex Fridman (2:16:27.900)

shows. It could be through bureaucracy. It could

Dmitri Dolgov (2:16:29.820)

be through incompetence. It could be through misinformation.

Lex Fridman (2:16:33.180)

It could be through division and toxicity.

Dmitri Dolgov (2:16:36.460)

I'm a huge believer in, like, that love will be

Lex Fridman (2:16:39.980)

the, somehow, the solution. So, love and robots. Love and robots, yeah.

Dmitri Dolgov (2:16:46.460)

I think you're exactly right. Unfortunately, I think it's less of a

Lex Fridman (2:16:49.340)

flamethrower type of thing. It's more of a,

Dmitri Dolgov (2:16:51.980)

in many cases, it's going to be more of a slow boil. And that's the

Lex Fridman (2:16:55.100)

danger. Let me ask, it's a fun thing to make

Dmitri Dolgov (2:17:00.220)

a world class roboticist, engineer, and leader uncomfortable with a

Lex Fridman (2:17:05.100)

ridiculous question about life. What is the meaning of life,

Lex Fridman (2:17:09.660)

Dimitri, from a robotics and a human perspective?

Lex Fridman (2:17:14.700)

You only have a couple minutes, or one minute to answer, so.

Dmitri Dolgov (2:17:19.820)

I don't know if that makes it more difficult or easier, actually.

Lex Fridman (2:17:23.180)

You know, they're very tempted to quote one of the

Dmitri Dolgov (2:17:29.740)

stories by Isaac Asimov, actually. Actually, titled,

Dmitri Dolgov (2:17:36.060)

appropriately titled, The Last Question. It's a short story where, you know, the

Dmitri Dolgov (2:17:39.900)

plot is that, you know, humans build this supercomputer,

Lex Fridman (2:17:42.860)

you know, this AI intelligence, and, you know, once it

Dmitri Dolgov (2:17:46.220)

gets powerful enough, they pose this question to it, you know,

Lex Fridman (2:17:49.660)

how can the entropy in the universe be reduced, right? So the computer replies,

Dmitri Dolgov (2:17:54.380)

as of yet, insufficient information to give a meaningful answer,

Lex Fridman (2:17:58.140)

right? And then, you know, thousands of years go by, and they keep posing the

Dmitri Dolgov (2:18:00.940)

same question, and the computer, you know, gets more and more powerful, and keeps

Lex Fridman (2:18:03.980)

giving the same answer, you know, as of yet, insufficient

Dmitri Dolgov (2:18:06.540)

information to give a meaningful answer, or something along those lines,

Lex Fridman (2:18:09.580)

right? And then, you know, it keeps, you know, happening, and

Dmitri Dolgov (2:18:12.940)

happening, you fast forward, like, millions of years into the future, and,

Dmitri Dolgov (2:18:16.060)

you know, billions of years, and, like, at some point, it's just the only entity in

Dmitri Dolgov (2:18:19.100)

the universe, it's, like, absorbed all humanity,

Lex Fridman (2:18:21.580)

and all knowledge in the universe, and it, like, keeps posing the same question

Dmitri Dolgov (2:18:24.460)

to itself, and, you know, finally, it gets to the

Lex Fridman (2:18:28.700)

point where it is able to answer that question, but, of course, at that point,

Dmitri Dolgov (2:18:31.900)

you know, there's, you know, the heat death of the universe has occurred, and

Lex Fridman (2:18:34.700)

that's the only entity, and there's nobody else to provide that

Dmitri Dolgov (2:18:37.500)

answer to, so the only thing it can do is to,

Dmitri Dolgov (2:18:40.140)

you know, answer it by demonstration, so, like, you know, it recreates the big bang,

Lex Fridman (2:18:43.980)

right, and resets the clock, right?

Lex Fridman (2:18:47.100)

But, like, you know, I can try to give kind of a

Dmitri Dolgov (2:18:50.540)

different version of the answer, you know, maybe

Lex Fridman (2:18:53.340)

not on the behalf of all humanity, I think that that might be a little

Dmitri Dolgov (2:18:56.780)

presumptuous for me to speak about the meaning of life on the behalf of all

Lex Fridman (2:19:00.300)

humans, but at least, you know, personally,

Dmitri Dolgov (2:19:03.420)

it changes, right? I think if you think about kind of what

Lex Fridman (2:19:06.940)

gives, you know, you and your life meaning and purpose, and kind of

Lex Fridman (2:19:13.660)

what drives you, it seems to

Lex Fridman (2:19:18.460)

change over time, right, and that lifespan

Dmitri Dolgov (2:19:22.060)

of, you know, kind of your existence, you know, when

Lex Fridman (2:19:25.180)

just when you just enter this world, right, it's all about kind of new

Dmitri Dolgov (2:19:27.980)

experiences, right? You get, like, new smells, new sounds, new emotions, right,

Lex Fridman (2:19:33.180)

and, like, that's what's driving you, right? You're experiencing

Dmitri Dolgov (2:19:36.380)

new amazing things, right, and that's magical, right? That's pretty

Lex Fridman (2:19:40.140)

pretty awesome, right? That gives you kind of meaning.

Dmitri Dolgov (2:19:43.100)

Then, you know, you get a little bit older, you start more intentionally

Lex Fridman (2:19:47.740)

learning about things, right? I guess, actually, before you start intentionally

Dmitri Dolgov (2:19:51.020)

learning, it's probably fun. Fun is a thing that gives you kind of

Lex Fridman (2:19:53.740)

meaning and purpose and purpose and the thing you optimize for, right?

Dmitri Dolgov (2:19:56.780)

And, like, fun is good. Then you get, you know, start learning, and I guess that

Lex Fridman (2:20:01.020)

this joy of comprehension

Lex Fridman (2:20:05.660)

and discovery is another thing that, you know, gives you

Lex Fridman (2:20:09.500)

meaning and purpose and drives you, right? Then, you know, you

Dmitri Dolgov (2:20:12.940)

learn enough stuff and you want to give some of it back, right? And so

Lex Fridman (2:20:17.420)

impact and contributions back to, you know, technology or society,

Dmitri Dolgov (2:20:20.460)

you know, people, you know, local or more globally

Lex Fridman (2:20:24.860)

becomes a new thing that, you know, drives a lot of kind of your behavior

Lex Fridman (2:20:28.620)

and is something that gives you purpose and

Lex Fridman (2:20:31.900)

that you derive, you know, positive feedback from, right?

Dmitri Dolgov (2:20:35.260)

You know, then you go and so on and so forth. You go through various stages of

Lex Fridman (2:20:38.460)

life. If you have kids,

Dmitri Dolgov (2:20:43.420)

like, that definitely changes your perspective on things. You know, I have

Lex Fridman (2:20:46.220)

three that definitely flips some bits in your

Dmitri Dolgov (2:20:48.940)

head in terms of, you know, what you care about and what you

Lex Fridman (2:20:52.220)

optimize for and, you know, what matters, what doesn't matter, right?

Dmitri Dolgov (2:20:54.940)

So, you know, and so on and so forth, right? And I,

Lex Fridman (2:20:58.140)

it seems to me that, you know, it's all of those things and as

Dmitri Dolgov (2:21:02.380)

kind of you go through life, you know,

Lex Fridman (2:21:06.700)

you want these to be additive, right? New experiences,

Dmitri Dolgov (2:21:10.140)

fun, learning, impact. Like, you want to, you know, be accumulating.

Dmitri Dolgov (2:21:14.460)

I don't want to, you know, stop having fun or, you know, experiencing new things and

Dmitri Dolgov (2:21:17.820)

I think it's important that, you know, it just kind of becomes

Lex Fridman (2:21:20.300)

additive as opposed to a replacement or subtraction.

Dmitri Dolgov (2:21:23.660)

But, you know, those fewest problems as far as I got, but, you know, ask me in a

Lex Fridman (2:21:27.500)

few years, I might have one or two more to add to the list.

Lex Fridman (2:21:30.220)

And before you know it, time is up, just like it is for this conversation,

Lex Fridman (2:21:34.540)

but hopefully it was a fun ride. It was a huge honor to meet you.

Dmitri Dolgov (2:21:38.460)

As you know, I've been a fan of yours and a fan of Google Self Driving Car and

Lex Fridman (2:21:43.900)

Waymo for a long time. I can't wait. I mean, it's one of the

Dmitri Dolgov (2:21:47.420)

most exciting, if we look back in the 21st century, I

Lex Fridman (2:21:50.300)

truly believe it'll be one of the most exciting things we

Dmitri Dolgov (2:21:53.180)

descendants of apes have created on this earth. So,

Lex Fridman (2:21:57.100)

I'm a huge fan and I can't wait to see what you do

Dmitri Dolgov (2:22:00.540)

next. Thanks so much for talking to me. Thanks, thanks for having me and it's a

Lex Fridman (2:22:04.460)

also a huge fan doing work, honestly, and I really

Dmitri Dolgov (2:22:08.540)

enjoyed it. Thank you. Thanks for listening to this

Lex Fridman (2:22:11.260)

conversation with Dmitry Dolgov and thank you to our sponsors,

Dmitri Dolgov (2:22:14.620)

Triolabs, a company that helps businesses apply machine learning to

Lex Fridman (2:22:19.340)

solve real world problems, Blinkist, an app I use for reading

Dmitri Dolgov (2:22:23.100)

through summaries of books, BetterHelp, online therapy with a licensed

Lex Fridman (2:22:27.420)

professional, and CashApp, the app I use to send money to

Dmitri Dolgov (2:22:30.860)

friends. Please check out these sponsors in the

Lex Fridman (2:22:33.260)

description to get a discount and to support this podcast. If you

Dmitri Dolgov (2:22:37.180)

enjoy this thing, subscribe on YouTube, review it with Five Stars

Lex Fridman (2:22:40.380)

and Upper Podcast, follow on Spotify, support on Patreon,

Dmitri Dolgov (2:22:44.140)

or connect with me on Twitter at Lex Friedman. And now,

Lex Fridman (2:22:47.740)

let me leave you with some words from Isaac Asimov.

Dmitri Dolgov (2:22:51.420)

Science can amuse and fascinate us all, but it is engineering

Lex Fridman (2:22:55.660)

that changes the world. Thank you for listening and hope to see you

Dmitri Dolgov (2:22:59.980)

next time.

Lex Fridman (30:02.940)

So, okay.

Lex Fridman (30:04.140)

So the next step, you mentioned the milestones that you had in the, in those

Lex Fridman (30:09.260)

two years, what are the next milestones that then led to the creation of Waymo

Lex Fridman (30:13.500)

and beyond?

Lex Fridman (30:14.780)

Yeah, we had a, it was a really interesting journey and, you know, Waymo

Dmitri Dolgov (30:18.140)

came a little bit later, then, you know, we completed those milestones in 2010.

Lex Fridman (30:25.020)

That was the pivot when we decided to focus on actually building a product

Dmitri Dolgov (30:30.300)

using this technology.

Lex Fridman (30:32.460)

The initial couple of years after that, we were focused on a freeway, you

Dmitri Dolgov (30:37.660)

know, what you would call a driver assist, maybe, you know, an L3 driver

Lex Fridman (30:41.180)

assist program.

Dmitri Dolgov (30:42.780)

Then around 2013, we've learned enough about the space and thought more deeply

Lex Fridman (30:49.500)

about, you know, the product that we wanted to build, that we pivoted, we

Dmitri Dolgov (30:54.940)

pivoted towards this vision of building a driver and deploying it fully driverless

Lex Fridman (31:01.900)

vehicles without a person.

Lex Fridman (31:02.940)

And that that's the path that we've been on since then.

Lex Fridman (31:05.100)

And very, it was exactly the right decision for us.

Lex Fridman (31:08.540)

So there was a moment where you're also considered like, what is the right

Lex Fridman (31:13.580)

trajectory here?

Lex Fridman (31:14.780)

What is the right role of automation in the, in the task of driving?

Lex Fridman (31:18.140)

There's still, it wasn't from the early days, obviously you want to go fully

Dmitri Dolgov (31:23.180)

autonomous.

Lex Fridman (31:24.060)

From the early days, it was not.

Dmitri Dolgov (31:25.100)

I think it was in 20, around 2013, maybe that we've, that became very clear and

Lex Fridman (31:31.740)

we made that pivot and also became very clear and that it's either the way you

Dmitri Dolgov (31:36.860)

go building a driver assist system is, you know, fundamentally different from

Lex Fridman (31:41.500)

how you go building a fully driverless vehicle.

Dmitri Dolgov (31:43.500)

So, you know, we've pivoted towards the ladder and that's what we've been

Lex Fridman (31:48.700)

working on ever since.

Lex Fridman (31:50.620)

And so that was around 2013, then there's sequence of really meaningful for us

Lex Fridman (31:57.900)

really important defining milestones since then.

Lex Fridman (32:00.540)

And in 2015, we had our first, actually the world's first fully driverless

Lex Fridman (32:12.220)

trade on public roads.

Dmitri Dolgov (32:15.020)

It was in a custom built vehicle that we had.

Lex Fridman (32:17.500)

I must've seen those.

Dmitri Dolgov (32:18.380)

We called them the Firefly, that, you know, funny looking marshmallow looking

Lex Fridman (32:21.100)

thing.

Lex Fridman (32:22.700)

And we put a passenger, his name was Steve Mann, you know, great friend of

Lex Fridman (32:30.060)

our project from the early days, the man happens to be blind.

Lex Fridman (32:34.540)

So we put them in that vehicle.

Lex Fridman (32:36.060)

The car had no steering wheel, no pedals.

Dmitri Dolgov (32:38.140)

It was an uncontrolled environment.

Lex Fridman (32:40.460)

You know, no, you know, lead or chase cars, no police escorts.

Dmitri Dolgov (32:44.540)

And, you know, we did that trip a few times in Austin, Texas.

Lex Fridman (32:47.900)

So that was a really big milestone.

Lex Fridman (32:49.500)

But that was in Austin.

Lex Fridman (32:50.620)

Yeah.

Dmitri Dolgov (32:51.180)

Okay.

Lex Fridman (32:52.860)

And, you know, we only, but at that time we're only, it took a tremendous

Dmitri Dolgov (32:56.620)

amount of engineering.

Lex Fridman (32:57.340)

It took a tremendous amount of validation to get to that point.

Dmitri Dolgov (33:01.020)

But, you know, we only did it a few times.

Lex Fridman (33:03.820)

We only did that.

Dmitri Dolgov (33:04.540)

It was a fixed route.

Lex Fridman (33:05.500)

It was not kind of a controlled environment, but it was a fixed route.

Lex Fridman (33:08.060)

And we only did a few times.

Lex Fridman (33:10.220)

Then in 2016, end of 2016, beginning of 2017 is when we founded Waymo, the

Dmitri Dolgov (33:19.820)

company.

Lex Fridman (33:20.220)

That's when we kind of, that was the next phase of the project where I

Dmitri Dolgov (33:25.100)

wanted, we believed in kind of the commercial vision of this technology.

Lex Fridman (33:30.460)

And it made sense to create an independent entity, you know, within

Dmitri Dolgov (33:33.420)

that alphabet umbrella to pursue this product at scale.

Lex Fridman (33:39.420)

Beyond that in 2017, later in 2017 was another really huge step for us.

Dmitri Dolgov (33:46.540)

Really big milestone where we started, I think it was October of 2017 where

Lex Fridman (33:52.460)

when we started regular driverless operations on public roads, that first

Dmitri Dolgov (33:59.340)

day of operations, we drove in one day.

Lex Fridman (34:02.780)

And that first day, a hundred miles and driverless fashion.

Lex Fridman (34:05.980)

And then we've now the most, the most important thing about that milestone

Lex Fridman (34:08.460)

was not that, you know, a hundred miles in one day, but that it was the

Dmitri Dolgov (34:11.500)

start of kind of regular ongoing driverless operations.

Lex Fridman (34:14.940)

And when you say driverless, it means no driver.

Dmitri Dolgov (34:19.100)

That's exactly right.

Lex Fridman (34:19.740)

So on that first day, we actually hit a mix and in some, we didn't want

Dmitri Dolgov (34:24.780)

to like, you know, be on YouTube and Twitter that same day.

Lex Fridman (34:27.100)

So in, in many of the rides we had somebody in the driver's seat, but

Dmitri Dolgov (34:32.460)

they could not disengage like the car, not disengage, but actually on that

Dmitri Dolgov (34:36.860)

first day, some of the miles were driven and just completely empty driver's seat.

Lex Fridman (34:42.540)

And this is the key distinction that I think people don't realize it's, you

Lex Fridman (34:46.780)

know, that oftentimes when you talk about autonomous vehicles, you're, there's

Dmitri Dolgov (34:53.020)

often a driver in the seat that's ready to to take over what's called a safety

Lex Fridman (34:59.420)

driver and then Waymo is really one of the only companies at least that I'm

Dmitri Dolgov (35:05.420)

aware of, or at least as like boldly and carefully and all, and all of that is

Lex Fridman (35:10.940)

actually has cases.

Lex Fridman (35:12.540)

And now we'll talk about more and more where there's literally no driver.

Lex Fridman (35:17.100)

So that's another, the interesting case of where the driver's not supposed

Dmitri Dolgov (35:21.500)

to disengage, that's like a nice middle ground, they're still there, but

Lex Fridman (35:24.700)

they're not supposed to disengage, but really there's the case when there's

Dmitri Dolgov (35:28.380)

no, okay, there's something magical about there being nobody in the driver's seat.

Dmitri Dolgov (35:34.540)

Like, just like to me, you mentioned the first time you wrote some code for free

Dmitri Dolgov (35:41.260)

space navigation of the parking lot, that was like a magical moment to me, just

Lex Fridman (35:46.700)

sort of as an observer of robots, the first magical moment is seeing an

Dmitri Dolgov (35:53.900)

autonomous vehicle turn, like make a left turn, like apply sufficient torque to

Lex Fridman (36:01.660)

the steering wheel to where it, like, there's a lot of rotation and for some

Dmitri Dolgov (36:05.740)

reason, and there's nobody in the driver's seat, for some reason that

Lex Fridman (36:10.300)

communicates that here's a being with power that makes a decision.

Dmitri Dolgov (36:16.060)

There's something about like the steering wheel, cause we perhaps romanticize

Lex Fridman (36:19.660)

the notion of the steering wheel, it's so essential to our conception, our 20th

Dmitri Dolgov (36:24.300)

century conception of a car and it turning the steering wheel with nobody

Lex Fridman (36:28.380)

in driver's seat, that to me, I think maybe to others, it's really powerful.

Dmitri Dolgov (36:34.460)

Like this thing is in control and then there's this leap of trust that you give.

Lex Fridman (36:39.100)

Like I'm going to put my life in the hands of this thing that's in control.

Lex Fridman (36:42.620)

So in that sense, when there's no, but no driver in the driver's seat, that's a

Lex Fridman (36:47.420)

magical moment for robots.

Lex Fridman (36:49.820)

So I'm, I've gotten a chance to last year to take a ride in a, in a

Lex Fridman (36:54.700)

way more vehicle and that, that was the magical moment. There's like nobody in

Dmitri Dolgov (36:58.700)

the driver's seat. It's, it's like the little details. You would think it

Lex Fridman (37:03.180)

doesn't matter whether there's a driver or not, but like if there's no driver

Lex Fridman (37:07.500)

and the steering wheel is turning on its own, I don't know. That's magical.

Dmitri Dolgov (37:13.260)

It's absolutely magical. I, I have taken many of these rides and like completely

Dmitri Dolgov (37:17.260)

empty car, no human in the car pulls up, you know, you call it on your cell phone.

Dmitri Dolgov (37:22.220)

It pulls up, you get in, it takes you on its way. There's nobody in the car, but

Dmitri Dolgov (37:27.740)

you, right? That's something called, you know, fully driverless, you know, our

Lex Fridman (37:31.980)

writer only mode of operation. Yeah. It, it is magical. It is, you know,

Dmitri Dolgov (37:39.900)

transformative. This is what we hear from our writers. It kind of really

Lex Fridman (37:44.780)

changes your experience. And not like that, that really is what unlocks the

Dmitri Dolgov (37:48.780)

real potential of this technology. But, you know, coming back to our journey,

Lex Fridman (37:53.580)

you know, that was 2017 when we started, you know, truly driverless operations.

Dmitri Dolgov (37:58.780)

Then in 2018, we've launched our public commercial service that we called

Lex Fridman (38:05.740)

Waymo One in Phoenix. In 2019, we started offering truly driverless writer

Dmitri Dolgov (38:13.820)

only rides to our early rider population of users. And then, you know, 2020 has

Lex Fridman (38:22.940)

also been a pretty interesting year. One of the first ones, less about

Dmitri Dolgov (38:26.700)

technology, but more about the maturing and the growth of Waymo as a company.

Lex Fridman (38:31.980)

We raised our first round of external financing this year, you know, we were

Dmitri Dolgov (38:37.500)

part of Alphabet. So obviously we have access to, you know, significant resources

Lex Fridman (38:42.060)

but as kind of on the journey of Waymo maturing as a company, it made sense

Dmitri Dolgov (38:45.900)

for us to, you know, partially go externally in this round. So, you know,

Lex Fridman (38:50.620)

we're raised about $3.2 billion from that round. We've also started putting

Dmitri Dolgov (38:59.740)

our fifth generation of our driver, our hardware, that is on the new vehicle,

Lex Fridman (39:05.420)

but it's also a qualitatively different set of self driving hardware.

Dmitri Dolgov (39:10.380)

That is now on the JLR pace. So that was a very important step for us.

Lex Fridman (39:19.340)

Hardware specs, fifth generation. I think it'd be fun to maybe, I apologize if

Dmitri Dolgov (39:25.580)

I'm interrupting, but maybe talk about maybe the generations with a focus on

Lex Fridman (39:31.980)

what we're talking about on the fifth generation in terms of hardware specs,

Dmitri Dolgov (39:35.660)

like what's on this car.

Lex Fridman (39:36.700)

Sure. So we separated out, you know, the actual car that we are driving from

Dmitri Dolgov (39:41.580)

the self driving hardware we put on it. Right now we have, so this is, as I

Lex Fridman (39:45.820)

mentioned, the fifth generation, you know, we've gone through, we started,

Dmitri Dolgov (39:49.980)

you know, building our own hardware, you know, many, many years ago. And

Lex Fridman (39:56.060)

that, you know, Firefly vehicle also had the hardware suite that was mostly

Dmitri Dolgov (40:01.020)

designed, engineered, and built in house. Lighters are one of the more important

Lex Fridman (40:07.580)

components that we design and build from the ground up. So on the fifth

Dmitri Dolgov (40:11.820)

generation of our drivers of our self driving hardware that we're switching

Lex Fridman (40:18.700)

to right now, we have, as with previous generations, in terms of sensing,

Dmitri Dolgov (40:24.220)

we have lighters, cameras, and radars, and we have a pretty beefy computer

Lex Fridman (40:29.580)

that processes all that information and makes decisions in real time on

Dmitri Dolgov (40:33.420)

board the car. So in all of the, and it's really a qualitative jump forward

Lex Fridman (40:41.180)

in terms of the capabilities and the various parameters and the specs of

Dmitri Dolgov (40:45.660)

the hardware compared to what we had before and compared to what you can

Lex Fridman (40:49.260)

kind of get off the shelf in the market today.

Lex Fridman (40:51.580)

Meaning from fifth to fourth or from fifth to first?

Lex Fridman (40:54.700)

Definitely from first to fifth, but also from the fourth.

Dmitri Dolgov (40:57.340)

That was the world's dumbest question.

Dmitri Dolgov (40:58.700)

Definitely from fourth to fifth, as well as the last step is a big step forward.

Lex Fridman (41:07.500)

So everything's in house. So like LIDAR is built in house and cameras are

Lex Fridman (41:13.900)

built in house?

Dmitri Dolgov (41:15.740)

You know, it's different. We work with partners and there's some components

Lex Fridman (41:19.340)

that we get from our manufacturing and supply chain partners. What exactly

Dmitri Dolgov (41:26.780)

is in house is a bit different. We do a lot of custom design on all of

Lex Fridman (41:34.140)

our sensing modalities, lighters, radars, cameras, you know, exactly.

Dmitri Dolgov (41:37.580)

There's lighters are almost exclusively in house and some of the

Lex Fridman (41:43.180)

technologies that we have, some of the fundamental technologies there

Dmitri Dolgov (41:45.980)

are completely unique to Waymo. That is also largely true about radars

Lex Fridman (41:51.420)

and cameras. It's a little bit more of a mix in terms of what we do

Dmitri Dolgov (41:55.180)

ourselves versus what we get from partners.

Lex Fridman (41:57.980)

Is there something super sexy about the computer that you can mention

Dmitri Dolgov (42:01.580)

that's not top secret? Like for people who enjoy computers for, I

Lex Fridman (42:08.300)

mean, there's a lot of machine learning involved, but there's a lot

Dmitri Dolgov (42:12.700)

of just basic compute. You have to probably do a lot of signal

Lex Fridman (42:17.260)

processing on all the different sensors. You have to integrate everything

Dmitri Dolgov (42:20.780)

has to be in real time. There's probably some kind of redundancy

Lex Fridman (42:23.820)

type of situation. Is there something interesting you can say about

Dmitri Dolgov (42:27.420)

the computer for the people who love hardware? It does have all of

Lex Fridman (42:31.820)

the characteristics, all the properties that you just mentioned.

Dmitri Dolgov (42:34.380)

Redundancy, very beefy compute for general processing, as well as

Lex Fridman (42:41.020)

inference and ML models. It is some of the more sensitive stuff that

Dmitri Dolgov (42:45.260)

I don't want to get into for IP reasons, but it can be shared a

Lex Fridman (42:49.420)

little bit in terms of the specs of the sensors that we have on the

Dmitri Dolgov (42:54.860)

car. We actually shared some videos of what our

Lex Fridman (43:00.700)

lighters see in the world. We have 29 cameras. We have five lighters.

Dmitri Dolgov (43:05.100)

We have six radars on these vehicles, and you can get a feel for

Lex Fridman (43:09.260)

the amount of data that they're producing. That all has to be

Dmitri Dolgov (43:12.380)

processed in real time to do perception, to do complex

Lex Fridman (43:16.860)

reasoning. That kind of gives you some idea of how beefy those computers

Dmitri Dolgov (43:19.740)

are, but I don't want to get into specifics of exactly how we build

Lex Fridman (43:22.540)

them. Okay, well, let me try some more questions that you can get

Dmitri Dolgov (43:25.500)

into the specifics of, like GPU wise. Is that something you can get

Lex Fridman (43:28.780)

into? I know that Google works with GPUs and so on. I mean, for

Dmitri Dolgov (43:33.020)

machine learning folks, it's kind of interesting. Or is there no...

Lex Fridman (43:38.780)

How do I ask it? I've been talking to people in the government about

Dmitri Dolgov (43:43.340)

UFOs and they don't answer any questions. So this is how I feel

Lex Fridman (43:46.860)

right now asking about GPUs. But is there something interesting that

Dmitri Dolgov (43:51.980)

you could reveal? Or is it just... Or leave it up to our

Lex Fridman (43:57.500)

imagination, some of the compute. Is there any, I guess, is there any

Dmitri Dolgov (44:02.060)

fun trickery? Like I talked to Chris Latner for a second time and he

Lex Fridman (44:05.820)

was a key person about GPUs, and there's a lot of fun stuff going

Dmitri Dolgov (44:09.580)

on in Google in terms of hardware that optimizes for machine

Lex Fridman (44:15.580)

learning. Is there something you can reveal in terms of how much,

Dmitri Dolgov (44:19.420)

you mentioned customization, how much customization there is for

Lex Fridman (44:23.100)

hardware for machine learning purposes? I'm going to be like that

Dmitri Dolgov (44:26.220)

government person who bought UFOs. I guess I will say that it's

Lex Fridman (44:34.540)

really... Compute is really important. We have very data hungry

Lex Fridman (44:41.340)

and compute hungry ML models all over our stack. And this is where

Lex Fridman (44:48.300)

both being part of Alphabet, as well as designing our own sensors

Lex Fridman (44:52.060)

and the entire hardware suite together, where on one hand you

Lex Fridman (44:55.820)

get access to really rich raw sensor data that you can pipe

Dmitri Dolgov (45:01.740)

from your sensors into your compute platform and build like

Lex Fridman (45:07.820)

build the whole pipe from sensor raw sensor data to the big

Dmitri Dolgov (45:11.020)

compute as then have the massive compute to process all that

Lex Fridman (45:14.060)

data. And this is where we're finding that having a lot of

Dmitri Dolgov (45:17.420)

control of that hardware part of the stack is really

Lex Fridman (45:21.260)

advantageous. One of the fascinating magical places to me

Dmitri Dolgov (45:25.340)

again, might not be able to speak to the details, but it is

Lex Fridman (45:29.980)

the other compute, which is like, we're just talking about a

Dmitri Dolgov (45:32.940)

single car, but the driving experience is a source of a lot

Lex Fridman (45:39.340)

of fascinating data. And you have a huge amount of data

Dmitri Dolgov (45:42.060)

coming in on the car and the infrastructure of storing some

Lex Fridman (45:47.820)

of that data to then train or to analyze or so on. That's a

Dmitri Dolgov (45:52.460)

fascinating piece of it that I understand a single car. I

Lex Fridman (45:58.220)

don't understand how you pull it all together in a nice way.

Dmitri Dolgov (46:00.940)

Is that something that you could speak to in terms of the

Lex Fridman (46:03.100)

challenges of seeing the network of cars and then

Dmitri Dolgov (46:08.460)

bringing the data back and analyzing things that like edge

Lex Fridman (46:12.620)

cases of driving, be able to learn on them to improve the

Dmitri Dolgov (46:15.340)

system to see where things went wrong, where things went right

Lex Fridman (46:20.060)

and analyze all that kind of stuff. Is there something

Lex Fridman (46:22.220)

interesting there from an engineering perspective?

Lex Fridman (46:25.340)

Oh, there's an incredible amount of really interesting

Dmitri Dolgov (46:30.780)

work that's happening there, both in the real time operation

Lex Fridman (46:35.100)

of the fleet of cars and the information that they exchange

Dmitri Dolgov (46:38.220)

with each other in real time to make better decisions as well

Lex Fridman (46:43.340)

as on the kind of the off board component where you have to

Dmitri Dolgov (46:46.700)

deal with massive amounts of data for training your ML

Lex Fridman (46:50.380)

models, evaluating the ML models for simulating the entire

Dmitri Dolgov (46:54.620)

system and for evaluating your entire system. And this is

Lex Fridman (46:57.980)

where being part of Alphabet has once again been tremendously

Dmitri Dolgov (47:03.420)

advantageous because we consume an incredible amount of

Lex Fridman (47:06.460)

compute for ML infrastructure. We build a lot of custom

Dmitri Dolgov (47:10.140)

frameworks to get good at data mining, finding the

Lex Fridman (47:16.460)

interesting edge cases for training and for evaluation of

Dmitri Dolgov (47:19.180)

the system for both training and evaluating some components

Lex Fridman (47:23.900)

and your sub parts of the system and various ML models,

Dmitri Dolgov (47:27.020)

as well as the evaluating the entire system and simulation.

Lex Fridman (47:31.020)

Okay. That first piece that you mentioned that cars

Dmitri Dolgov (47:33.820)

communicating to each other, essentially, I mean, through

Lex Fridman (47:36.700)

perhaps through a centralized point, but what that's

Dmitri Dolgov (47:40.060)

fascinating too, how much does that help you? Like if you

Lex Fridman (47:43.420)

imagine, you know, right now the number of way more vehicles

Dmitri Dolgov (47:46.940)

is whatever X. I don't know if you can talk to what that

Lex Fridman (47:50.460)

number is, but it's not in the hundreds of millions yet. And

Dmitri Dolgov (47:55.100)

imagine if the whole world is way more vehicles, like that

Lex Fridman (47:59.340)

changes potentially the power of connectivity. Like the more

Dmitri Dolgov (48:03.660)

cars you have, I guess, actually, if you look at

Lex Fridman (48:06.300)

Phoenix, cause there's enough vehicles, there's enough, when

Dmitri Dolgov (48:09.980)

there's like some level of density, you can start to

Lex Fridman (48:13.100)

probably do some really interesting stuff with the fact

Dmitri Dolgov (48:15.900)

that cars can negotiate, can be, can communicate with each

Lex Fridman (48:21.420)

other and thereby make decisions. Is there something

Dmitri Dolgov (48:24.300)

interesting there that you can talk to about like, how does

Lex Fridman (48:27.820)

that help with the driving problem from, as compared to

Lex Fridman (48:31.100)

just a single car solving the driving problem by itself?

Lex Fridman (48:35.660)

Yeah, it's a spectrum. I first and say that, you know, it's,

Dmitri Dolgov (48:40.460)

it helps and it helps in various ways, but it's not required

Lex Fridman (48:44.140)

right now with the way we build our system, like each cars can

Dmitri Dolgov (48:46.700)

operate independently. They can operate with no connectivity.

Lex Fridman (48:49.660)

So I think it is important that, you know, you have a fully

Dmitri Dolgov (48:53.740)

autonomous, fully capable driver that, you know, computerized

Lex Fridman (48:59.580)

driver that each car has. Then, you know, they do share

Dmitri Dolgov (49:03.340)

information and they share information in real time. It

Lex Fridman (49:06.140)

really, really helps. So the way we do this today is, you know,

Dmitri Dolgov (49:11.820)

whenever one car encounters something interesting in the

Lex Fridman (49:15.180)

world, whether it might be an accident or a new construction

Dmitri Dolgov (49:17.980)

zone, that information immediately gets, you know,

Lex Fridman (49:21.420)

uploaded over the air and it's propagated to the rest of the

Dmitri Dolgov (49:23.900)

fleet. So, and that's kind of how we think about maps as

Lex Fridman (49:27.420)

priors in terms of the knowledge of our drivers, of our fleet of

Dmitri Dolgov (49:32.940)

drivers that is distributed across the fleet and it's

Lex Fridman (49:38.140)

updated in real time. So that's one use case. And

Dmitri Dolgov (49:41.740)

you know, you can imagine as the, you know, the density of

Lex Fridman (49:46.940)

these vehicles go up, that they can exchange more information

Dmitri Dolgov (49:50.060)

in terms of what they're planning to do and start

Lex Fridman (49:53.820)

influencing how they interact with each other, as well as,

Dmitri Dolgov (49:56.540)

you know, potentially sharing some observations, right, to

Lex Fridman (49:59.660)

help with, you know, if you have enough density of these

Dmitri Dolgov (50:01.820)

vehicles where, you know, one car might be seeing something

Lex Fridman (50:04.060)

that another is relevant to another car that is very

Dmitri Dolgov (50:06.780)

dynamic. You know, it's not part of kind of your updating

Lex Fridman (50:08.940)

your static prior of the map of the world, but it's more of a

Dmitri Dolgov (50:11.500)

dynamic information that could be relevant to the decisions

Lex Fridman (50:14.220)

that another car is making real time. So you can see them

Dmitri Dolgov (50:16.380)

exchanging that information and you can build on that. But

Lex Fridman (50:18.860)

again, I see that as an advantage, but it's not a

Dmitri Dolgov (50:23.660)

requirement. So what about the human in the loop? So when I

Lex Fridman (50:28.460)

got a chance to drive with a ride in a Waymo, you know,

Dmitri Dolgov (50:34.780)

there's customer service. So like there is somebody that's

Lex Fridman (50:39.740)

able to dynamically like tune in and help you out. What role

Dmitri Dolgov (50:48.300)

does the human play in that picture? That's a fascinating

Lex Fridman (50:51.500)

like, you know, the idea of teleoperation, be able to

Dmitri Dolgov (50:53.980)

remotely control a vehicle. So here, what we're talking

Lex Fridman (50:57.180)

about is like, like frictionless, like a human being

Dmitri Dolgov (51:03.900)

able to in a in a frictionless way, sort of help you out. I

Lex Fridman (51:08.460)

don't know if they're able to actually control the vehicle.

Dmitri Dolgov (51:10.780)

Is that something you could talk to? Yes. Okay. To be clear,

Lex Fridman (51:14.300)

we don't do teleporation. I kind of believe in

Dmitri Dolgov (51:16.300)

teleporation for various reasons. That's not what we

Lex Fridman (51:19.100)

have in our cars. We do, as you mentioned, have, you know,

Dmitri Dolgov (51:22.300)

version of, you know, customer support. You know, we call it

Lex Fridman (51:24.780)

life health. In fact, we find it that it's very important for

Dmitri Dolgov (51:28.860)

our ride experience, especially if it's your first trip, you've

Lex Fridman (51:32.300)

never been in a fully driverless ride or only way more

Dmitri Dolgov (51:35.020)

vehicle you get in, there's nobody there. And so you can

Lex Fridman (51:37.660)

imagine having all kinds of, you know, questions in your head,

Dmitri Dolgov (51:40.460)

like how this thing works. So we've put a lot of thought into

Lex Fridman (51:43.260)

kind of guiding our, our writers or customers through that

Dmitri Dolgov (51:47.420)

experience, especially for the first time they get some

Lex Fridman (51:49.500)

information on the phone. If the fully driverless vehicle is

Dmitri Dolgov (51:54.380)

used to service their trip, when you get into the car, we

Lex Fridman (51:58.060)

have an in car, you know, screen and audio that kind of guides

Dmitri Dolgov (52:01.260)

them and explains what to expect. They also have a button

Lex Fridman (52:05.820)

that they can push that will connect them to, you know, a

Dmitri Dolgov (52:09.660)

real life human being that they can talk to, right, about this

Lex Fridman (52:13.260)

whole process. So that's one aspect of it. There is, you

Dmitri Dolgov (52:16.460)

know, I should mention that there is another function that

Lex Fridman (52:21.100)

humans provide to our cars, but it's not teleoperation. You can

Dmitri Dolgov (52:24.700)

think of it a little bit more like, you know, fleet

Lex Fridman (52:26.620)

assistance, kind of like, you know, traffic control that you

Dmitri Dolgov (52:29.980)

have, where our cars, again, they're responsible on their own

Lex Fridman (52:34.860)

for making all of the decisions, all of the driving decisions

Dmitri Dolgov (52:37.740)

that don't require connectivity. They, you know,

Lex Fridman (52:40.460)

anything that is safety or latency critical is done, you

Dmitri Dolgov (52:44.300)

know, purely autonomously by onboard, our onboard system.

Lex Fridman (52:49.020)

But there are situations where, you know, if connectivity is

Dmitri Dolgov (52:51.260)

available, when a car encounters a particularly challenging

Lex Fridman (52:53.980)

situation, you can imagine like a super hairy scene of an

Dmitri Dolgov (52:57.100)

accident, the cars will do their best, they will recognize that

Lex Fridman (53:00.780)

it's an off nominal situation, they will do their best to come

Dmitri Dolgov (53:05.660)

up with the right interpretation, the best course

Lex Fridman (53:07.500)

of action in that scenario. But if connectivity is available,

Dmitri Dolgov (53:09.980)

they can ask for confirmation from, you know, human

Lex Fridman (53:15.660)

assistant to kind of confirm those actions and perhaps

Dmitri Dolgov (53:19.580)

provide a little bit of kind of contextual information and

Lex Fridman (53:22.140)

guidance. So October 8th was when you're talking about the

Dmitri Dolgov (53:26.380)

was Waymo launched the fully self, the public version of

Lex Fridman (53:33.500)

its fully driverless, that's the right term, I think, service

Dmitri Dolgov (53:38.300)

in Phoenix. Is that October 8th? That's right. It was the

Lex Fridman (53:41.580)

introduction of fully driverless, right, our only

Dmitri Dolgov (53:43.820)

vehicles into our public Waymo One service. Okay, so that's

Lex Fridman (53:47.660)

that's amazing. So it's like anybody can get into Waymo in

Dmitri Dolgov (53:51.420)

Phoenix. So we previously had early people in our early

Lex Fridman (53:57.100)

rider program, taking fully driverless rides in Phoenix.

Lex Fridman (54:01.100)

And just this a little while ago, we opened on October 8th,

Lex Fridman (54:06.220)

we opened that mode of operation to the public. So I

Dmitri Dolgov (54:09.500)

can download the app and go on a ride. There's a lot more

Lex Fridman (54:14.300)

demand right now for that service. And then we have

Dmitri Dolgov (54:17.100)

capacity. So we're kind of managing that. But that's

Lex Fridman (54:20.300)

exactly the way to describe it. Yeah, that's interesting. So

Dmitri Dolgov (54:22.540)

there's more demand than you can handle. Like what has been

Lex Fridman (54:28.700)

reception so far? I mean, okay, so this is a product,

Dmitri Dolgov (54:34.620)

right? That's a whole nother discussion of like how

Lex Fridman (54:38.140)

compelling of a product it is. Great. But it's also like one

Dmitri Dolgov (54:41.420)

of the most kind of transformational technologies of

Lex Fridman (54:43.980)

the 21st century. So it's also like a tourist attraction.

Dmitri Dolgov (54:48.300)

Like it's fun to, you know, to be a part of it. So it'd be

Lex Fridman (54:52.380)

interesting to see like, what do people say? What do people,

Lex Fridman (54:56.540)

what have been the feedback so far? You know, still early

Lex Fridman (54:59.500)

days, but so far, the feedback has been incredible, incredibly

Dmitri Dolgov (55:04.140)

positive. They, you know, we asked them for feedback during

Lex Fridman (55:07.180)

the ride, we asked them for feedback after the ride as part

Dmitri Dolgov (55:10.700)

of their trip. We asked them some questions, we asked them

Lex Fridman (55:12.780)

to rate the performance of our driver. Most by far, you know,

Dmitri Dolgov (55:17.100)

most of our drivers give us five stars in our app, which is

Lex Fridman (55:21.740)

absolutely great to see. And you know, that's and we're

Dmitri Dolgov (55:24.700)

they're also giving us feedback on you know, things we can

Lex Fridman (55:26.620)

improve. And you know, that's that's one of the main reasons

Dmitri Dolgov (55:29.180)

we're doing this as Phoenix and you know, over the last couple

Lex Fridman (55:31.340)

of years, and every day today, we are just learning a

Dmitri Dolgov (55:35.420)

tremendous amount of new stuff from our users. There's there's

Lex Fridman (55:38.300)

no substitute for actually doing the real thing, actually

Dmitri Dolgov (55:41.980)

having a fully driverless product out there in the field

Lex Fridman (55:44.780)

with, you know, users that are actually paying us money to

Dmitri Dolgov (55:48.140)

get from point A to point B. So this is a legitimate like,

Lex Fridman (55:51.740)

there's a paid service. That's right. And the idea is you use

Dmitri Dolgov (55:56.140)

the app to go from point A to point B. And then what what are

Lex Fridman (55:59.340)

the A's? What are the what's the freedom of the of the starting

Lex Fridman (56:03.260)

and ending places? It's an area of geography where that

Lex Fridman (56:07.900)

service is enabled. It's a decent size of geography of

Dmitri Dolgov (56:12.140)

territory. It's actually larger than the size of San Francisco.

Lex Fridman (56:16.300)

And you know, within that, you have full freedom of, you know,

Dmitri Dolgov (56:20.220)

selecting where you want to go. You know, of course, there's

Lex Fridman (56:22.540)

some and you on your app, you get a map, you tell the car

Dmitri Dolgov (56:27.100)

where you want to be picked up, where you want the car to pull

Lex Fridman (56:31.340)

over and pick you up. And then you tell it where you want to

Dmitri Dolgov (56:33.020)

be dropped off. All right. And of course, there are some

Lex Fridman (56:34.940)

exclusions, right? You want to be you know, you were in terms

Dmitri Dolgov (56:37.740)

of where the car is allowed to pull over, right? So that you

Lex Fridman (56:40.860)

can do. But you know, besides that, it's amazing. It's not

Dmitri Dolgov (56:43.820)

like a fixed just would be very I guess. I don't know. Maybe

Lex Fridman (56:45.900)

that's what's the question behind your question. But it's

Dmitri Dolgov (56:47.740)

not a, you know, preset set of yes, I guess. So within the

Lex Fridman (56:51.420)

geographic constraints with that within that area anywhere

Dmitri Dolgov (56:54.220)

else, it can be you can be picked up and dropped off

Lex Fridman (56:56.780)

anywhere. That's right. And you know, people use them on like

Dmitri Dolgov (56:59.740)

all kinds of trips. They we have and we have an incredible

Lex Fridman (57:02.540)

spectrum of riders. We I think the youngest actually have car

Dmitri Dolgov (57:05.340)

seats them and we have, you know, people taking their kids

Lex Fridman (57:07.180)

and rides. I think the youngest riders we had on cars are, you

Dmitri Dolgov (57:09.900)

know, one or two years old, you know, and the full spectrum of

Lex Fridman (57:12.220)

use cases people you can take them to, you know, schools to,

Dmitri Dolgov (57:17.020)

you know, go grocery shopping, to restaurants, to bars, you

Lex Fridman (57:21.500)

know, run errands, you know, go shopping, etc, etc. You can go

Dmitri Dolgov (57:24.220)

to your office, right? Like the full spectrum of use cases,

Lex Fridman (57:27.180)

and people are going to use them in their daily lives to get

Dmitri Dolgov (57:31.740)

around. And we see all kinds of really interesting use cases

Lex Fridman (57:37.020)

and that that that's providing us incredibly valuable

Dmitri Dolgov (57:40.140)

experience that we then, you know, use to improve our

Lex Fridman (57:43.740)

product. So as somebody who's been on done a few long rants

Dmitri Dolgov (57:50.220)

with Joe Rogan and others about the toxicity of the internet

Lex Fridman (57:53.740)

and the comments and the negativity in the comments, I'm

Dmitri Dolgov (57:56.860)

fascinated by feedback. I believe that most people are

Lex Fridman (58:01.740)

good and kind and intelligent and can provide, like, even in

Dmitri Dolgov (58:07.420)

disagreement, really fascinating ideas. So on a product

Lex Fridman (58:11.100)

side, it's fascinating to me, like, how do you get the richest

Dmitri Dolgov (58:14.540)

possible user feedback, like, to improve? What's, what are the

Lex Fridman (58:19.500)

channels that you use to measure? Because, like, you're

Dmitri Dolgov (58:23.980)

no longer, that's one of the magical things about autonomous

Lex Fridman (58:28.540)

vehicles is it's not like it's frictionless interaction with

Dmitri Dolgov (58:32.300)

the human. So like, you don't get to, you know, it's just

Lex Fridman (58:35.820)

giving a ride. So like, how do you get feedback from people

Lex Fridman (58:39.100)

to in order to improve?

Lex Fridman (58:40.780)

Yeah, great question, various mechanisms. So as part of the

Dmitri Dolgov (58:44.940)

normal flow, we ask people for feedback, they as the car is

Lex Fridman (58:48.220)

driving around, we have on the phone and in the car, and we

Dmitri Dolgov (58:51.260)

have a touchscreen in the car, you can actually click some

Lex Fridman (58:54.060)

buttons and provide real time feedback on how the car is

Dmitri Dolgov (58:57.660)

doing, and how the car is handling a particular situation,

Lex Fridman (59:00.460)

you know, both positive and negative. So that's one

Dmitri Dolgov (59:02.540)

channel, we have, as we discussed, customer support or

Lex Fridman (59:05.900)

life help, where, you know, if a customer wants to, has a

Dmitri Dolgov (59:09.020)

question, or he has some sort of concern, they can talk to a

Lex Fridman (59:13.660)

person in real time. So that that is another mechanism that

Dmitri Dolgov (59:16.460)

gives us feedback. At the end of a trip, you know, we also ask

Lex Fridman (59:21.340)

them how things went, they give us comments, and you know, star

Dmitri Dolgov (59:25.100)

rating. And you know, if it's, we also, you know, ask them to

Lex Fridman (59:30.780)

explain what you know, one, well, and you know, what could

Dmitri Dolgov (59:33.420)

be improved. And we have our writers providing very rich

Lex Fridman (59:40.060)

feedback, they're a lot, a large fraction is very passionate,

Dmitri Dolgov (59:44.300)

very excited about this technology. So we get really

Lex Fridman (59:45.980)

good feedback. We also run UXR studies, right, you know,

Dmitri Dolgov (59:49.660)

specific and that are kind of more, you know, go more in

Lex Fridman (59:53.340)

depth. And we will run both kind of lateral and longitudinal

Dmitri Dolgov (59:56.220)

studies, where we have deeper engagement with our customers,