Chris Lattner: The Future of Computing and Programming Languages
技术与编程AI 与机器学习政治与社会心理与人性音乐与艺术
📋 章节目录
暂无章节信息
🔑 关键词
donswiftlanguageprogrammingpythonlearninglanguagescodetalkingharddesigncompilerinterestingstuffdoingtalkbettergotdoesnvalue
💬 精彩语录
暂无语录
🎙️ 完整对话（4046 条）
Lex Fridman (00:00.000)
The following is a conversation with Chris Latner,
以下是与克里斯·拉特纳的对话，
Lex Fridman (00:02.640)
his second time on the podcast.
他第二次参加播客。
Lex Fridman (00:04.680)
He's one of the most brilliant engineers
他是最杰出的工程师之一
Lex Fridman (00:06.600)
in modern computing, having created
在现代计算中，创建了
Lex Fridman (00:08.780)
LLVM compiler infrastructure project,
LLVM编译器基础设施项目，
Chris Lattner (00:11.460)
the Clang compiler, the Swift programming language,
Lex Fridman (00:14.640)
a lot of key contributions to TensorFlow and TPUs
对 TensorFlow 和 TPU 做出了许多重要贡献
Chris Lattner (00:17.640)
as part of Google.
作为谷歌的一部分。
Lex Fridman (00:19.040)
He served as vice president of autopilot software at Tesla,
他曾担任特斯拉自动驾驶软件副总裁，
Chris Lattner (00:23.500)
was a software innovator and leader at Apple,
是苹果公司的软件创新者和领导者，
Lex Fridman (00:26.180)
and now is at SciFive as senior vice president
现在在 SciFive 担任高级副总裁
Chris Lattner (00:29.320)
of platform engineering, looking to revolutionize
平台工程，寻求彻底变革
Lex Fridman (00:32.500)
chip design to make it faster, better, and cheaper.
芯片设计使其更快、更好、更便宜。
Chris Lattner (00:36.560)
Quick mention of each sponsor, followed by some thoughts
快速提及每个赞助商，然后是一些想法
Lex Fridman (00:39.180)
related to the episode.
与情节相关。
Chris Lattner (00:40.900)
First sponsor is Blinkist, an app that summarizes
第一个赞助商是 Blinkist，一个总结的应用程序
Lex Fridman (00:43.480)
key ideas from thousands of books.
来自数千本书的关键思想。
Chris Lattner (00:45.380)
I use it almost every day to learn new things
我几乎每天都用它来学习新东西
Lex Fridman (00:48.020)
or to pick which books I want to read or listen to next.
或者选择我接下来想读或听的书。
Chris Lattner (00:52.280)
Second is Neuro, the maker of functional sugar free gum
其次是功能性无糖口香糖制造商Neuro
Lex Fridman (00:55.860)
and mints that I use to supercharge my mind
Chris Lattner (00:58.520)
with caffeine, altheanine, and B vitamins.
Lex Fridman (01:01.620)
Third is Masterclass, online courses from the best people
Chris Lattner (01:05.940)
in the world on each of the topics covered,
Lex Fridman (01:08.360)
from rockets, to game design, to poker,
Chris Lattner (01:11.140)
to writing, and to guitar.
Lex Fridman (01:13.920)
And finally, Cash App, the app I use to send money
Chris Lattner (01:16.960)
to friends for food, drinks, and unfortunately, lost bets.
Lex Fridman (01:21.780)
Please check out the sponsors in the description
Chris Lattner (01:23.740)
to get a discount and to support this podcast.
Lex Fridman (01:27.320)
As a side note, let me say that Chris has been
Chris Lattner (01:29.800)
an inspiration to me on a human level
Lex Fridman (01:32.560)
because he is so damn good as an engineer
Lex Fridman (01:35.240)
and leader of engineers, and yet he's able to stay humble,
Lex Fridman (01:38.600)
especially humble enough to hear the voices
Chris Lattner (01:41.040)
of disagreement and to learn from them.
Lex Fridman (01:43.800)
He was supportive of me and this podcast
Chris Lattner (01:46.080)
from the early days, and for that, I'm forever grateful.
Lex Fridman (01:49.520)
To be honest, most of my life, no one really believed
Chris Lattner (01:52.280)
that I would amount to much.
Lex Fridman (01:53.920)
So when another human being looks at me,
Chris Lattner (01:56.500)
it makes me feel like I might be someone special,
Lex Fridman (01:58.920)
it can be truly inspiring.
Chris Lattner (02:00.840)
That's a lesson for educators.
Lex Fridman (02:02.780)
The weird kid in the corner with a dream
Chris Lattner (02:05.640)
is someone who might need your love and support
Lex Fridman (02:08.160)
in order for that dream to flourish.
Chris Lattner (02:10.920)
If you enjoy this thing, subscribe on YouTube,
Lex Fridman (02:13.320)
review it with five stars on Apple Podcast,
Chris Lattner (02:15.480)
follow on Spotify, support on Patreon,
Lex Fridman (02:17.960)
or connect with me on Twitter at Lex Friedman.
Lex Fridman (02:21.320)
And now, here's my conversation with Chris Latner.
Lex Fridman (02:24.780)
What are the strongest qualities of Steve Jobs,
Chris Lattner (02:28.960)
Elon Musk, and the great and powerful Jeff Dean
Lex Fridman (02:32.960)
since you've gotten the chance to work with each?
Chris Lattner (02:36.000)
You're starting with an easy question there.
Lex Fridman (02:38.560)
These are three very different people.
Chris Lattner (02:40.680)
I guess you could do maybe a pairwise comparison
Lex Fridman (02:43.840)
between them instead of a group comparison.
Lex Fridman (02:45.720)
So if you look at Steve Jobs and Elon,
Lex Fridman (02:48.200)
I worked a lot more with Elon than I did with Steve.
Chris Lattner (02:51.000)
They have a lot of commonality.
Lex Fridman (02:52.360)
They're both visionary in their own way.
Chris Lattner (02:55.400)
They're both very demanding in their own way.
Lex Fridman (02:58.680)
My sense is Steve is much more human factor focused
Chris Lattner (03:02.400)
where Elon is more technology focused.
Lex Fridman (03:04.620)
What does human factor mean?
Chris Lattner (03:05.960)
Steve's trying to build things that feel good,
Lex Fridman (03:08.440)
that people love, that affect people's lives, how they live.
Chris Lattner (03:11.560)
He's looking into the future a little bit
Lex Fridman (03:14.640)
in terms of what people want.
Chris Lattner (03:17.760)
Where I think that Elon focuses more on
Lex Fridman (03:20.240)
learning how exponentials work and predicting
Chris Lattner (03:22.360)
the development of those.
Lex Fridman (03:24.080)
Steve worked with a lot of engineers.
Chris Lattner (03:26.240)
That was one of the things that are reading the biography.
Lex Fridman (03:29.480)
How can a designer essentially talk to engineers
Lex Fridman (03:33.280)
and get their respect?
Lex Fridman (03:35.580)
I think, so I did not work very closely with Steve.
Chris Lattner (03:37.760)
I'm not an expert at all.
Lex Fridman (03:38.600)
My sense is that he pushed people really hard,
Lex Fridman (03:41.840)
but then when he got an explanation that made sense to him,
Lex Fridman (03:44.420)
then he would let go.
Lex Fridman (03:45.720)
And he did actually have a lot of respect for engineering,
Lex Fridman (03:49.200)
but he also knew when to push.
Lex Fridman (03:51.480)
And when you can read people well,
Lex Fridman (03:54.160)
you can know when they're holding back
Lex Fridman (03:56.880)
and when you can get a little bit more out of them.
Lex Fridman (03:58.440)
And I think he was very good at that.
Chris Lattner (04:01.200)
I mean, if you compare the other folks,
Lex Fridman (04:03.240)
so Jeff Dean, right?
Chris Lattner (04:05.180)
Jeff Dean's an amazing guy.
Lex Fridman (04:06.280)
He's super smart, as are the other guys.
Chris Lattner (04:10.440)
Jeff is a really, really, really nice guy, well meaning.
Lex Fridman (04:13.820)
He's a classic Googler.
Chris Lattner (04:15.280)
He wants people to be happy.
Lex Fridman (04:17.720)
He combines it with brilliance
Lex Fridman (04:19.760)
so he can pull people together in a really great way.
Lex Fridman (04:22.580)
He's definitely not a CEO type.
Chris Lattner (04:24.640)
I don't think he would even want to be that.
Lex Fridman (04:28.040)
Do you know if he still programs?
Chris Lattner (04:29.280)
Oh yeah, he definitely programs.
Lex Fridman (04:30.560)
Jeff is an amazing engineer today, right?
Lex Fridman (04:32.840)
And that has never changed.
Lex Fridman (04:34.080)
So it's really hard to compare Jeff to either of those two.
Chris Lattner (04:40.360)
I think that Jeff leads through technology
Lex Fridman (04:43.680)
and building it himself and then pulling people in
Lex Fridman (04:45.800)
and inspiring them.
Lex Fridman (04:46.800)
And so I think that that's one of the amazing things
Chris Lattner (04:50.080)
about Jeff.
Lex Fridman (04:50.920)
But each of these people, with their pros and cons,
Chris Lattner (04:53.240)
all are really inspirational
Lex Fridman (04:55.040)
and have achieved amazing things.
Lex Fridman (04:56.800)
So I've been very fortunate to get to work with these guys.
Lex Fridman (05:00.760)
For yourself, you've led large teams,
Chris Lattner (05:03.880)
you've done so many incredible,
Lex Fridman (05:06.240)
difficult technical challenges.
Chris Lattner (05:08.440)
Is there something you've picked up from them
Lex Fridman (05:10.940)
about how to lead?
Chris Lattner (05:12.560)
Yeah, so I mean, I think leadership is really hard.
Lex Fridman (05:14.700)
It really depends on what you're looking for there.
Chris Lattner (05:17.240)
I think you really need to know what you're talking about.
Lex Fridman (05:20.240)
So being grounded on the product, on the technology,
Chris Lattner (05:23.040)
on the business, on the mission is really important.
Lex Fridman (05:28.360)
Understanding what people are looking for,
Lex Fridman (05:29.880)
why they're there.
Lex Fridman (05:30.800)
One of the most amazing things about Tesla
Lex Fridman (05:32.440)
is the unifying vision, right?
Lex Fridman (05:34.680)
People are there because they believe in clean energy
Lex Fridman (05:37.280)
and electrification, all these kinds of things.
Lex Fridman (05:39.700)
The other is to understand what really motivates people,
Lex Fridman (05:42.700)
how to get the best people,
Lex Fridman (05:43.860)
how to build a plan that actually can be executed, right?
Chris Lattner (05:46.820)
There's so many different aspects of leadership
Lex Fridman (05:48.420)
and it really depends on the time, the place, the problems.
Chris Lattner (05:52.820)
There's a lot of issues that don't need to be solved.
Lex Fridman (05:54.820)
And so if you focus on the right things and prioritize well,
Chris Lattner (05:57.780)
that can really help move things.
Lex Fridman (05:59.380)
Two interesting things you mentioned.
Chris Lattner (06:01.140)
One is you really have to know what you're talking about.
Lex Fridman (06:03.940)
How you've worked on your business,
Chris Lattner (06:08.940)
you've worked on a lot of very challenging technical things.
Lex Fridman (06:12.260)
So I kind of assume you were born technically savvy,
Lex Fridman (06:18.000)
but assuming that's not the case,
Lex Fridman (06:20.760)
how did you develop technical expertise?
Chris Lattner (06:24.980)
Like even at Google you worked on,
Lex Fridman (06:27.380)
I don't know how many projects,
Lex Fridman (06:28.980)
but really challenging, very varied.
Lex Fridman (06:32.260)
Compilers, TPUs, hardware, cloud stuff,
Chris Lattner (06:34.660)
bunch of different things.
Lex Fridman (06:36.420)
The thing that I've become comfortable
Chris Lattner (06:37.780)
as I've more comfortable with as I've gained experience
Lex Fridman (06:42.300)
is being okay with not knowing.
Lex Fridman (06:45.980)
And so a major part of leadership is actually,
Lex Fridman (06:49.100)
it's not about having the right answer,
Chris Lattner (06:50.860)
it's about getting the right answer.
Lex Fridman (06:52.860)
And so if you're working in a team of amazing people, right?
Lex Fridman (06:56.340)
And many of these places, many of these companies
Lex Fridman (06:58.740)
all have amazing people.
Lex Fridman (07:00.320)
It's the question of how do you get people together?
Lex Fridman (07:02.100)
How do you build trust?
Lex Fridman (07:04.140)
How do you get people to open up?
Lex Fridman (07:05.900)
How do you get people to be vulnerable sometimes
Chris Lattner (07:10.000)
with an idea that maybe isn't good enough,
Lex Fridman (07:11.760)
but it's the start of something beautiful?
Lex Fridman (07:13.880)
How do you provide an environment
Lex Fridman (07:17.380)
where you're not just like top down,
Lex Fridman (07:18.820)
thou shalt do the thing that I tell you to do, right?
Lex Fridman (07:21.100)
But you're encouraging people to be part of the solution
Lex Fridman (07:23.720)
and providing a safe space
Lex Fridman (07:26.420)
where if you're not doing the right thing,
Lex Fridman (07:27.900)
they're willing to tell you about it, right?
Lex Fridman (07:29.660)
So you're asking dumb questions?
Chris Lattner (07:31.420)
Yeah, dumb questions are my specialty, yeah.
Lex Fridman (07:33.520)
Well, so I've been in the hardware realm recently
Lex Fridman (07:35.820)
and I don't know much at all about how chips are designed.
Lex Fridman (07:39.060)
I know a lot about using them.
Chris Lattner (07:40.060)
I know some of the principles
Lex Fridman (07:41.100)
and the art's technical level of this,
Lex Fridman (07:43.260)
but it turns out that if you ask a lot of dumb questions,
Lex Fridman (07:47.220)
you get smarter really, really quick.
Lex Fridman (07:48.940)
And when you're surrounded by people that want to teach
Lex Fridman (07:51.040)
and learn themselves, it can be a beautiful thing.
Lex Fridman (07:56.100)
So let's talk about programming languages, if it's okay.
Lex Fridman (07:58.460)
Sure, sure.
Chris Lattner (07:59.300)
At the highest absurd philosophical level,
Lex Fridman (08:01.460)
because I...
Chris Lattner (08:02.300)
Don't get romantic on me, Lex.
Lex Fridman (08:03.380)
I will forever get romantic and torture you, I apologize.
Lex Fridman (08:09.980)
Why do programming languages even matter?
Lex Fridman (08:14.140)
Okay, well, thank you very much.
Chris Lattner (08:15.700)
You're saying why should you care
Lex Fridman (08:17.420)
about any one programming language
Lex Fridman (08:18.620)
or why do we care about programming computers or?
Lex Fridman (08:20.940)
No, why do we care about programming language design,
Chris Lattner (08:25.180)
creating effective programming languages,
Lex Fridman (08:30.060)
choosing one programming languages
Chris Lattner (08:32.620)
such as another programming language,
Lex Fridman (08:34.580)
why we keep struggling and improving
Chris Lattner (08:37.820)
through the evolution of these programming languages.
Lex Fridman (08:39.820)
Sure, sure, sure.
Chris Lattner (08:40.660)
Okay, so I mean, I think you have to come back
Lex Fridman (08:42.100)
to what are we trying to do here, right?
Lex Fridman (08:43.660)
So we have these beasts called computers
Lex Fridman (08:47.120)
that are very good at specific kinds of things
Lex Fridman (08:48.820)
and we think it's useful to have them do it for us, right?
Lex Fridman (08:52.020)
Now you have this question of how best to express that
Chris Lattner (08:55.540)
because you have a human brain still
Lex Fridman (08:57.180)
that has an idea in its head
Lex Fridman (08:58.860)
and you want to achieve something, right?
Lex Fridman (09:00.580)
So, well, there's lots of ways of doing this.
Chris Lattner (09:03.220)
You can go directly to the machine
Lex Fridman (09:04.740)
and speak assembly language
Lex Fridman (09:06.020)
and then you can express directly
Lex Fridman (09:07.660)
what the computer understands, that's fine.
Chris Lattner (09:10.740)
You can then have higher and higher and higher levels
Lex Fridman (09:12.840)
of abstraction up until machine learning
Lex Fridman (09:14.900)
and you're designing a neural net to do the work for you.
Lex Fridman (09:18.060)
The question is where along this way do you want to stop
Lex Fridman (09:21.260)
and what benefits do you get out of doing so?
Lex Fridman (09:23.480)
And so programming languages in general,
Chris Lattner (09:25.300)
you have C, you have Fortran, Java and Ada, Pascal, Swift,
Lex Fridman (09:31.260)
you have lots of different things.
Chris Lattner (09:33.360)
They'll have different trade offs
Lex Fridman (09:34.340)
and they're tackling different parts of the problems.
Chris Lattner (09:36.540)
Now, one of the things that most programming languages do
Lex Fridman (09:39.940)
is they're trying to make it
Lex Fridman (09:40.820)
so that you have pretty basic things
Lex Fridman (09:42.780)
like portability across different hardware.
Lex Fridman (09:45.080)
So you've got, I'm gonna run on an Intel PC,
Lex Fridman (09:47.660)
I'm gonna run on a RISC 5 PC,
Chris Lattner (09:49.220)
I'm gonna run on a ARM phone or something like that, fine.
Lex Fridman (09:53.500)
I wanna write one program and have it portable.
Lex Fridman (09:55.580)
And this is something that assembly doesn't do.
Lex Fridman (09:57.780)
Now, when you start looking
Chris Lattner (09:59.060)
at the space of programming languages,
Lex Fridman (10:00.900)
this is where I think it's fun
Chris Lattner (10:02.460)
because programming languages all have trade offs
Lex Fridman (10:06.180)
and most people will walk up to them
Lex Fridman (10:07.940)
and they look at the surface level of syntax and say,
Lex Fridman (10:11.000)
oh, I like curly braces or I like tabs
Lex Fridman (10:13.860)
or I like semi colons or not or whatever, right?
Lex Fridman (10:17.180)
Subjective, fairly subjective, very shallow things.
Lex Fridman (10:21.300)
But programming languages when done right
Lex Fridman (10:23.180)
can actually be very powerful.
Lex Fridman (10:24.620)
And the benefit they bring is expression.
Lex Fridman (10:30.220)
Okay, and if you look at programming languages,
Chris Lattner (10:32.580)
there's really kind of two different levels to them.
Lex Fridman (10:34.420)
One is the down in the dirt, nuts and bolts
Chris Lattner (10:37.940)
of how do you get the computer to be efficient,
Lex Fridman (10:39.380)
stuff like that, how they work,
Chris Lattner (10:40.660)
type systems, compiler stuff, things like that.
Lex Fridman (10:43.520)
The other is the UI.
Lex Fridman (10:45.860)
And the UI for programming language
Lex Fridman (10:47.220)
is really a design problem
Lex Fridman (10:48.620)
and a lot of people don't think about it that way.
Lex Fridman (10:50.620)
And the UI, you mean all that stuff with the braces and?
Chris Lattner (10:53.660)
Yeah, all that stuff's the UI and what it is
Lex Fridman (10:55.980)
and UI means user interface.
Lex Fridman (10:58.020)
And so what's really going on is
Lex Fridman (11:00.380)
it's the interface between the guts and the human.
Lex Fridman (11:04.340)
And humans are hard, right?
Lex Fridman (11:05.860)
Humans have feelings, they have things they like,
Chris Lattner (11:09.500)
they have things they don't like.
Lex Fridman (11:10.700)
And a lot of people treat programming languages
Chris Lattner (11:12.700)
as though humans are just kind of abstract creatures
Lex Fridman (11:16.300)
that cannot be predicted.
Lex Fridman (11:17.520)
But it turns out that actually there is better and worse.
Lex Fridman (11:21.620)
Like people can tell when a programming language is good
Lex Fridman (11:24.960)
or when it was an accident, right?
Lex Fridman (11:26.860)
And one of the things with Swift in particular
Chris Lattner (11:29.340)
is that a tremendous amount of time
Lex Fridman (11:30.980)
by a tremendous number of people
Chris Lattner (11:33.260)
have been put into really polishing and making it feel good.
Lex Fridman (11:36.660)
But it also has really good nuts and bolts underneath it.
Chris Lattner (11:39.080)
You said that Swift makes a lot of people feel good.
Lex Fridman (11:42.480)
How do you get to that point?
Lex Fridman (11:45.500)
So how do you predict that tens of thousands,
Lex Fridman (11:51.660)
hundreds of thousands of people are going to enjoy
Lex Fridman (11:53.620)
using this user experience of this programming language?
Lex Fridman (11:57.180)
Well, you can look at it in terms of better and worse, right?
Lex Fridman (11:59.540)
So if you have to write lots of boilerplate
Lex Fridman (12:01.340)
or something like that, you will feel unproductive.
Lex Fridman (12:03.540)
And so that's a bad thing.
Lex Fridman (12:05.060)
You can look at it in terms of safety.
Chris Lattner (12:06.700)
If like C for example,
Lex Fridman (12:08.140)
is what's called a memory unsafe language.
Lex Fridman (12:10.060)
And so you get dangling pointers
Lex Fridman (12:11.580)
and you get all these kinds of bugs
Chris Lattner (12:13.300)
that then you have spent tons of time debugging
Lex Fridman (12:15.020)
and it's a real pain in the butt and you feel unproductive.
Lex Fridman (12:17.740)
And so by subtracting these things from the experience,
Lex Fridman (12:19.940)
you get happier people.
Lex Fridman (12:22.620)
But again, keep interrupting.
Lex Fridman (12:25.360)
I'm sorry, but so hard to deal with.
Chris Lattner (12:29.180)
If you look at the people that are most productive
Lex Fridman (12:31.820)
on Stack Overflow, they have a set of priorities
Chris Lattner (12:37.440)
that may not always correlate perfectly
Lex Fridman (12:39.860)
with the experience of the majority of users.
Chris Lattner (12:43.640)
If you look at the most upvoted,
Lex Fridman (12:46.260)
quote unquote, correct answer on Stack Overflow,
Chris Lattner (12:49.100)
it usually really sort of prioritizes
Lex Fridman (12:55.460)
like safe code, proper code, stable code,
Chris Lattner (13:00.820)
you know, that kind of stuff.
Lex Fridman (13:01.860)
As opposed to like,
Lex Fridman (13:02.980)
if I wanna use go to statements in my basic, right?
Lex Fridman (13:08.660)
I wanna use go to statements.
Lex Fridman (13:09.860)
Like what if 99% of people wanna use go to statements?
Lex Fridman (13:12.700)
So you use completely improper, you know, unsafe syntax.
Chris Lattner (13:16.620)
I don't think that people actually,
Lex Fridman (13:17.940)
like if you boil it down and you get below
Chris Lattner (13:19.460)
the surface level, people don't actually care
Lex Fridman (13:21.180)
about go tos or if statements or things like this.
Lex Fridman (13:24.180)
They care about achieving a goal, right?
Lex Fridman (13:26.780)
So the real question is I wanna set up a web server
Lex Fridman (13:30.020)
and I wanna do a thing, whatever.
Lex Fridman (13:32.300)
Like how quickly can I achieve that, right?
Lex Fridman (13:34.300)
And so from a programming language perspective,
Lex Fridman (13:36.460)
there's really two things that matter there.
Chris Lattner (13:39.060)
One is what libraries exist
Lex Fridman (13:41.980)
and then how quickly can you put it together
Lex Fridman (13:44.460)
and what are the tools around that look like, right?
Lex Fridman (13:47.260)
And when you wanna build a library that's missing,
Lex Fridman (13:49.740)
what do you do?
Lex Fridman (13:50.580)
Okay, now this is where you see huge divergence
Lex Fridman (13:53.280)
in the force between worlds, okay?
Lex Fridman (13:55.820)
And so you look at Python, for example.
Chris Lattner (13:57.340)
Python is really good at assembling things,
Lex Fridman (13:59.220)
but it's not so great at building all the libraries.
Lex Fridman (14:02.500)
And so what you get because of performance reasons,
Lex Fridman (14:04.340)
other things like this,
Chris Lattner (14:05.580)
is you get Python layered on top of C, for example,
Lex Fridman (14:09.260)
and that means that doing certain kinds of things
Chris Lattner (14:11.540)
well, it doesn't really make sense to do in Python.
Lex Fridman (14:13.380)
Instead you do it in C and then you wrap it
Lex Fridman (14:15.580)
and then you have, you're living in two worlds
Lex Fridman (14:17.660)
and two worlds never is really great
Chris Lattner (14:19.300)
because tooling and the debugger doesn't work right
Lex Fridman (14:21.900)
and like all these kinds of things.
Lex Fridman (14:23.800)
Can you clarify a little bit what you mean
Lex Fridman (14:25.940)
by Python is not good at building libraries,
Chris Lattner (14:28.580)
meaning it doesn't make it conducive.
Lex Fridman (14:30.460)
Certain kinds of libraries.
Chris Lattner (14:31.540)
No, but just the actual meaning of the sentence,
Lex Fridman (14:35.900)
meaning like it's not conducive to developers
Chris Lattner (14:38.400)
to come in and add libraries
Lex Fridman (14:40.520)
or is it the duality of the,
Chris Lattner (14:44.780)
it's a dance between Python and C and...
Lex Fridman (14:48.100)
Well, so Python's amazing.
Chris Lattner (14:49.460)
Python's a great language.
Lex Fridman (14:50.420)
I did not mean to say that Python is bad for libraries.
Lex Fridman (14:53.420)
What I meant to say is there are libraries
Lex Fridman (14:56.820)
that Python's really good at that you can write in Python,
Lex Fridman (15:00.440)
but there are other things,
Lex Fridman (15:01.280)
like if you wanna build a machine learning framework,
Chris Lattner (15:03.600)
you're not gonna build a machine learning framework
Lex Fridman (15:05.020)
in Python because of performance, for example,
Chris Lattner (15:07.380)
or you want GPU acceleration or things like this.
Lex Fridman (15:10.180)
Instead, what you do is you write a bunch of C
Chris Lattner (15:13.260)
or C++ code or something like that,
Lex Fridman (15:15.300)
and then you talk to it from Python, right?
Lex Fridman (15:18.460)
And so this is because of decisions
Lex Fridman (15:21.100)
that were made in the Python design
Lex Fridman (15:23.140)
and those decisions have other counterbalancing forces.
Lex Fridman (15:27.140)
But the trick when you start looking at this
Chris Lattner (15:29.880)
from a programming language perspective,
Lex Fridman (15:31.340)
you start to say, okay, cool.
Lex Fridman (15:33.220)
How do I build this catalog of libraries
Lex Fridman (15:36.380)
that are really powerful?
Lex Fridman (15:37.860)
And how do I make it so that then they can be assembled
Lex Fridman (15:40.520)
into ways that feel good
Lex Fridman (15:42.100)
and they generally work the first time?
Lex Fridman (15:44.020)
Because when you're talking about building a thing,
Chris Lattner (15:46.900)
you have to include the debugging, the fixing,
Lex Fridman (15:50.220)
the turnaround cycle, the development cycle,
Chris Lattner (15:51.900)
all that kind of stuff
Lex Fridman (15:53.940)
into the process of building the thing.
Chris Lattner (15:56.040)
It's not just about pounding out the code.
Lex Fridman (15:58.300)
And so this is where things like catching bugs
Chris Lattner (16:01.300)
at compile time is valuable, for example.
Lex Fridman (16:04.220)
But if you dive into the details in this,
Chris Lattner (16:07.600)
Swift, for example, has certain things like value semantics,
Lex Fridman (16:10.580)
which is this fancy way of saying
Chris Lattner (16:11.980)
that when you treat a variable like a value,
Lex Fridman (16:18.460)
it acts like a mathematical object would.
Chris Lattner (16:21.460)
Okay, so you have used PyTorch a little bit.
Lex Fridman (16:25.180)
In PyTorch, you have tensors.
Chris Lattner (16:26.620)
Tensors are n dimensional grid of numbers, very simple.
Lex Fridman (16:31.940)
You can do plus and other operators on them.
Chris Lattner (16:34.660)
It's all totally fine.
Lex Fridman (16:35.840)
But why do you need to clone a tensor sometimes?
Lex Fridman (16:39.140)
Have you ever run into that?
Lex Fridman (16:40.820)
Yeah.
Lex Fridman (16:41.660)
Okay, and so why is that?
Lex Fridman (16:42.780)
Why do you need to clone a tensor?
Chris Lattner (16:43.900)
It's the usual object thing that's in Python.
Lex Fridman (16:46.820)
So in Python, and just like with Java
Lex Fridman (16:49.300)
and many other languages, this isn't unique to Python.
Lex Fridman (16:51.540)
In Python, it has a thing called reference semantics,
Chris Lattner (16:53.740)
which is the nerdy way of explaining this.
Lex Fridman (16:55.700)
And what that means is you actually have a pointer
Lex Fridman (16:58.060)
do a thing instead of the thing, okay?
Lex Fridman (17:01.080)
Now, this is due to a bunch of implementation details
Chris Lattner (17:05.220)
that you don't want to go into.
Lex Fridman (17:06.780)
But in Swift, you have this thing called value semantics.
Lex Fridman (17:09.540)
And so when you have a tensor in Swift, it is a value.
Lex Fridman (17:12.140)
If you copy it, it looks like you have a unique copy.
Lex Fridman (17:15.060)
And if you go change one of those copies,
Lex Fridman (17:16.780)
then it doesn't update the other one
Lex Fridman (17:19.340)
because you just made a copy of this thing, right?
Lex Fridman (17:21.420)
So that's like highly error prone
Chris Lattner (17:24.300)
in at least computer science, math centric disciplines
Lex Fridman (17:29.180)
about Python, that like the thing you would expect
Chris Lattner (17:33.520)
to behave like math.
Lex Fridman (17:35.420)
Like math, it doesn't behave like math.
Lex Fridman (17:38.260)
And in fact, quietly it doesn't behave like math
Lex Fridman (17:41.660)
and then can ruin the entirety of your math thing.
Chris Lattner (17:43.220)
Exactly.
Lex Fridman (17:44.060)
Well, and then it puts you in debugging land again.
Chris Lattner (17:45.980)
Yeah.
Lex Fridman (17:46.820)
Right now, you just want to get something done
Lex Fridman (17:48.580)
and you're like, wait a second, where do I need to put clone?
Lex Fridman (17:51.500)
And what level of the stack, which is very complicated,
Chris Lattner (17:54.180)
which I thought I was reusing somebody's library
Lex Fridman (17:56.780)
and now I need to understand it
Lex Fridman (17:57.860)
to know where to clone a thing, right?
Lex Fridman (17:59.620)
And hard to debug, by the way.
Chris Lattner (18:01.280)
Exactly, right.
Lex Fridman (18:02.120)
And so this is where programming languages really matter.
Chris Lattner (18:04.340)
Right, and so in Swift having value semantics
Lex Fridman (18:06.300)
so that both you get the benefit of math,
Lex Fridman (18:10.300)
working like math, right?
Lex Fridman (18:12.340)
But also the efficiency that comes with certain advantages
Chris Lattner (18:15.120)
there, certain implementation details there
Lex Fridman (18:17.340)
really benefit you as a programmer, right?
Lex Fridman (18:18.860)
Can you clarify the value semantics?
Lex Fridman (18:20.620)
Like how do you know that a thing should be treated
Lex Fridman (18:22.900)
like a value?
Lex Fridman (18:23.740)
Yeah, so Swift has a pretty strong culture
Lex Fridman (18:27.740)
and good language support for defining values.
Lex Fridman (18:30.420)
And so if you have an array,
Lex Fridman (18:31.960)
so tensors are one example that the machine learning folks
Lex Fridman (18:34.860)
are very used to.
Chris Lattner (18:36.500)
Just think about arrays, same thing,
Lex Fridman (18:38.280)
where you have an array, you create an array,
Chris Lattner (18:41.640)
you put two or three or four things into it,
Lex Fridman (18:43.900)
and then you pass it off to another function.
Lex Fridman (18:46.940)
What happens if that function adds some more things to it?
Lex Fridman (18:51.380)
Well, you'll see it on the side that you pass it in, right?
Chris Lattner (18:54.300)
This is called reference semantics.
Lex Fridman (18:56.680)
Now, what if you pass an array off to a function,
Chris Lattner (19:01.220)
it scrolls it away in some dictionary
Lex Fridman (19:02.860)
or some other data structure somewhere, right?
Chris Lattner (19:04.880)
Well, it thought that you just handed it that array,
Lex Fridman (19:07.980)
then you return back and that reference to that array
Chris Lattner (19:10.780)
still exists in the caller,
Lex Fridman (19:12.820)
and they go and put more stuff in it, right?
Chris Lattner (19:15.780)
The person you handed it off to
Lex Fridman (19:17.860)
may have thought they had the only reference to that,
Lex Fridman (19:20.260)
and so they didn't know that this was gonna change
Lex Fridman (19:22.860)
underneath the covers.
Lex Fridman (19:23.940)
And so this is where you end up having to do clone.
Lex Fridman (19:26.220)
So like I was passed a thing,
Chris Lattner (19:27.820)
I'm not sure if I have the only version of it,
Lex Fridman (19:30.240)
so now I have to clone it.
Lex Fridman (19:32.260)
So what value semantics does is it allows you to say,
Lex Fridman (19:34.680)
hey, I have a, so in Swift, it defaults to value semantics.
Chris Lattner (19:38.380)
Oh, so it defaults to value semantics,
Lex Fridman (19:40.260)
and then because most things
Chris Lattner (19:42.460)
should end up being like values,
Lex Fridman (19:44.100)
then it makes sense for that to be the default.
Lex Fridman (19:46.100)
And one of the important things about that
Lex Fridman (19:47.240)
is that arrays and dictionaries
Lex Fridman (19:48.740)
and all these other collections
Lex Fridman (19:49.940)
that are aggregations of other things
Chris Lattner (19:51.300)
also have value semantics.
Lex Fridman (19:53.020)
And so when you pass this around
Chris Lattner (19:55.060)
to different parts of your program,
Lex Fridman (19:56.680)
you don't have to do these defensive copies.
Lex Fridman (19:59.180)
And so this is great for two sides, right?
Lex Fridman (1:00:00.040)
they could theoretically do whatever.
Chris Lattner (1:00:02.140)
There's two reasons for that, in my opinion,
Lex Fridman (1:00:04.460)
or in many cases, it's always different.
Lex Fridman (1:00:07.340)
But one of which is they weren't there
Lex Fridman (1:00:09.760)
for all the decisions that were made.
Lex Fridman (1:00:11.620)
And so they don't know the principles
Lex Fridman (1:00:13.360)
in which those decisions were made.
Lex Fridman (1:00:15.340)
And once the principles change,
Lex Fridman (1:00:17.620)
you should be obligated to change what you're doing
Lex Fridman (1:00:20.740)
and change direction, right?
Lex Fridman (1:00:22.700)
And so if you don't know how you got to where you are,
Chris Lattner (1:00:25.860)
it just seems like gospel
Lex Fridman (1:00:27.400)
and you're not gonna question it.
Chris Lattner (1:00:29.820)
You may not understand
Lex Fridman (1:00:30.740)
that it really is the right thing to do,
Lex Fridman (1:00:32.420)
so you just may not see it.
Lex Fridman (1:00:33.460)
That's so brilliant.
Chris Lattner (1:00:34.300)
I never thought of it that way.
Lex Fridman (1:00:35.940)
Like it's so much higher burden
Chris Lattner (1:00:38.660)
when as a leader you step into a thing
Lex Fridman (1:00:40.400)
that's already worked for a long time.
Chris Lattner (1:00:41.740)
Yeah, yeah.
Lex Fridman (1:00:42.560)
Well, and if you change it and it doesn't work out,
Chris Lattner (1:00:44.100)
now you're the person who screwed it up.
Lex Fridman (1:00:46.340)
People always second guess it.
Chris Lattner (1:00:47.660)
Yeah.
Lex Fridman (1:00:48.500)
And the second thing is that
Chris Lattner (1:00:49.340)
even if you decide to make a change,
Lex Fridman (1:00:51.140)
even if you're theoretically in charge,
Chris Lattner (1:00:53.540)
you're just a person that thinks they're in charge.
Lex Fridman (1:00:57.480)
Meanwhile, you have to motivate the troops.
Chris Lattner (1:00:58.860)
You have to explain it to them
Lex Fridman (1:00:59.700)
in terms they'll understand.
Chris Lattner (1:01:00.540)
You have to get them to buy into it and believe in it,
Lex Fridman (1:01:02.180)
because if they don't,
Chris Lattner (1:01:03.620)
then they're not gonna be able to make the turn
Lex Fridman (1:01:05.940)
even if you tell them their bonuses are gonna be curtailed.
Lex Fridman (1:01:08.460)
They're just not gonna like buy into it, you know?
Lex Fridman (1:01:10.700)
And so there's only so much power you have as a leader,
Lex Fridman (1:01:12.980)
and you have to understand what those limitations are.
Lex Fridman (1:01:16.400)
Are you still BDFL?
Chris Lattner (1:01:18.220)
You've been a BDFL of some stuff.
Lex Fridman (1:01:21.540)
You're very heavy on the B,
Chris Lattner (1:01:24.660)
the benevolent, benevolent dictator for life.
Lex Fridman (1:01:27.900)
I guess LLVM?
Chris Lattner (1:01:29.180)
Yeah, so I still lead the LLVM world.
Lex Fridman (1:01:32.560)
I mean, what's the role of,
Lex Fridman (1:01:35.460)
so then on Swift you said that there's a group of people.
Lex Fridman (1:01:38.460)
Yeah, so if you contrast Python with Swift, right,
Chris Lattner (1:01:41.660)
one of the reasons,
Lex Fridman (1:01:43.420)
so everybody on the core team takes the role
Chris Lattner (1:01:45.620)
really seriously, and I think we all really care
Lex Fridman (1:01:47.580)
about where Swift goes,
Lex Fridman (1:01:49.220)
but you're almost delegating the final decision making
Lex Fridman (1:01:52.940)
to the wisdom of the group,
Lex Fridman (1:01:54.940)
and so it doesn't become personal.
Lex Fridman (1:01:57.580)
And also, when you're talking with the community,
Lex Fridman (1:01:59.620)
so yeah, some people are very annoyed
Lex Fridman (1:02:02.060)
as certain decisions get made.
Chris Lattner (1:02:04.300)
There's a certain faith in the process,
Lex Fridman (1:02:06.260)
because it's a very transparent process,
Lex Fridman (1:02:08.100)
and when a decision gets made,
Lex Fridman (1:02:09.940)
a full rationale is provided, things like this.
Chris Lattner (1:02:12.180)
These are almost defense mechanisms
Lex Fridman (1:02:14.420)
to help both guide future discussions
Lex Fridman (1:02:16.500)
and provide case law, kind of like Supreme Court does
Lex Fridman (1:02:18.820)
about this decision was made for this reason,
Lex Fridman (1:02:20.980)
and here's the rationale
Lex Fridman (1:02:21.900)
and what we want to see more of or less of.
Lex Fridman (1:02:25.540)
But it's also a way to provide a defense mechanism,
Lex Fridman (1:02:27.600)
so that when somebody's griping about it,
Chris Lattner (1:02:28.980)
they're not saying that person did the wrong thing.
Lex Fridman (1:02:31.980)
They're saying, well, this thing sucks,
Lex Fridman (1:02:33.980)
and later they move on and they get over it.
Lex Fridman (1:02:38.540)
Yeah, the analogy of the Supreme Court,
Chris Lattner (1:02:40.140)
I think, is really good.
Lex Fridman (1:02:42.660)
But then, okay, not to get personal on the SWIFT team,
Lex Fridman (1:02:45.700)
but it just seems like it's impossible
Lex Fridman (1:02:50.020)
for division not to emerge.
Chris Lattner (1:02:52.820)
Well, each of the humans on the SWIFT Core Team,
Lex Fridman (1:02:55.340)
for example, are different,
Lex Fridman (1:02:56.980)
and the membership of the SWIFT Core Team
Lex Fridman (1:02:58.380)
changes slowly over time, which is, I think, a healthy thing.
Lex Fridman (1:03:02.340)
And so each of these different humans
Lex Fridman (1:03:04.020)
have different opinions.
Chris Lattner (1:03:05.220)
Trust me, it's not a singular consciousness
Lex Fridman (1:03:09.380)
by any stretch of the imagination.
Chris Lattner (1:03:11.000)
You've got three major organizations,
Lex Fridman (1:03:12.840)
including Apple, Google, and SciFive,
Chris Lattner (1:03:14.580)
all kind of working together.
Lex Fridman (1:03:16.380)
And it's a small group of people, but you need high trust.
Chris Lattner (1:03:20.180)
You need, again, it comes back to the principles
Lex Fridman (1:03:21.940)
of what you're trying to achieve
Lex Fridman (1:03:23.360)
and understanding what you're optimizing for.
Lex Fridman (1:03:27.480)
And I think that starting with strong principles
Lex Fridman (1:03:30.500)
and working towards decisions
Lex Fridman (1:03:32.300)
is always a good way to both make wise decisions in general
Lex Fridman (1:03:36.300)
but then be able to communicate them to people
Lex Fridman (1:03:37.940)
so that they can buy into them.
Lex Fridman (1:03:39.300)
And that is hard.
Lex Fridman (1:03:41.420)
And so you mentioned LLVM.
Chris Lattner (1:03:42.700)
LLVM is gonna be 20 years old this December,
Lex Fridman (1:03:46.780)
so it's showing its own age.
Lex Fridman (1:03:49.500)
Do you have like a dragon cake plan?
Lex Fridman (1:03:53.580)
No, I should definitely do that.
Chris Lattner (1:03:54.740)
Yeah, if we can have a pandemic cake.
Lex Fridman (1:03:57.820)
Pandemic cake.
Chris Lattner (1:03:58.980)
Everybody gets a slice of cake
Lex Fridman (1:04:00.380)
and it gets sent through email.
Lex Fridman (1:04:04.300)
But LLVM has had tons of its own challenges
Lex Fridman (1:04:08.140)
over time too, right?
Lex Fridman (1:04:09.160)
And one of the challenges that the LLVM community has,
Lex Fridman (1:04:12.620)
in my opinion, is that it has a whole bunch of people
Lex Fridman (1:04:15.220)
that have been working on LLVM for 10 years, right?
Lex Fridman (1:04:19.060)
Because this happens somehow.
Lex Fridman (1:04:20.900)
And LLVM has always been one way,
Lex Fridman (1:04:22.780)
but it needs to be a different way, right?
Lex Fridman (1:04:25.060)
And they've worked on it for like 10 years.
Lex Fridman (1:04:26.620)
It's a long time to work on something.
Lex Fridman (1:04:28.540)
And you suddenly can't see the faults
Lex Fridman (1:04:32.140)
in the thing that you're working on.
Lex Fridman (1:04:33.460)
And LLVM has lots of problems and we need to address them
Lex Fridman (1:04:35.780)
and we need to make it better.
Lex Fridman (1:04:36.700)
And if we don't make it better,
Lex Fridman (1:04:37.700)
then somebody else will come up with a better idea, right?
Lex Fridman (1:04:40.220)
And so it's just kind of of that age
Lex Fridman (1:04:42.500)
where the community is like in danger
Chris Lattner (1:04:45.060)
of getting too calcified.
Lex Fridman (1:04:46.580)
And so I'm happy to see new projects joining
Lex Fridman (1:04:50.420)
and new things mixing it up.
Lex Fridman (1:04:51.980)
Fortran is now a new thing in the LLVM community,
Chris Lattner (1:04:54.520)
which is hilarious and good.
Lex Fridman (1:04:56.300)
I've been trying to find, on a little tangent,
Chris Lattner (1:04:58.980)
find people who program in Cobalt or Fortran,
Lex Fridman (1:05:02.340)
Fortran especially, to talk to, they're hard to find.
Chris Lattner (1:05:06.460)
Yeah, look to the scientific community.
Lex Fridman (1:05:09.820)
They still use Fortran quite a bit.
Chris Lattner (1:05:11.660)
Well, interesting thing you kind of mentioned with LLVM,
Lex Fridman (1:05:14.260)
or just in general, that as something evolves,
Chris Lattner (1:05:16.980)
you're not able to see the faults.
Lex Fridman (1:05:19.700)
So do you fall in love with the thing over time?
Chris Lattner (1:05:23.100)
Or do you start hating everything
Lex Fridman (1:05:24.580)
about the thing over time?
Chris Lattner (1:05:26.300)
Well, so my personal folly is that I see,
Lex Fridman (1:05:31.020)
maybe not all, but many of the faults,
Lex Fridman (1:05:33.460)
and they grate on me, and I don't have time to go fix them.
Lex Fridman (1:05:35.580)
Yeah, and they get magnified over time.
Chris Lattner (1:05:37.540)
Well, and they may not get magnified,
Lex Fridman (1:05:38.900)
but they never get fixed.
Lex Fridman (1:05:39.740)
And it's like sand underneath,
Lex Fridman (1:05:41.300)
you know, it's just like grating against you.
Lex Fridman (1:05:43.540)
And it's like sand underneath your fingernails or something.
Lex Fridman (1:05:45.820)
It's just like, you know it's there,
Chris Lattner (1:05:46.880)
you can't get rid of it.
Lex Fridman (1:05:49.660)
And so the problem is that if other people don't see it,
Chris Lattner (1:05:52.980)
like I don't have time to go write the code
Lex Fridman (1:05:56.860)
and fix it anymore,
Lex Fridman (1:05:58.420)
but then people are resistant to change.
Lex Fridman (1:06:01.420)
And so you say, hey, we should go fix this thing.
Chris Lattner (1:06:03.020)
They're like, oh yeah, that sounds risky.
Lex Fridman (1:06:05.260)
It's like, well, is it the right thing or not?
Chris Lattner (1:06:07.180)
Are the challenges the group dynamics,
Lex Fridman (1:06:10.180)
or is it also just technical?
Chris Lattner (1:06:11.620)
I mean, some of these features like,
Lex Fridman (1:06:14.260)
I think as an observer, it's almost like a fan
Chris Lattner (1:06:17.180)
in the, you know, as a spectator of the whole thing,
Lex Fridman (1:06:21.220)
I don't often think about, you know,
Chris Lattner (1:06:23.820)
some things might actually be
Lex Fridman (1:06:24.980)
technically difficult to implement.
Chris Lattner (1:06:27.540)
An example of this is we built this new compiler framework
Lex Fridman (1:06:30.040)
called MLIR.
Chris Lattner (1:06:31.300)
Yes.
Lex Fridman (1:06:32.140)
MLIR is a whole new framework.
Chris Lattner (1:06:34.180)
It's not, many people think it's about machine learning.
Lex Fridman (1:06:37.300)
The ML stands for multi level
Chris Lattner (1:06:39.140)
because compiler people can't name things very well,
Lex Fridman (1:06:41.380)
I guess.
Lex Fridman (1:06:42.220)
Do we dig into what MLIR is?
Lex Fridman (1:06:45.240)
Yeah, so when you look at compilers,
Chris Lattner (1:06:47.700)
compilers have historically been solutions for a given space.
Lex Fridman (1:06:51.700)
So LLVM is a, it's really good for dealing with CPUs,
Chris Lattner (1:06:56.540)
let's just say, at a high level.
Lex Fridman (1:06:58.100)
You look at Java, Java has a JVM.
Chris Lattner (1:07:01.620)
The JVM is very good for garbage collected languages
Lex Fridman (1:07:04.300)
that need dynamic compilation,
Lex Fridman (1:07:05.540)
and it's very optimized for a specific space.
Lex Fridman (1:07:08.380)
And so hotspot is one of the compilers
Chris Lattner (1:07:09.980)
that gets used in that space,
Lex Fridman (1:07:11.000)
and that compiler is really good at that kind of stuff.
Chris Lattner (1:07:14.060)
Usually when you build these domain specific compilers,
Lex Fridman (1:07:16.740)
you end up building the whole thing from scratch
Chris Lattner (1:07:19.620)
for each domain.
Lex Fridman (1:07:22.220)
What's a domain?
Lex Fridman (1:07:23.380)
So what's the scope of a domain?
Lex Fridman (1:07:26.660)
Well, so here I would say, like, if you look at Swift,
Chris Lattner (1:07:29.160)
there's several different parts to the Swift compiler,
Lex Fridman (1:07:31.940)
one of which is covered by the LLVM part of it.
Chris Lattner (1:07:36.100)
There's also a high level piece that's specific to Swift,
Lex Fridman (1:07:39.420)
and there's a huge amount of redundancy
Chris Lattner (1:07:41.540)
between those two different infrastructures
Lex Fridman (1:07:44.060)
and a lot of re implemented stuff
Chris Lattner (1:07:46.380)
that is similar but different.
Lex Fridman (1:07:48.300)
What does LLVM define?
Chris Lattner (1:07:49.980)
LLVM is effectively an infrastructure.
Lex Fridman (1:07:53.020)
So you can mix and match it in different ways.
Chris Lattner (1:07:55.140)
It's built out of libraries.
Lex Fridman (1:07:56.060)
You can use it for different things,
Lex Fridman (1:07:57.620)
but it's really good at CPUs and GPUs.
Lex Fridman (1:07:59.820)
CPUs and like the tip of the iceberg on GPUs.
Chris Lattner (1:08:02.500)
It's not really great at GPUs.
Lex Fridman (1:08:04.340)
Okay.
Lex Fridman (1:08:05.660)
But it turns out. A bunch of languages that.
Lex Fridman (1:08:07.860)
That then use it to talk to CPUs.
Chris Lattner (1:08:10.100)
Got it.
Lex Fridman (1:08:11.060)
And so it turns out there's a lot of hardware out there
Chris Lattner (1:08:13.100)
that is custom accelerators.
Lex Fridman (1:08:14.820)
So machine learning, for example.
Chris Lattner (1:08:16.140)
There are a lot of matrix multiply accelerators
Lex Fridman (1:08:18.780)
and things like this.
Chris Lattner (1:08:20.580)
There's a whole world of hardware synthesis.
Lex Fridman (1:08:22.820)
So we're using MLIR to build circuits.
Chris Lattner (1:08:26.620)
Okay.
Lex Fridman (1:08:27.460)
And so you're compiling for a domain of transistors.
Lex Fridman (1:08:30.860)
And so what MLIR does is it provides
Lex Fridman (1:08:32.460)
a tremendous amount of compiler infrastructure
Chris Lattner (1:08:34.500)
that allows you to build these domain specific compilers
Lex Fridman (1:08:37.500)
in a much faster way and have the result be good.
Chris Lattner (1:08:41.900)
If we're thinking about the future,
Lex Fridman (1:08:44.380)
now we're talking about like ASICs.
Lex Fridman (1:08:45.980)
So anything.
Lex Fridman (1:08:46.900)
Yeah, yeah.
Lex Fridman (1:08:47.740)
So if we project into the future,
Lex Fridman (1:08:50.540)
it's very possible that the number of these kinds of ASICs,
Chris Lattner (1:08:54.500)
very specific infrastructure architecture things
Lex Fridman (1:09:02.740)
like multiplies exponentially.
Chris Lattner (1:09:05.340)
I hope so.
Lex Fridman (1:09:06.340)
So that's MLIR.
Lex Fridman (1:09:08.620)
So what MLIR does is it allows you
Lex Fridman (1:09:10.780)
to build these compilers very efficiently.
Chris Lattner (1:09:13.260)
Right now, one of the things that coming back
Lex Fridman (1:09:15.820)
to the LLVM thing, and then we'll go to hardware,
Chris Lattner (1:09:17.980)
is LLVM is a specific compiler for a specific domain.
Lex Fridman (1:09:23.980)
MLIR is now this very general, very flexible thing
Chris Lattner (1:09:26.860)
that can solve lots of different kinds of problems.
Lex Fridman (1:09:29.260)
So LLVM is a subset of what MLIR does.
Lex Fridman (1:09:32.380)
So MLIR is, I mean, it's an ambitious project then.
Lex Fridman (1:09:35.340)
Yeah, it's a very ambitious project, yeah.
Lex Fridman (1:09:36.980)
And so to make it even more confusing,
Lex Fridman (1:09:39.820)
MLIR has joined the LLVM Umbrella Project.
Lex Fridman (1:09:42.420)
So it's part of the LLVM family.
Lex Fridman (1:09:44.340)
Right.
Lex Fridman (1:09:45.180)
But where this comes full circle is now folks
Lex Fridman (1:09:47.620)
that work on the LLVM part,
Chris Lattner (1:09:49.380)
the classic part that's 20 years old,
Lex Fridman (1:09:51.980)
aren't aware of all the cool new things
Chris Lattner (1:09:54.100)
that have been done in the new thing,
Lex Fridman (1:09:56.140)
that MLIR was built by me and many other people
Chris Lattner (1:09:59.620)
that knew a lot about LLVM,
Lex Fridman (1:10:01.860)
and so we fixed a lot of the mistakes that lived in LLVM.
Lex Fridman (1:10:05.140)
And so now you have this community dynamic
Lex Fridman (1:10:07.140)
where it's like, well, there's this new thing,
Lex Fridman (1:10:08.540)
but it's not familiar, nobody knows it,
Lex Fridman (1:10:10.340)
it feels like it's new, and so let's not trust it.
Lex Fridman (1:10:12.820)
And so it's just really interesting
Lex Fridman (1:10:13.980)
to see the cultural social dynamic that comes out of that.
Lex Fridman (1:10:16.900)
And I think it's super healthy
Lex Fridman (1:10:19.500)
because we're seeing the ideas percolate
Lex Fridman (1:10:21.540)
and we're seeing the technology diffusion happen
Lex Fridman (1:10:24.020)
as people get more comfortable with it,
Chris Lattner (1:10:25.260)
they start to understand things in their own terms.
Lex Fridman (1:10:27.220)
And this just gets to the,
Chris Lattner (1:10:28.820)
it takes a while for ideas to propagate,
Lex Fridman (1:10:31.220)
even though they may be very different
Chris Lattner (1:10:33.980)
than what people are used to.
Lex Fridman (1:10:35.260)
So maybe let's talk about that a little bit,
Chris Lattner (1:10:37.220)
the world of Asics.
Lex Fridman (1:10:38.220)
Yeah.
Chris Lattner (1:10:39.060)
Actually, you have a new role at SciFive.
Lex Fridman (1:10:45.420)
What's that place about?
Lex Fridman (1:10:47.380)
What is the vision for their vision
Lex Fridman (1:10:50.980)
for, I would say, the future of computer?
Chris Lattner (1:10:52.940)
Yeah, so I lead the engineering and product teams at SciFive.
Lex Fridman (1:10:55.900)
SciFive is a company who was founded
Chris Lattner (1:10:59.660)
with this architecture called RISC5.
Lex Fridman (1:11:02.580)
RISC5 is a new instruction set.
Chris Lattner (1:11:04.380)
Instruction sets are the things inside of your computer
Lex Fridman (1:11:06.300)
that tell it how to run things.
Chris Lattner (1:11:08.420)
X86 from Intel and ARM from the ARM company
Lex Fridman (1:11:12.020)
and things like this are other instruction sets.
Chris Lattner (1:11:13.860)
I've talked to, sorry to interrupt,
Lex Fridman (1:11:15.020)
I've talked to Dave Patterson,
Chris Lattner (1:11:15.980)
who's super excited about RISC5.
Lex Fridman (1:11:17.980)
Dave is awesome.
Chris Lattner (1:11:18.860)
Yeah, he's brilliant, yeah.
Lex Fridman (1:11:20.540)
The RISC5 is distinguished by not being proprietary.
Lex Fridman (1:11:24.540)
And so X86 can only be made by Intel and AMD.
Lex Fridman (1:11:28.820)
ARM can only be made by ARM.
Chris Lattner (1:11:30.380)
They sell licenses to build ARM chips to other companies,
Lex Fridman (1:11:33.340)
things like this.
Chris Lattner (1:11:34.180)
MIPS is another instruction set
Lex Fridman (1:11:35.540)
that is owned by the MIPS company, now Wave.
Lex Fridman (1:11:38.300)
And then it gets licensed out, things like that.
Lex Fridman (1:11:40.860)
And so RISC5 is an open standard
Chris Lattner (1:11:43.340)
that anybody can build chips for.
Lex Fridman (1:11:45.140)
And so SciFive was founded by three of the founders
Chris Lattner (1:11:48.220)
of RISC5 that designed and built it in Berkeley,
Lex Fridman (1:11:51.580)
working with Dave.
Lex Fridman (1:11:52.860)
And so that was the genesis of the company.
Lex Fridman (1:11:56.780)
SciFive today has some of the world's best RISC5 cores
Lex Fridman (1:11:59.060)
and we're selling them and that's really great.
Lex Fridman (1:12:01.420)
They're going to tons of products, it's very exciting.
Lex Fridman (1:12:04.020)
So they're taking this thing that's open source
Lex Fridman (1:12:06.100)
and just trying to be or are the best in the world
Chris Lattner (1:12:09.620)
at building these things.
Lex Fridman (1:12:10.780)
Yeah, so here it's the specifications open source.
Chris Lattner (1:12:13.260)
It's like saying TCP IP is an open standard
Lex Fridman (1:12:15.940)
or C is an open standard,
Lex Fridman (1:12:18.020)
but then you have to build an implementation
Lex Fridman (1:12:19.620)
of the standard.
Lex Fridman (1:12:20.780)
And so SciFive, on the one hand, pushes forward
Lex Fridman (1:12:23.660)
and defined and pushes forward the standard.
Chris Lattner (1:12:26.260)
On the other hand, we have implementations
Lex Fridman (1:12:28.100)
that are best in class for different points in the space,
Chris Lattner (1:12:30.980)
depending on if you want a really tiny CPU
Lex Fridman (1:12:33.620)
or if you want a really big, beefy one that is faster,
Lex Fridman (1:12:36.980)
but it uses more area and things like this.
Lex Fridman (1:12:38.860)
What about the actual manufacturer chips?
Lex Fridman (1:12:41.220)
So like, where does that all fit?
Lex Fridman (1:12:43.580)
I'm going to ask a bunch of dumb questions.
Lex Fridman (1:12:45.340)
That's okay, this is how we learn, right?
Lex Fridman (1:12:48.180)
And so the way this works is that there's generally
Chris Lattner (1:12:52.540)
a separation of the people who designed the circuits
Lex Fridman (1:12:55.140)
and then people who manufacture them.
Lex Fridman (1:12:56.900)
And so you'll hear about fabs like TSMC and Samsung
Lex Fridman (1:13:00.780)
and things like this that actually produce the chips,
Lex Fridman (1:13:03.780)
but they take a design coming in
Lex Fridman (1:13:05.820)
and that design specifies how the,
Chris Lattner (1:13:09.940)
you turn code for the chip into little rectangles
Lex Fridman (1:13:16.300)
that then use photolithography to make mask sets
Lex Fridman (1:13:20.260)
and then burn transistors onto a chip
Lex Fridman (1:13:22.260)
or onto a, onto silicon rather.
Chris Lattner (1:13:24.700)
So, and we're talking about mass manufacturing, so.
Lex Fridman (1:13:28.340)
Yeah, they're talking about making hundreds of millions
Chris Lattner (1:13:29.940)
of parts and things like that, yeah.
Lex Fridman (1:13:31.380)
And so the fab handles the volume production,
Chris Lattner (1:13:33.580)
things like that.
Lex Fridman (1:13:34.660)
But when you look at this problem,
Chris Lattner (1:13:37.260)
the interesting thing about the space when you look at it
Lex Fridman (1:13:39.740)
is that these, the steps that you go from designing a chip
Lex Fridman (1:13:44.380)
and writing the quote unquote code for it
Lex Fridman (1:13:46.300)
and things like Verilog and languages like that,
Chris Lattner (1:13:49.220)
down to what you hand off to the fab
Lex Fridman (1:13:51.660)
is a really well studied, really old problem, okay?
Chris Lattner (1:13:56.260)
Tons of people have worked on it.
Lex Fridman (1:13:57.580)
Lots of smart people have built systems and tools.
Chris Lattner (1:14:00.580)
These tools then have generally gone through acquisitions.
Lex Fridman (1:14:03.500)
And so they've ended up at three different major companies
Chris Lattner (1:14:06.180)
that build and sell these tools.
Lex Fridman (1:14:07.780)
They're called the EDA tools
Chris Lattner (1:14:08.980)
like for electronic design automation.
Lex Fridman (1:14:11.660)
The problem with this is you have huge amounts
Chris Lattner (1:14:13.220)
of fragmentation, you have loose standards
Lex Fridman (1:14:17.900)
and the tools don't really work together.
Lex Fridman (1:14:20.060)
So you have tons of duct tape
Lex Fridman (1:14:21.300)
and you have tons of loss productivity.
Chris Lattner (1:14:24.260)
Now these are, these are tools for designing.
Lex Fridman (1:14:26.740)
So the RISC 5 is a instruction.
Lex Fridman (1:14:30.260)
Like what is RISC 5?
Lex Fridman (1:14:32.060)
Like how deep does it go?
Lex Fridman (1:14:33.260)
How much does it touch the hardware?
Lex Fridman (1:14:35.940)
How much does it define how much of the hardware is?
Chris Lattner (1:14:38.460)
Yeah, so RISC 5 is all about given a CPU.
Lex Fridman (1:14:41.900)
So the processor and your computer,
Lex Fridman (1:14:44.900)
how does the compiler like Swift compiler,
Lex Fridman (1:14:47.420)
the C compiler, things like this, how does it make it work?
Lex Fridman (1:14:50.500)
So it's, what is the assembly code?
Lex Fridman (1:14:52.700)
And so you write RISC 5 assembly
Chris Lattner (1:14:54.180)
instead of XA6 assembly, for example.
Lex Fridman (1:14:57.060)
But it's a set of instructions
Chris Lattner (1:14:58.620)
as opposed to instructions.
Lex Fridman (1:15:00.060)
Why do you say it tells you how the compiler works?
Chris Lattner (1:15:03.660)
Sorry, it's what the compiler talks to.
Lex Fridman (1:15:05.380)
Okay. Yeah.
Lex Fridman (1:15:06.220)
And then the tooling you mentioned
Lex Fridman (1:15:08.500)
that the disparate tools are for what?
Chris Lattner (1:15:10.700)
For when you're building a specific chip.
Lex Fridman (1:15:13.340)
So RISC 5. In hardware.
Chris Lattner (1:15:14.860)
In hardware, yeah.
Lex Fridman (1:15:15.740)
So RISC 5, you can buy a RISC 5 core from SciFive
Lex Fridman (1:15:19.140)
and say, hey, I want to have a certain number of,
Lex Fridman (1:15:21.660)
run a certain number of gigahertz.
Chris Lattner (1:15:23.380)
I want it to be this big.
Lex Fridman (1:15:24.660)
I want it to be, have these features.
Chris Lattner (1:15:26.820)
I want to have like, I want floating point or not,
Lex Fridman (1:15:29.860)
for example.
Lex Fridman (1:15:31.860)
And then what you get is you get a description
Lex Fridman (1:15:34.180)
of a CPU with those characteristics.
Chris Lattner (1:15:36.620)
Now, if you want to make a chip,
Lex Fridman (1:15:38.140)
you want to build like an iPhone chip
Lex Fridman (1:15:39.940)
or something like that, right?
Lex Fridman (1:15:41.180)
You have to take both the CPU,
Lex Fridman (1:15:42.740)
but then you have to talk to memory.
Lex Fridman (1:15:44.380)
You have to have timers, IOs, a GPU, other components.
Lex Fridman (1:15:49.300)
And so you need to pull all those things together
Lex Fridman (1:15:51.380)
into what's called an ASIC,
Chris Lattner (1:15:53.900)
an Application Specific Integrated Circuit.
Lex Fridman (1:15:55.500)
So a custom chip.
Lex Fridman (1:15:56.860)
And then you take that design
Lex Fridman (1:15:58.980)
and then you have to transform it into something
Chris Lattner (1:16:00.900)
that the fabs, like TSMC, for example,
Lex Fridman (1:16:03.980)
know how to take to production.
Chris Lattner (1:16:06.740)
Got it.
Lex Fridman (1:16:07.580)
So, but yeah, okay.
Lex Fridman (1:16:08.580)
And so that process, I will,
Lex Fridman (1:16:11.820)
I can't help but see it as, is a big compiler.
Chris Lattner (1:16:15.620)
Yeah, yeah.
Lex Fridman (1:16:16.940)
It's a whole bunch of compilers written
Chris Lattner (1:16:18.860)
without thinking about it through that lens.
Lex Fridman (1:16:21.420)
Isn't the universe a compiler?
Chris Lattner (1:16:23.700)
Yeah, compilers do two things.
Lex Fridman (1:16:26.820)
They represent things and transform them.
Lex Fridman (1:16:29.140)
And so there's a lot of things that end up being compilers.
Lex Fridman (1:16:31.780)
But this is a space where we're talking about design
Lex Fridman (1:16:34.700)
and usability and the way you think about things,
Lex Fridman (1:16:37.460)
the way things compose correctly, it matters a lot.
Lex Fridman (1:16:40.900)
And so SciFi is investing a lot into that space.
Lex Fridman (1:16:43.460)
And we think that there's a lot of benefit
Chris Lattner (1:16:45.900)
that can be made by allowing people to design chips faster,
Lex Fridman (1:16:48.980)
get them to market quicker and scale out
Chris Lattner (1:16:52.020)
because at the alleged end of Moore's law,
Lex Fridman (1:16:56.420)
you've got this problem of you're not getting
Chris Lattner (1:16:59.300)
free performance just by waiting another year
Lex Fridman (1:17:01.980)
for a faster CPU.
Lex Fridman (1:17:03.540)
And so you have to find performance in other ways.
Lex Fridman (1:17:06.540)
And one of the ways to do that is with custom accelerators
Lex Fridman (1:17:09.100)
and other things and hardware.
Lex Fridman (1:17:11.460)
And so, well, we'll talk a little more about ASICs,
Lex Fridman (1:17:17.420)
but do you see that a lot of people,
Lex Fridman (1:17:21.980)
a lot of companies will try to have
Chris Lattner (1:17:25.300)
different sets of requirements
Lex Fridman (1:17:26.980)
that this whole process to go for?
Lex Fridman (1:17:28.380)
So like almost different car companies might use different
Lex Fridman (1:17:32.540)
and like different PC manufacturers.
Lex Fridman (1:17:35.660)
So is RISC 5 in this whole process,
Lex Fridman (1:17:40.660)
is it potentially the future of all computing devices?
Chris Lattner (1:17:44.860)
Yeah, I think that, so if you look at RISC 5
Lex Fridman (1:17:47.460)
and step back from the Silicon side of things,
Chris Lattner (1:17:49.660)
RISC 5 is an open standard.
Lex Fridman (1:17:51.580)
And one of the things that has happened
Chris Lattner (1:17:53.900)
over the course of decades,
Lex Fridman (1:17:55.460)
if you look over the long arc of computing,
Chris Lattner (1:17:57.820)
somehow became decades old.
Lex Fridman (1:17:59.220)
Yeah.
Chris Lattner (1:18:00.060)
Is that you have companies that come and go
Lex Fridman (1:18:02.700)
and you have instruction sets that come and go.
Chris Lattner (1:18:04.900)
Like one example of this out of many is Sun with Spark.
Lex Fridman (1:18:09.900)
Yeah, it's on one way.
Chris Lattner (1:18:11.260)
Spark still lives on at Fujitsu,
Lex Fridman (1:18:12.980)
but we have HP had this instruction set called PA RISC.
Lex Fridman (1:18:18.140)
So PA RISC was this big server business
Lex Fridman (1:18:21.020)
and had tons of customers.
Chris Lattner (1:18:22.900)
They decided to move to this architecture
Lex Fridman (1:18:25.100)
called Itanium from Intel.
Chris Lattner (1:18:27.140)
Yeah.
Lex Fridman (1:18:27.980)
This didn't work out so well.
Chris Lattner (1:18:29.620)
Yeah.
Lex Fridman (1:18:30.460)
Right, and so you have this issue of
Chris Lattner (1:18:32.940)
you're making many billion dollar investments
Lex Fridman (1:18:35.380)
on instruction sets that are owned by a company.
Lex Fridman (1:18:38.220)
And even companies as big as Intel
Lex Fridman (1:18:39.740)
don't always execute as well as they could.
Chris Lattner (1:18:42.460)
They even have their own issues.
Lex Fridman (1:18:44.700)
HP, for example, decided that it wasn't
Chris Lattner (1:18:46.700)
in their best interest to continue investing in the space
Lex Fridman (1:18:48.620)
because it was very expensive.
Lex Fridman (1:18:49.700)
And so they make technology decisions
Lex Fridman (1:18:52.180)
or they make their own business decisions.
Lex Fridman (1:18:54.180)
And this means that as a customer, what do you do?
Lex Fridman (1:18:57.860)
You've sunk all this time, all this engineering,
Chris Lattner (1:18:59.660)
all this software work, all these,
Lex Fridman (1:19:01.300)
you've built other products around them
Lex Fridman (1:19:02.540)
and now you're stuck, right?
Lex Fridman (1:19:05.020)
What RISC 5 does is provide you more optionality
Chris Lattner (1:19:07.580)
in the space because if you buy an implementation
Lex Fridman (1:19:10.860)
of RISC 5 from SciFive, and you should,
Chris Lattner (1:19:13.540)
they're the best ones.
Lex Fridman (1:19:14.380)
Yeah.
Lex Fridman (1:19:16.380)
But if something bad happens to SciFive in 20 years, right?
Lex Fridman (1:19:19.460)
Well, great, you can turn around
Lex Fridman (1:19:21.220)
and buy a RISC 5 core from somebody else.
Lex Fridman (1:19:23.300)
And there's an ecosystem of people
Chris Lattner (1:19:25.020)
that are all making different RISC 5 cores
Lex Fridman (1:19:26.620)
with different trade offs, which means that
Chris Lattner (1:19:29.180)
if you have more than one requirement,
Lex Fridman (1:19:30.620)
if you have a family of products,
Chris Lattner (1:19:31.900)
you can probably find something in the RISC 5 space
Lex Fridman (1:19:34.700)
that fits your needs.
Chris Lattner (1:19:36.020)
Whereas with, if you're talking about XA6, for example,
Lex Fridman (1:19:39.620)
it's Intel's only gonna bother
Lex Fridman (1:19:41.340)
to make certain classes of devices, right?
Lex Fridman (1:19:45.060)
I see, so maybe a weird question,
Lex Fridman (1:19:47.740)
but like if SciFive is like infinitely successful
Lex Fridman (1:19:54.820)
in the next 20, 30 years, what does the world look like?
Lex Fridman (1:19:58.100)
So like how does the world of computing change?
Lex Fridman (1:20:01.900)
So too much diversity in hardware instruction sets,
Chris Lattner (1:20:05.340)
I think is bad.
Lex Fridman (1:20:06.540)
Like we have a lot of people that are using
Chris Lattner (1:20:09.700)
lots of different instruction sets,
Lex Fridman (1:20:10.980)
particularly in the embedded,
Chris Lattner (1:20:12.260)
the like very tiny microcontroller space,
Lex Fridman (1:20:14.340)
the thing in your toaster that are just weird
Lex Fridman (1:20:19.580)
and different for historical reasons.
Lex Fridman (1:20:21.060)
And so the compilers and the tool chains
Lex Fridman (1:20:23.100)
and the languages on top of them aren't there.
Lex Fridman (1:20:27.140)
And so the developers for that software
Chris Lattner (1:20:29.220)
have to use really weird tools
Lex Fridman (1:20:31.060)
because the ecosystem that supports is not big enough.
Lex Fridman (1:20:34.220)
So I expect that will change, right?
Lex Fridman (1:20:35.460)
People will have better tools and better languages,
Chris Lattner (1:20:38.060)
better features everywhere
Lex Fridman (1:20:39.460)
that then can serve as many different points in the space.
Lex Fridman (1:20:43.300)
And I think RISC5 will progressively
Lex Fridman (1:20:46.300)
eat more of the ecosystem because it can scale up,
Chris Lattner (1:20:49.420)
it can scale down, sideways, left, right.
Lex Fridman (1:20:51.620)
It's very flexible and very well considered
Lex Fridman (1:20:53.860)
and well designed instruction set.
Lex Fridman (1:20:56.420)
I think when you look at SciFive tackling silicon
Lex Fridman (1:20:58.820)
and how people build chips,
Lex Fridman (1:21:00.020)
which is a very different space,
Chris Lattner (1:21:03.980)
that's where you say,
Lex Fridman (1:21:05.180)
I think we'll see a lot more custom chips.
Lex Fridman (1:21:07.540)
And that means that you get much more battery life,
Lex Fridman (1:21:09.820)
you get better tuned solutions for your IoT thingy.
Chris Lattner (1:21:16.220)
You get people that move faster,
Lex Fridman (1:21:18.260)
you get the ability to have faster time to market,
Chris Lattner (1:21:20.700)
for example.
Lex Fridman (1:21:21.540)
So how many custom...
Lex Fridman (1:21:22.620)
So first of all, on the IoT side of things,
Lex Fridman (1:21:25.020)
do you see the number of smart toasters
Lex Fridman (1:21:29.100)
increasing exponentially?
Lex Fridman (1:21:30.260)
So, and if you do,
Lex Fridman (1:21:35.460)
like how much customization per toaster is there?
Lex Fridman (1:21:38.940)
Do all toasters in the world run the same silicon,
Chris Lattner (1:21:42.700)
like the same design,
Lex Fridman (1:21:44.060)
or is it different companies have different design?
Lex Fridman (1:21:46.060)
Like how much customization is possible here?
Lex Fridman (1:21:49.700)
Well, a lot of it comes down to cost, right?
Lex Fridman (1:21:52.460)
And so the way that chips work is you end up paying by the...
Lex Fridman (1:21:56.100)
One of the factors is the size of the chip.
Lex Fridman (1:21:58.780)
And so what ends up happening
Lex Fridman (1:22:01.380)
just from an economic perspective is
Chris Lattner (1:22:03.180)
there's only so many chips that get made in a year
Lex Fridman (1:22:06.100)
of a given design.
Lex Fridman (1:22:07.340)
And so often what customers end up having to do
Lex Fridman (1:22:10.220)
is they end up having to pick up a chip that exists
Chris Lattner (1:22:12.260)
that was built for somebody else
Lex Fridman (1:22:14.140)
so that they can then ship their product.
Lex Fridman (1:22:16.500)
And the reason for that
Lex Fridman (1:22:17.340)
is they don't have the volume of the iPhone.
Chris Lattner (1:22:19.220)
They can't afford to build a custom chip.
Lex Fridman (1:22:21.700)
However, what that means is they're now buying
Chris Lattner (1:22:23.820)
an off the shelf chip that isn't really good,
Lex Fridman (1:22:26.900)
isn't a perfect fit for their needs.
Lex Fridman (1:22:28.220)
And so they're paying a lot of money for it
Lex Fridman (1:22:30.060)
because they're buying silicon that they're not using.
Chris Lattner (1:22:33.500)
Well, if you now reduce the cost of designing the chip,
Lex Fridman (1:22:36.580)
now you get a lot more chips.
Lex Fridman (1:22:37.780)
And the more you reduce it,
Lex Fridman (1:22:39.500)
the easier it is to design chips.
Chris Lattner (1:22:42.340)
The more the world keeps evolving
Lex Fridman (1:22:44.340)
and we get more AI accelerators,
Chris Lattner (1:22:45.740)
we get more other things,
Lex Fridman (1:22:46.740)
we get more standards to talk to,
Lex Fridman (1:22:48.780)
we get 6G, right?
Lex Fridman (1:22:50.980)
You get changes in the world
Chris Lattner (1:22:53.180)
that you wanna be able to talk to these different things.
Lex Fridman (1:22:54.780)
There's more diversity in the cross product of features
Chris Lattner (1:22:57.220)
that people want.
Lex Fridman (1:22:58.460)
And that drives differentiated chips
Chris Lattner (1:23:02.140)
in another direction.
Lex Fridman (1:23:03.300)
And so nobody really knows what the future looks like,
Lex Fridman (1:23:05.620)
but I think that there's a lot of silicon in the future.
Lex Fridman (1:23:09.780)
Speaking of the future,
Chris Lattner (1:23:11.180)
you said Moore's law allegedly is dead.
Lex Fridman (1:23:13.740)
So do you agree with Dave Patterson and many folks
Lex Fridman (1:23:20.340)
that Moore's law is dead?
Lex Fridman (1:23:22.100)
Or do you agree with Jim Keller,
Chris Lattner (1:23:23.940)
who's standing at the helm of the pirate ship
Lex Fridman (1:23:28.620)
saying it's still alive?
Chris Lattner (1:23:31.620)
Yeah.
Lex Fridman (1:23:32.460)
Well, so I agree with what they're saying
Lex Fridman (1:23:35.700)
and different people are interpreting
Lex Fridman (1:23:37.780)
the end of Moore's law in different ways.
Chris Lattner (1:23:39.340)
Yeah.
Lex Fridman (1:23:40.180)
So Jim would say,
Chris Lattner (1:23:41.020)
there's another thousand X left in physics
Lex Fridman (1:23:44.180)
and we can continue to squeeze the stone
Lex Fridman (1:23:46.900)
and make it faster and smaller and smaller geometries
Lex Fridman (1:23:50.060)
and all that kind of stuff.
Chris Lattner (1:23:52.340)
He's right.
Lex Fridman (1:23:53.500)
So Jim is absolutely right
Chris Lattner (1:23:55.220)
that there's a ton of progress left
Lex Fridman (1:23:57.820)
and we're not at the limit of physics yet.
Chris Lattner (1:24:01.700)
That's not really what Moore's law is though.
Lex Fridman (1:24:04.940)
If you look at what Moore's law is,
Chris Lattner (1:24:06.620)
is that it's a very simple evaluation of,
Lex Fridman (1:24:10.660)
okay, well you look at the cost per,
Chris Lattner (1:24:13.580)
I think it was cost per area
Lex Fridman (1:24:14.980)
and the most economic point in that space.
Lex Fridman (1:24:17.020)
And if you go look at the now quite old paper
Lex Fridman (1:24:20.020)
that describes this,
Chris Lattner (1:24:21.860)
Moore's law has a specific economic aspect to it
Lex Fridman (1:24:25.460)
and I think this is something
Chris Lattner (1:24:26.380)
that Dave and others often point out.
Lex Fridman (1:24:28.220)
And so on a technicality, that's right.
Chris Lattner (1:24:31.380)
I look at it from,
Lex Fridman (1:24:33.300)
so I can acknowledge both of those viewpoints.
Chris Lattner (1:24:34.980)
They're both right.
Lex Fridman (1:24:35.820)
They're both right.
Chris Lattner (1:24:36.660)
I'll give you a third wrong viewpoint
Lex Fridman (1:24:39.140)
that may be right in its own way,
Chris Lattner (1:24:40.300)
which is single threaded performance
Lex Fridman (1:24:44.060)
doesn't improve like it used to.
Lex Fridman (1:24:46.020)
And it used to be back when you got a,
Lex Fridman (1:24:48.460)
you know, a Pentium 66 or something
Lex Fridman (1:24:50.580)
and the year before you had a Pentium 33
Lex Fridman (1:24:53.820)
and now it's twice as fast, right?
Chris Lattner (1:24:56.740)
Well, it was twice as fast at doing exactly the same thing.
Lex Fridman (1:25:00.380)
Okay, like literally the same program ran twice as fast.
Chris Lattner (1:25:03.820)
You just wrote a check and waited a year, year and a half.
Lex Fridman (1:25:07.020)
Well, so that's what a lot of people think about Moore's law
Lex Fridman (1:25:10.100)
and I think that is dead.
Lex Fridman (1:25:11.780)
And so what we're seeing instead is we're pushing,
Chris Lattner (1:25:15.260)
we're pushing people to write software in different ways.
Lex Fridman (1:25:17.260)
And so we're pushing people to write CUDA
Lex Fridman (1:25:19.060)
so they can get GPU compute
Lex Fridman (1:25:20.980)
and the thousands of cores on GPU.
Chris Lattner (1:25:23.420)
We're talking about C programmers having to use P threads
Lex Fridman (1:25:26.380)
because they now have, you know,
Chris Lattner (1:25:27.860)
a hundred threads or 50 cores in a machine
Lex Fridman (1:25:30.460)
or something like that.
Chris Lattner (1:25:31.980)
You're now talking about machine learning accelerators
Lex Fridman (1:25:33.660)
that are now domain specific.
Lex Fridman (1:25:35.100)
And when you look at these kinds of use cases,
Lex Fridman (1:25:38.460)
you can still get performance
Lex Fridman (1:25:40.460)
and Jim will come up with cool things
Lex Fridman (1:25:42.660)
that utilize the silicon in new ways for sure,
Lex Fridman (1:25:45.780)
but you're also gonna change the programming model.
Lex Fridman (1:25:48.420)
Right.
Lex Fridman (1:25:49.260)
And now when you start talking about changing
Lex Fridman (1:25:50.140)
the programming model,
Chris Lattner (1:25:50.980)
that's when you come back to languages
Lex Fridman (1:25:53.060)
and things like this too,
Chris Lattner (1:25:54.020)
because often what you see is like you take
Lex Fridman (1:25:58.500)
the C programming language, right?
Chris Lattner (1:25:59.820)
The C programming language is designed for CPUs.
Lex Fridman (1:26:03.340)
And so if you want to talk to a GPU,
Lex Fridman (1:26:04.980)
now you're talking to its cousin CUDA, okay?
Lex Fridman (1:26:08.820)
CUDA is a different thing with a different set of tools,
Chris Lattner (1:26:11.900)
a different world, a different way of thinking.
Lex Fridman (1:26:14.380)
And we don't have one world that scales.
Lex Fridman (1:26:16.940)
And I think that we can get there.
Lex Fridman (1:26:18.460)
We can have one world that scales in a much better way.
Lex Fridman (1:26:21.020)
And a small tangent then,
Lex Fridman (1:26:22.500)
I think most programming languages are designed for CPUs,
Chris Lattner (1:26:25.940)
for single core, even just in their spirit,
Lex Fridman (1:26:28.900)
even if they allow for parallelization.
Lex Fridman (1:26:30.460)
So what does it look like for a programming language
Lex Fridman (1:26:34.140)
to have parallelization or massive parallelization
Lex Fridman (1:26:38.660)
as it's like first principle?
Lex Fridman (1:26:41.300)
So the canonical example of this
Chris Lattner (1:26:43.540)
is the hardware design world.
Lex Fridman (1:26:46.380)
So Verilog, VHDL, these kinds of languages,
Chris Lattner (1:26:50.020)
they're what's called a high level synthesis language.
Lex Fridman (1:26:53.500)
This is the thing people design chips in.
Lex Fridman (1:26:56.860)
And when you're designing a chip,
Lex Fridman (1:26:58.140)
it's kind of like a brain where you have infinite parallelism.
Chris Lattner (1:27:02.660)
Like you're like laying down transistors.
Lex Fridman (1:27:05.580)
Transistors are always running, okay?
Lex Fridman (1:27:08.340)
And so you're not saying run this transistor,
Lex Fridman (1:27:10.260)
then this transistor, then this transistor.
Chris Lattner (1:27:12.300)
It's like your brain,
Lex Fridman (1:27:13.140)
like your neurons are always just doing something.
Lex Fridman (1:27:15.300)
They're not clocked, right?
Lex Fridman (1:27:16.780)
They're just doing their thing.
Lex Fridman (1:27:20.180)
And so when you design a chip or when you design a CPU,
Lex Fridman (1:27:23.540)
when you design a GPU, when you design,
Chris Lattner (1:27:25.180)
when you're laying down the transistors,
Lex Fridman (1:27:27.260)
similarly, you're talking about,
Lex Fridman (1:27:28.460)
well, okay, well, how do these things communicate?
Lex Fridman (1:27:31.300)
And so these languages exist.
Chris Lattner (1:27:32.740)
Verilog is a kind of mixed example of that.
Lex Fridman (1:27:36.140)
None of these languages are really great.
Chris Lattner (1:27:37.620)
You have a very low level, yeah.
Lex Fridman (1:27:39.580)
Yeah, they're very low level
Lex Fridman (1:27:40.660)
and abstraction is necessary here.
Lex Fridman (1:27:42.540)
And there's different approaches with that.
Lex Fridman (1:27:44.500)
And it's itself a very complicated world,
Lex Fridman (1:27:47.340)
but it's implicitly parallel.
Lex Fridman (1:27:50.620)
And so having that as the domain that you program towards
Lex Fridman (1:27:56.220)
makes it so that by default, you get parallel systems.
Chris Lattner (1:27:59.460)
If you look at CUDA,
Lex Fridman (1:28:00.300)
CUDA is a point halfway in the space where in CUDA,
Chris Lattner (1:28:03.660)
when you write a CUDA kernel for your GPU,
Lex Fridman (1:28:05.940)
it feels like you're writing a scalar program.
Lex Fridman (1:28:08.100)
So you're like, you have ifs, you have for loops,
Lex Fridman (1:28:10.020)
stuff like this, you're just writing normal code.
Lex Fridman (1:28:12.620)
But what happens outside of that in your driver
Lex Fridman (1:28:14.820)
is that it actually is running you
Lex Fridman (1:28:16.180)
on like a thousand things at once, right?
Lex Fridman (1:28:18.900)
And so it's parallel,
Lex Fridman (1:28:20.580)
but it has pulled it out of the programming model.
Lex Fridman (1:28:23.900)
And so now you as a programmer are working in a simpler world
Lex Fridman (1:28:28.460)
and it's solved that for you, right?
Lex Fridman (1:28:31.540)
How do you take the language like Swift?
Chris Lattner (1:28:36.420)
If we think about GPUs, but also ASICs,
Lex Fridman (1:28:39.060)
maybe if we can dance back and forth
Chris Lattner (1:28:40.900)
between hardware and software.
Lex Fridman (1:28:42.500)
How do you design for these features
Chris Lattner (1:28:46.740)
to be able to program and get a first class citizen
Lex Fridman (1:28:50.060)
to be able to do like Swift for TensorFlow
Chris Lattner (1:28:53.100)
to be able to do machine learning on current hardware,
Lex Fridman (1:28:56.660)
but also future hardware like TPUs
Lex Fridman (1:28:59.700)
and all kinds of ASICs
Lex Fridman (1:29:00.660)
that I'm sure will be popping up more and more.
Chris Lattner (1:29:02.220)
Yeah, well, so a lot of this comes down
Lex Fridman (1:29:04.380)
to this whole idea of having the nuts and bolts
Chris Lattner (1:29:06.540)
underneath the covers that work really well.
Lex Fridman (1:29:08.660)
So you need, if you're talking to TPUs,
Chris Lattner (1:29:10.420)
you need MLIR or XLA or one of these compilers
Lex Fridman (1:29:13.780)
that talks to TPUs to build on top of, okay?
Lex Fridman (1:29:17.420)
And if you're talking to circuits,
Lex Fridman (1:29:19.340)
you need to figure out how to lay down the transistors
Lex Fridman (1:29:21.500)
and how to organize it and how to set up clocking
Lex Fridman (1:29:23.300)
and like all the domain problems
Chris Lattner (1:29:24.460)
that you get with circuits.
Lex Fridman (1:29:27.420)
Then you have to decide how to explain it to a human.
Lex Fridman (1:29:29.780)
What is ZY, right?
Lex Fridman (1:29:31.820)
And if you do it right, that's a library problem,
Chris Lattner (1:29:34.460)
not a language problem.
Lex Fridman (1:29:36.420)
And that works if you have a library or a language
Chris Lattner (1:29:39.060)
which allows your library to write things
Lex Fridman (1:29:42.140)
that feel native in the language by implementing libraries,
Chris Lattner (1:29:45.820)
because then you can innovate in programming models
Lex Fridman (1:29:49.220)
without having to change your syntax again.
Chris Lattner (1:29:51.220)
Like you have to invent new code formatting tools
Lex Fridman (1:29:54.860)
and like all the other things that languages come with.
Lex Fridman (1:29:57.500)
And this gets really interesting.
Lex Fridman (1:29:59.940)
And so if you look at the space,
Chris Lattner (1:30:02.300)
the interesting thing once you separate out syntax
Lex Fridman (1:30:05.820)
becomes what is that programming model?
Lex Fridman (1:30:07.860)
And so do you want the CUDA style?
Lex Fridman (1:30:10.260)
I write one program and it runs many places.
Lex Fridman (1:30:14.340)
Do you want the implicitly parallel model?
Lex Fridman (1:30:16.820)
How do you reason about that?
Lex Fridman (1:30:17.780)
How do you give developers, chip architects,
Lex Fridman (1:30:20.780)
the ability to express their intent?
Lex Fridman (1:30:24.100)
And that comes into this whole design question
Lex Fridman (1:30:26.300)
of how do you detect bugs quickly?
Lex Fridman (1:30:29.180)
So you don't have to tape out a chip
Lex Fridman (1:30:30.260)
to find out it's wrong, ideally, right?
Lex Fridman (1:30:32.620)
How do you, and this is a spectrum,
Lex Fridman (1:30:35.540)
how do you make it so that people feel productive?
Lex Fridman (1:30:38.500)
So their turnaround time is very quick.
Lex Fridman (1:30:40.460)
All these things are really hard problems.
Lex Fridman (1:30:42.420)
And in this world, I think that not a lot of effort
Lex Fridman (1:30:46.100)
has been put into that design problem
Lex Fridman (1:30:48.060)
and thinking about the layering in other pieces.
Lex Fridman (1:30:51.140)
Well, you've, on the topic of concurrency,
Chris Lattner (1:30:53.500)
you've written the Swift concurrency manifest.
Lex Fridman (1:30:55.580)
I think it's kind of interesting.
Chris Lattner (1:30:57.620)
Anything that has the word manifest on it
Lex Fridman (1:31:00.620)
is very interesting.
Lex Fridman (1:31:02.380)
Can you summarize the key ideas of each of the five parts
Lex Fridman (1:31:06.140)
you've written about?
Lex Fridman (1:31:07.380)
So what is a manifesto?
Lex Fridman (1:31:08.900)
Yes.
Lex Fridman (1:31:09.740)
How about, we start there.
Lex Fridman (1:31:11.820)
So in the Swift community, we have this problem,
Chris Lattner (1:31:15.180)
which is on the one hand,
Lex Fridman (1:31:16.100)
you wanna have relatively small proposals
Chris Lattner (1:31:19.300)
that you can kind of fit in your head,
Lex Fridman (1:31:21.420)
you can understand the details at a very fine grain level
Chris Lattner (1:31:24.100)
that move the world forward.
Lex Fridman (1:31:26.020)
But then you also have these big arcs, okay?
Lex Fridman (1:31:28.900)
And often when you're working on something
Lex Fridman (1:31:30.820)
that is a big arc, but you're tackling it in small pieces,
Chris Lattner (1:31:34.060)
you have this question of,
Lex Fridman (1:31:35.180)
how do I know I'm not doing a random walk?
Lex Fridman (1:31:37.580)
Where are we going?
Lex Fridman (1:31:38.780)
How does this add up?
Chris Lattner (1:31:39.740)
Furthermore, when you start the first small step,
Lex Fridman (1:31:43.580)
what terminology do you use?
Lex Fridman (1:31:45.300)
How do we think about it?
Lex Fridman (1:31:46.580)
What is better and worse in the space?
Lex Fridman (1:31:47.940)
What are the principles?
Lex Fridman (1:31:48.780)
What are we trying to achieve?
Lex Fridman (1:31:50.100)
And so what a manifesto in the Swift community does
Lex Fridman (1:31:52.060)
is it starts to say,
Chris Lattner (1:31:53.220)
hey, well, let's step back from the details of everything.
Lex Fridman (1:31:56.620)
Let's paint a broad picture to talk about
Lex Fridman (1:31:58.700)
what we're trying to achieve.
Lex Fridman (1:32:01.300)
Let's give an example design point.
Chris Lattner (1:32:02.780)
Let's try to paint the big picture
Lex Fridman (1:32:05.260)
so that then we can zero in on the individual steps
Lex Fridman (1:32:07.380)
and make sure that we're making good progress.
Lex Fridman (1:32:09.660)
And so the Swift concurrency manifesto
Chris Lattner (1:32:11.220)
is something I wrote three years ago.
Lex Fridman (1:32:13.860)
It's been a while, maybe more.
Chris Lattner (1:32:16.260)
Trying to do that for Swift and concurrency.
Lex Fridman (1:32:19.740)
It starts with some fairly simple things
Chris Lattner (1:32:22.420)
like making the observation that
Lex Fridman (1:32:25.060)
when you have multiple different computers
Lex Fridman (1:32:26.740)
and multiple different threads that are communicating,
Lex Fridman (1:32:28.940)
it's best for them to be asynchronous.
Lex Fridman (1:32:32.020)
And so you need things to be able to run separately
Lex Fridman (1:32:34.540)
and then communicate with each other.
Lex Fridman (1:32:35.820)
And this means asynchrony.
Lex Fridman (1:32:37.420)
And this means that you need a way
Chris Lattner (1:32:38.980)
to modeling asynchronous communication.
Lex Fridman (1:32:41.740)
Many languages have features like this.
Chris Lattner (1:32:43.700)
Async await is a popular one.
Lex Fridman (1:32:45.420)
And so that's what I think is very likely in Swift.
Lex Fridman (1:32:49.220)
But as you start building this tower of abstractions,
Lex Fridman (1:32:51.380)
it's not just about how do you write this,
Chris Lattner (1:32:53.660)
you then reach into the how do you get memory safety
Lex Fridman (1:32:57.460)
because you want correctness,
Chris Lattner (1:32:58.380)
you want debuggability and sanity for developers.
Lex Fridman (1:33:01.700)
And how do you get that memory safety into the language?
Lex Fridman (1:33:06.620)
So if you take a language like Go or C
Lex Fridman (1:33:09.020)
or any of these languages,
Chris Lattner (1:33:10.420)
you get what's called a race condition
Lex Fridman (1:33:11.940)
when two different threads or Go routines or whatever
Lex Fridman (1:33:14.900)
touch the same point in memory, right?
Lex Fridman (1:33:17.380)
This is a huge like maddening problem to debug
Chris Lattner (1:33:21.260)
because it's not reproducible generally.
Lex Fridman (1:33:24.500)
And so there's tools,
Chris Lattner (1:33:25.540)
there's a whole ecosystem of solutions
Lex Fridman (1:33:27.020)
that built up around this,
Lex Fridman (1:33:28.300)
but it's a huge problem
Lex Fridman (1:33:29.700)
when you're writing concurrent code.
Lex Fridman (1:33:31.060)
And so with Swift,
Lex Fridman (1:33:33.100)
this whole value semantics thing is really powerful there
Chris Lattner (1:33:35.460)
because it turns out that math and copies actually work
Lex Fridman (1:33:39.060)
even in concurrent worlds.
Lex Fridman (1:33:40.700)
And so you get a lot of safety just out of the box,
Lex Fridman (1:33:43.300)
but there are also some hard problems.
Lex Fridman (1:33:44.620)
And it talks about some of that.
Lex Fridman (1:33:47.020)
When you start building up to the next level up
Lex Fridman (1:33:48.820)
and you start talking beyond memory safety,
Lex Fridman (1:33:50.540)
you have to talk about what is the programmer model?
Lex Fridman (1:33:53.020)
How does a human think about this?
Lex Fridman (1:33:54.300)
So a developer that's trying to build a program
Chris Lattner (1:33:56.820)
think about this,
Lex Fridman (1:33:57.660)
and it proposes a really old model with a new spin
Chris Lattner (1:34:01.060)
called Actors.
Lex Fridman (1:34:02.100)
Actors are about saying,
Chris Lattner (1:34:03.980)
we have islands of single threadedness logically.
Lex Fridman (1:34:08.180)
So you write something that feels like
Chris Lattner (1:34:09.380)
it's one program running in a unit,
Lex Fridman (1:34:13.260)
and then it communicates asynchronously with other things.
Lex Fridman (1:34:16.740)
And so making that expressive and natural feel good
Lex Fridman (1:34:19.860)
be the first thing you reach for and being safe by default
Chris Lattner (1:34:22.420)
is a big part of the design of that proposal.
Lex Fridman (1:34:25.620)
When you start going beyond that,
Chris Lattner (1:34:26.740)
now you start to say, cool,
Lex Fridman (1:34:27.740)
well, these things that communicate asynchronously,
Chris Lattner (1:34:30.140)
they don't have to share memory.
Lex Fridman (1:34:32.020)
Well, if they don't have to share memory
Lex Fridman (1:34:33.260)
and they're sending messages to each other,
Lex Fridman (1:34:35.100)
why do they have to be in the same process?
Chris Lattner (1:34:38.220)
These things should be able to be in different processes
Lex Fridman (1:34:40.500)
on your machine.
Lex Fridman (1:34:41.740)
And why just processes?
Lex Fridman (1:34:43.060)
Well, why not different machines?
Lex Fridman (1:34:44.660)
And so now you have a very nice gradual transition
Lex Fridman (1:34:47.740)
towards distributed programming.
Lex Fridman (1:34:49.780)
And of course, when you start talking about the big future,
Lex Fridman (1:34:53.060)
the manifesto doesn't go into it,
Lex Fridman (1:34:55.020)
but accelerators are things you talk to asynchronously
Lex Fridman (1:35:00.180)
by sending messages to them.
Lex Fridman (1:35:01.900)
And how do you program those?
Lex Fridman (1:35:03.900)
Well, that gets very interesting.
Chris Lattner (1:35:05.820)
That's not in the proposal.
Lex Fridman (1:35:07.580)
And how much do you wanna make that explicit
Chris Lattner (1:35:12.500)
like the control of that whole process
Lex Fridman (1:35:14.700)
explicit to the program?
Chris Lattner (1:35:15.780)
Yeah, good question.
Lex Fridman (1:35:16.860)
So when you're designing any of these kinds of features
Chris Lattner (1:35:20.540)
or language features or even libraries,
Lex Fridman (1:35:22.900)
you have this really hard trade off you have to make,
Chris Lattner (1:35:25.300)
which is how much is it magic
Lex Fridman (1:35:27.420)
or how much is it in the human's control?
Lex Fridman (1:35:29.740)
How much can they predict and control it?
Lex Fridman (1:35:32.380)
What do you do when the default case is the wrong case?
Lex Fridman (1:35:37.820)
And so when you're designing a system,
Lex Fridman (1:35:39.820)
and so when you're designing a system, I won't name names,
Lex Fridman (1:35:45.060)
but there are systems where it's really easy to get started
Lex Fridman (1:35:50.980)
and then you jump.
Lex Fridman (1:35:52.620)
So let's pick like logo.
Lex Fridman (1:35:54.100)
Okay, so something like this.
Lex Fridman (1:35:55.580)
So it's really easy to get started.
Lex Fridman (1:35:57.140)
It's really designed for teaching kids,
Lex Fridman (1:35:59.540)
but as you get into it, you hit a ceiling
Lex Fridman (1:36:02.060)
and then you can't go any higher.
Lex Fridman (1:36:03.180)
And then what do you do?
Lex Fridman (1:36:04.100)
Well, you have to go switch to a different world
Lex Fridman (1:36:05.580)
and rewrite all your code.
Lex Fridman (1:36:07.140)
And this logo is a silly example here.
Chris Lattner (1:36:09.100)
This exists in many other languages.
Lex Fridman (1:36:11.380)
With Python, you would say like concurrency, right?
Lex Fridman (1:36:15.260)
So Python has the global interpreter block.
Lex Fridman (1:36:17.340)
So threading is challenging in Python.
Lex Fridman (1:36:19.460)
And so if you start writing a large scale application
Lex Fridman (1:36:22.620)
in Python, and then suddenly you need concurrency,
Lex Fridman (1:36:25.140)
you're kind of stuck with a series of bad trade offs, right?
Lex Fridman (1:36:30.420)
There's other ways to go where you say like,
Lex Fridman (1:36:32.220)
foist all the complexity on the user all at once, right?
Lex Fridman (1:36:37.020)
And that's also bad in a different way.
Lex Fridman (1:36:38.820)
And so what I prefer is building a simple model
Lex Fridman (1:36:43.460)
that you can explain that then has an escape hatch.
Lex Fridman (1:36:46.940)
So you get in, you have guardrails,
Lex Fridman (1:36:50.580)
memory safety works like this in Swift,
Chris Lattner (1:36:52.100)
where you can start with, like by default,
Lex Fridman (1:36:54.860)
if you use all the standard things, it's memory safe,
Chris Lattner (1:36:57.140)
you're not gonna shoot your foot off.
Lex Fridman (1:36:58.620)
But if you wanna get a C level pointer to something,
Chris Lattner (1:37:02.300)
you can explicitly do that.
Lex Fridman (1:37:04.300)
But by default, there's guardrails.
Chris Lattner (1:37:07.740)
There's guardrails.
Lex Fridman (1:37:08.900)
Okay, so but like, whose job is it to figure out
Lex Fridman (1:37:14.300)
which part of the code is parallelizable?
Lex Fridman (1:37:17.420)
So in the case of the proposal, it is the human's job.
Lex Fridman (1:37:21.020)
So they decide how to architect their application.
Lex Fridman (1:37:24.220)
And then the runtime in the compiler is very predictable.
Lex Fridman (1:37:29.060)
And so this is in contrast to like,
Lex Fridman (1:37:31.860)
there's a long body of work, including on Fortran
Chris Lattner (1:37:34.180)
for auto parallelizing compilers.
Lex Fridman (1:37:36.020)
And this is an example of a bad thing in my,
Lex Fridman (1:37:39.540)
so as a compiler person, I can drag on compiler people.
Lex Fridman (1:37:42.940)
Often compiler people will say,
Chris Lattner (1:37:45.100)
cool, since I can't change the code,
Lex Fridman (1:37:46.740)
I'm gonna write my compiler that then takes
Chris Lattner (1:37:48.500)
this unmodified code and makes go way faster on this machine.
Lex Fridman (1:37:52.060)
Okay, application, and so it does pattern matching.
Chris Lattner (1:37:55.700)
It does like really deep analysis.
Lex Fridman (1:37:57.780)
Compiler people are really smart.
Lex Fridman (1:37:58.980)
And so they like wanna like do something
Lex Fridman (1:38:00.820)
really clever and tricky.
Lex Fridman (1:38:01.820)
And you get like 10X speed up by taking
Lex Fridman (1:38:04.500)
like an array of structures and turn it
Chris Lattner (1:38:06.540)
into a structure of arrays or something,
Lex Fridman (1:38:08.100)
because it's so much better for memory.
Chris Lattner (1:38:09.340)
Like there's bodies, like tons of tricks.
Lex Fridman (1:38:12.660)
They love optimization.
Chris Lattner (1:38:13.860)
Yeah, you love optimization.
Lex Fridman (1:38:14.980)
Everyone loves optimization.
Chris Lattner (1:38:15.820)
Everyone loves it.
Lex Fridman (1:38:16.660)
Well, and it's this promise of build with my compiler
Lex Fridman (1:38:19.100)
and your thing goes fast, right?
Lex Fridman (1:38:20.980)
But here's the problem, Lex, you write a program,
Chris Lattner (1:38:24.740)
you run it with my compiler, it goes fast.
Lex Fridman (1:38:26.580)
You're very happy.
Chris Lattner (1:38:27.420)
Wow, it's so much faster than the other compiler.
Lex Fridman (1:38:29.500)
Then you go and you add a feature to your program
Chris Lattner (1:38:31.260)
or you refactor some code.
Lex Fridman (1:38:32.700)
And suddenly you got a 10X loss in performance.
Lex Fridman (1:38:35.740)
Well, why?
Lex Fridman (1:38:36.580)
What just happened there?
Lex Fridman (1:38:37.580)
What just happened there is the heuristic,
Lex Fridman (1:38:39.860)
the pattern matching, the compiler,
Chris Lattner (1:38:41.980)
whatever analysis it was doing just got defeated
Lex Fridman (1:38:43.940)
because you didn't inline a function or something, right?
Chris Lattner (1:38:48.220)
As a user, you don't know, you don't wanna know.
Lex Fridman (1:38:50.260)
That was the whole point.
Chris Lattner (1:38:51.100)
You don't wanna know how the compiler works.
Lex Fridman (1:38:52.820)
You don't wanna know how the memory hierarchy works.
Chris Lattner (1:38:54.580)
You don't wanna know how it got parallelized
Lex Fridman (1:38:56.060)
across all these things.
Chris Lattner (1:38:57.380)
You wanted that abstracted away from you,
Lex Fridman (1:38:59.900)
but then the magic is lost as soon as you did something
Lex Fridman (1:39:02.900)
and you fall off a performance cliff.
Lex Fridman (1:39:05.060)
And now you're in this funny position
Lex Fridman (1:39:06.700)
where what do I do?
Lex Fridman (1:39:08.060)
I don't change my code.
Chris Lattner (1:39:09.060)
I don't fix that bug.
Lex Fridman (1:39:10.900)
It costs 10X performance.
Lex Fridman (1:39:12.380)
Now what do I do?
Lex Fridman (1:39:13.660)
Well, this is the problem with unpredictable performance.
Chris Lattner (1:39:16.340)
If you care about performance,
Lex Fridman (1:39:17.420)
predictability is a very important thing.
Lex Fridman (1:39:19.580)
And so what the proposal does is it provides
Lex Fridman (1:39:23.900)
architectural patterns for being able to lay out your code,
Chris Lattner (1:39:26.740)
gives you full control over that,
Lex Fridman (1:39:28.420)
makes it really simple so you can explain it.
Lex Fridman (1:39:30.220)
And then if you wanna scale out in different ways,
Lex Fridman (1:39:34.820)
you have full control over that.
Lex Fridman (1:39:36.620)
So in your sense, the intuition is for a compiler,
Lex Fridman (1:39:39.580)
it's too hard to do automated parallelization.
Chris Lattner (1:39:43.500)
Cause the compilers do stuff automatically
Lex Fridman (1:39:47.660)
that's incredibly impressive for other things,
Lex Fridman (1:39:50.540)
but for parallelization, we're not close to there.
Lex Fridman (1:39:54.660)
Well, it depends on the programming model.
Lex Fridman (1:39:56.300)
So there's many different kinds of compilers.
Lex Fridman (1:39:58.460)
And so if you talk about like a C compiler
Chris Lattner (1:40:00.380)
or Swift compiler or something like that,
Lex Fridman (1:40:01.940)
where you're writing imperative code,
Chris Lattner (1:40:04.980)
parallelizing that and reasoning about all the pointers
Lex Fridman (1:40:07.140)
and stuff like that is a very difficult problem.
Chris Lattner (1:40:10.140)
Now, if you switch domains,
Lex Fridman (1:40:12.220)
so there's this cool thing called machine learning, right?
Lex Fridman (1:40:15.540)
So machine learning nerds among other endearing things
Lex Fridman (1:40:18.940)
like solving cat detectors and other things like that
Chris Lattner (1:40:23.380)
have done this amazing breakthrough
Lex Fridman (1:40:25.420)
of producing a programming model,
Chris Lattner (1:40:27.540)
operations that you compose together
Lex Fridman (1:40:30.260)
that has raised levels of abstraction high enough
Chris Lattner (1:40:33.180)
that suddenly you can have auto parallelizing compilers.
Lex Fridman (1:40:36.780)
You can write a model using a TensorFlow
Lex Fridman (1:40:39.620)
and have it run on 1024 nodes of a TPU.
Lex Fridman (1:40:43.460)
Yeah, that's true.
Chris Lattner (1:40:44.300)
I didn't even think about like,
Lex Fridman (1:40:46.860)
cause there's so much flexibility
Chris Lattner (1:40:48.220)
in the design of architectures that ultimately boil down
Lex Fridman (1:40:50.860)
to a graph that's parallelized for you.
Lex Fridman (1:40:54.180)
And if you think about it, that's pretty cool.
Lex Fridman (1:40:56.660)
That's pretty cool, yeah.
Lex Fridman (1:40:57.660)
And you think about batching, for example,
Lex Fridman (1:40:59.780)
as a way of being able to exploit more parallelism.
Chris Lattner (1:41:02.460)
Like that's a very simple thing that now is very powerful.
Lex Fridman (1:41:05.420)
That didn't come out of the programming language nerds,
Chris Lattner (1:41:08.020)
those people, like that came out of people
Lex Fridman (1:41:10.140)
that are just looking to solve a problem
Lex Fridman (1:41:11.460)
and use a few GPUs and organically developed
Lex Fridman (1:41:14.060)
by the community of people focusing on machine learning.
Lex Fridman (1:41:16.860)
And it's an incredibly powerful abstraction layer
Lex Fridman (1:41:19.900)
that enables the compiler people to go and exploit that.
Lex Fridman (1:41:22.820)
And now you can drive supercomputers from Python.
Lex Fridman (1:41:26.420)
Well, that's pretty cool.
Chris Lattner (1:41:27.580)
That's amazing.
Lex Fridman (1:41:28.420)
So just to pause on that,
Chris Lattner (1:41:30.780)
cause I'm not sufficiently low level,
Lex Fridman (1:41:32.340)
I forget to admire the beauty and power of that,
Lex Fridman (1:41:35.420)
but maybe just to linger on it,
Lex Fridman (1:41:38.540)
like what does it take to run a neural network fast?
Lex Fridman (1:41:42.660)
Like how hard is that compilation?
Lex Fridman (1:41:44.100)
It's really hard.
Lex Fridman (1:41:45.700)
So we just skipped,
Lex Fridman (1:41:46.940)
you said like, it's amazing that that's a thing,
Lex Fridman (1:41:49.620)
but yeah, how hard is that of a thing?
Lex Fridman (1:41:51.540)
It's hard and I would say that not all of the systems
Chris Lattner (1:41:55.380)
are really great, including the ones I helped build.
Lex Fridman (1:41:58.620)
So there's a lot of work left to be done there.
Chris Lattner (1:42:00.740)
Is it the compiler nerds working on that
Lex Fridman (1:42:02.340)
or is it a whole new group of people?
Chris Lattner (1:42:04.620)
Well, it's a full stack problem,
Lex Fridman (1:42:05.900)
including compiler people, including APIs,
Lex Fridman (1:42:09.180)
so like Keras and the module API and PyTorch and Jax.
Lex Fridman (1:42:14.660)
And there's a bunch of people pushing
Chris Lattner (1:42:15.980)
on all the different parts of these things,
Lex Fridman (1:42:17.460)
because when you look at it as it's both,
Lex Fridman (1:42:19.660)
how do I express the computation?
Lex Fridman (1:42:21.300)
Do I stack up layers?
Chris Lattner (1:42:22.900)
Well, cool, like setting up a linear sequence of layers
Lex Fridman (1:42:25.620)
is great for the simple case,
Lex Fridman (1:42:26.780)
but how do I do the hard case?
Lex Fridman (1:42:28.220)
How do I do reinforcement learning?
Chris Lattner (1:42:29.500)
Well, now I need to integrate my application logic in this.
Lex Fridman (1:42:32.660)
Then it's the next level down of,
Lex Fridman (1:42:34.660)
how do you represent that for the runtime?
Lex Fridman (1:42:36.700)
How do you get hardware abstraction?
Lex Fridman (1:42:39.100)
And then you get to the next level down of saying like,
Lex Fridman (1:42:40.780)
forget about abstraction,
Lex Fridman (1:42:41.860)
how do I get the peak performance out of my TPU
Lex Fridman (1:42:44.540)
or my iPhone accelerator or whatever, right?
Lex Fridman (1:42:47.620)
And all these different things.
Lex Fridman (1:42:48.940)
And so this is a layered problem
Chris Lattner (1:42:50.260)
with a lot of really interesting design and work
Lex Fridman (1:42:53.620)
going on in the space
Lex Fridman (1:42:54.540)
and a lot of really smart people working on it.
Lex Fridman (1:42:56.940)
Machine learning is a very well funded area
Chris Lattner (1:42:59.460)
of investment right now.
Lex Fridman (1:43:00.820)
And so there's a lot of progress being made.
Lex Fridman (1:43:02.940)
So how much innovation is there on the lower level,
Lex Fridman (1:43:05.900)
so closer to the ASIC,
Lex Fridman (1:43:08.220)
so redesigning the hardware
Lex Fridman (1:43:09.780)
or redesigning concurrently compilers with that hardware?
Chris Lattner (1:43:13.180)
Is that like, if you were to predict the biggest,
Lex Fridman (1:43:16.100)
the equivalent of Moore's law improvements
Chris Lattner (1:43:20.540)
in the inference and the training of neural networks
Lex Fridman (1:43:24.620)
and just all of that,
Lex Fridman (1:43:25.700)
where is that gonna come from, you think?
Lex Fridman (1:43:26.900)
Sure, you get scalability of different things.
Lex Fridman (1:43:28.900)
And so you get Jim Keller shrinking process technology,
Lex Fridman (1:43:33.620)
you get three nanometer instead of five or seven or 10
Chris Lattner (1:43:36.620)
or 28 or whatever.
Lex Fridman (1:43:38.100)
And so that marches forward and that provides improvements.
Chris Lattner (1:43:41.300)
You get architectural level performance.
Lex Fridman (1:43:44.060)
And so the TPU with a matrix multiply unit
Lex Fridman (1:43:47.660)
and a systolic array is much more efficient
Lex Fridman (1:43:49.620)
than having a scalar core doing multiplies and adds
Lex Fridman (1:43:53.220)
and things like that.
Lex Fridman (1:43:54.380)
You then get system level improvements.
Lex Fridman (1:43:58.620)
So how you talk to memory,
Lex Fridman (1:43:59.860)
how you talk across a cluster of machines,
Lex Fridman (1:44:02.340)
how you scale out,
Lex Fridman (1:44:03.300)
how you have fast interconnects between machines.
Chris Lattner (1:44:06.060)
You then get system level programming models.
Lex Fridman (1:44:08.780)
So now that you have all this hardware, how to utilize it.
Chris Lattner (1:44:11.300)
You then have algorithmic breakthroughs where you say,
Lex Fridman (1:44:13.140)
hey, wow, cool.
Chris Lattner (1:44:14.340)
Instead of training in a resonant 50 in a week,
Lex Fridman (1:44:18.860)
I'm now training it in 25 seconds.
Lex Fridman (1:44:21.860)
And it's a combination of new optimizers
Lex Fridman (1:44:26.940)
and new just training regimens
Lex Fridman (1:44:29.620)
and different approaches to train.
Lex Fridman (1:44:32.100)
And all of these things come together
Chris Lattner (1:44:33.980)
to push the world forward.
Lex Fridman (1:44:36.020)
That was a beautiful exposition.
Lex Fridman (1:44:39.980)
But if you were to force to bet all your money
Lex Fridman (1:44:42.740)
on one of these.
Lex Fridman (1:44:45.220)
Why do we have to?
Lex Fridman (1:44:48.780)
Unfortunately, we have people working on all this.
Lex Fridman (1:44:50.740)
It's an exciting time, right?
Lex Fridman (1:44:52.220)
So, I mean, OpenAI did this little paper
Chris Lattner (1:44:56.140)
showing the algorithmic improvement you can get.
Lex Fridman (1:44:58.020)
It's been improving exponentially.
Chris Lattner (1:45:00.900)
I haven't quite seen the same kind of analysis
Lex Fridman (1:45:04.220)
on other layers of the stack.
Chris Lattner (1:45:06.700)
I'm sure it's also improving significantly.
Lex Fridman (1:45:09.300)
I just, it's a nice intuition builder.
Chris Lattner (1:45:12.300)
I mean, there's a reason why Moore's Law,
Lex Fridman (1:45:16.060)
that's the beauty of Moore's Law is
Chris Lattner (1:45:18.060)
somebody writes a paper that makes a ridiculous prediction.
Lex Fridman (1:45:22.500)
And it becomes reality in a sense.
Chris Lattner (1:45:27.100)
There's something about these narratives
Lex Fridman (1:45:28.820)
when you, when Chris Ladner on a silly little podcast
Chris Lattner (1:45:33.540)
makes, bets all his money on a particular thing,
Lex Fridman (1:45:37.180)
somehow it can have a ripple effect
Chris Lattner (1:45:39.220)
of actually becoming real.
Lex Fridman (1:45:40.700)
That's an interesting aspect of it.
Chris Lattner (1:45:43.220)
Cause like it might've been,
Lex Fridman (1:45:45.980)
we focus with Moore's Law,
Chris Lattner (1:45:47.460)
most of the computing industry
Lex Fridman (1:45:49.460)
really, really focused on the hardware.
Chris Lattner (1:45:53.500)
I mean, software innovation,
Lex Fridman (1:45:55.020)
I don't know how much software innovation
Chris Lattner (1:45:56.420)
there was in terms of efficient.
Lex Fridman (1:45:57.260)
What Intel Giveth Bill takes away, right?
Chris Lattner (1:46:00.060)
Yeah, I mean, compilers improved significantly also.
Lex Fridman (1:46:04.020)
Well, not really.
Lex Fridman (1:46:04.860)
So actually, I mean, so I'm joking
Lex Fridman (1:46:06.780)
about how software has gotten slower
Chris Lattner (1:46:08.900)
pretty much as fast as hardware got better,
Lex Fridman (1:46:11.500)
at least through the nineties.
Chris Lattner (1:46:13.140)
There's another joke, another law in compilers,
Lex Fridman (1:46:15.700)
which is called, I think it's called Probstine's Law,
Chris Lattner (1:46:18.140)
which is compilers double the performance
Lex Fridman (1:46:21.580)
of any given code every 18 years.
Lex Fridman (1:46:26.180)
So they move slowly.
Lex Fridman (1:46:27.860)
Yeah, also.
Chris Lattner (1:46:28.700)
Well, yeah, it's exponential also.
Lex Fridman (1:46:30.940)
Yeah, you're making progress,
Lex Fridman (1:46:32.380)
but there again, it's not about,
Lex Fridman (1:46:35.540)
the power of compilers is not just about
Lex Fridman (1:46:37.700)
how do you make the same thing go faster?
Lex Fridman (1:46:39.060)
It's how do you unlock the new hardware?
Lex Fridman (1:46:41.740)
A new chip came out, how do you utilize it?
Lex Fridman (1:46:43.500)
You say, oh, the programming model,
Lex Fridman (1:46:45.140)
how do we make people more productive?
Lex Fridman (1:46:46.980)
How do we have better error messages?
Chris Lattner (1:46:51.940)
Even such mundane things like how do I generate
Lex Fridman (1:46:55.460)
a very specific error message about your code
Chris Lattner (1:46:58.260)
actually makes people happy
Lex Fridman (1:46:59.780)
because then they know how to fix it, right?
Lex Fridman (1:47:01.860)
And it comes back to how do you help people
Lex Fridman (1:47:03.580)
get their job done.
Chris Lattner (1:47:04.580)
Yeah, and yeah, and then in this world
Lex Fridman (1:47:06.740)
of exponentially increasing smart toasters,
Lex Fridman (1:47:10.260)
how do you expand computing to all these kinds of devices?
Lex Fridman (1:47:16.100)
Do you see this world where just everything
Lex Fridman (1:47:18.420)
is a computing surface?
Lex Fridman (1:47:20.340)
You see that possibility?
Lex Fridman (1:47:22.100)
Just everything is a computer?
Lex Fridman (1:47:23.900)
Yeah, I don't see any reason
Chris Lattner (1:47:25.020)
that that couldn't be achieved.
Lex Fridman (1:47:26.940)
It turns out that sand goes into glass
Lex Fridman (1:47:30.380)
and glass is pretty useful too.
Lex Fridman (1:47:32.580)
And why not?
Lex Fridman (1:47:35.140)
Why not?
Lex Fridman (1:47:35.980)
So very important question then,
Chris Lattner (1:47:39.580)
if we're living in a simulation
Lex Fridman (1:47:44.580)
and the simulation is running a computer,
Lex Fridman (1:47:47.420)
like what's the architecture of that computer, do you think?
Lex Fridman (1:47:51.900)
So you're saying is it a quantum system?
Lex Fridman (1:47:54.940)
Yeah, like this whole quantum discussion, is it needed?
Lex Fridman (1:47:57.500)
Or can we run it with a RISC 5 architecture,
Lex Fridman (1:48:03.700)
a bunch of CPUs?
Lex Fridman (1:48:05.260)
I think it comes down to the right tool for the job.
Chris Lattner (1:48:07.540)
Yeah, and so.
Lex Fridman (1:48:08.620)
And what's the compiler?
Chris Lattner (1:48:10.100)
Yeah, exactly, that's my question.
Lex Fridman (1:48:12.540)
Did I get that job?
Chris Lattner (1:48:13.660)
Feed the universe compiler.
Lex Fridman (1:48:16.740)
And so there, as far as we know,
Chris Lattner (1:48:19.700)
quantum systems are the bottom of the pile of turtles
Lex Fridman (1:48:23.660)
so far.
Chris Lattner (1:48:24.500)
Yeah.
Lex Fridman (1:48:25.460)
And so we don't know efficient ways
Chris Lattner (1:48:28.260)
to implement quantum systems without using quantum computers.
Lex Fridman (1:48:32.180)
Yeah, and that's totally outside
Chris Lattner (1:48:33.540)
of everything we've talked about.
Lex Fridman (1:48:35.060)
But who runs that quantum computer?
Chris Lattner (1:48:37.060)
Yeah.
Lex Fridman (1:48:37.900)
Right, so if we really are living in a simulation,
Lex Fridman (1:48:41.460)
then is it bigger quantum computers?
Lex Fridman (1:48:44.420)
Is it different ones?
Lex Fridman (1:48:45.260)
Like how does that work out?
Lex Fridman (1:48:46.580)
How does that scale?
Chris Lattner (1:48:47.700)
Well, it's the same size.
Lex Fridman (1:48:49.940)
It's the same size.
Lex Fridman (1:48:50.780)
But then the thought of the simulation
Lex Fridman (1:48:52.660)
is that you don't have to run the whole thing,
Chris Lattner (1:48:54.260)
that we humans are cognitively very limited.
Lex Fridman (1:48:56.860)
We do checkpoints.
Chris Lattner (1:48:57.700)
We do checkpoints, yeah.
Lex Fridman (1:48:59.420)
And if we, the point at which we human,
Lex Fridman (1:49:03.020)
so you basically do minimal amount of,
Lex Fridman (1:49:06.860)
what is it, Swift does on right, copy on right.
Lex Fridman (1:49:12.220)
So you only adjust the simulation.
Lex Fridman (1:49:15.540)
Parallel universe theories, right?
Lex Fridman (1:49:17.060)
And so every time a decision's made,
Lex Fridman (1:49:20.580)
somebody opens the short end of your box,
Chris Lattner (1:49:22.060)
then there's a fork.
Lex Fridman (1:49:23.740)
And then this could happen.
Lex Fridman (1:49:24.980)
And then, thank you for considering the possibility.
Lex Fridman (1:49:29.980)
But yeah, so it may not require the entirety
Chris Lattner (1:49:32.900)
of the universe to simulate it.
Lex Fridman (1:49:34.260)
But it's interesting to think about
Chris Lattner (1:49:38.460)
as we create this higher and higher fidelity systems.
Lex Fridman (1:49:42.940)
But I do wanna ask on the quantum computer side,
Chris Lattner (1:49:46.180)
because everything we've talked about,
Lex Fridman (1:49:47.900)
whether you work with SciFive, with compilers,
Lex Fridman (1:49:51.660)
none of that includes quantum computers, right?
Lex Fridman (1:49:54.660)
That's true.
Lex Fridman (1:49:55.660)
So have you ever thought about this whole
Lex Fridman (1:50:03.140)
serious engineering work of quantum computers
Chris Lattner (1:50:06.180)
looks like of compilers, of architectures,
Lex Fridman (1:50:09.740)
all of that kind of stuff?
Lex Fridman (1:50:10.660)
So I've looked at it a little bit.
Lex Fridman (1:50:11.820)
I know almost nothing about it,
Chris Lattner (1:50:14.300)
which means that at some point,
Lex Fridman (1:50:15.540)
I will have to find an excuse to get involved,
Chris Lattner (1:50:17.860)
because that's how it works.
Lex Fridman (1:50:18.700)
But do you think that's a thing to be,
Chris Lattner (1:50:21.140)
like with your little tingly senses of the timing
Lex Fridman (1:50:24.620)
of when to be involved, is it not yet?
Chris Lattner (1:50:26.860)
Well, so the thing I do really well
Lex Fridman (1:50:28.820)
is I jump into messy systems
Lex Fridman (1:50:31.660)
and figure out how to make them,
Lex Fridman (1:50:33.700)
figure out what the truth in the situation is,
Chris Lattner (1:50:35.540)
try to figure out what the unifying theory is,
Lex Fridman (1:50:39.100)
how to like factor the complexity,
Lex Fridman (1:50:40.980)
how to find a beautiful answer to a problem
Lex Fridman (1:50:42.860)
that has been well studied
Lex Fridman (1:50:44.860)
and lots of people have bashed their heads against it.
Lex Fridman (1:50:47.060)
I don't know that quantum computers are mature enough
Lex Fridman (1:50:49.300)
and accessible enough to be figured out yet, right?
Lex Fridman (1:50:53.740)
And I think the open question with quantum computers is,
Chris Lattner (1:50:58.620)
is there a useful problem
Lex Fridman (1:51:00.180)
that gets solved with a quantum computer
Chris Lattner (1:51:02.340)
that makes it worth the economic cost
Lex Fridman (1:51:05.300)
of like having one of these things
Lex Fridman (1:51:06.780)
and having legions of people that set it up?
Lex Fridman (1:51:11.500)
You go back to the fifties, right?
Lex Fridman (1:51:12.780)
And there's the projections
Lex Fridman (1:51:13.740)
of the world will only need seven computers, right?
Chris Lattner (1:51:18.220)
Well, and part of that was that people hadn't figured out
Lex Fridman (1:51:20.740)
what they're useful for.
Lex Fridman (1:51:21.980)
What are the algorithms we wanna run?
Lex Fridman (1:51:23.260)
What are the problems that get solved?
Lex Fridman (1:51:24.340)
And this comes back to how do we make the world better,
Lex Fridman (1:51:27.620)
either economically or making somebody's life better
Chris Lattner (1:51:29.940)
or like solving a problem that wasn't solved before,
Lex Fridman (1:51:31.940)
things like this.
Lex Fridman (1:51:33.140)
And I think that just we're a little bit too early
Lex Fridman (1:51:36.020)
in that development cycle
Chris Lattner (1:51:36.860)
because it's still like literally a science project,
Lex Fridman (1:51:39.380)
not a negative connotation, right?
Chris Lattner (1:51:41.540)
It's literally a science project
Lex Fridman (1:51:42.860)
and the progress there is amazing.
Lex Fridman (1:51:45.420)
And so I don't know if it's 10 years away,
Lex Fridman (1:51:48.900)
if it's two years away,
Chris Lattner (1:51:50.100)
exactly where that breakthrough happens,
Lex Fridman (1:51:51.660)
but you look at machine learning,
Chris Lattner (1:51:56.300)
we went through a few winners
Lex Fridman (1:51:58.420)
before the AlexNet transition
Lex Fridman (1:52:00.140)
and then suddenly it had its breakout moment.
Lex Fridman (1:52:02.980)
And that was the catalyst
Chris Lattner (1:52:04.300)
that then drove the talent flocking into it.
Lex Fridman (1:52:07.580)
That's what drove the economic applications of it.
Chris Lattner (1:52:10.180)
That's what drove the technology to go faster
Lex Fridman (1:52:13.420)
because you now have more minds thrown at the problem.
Chris Lattner (1:52:15.940)
This is what caused like a serious knee in deep learning
Lex Fridman (1:52:20.180)
and the algorithms that we're using.
Lex Fridman (1:52:21.940)
And so I think that's what quantum needs to go through.
Lex Fridman (1:52:25.540)
And so right now it's in that formidable finding itself,
Chris Lattner (1:52:28.820)
getting the like literally the physics figured out.
Lex Fridman (1:52:32.980)
And then it has to figure out the application
Chris Lattner (1:52:36.100)
that makes this useful.
Lex Fridman (1:52:37.580)
Yeah, but I'm not skeptical that I think that will happen.
Chris Lattner (1:52:40.780)
I think it's just 10 years away, something like that.
Lex Fridman (1:52:43.420)
I forgot to ask,
Lex Fridman (1:52:44.500)
what programming language do you think
Lex Fridman (1:52:46.380)
the simulation is written in?
Chris Lattner (1:52:48.620)
Ooh, probably Lisp.
Lex Fridman (1:52:50.340)
So not Swift.
Chris Lattner (1:52:52.980)
Like if you're a Tibet, I'll just leave it at that.
Lex Fridman (1:52:58.060)
So, I mean, we've mentioned that you worked
Chris Lattner (1:53:00.380)
with all these companies,
Lex Fridman (1:53:01.220)
we've talked about all these projects.
Chris Lattner (1:53:03.900)
It's kind of like if we just step back and zoom out
Lex Fridman (1:53:07.220)
about the way you did that work.
Lex Fridman (1:53:10.100)
And we look at COVID times,
Lex Fridman (1:53:12.220)
this pandemic we're living through that may,
Chris Lattner (1:53:14.780)
if I look at the way Silicon Valley folks
Lex Fridman (1:53:17.020)
are talking about it, the way MIT is talking about it,
Chris Lattner (1:53:19.860)
this might last for a long time.
Lex Fridman (1:53:23.060)
Not just the virus, but the remote nature.
Chris Lattner (1:53:28.340)
The economic impact.
Lex Fridman (1:53:29.660)
I mean, all of it, yeah.
Chris Lattner (1:53:30.500)
Yeah, it's gonna be a mess.
Lex Fridman (1:53:32.140)
Do you think, what's your prediction?
Chris Lattner (1:53:34.500)
I mean, from sci fi to Google,
Lex Fridman (1:53:36.380)
to just all the places you worked in,
Chris Lattner (1:53:42.420)
just Silicon Valley, you're in the middle of it.
Lex Fridman (1:53:44.260)
What do you think is,
Lex Fridman (1:53:45.100)
how is this whole place gonna change?
Lex Fridman (1:53:46.620)
Yeah, so, I mean, I really can only speak
Chris Lattner (1:53:49.020)
to the tech perspective.
Lex Fridman (1:53:50.460)
I am in that bubble.
Chris Lattner (1:53:54.140)
I think it's gonna be really interesting
Lex Fridman (1:53:55.700)
because the Zoom culture of being remote
Lex Fridman (1:53:58.820)
and on video chat all the time
Lex Fridman (1:54:00.260)
has really interesting effects on people.
Lex Fridman (1:54:01.980)
So on the one hand, it's a great normalizer.
Lex Fridman (1:54:05.020)
It's a normalizer that I think will help communities
Chris Lattner (1:54:09.060)
of people that have traditionally been underrepresented
Lex Fridman (1:54:12.580)
because now you're taking, in some cases, a face off
Lex Fridman (1:54:16.380)
because you don't have to have a camera going, right?
Lex Fridman (1:54:18.780)
And so you can have conversations
Chris Lattner (1:54:20.020)
without physical appearance being part of the dynamic,
Lex Fridman (1:54:22.780)
which is pretty powerful.
Chris Lattner (1:54:24.500)
You're taking remote employees
Lex Fridman (1:54:25.940)
that have already been remote,
Lex Fridman (1:54:27.020)
and you're saying you're now on the same level
Lex Fridman (1:54:29.900)
and footing as everybody else.
Chris Lattner (1:54:31.380)
Nobody gets whiteboards.
Lex Fridman (1:54:33.460)
You're not gonna be the one person
Chris Lattner (1:54:34.620)
that doesn't get to be participating
Lex Fridman (1:54:36.020)
in the whiteboard conversation,
Lex Fridman (1:54:37.220)
and that's pretty powerful.
Lex Fridman (1:54:39.300)
You've got, you're forcing people to think asynchronously
Chris Lattner (1:54:44.140)
in some cases because it's harder to just get people
Lex Fridman (1:54:47.220)
physically together, and the bumping into each other
Chris Lattner (1:54:49.380)
forces people to find new ways to solve those problems.
Lex Fridman (1:54:52.740)
And I think that that leads to more inclusive behavior,
Chris Lattner (1:54:55.220)
which is good.
Lex Fridman (1:54:56.780)
On the other hand, it's also, it just sucks, right?
Lex Fridman (1:55:00.740)
And so the actual communication just sucks
Lex Fridman (1:55:05.940)
being not with people on a daily basis
Lex Fridman (1:55:09.820)
and collaborating with them.
Lex Fridman (1:55:11.380)
Yeah, all of that, right?
Chris Lattner (1:55:13.060)
I mean, everything, this whole situation is terrible.
Lex Fridman (1:55:15.660)
What I meant primarily was the,
Chris Lattner (1:55:19.060)
I think that most humans
Lex Fridman (1:55:20.860)
like working physically with humans.
Chris Lattner (1:55:22.980)
I think this is something that not everybody,
Lex Fridman (1:55:24.660)
but many people are programmed to do.
Lex Fridman (1:55:27.100)
And I think that we get something out of that
Lex Fridman (1:55:29.220)
that is very hard to express, at least for me.
Lex Fridman (1:55:31.460)
And so maybe this isn't true of everybody.
Lex Fridman (1:55:33.180)
But, and so the question to me is,
Chris Lattner (1:55:36.860)
when you get through that time of adaptation,
Lex Fridman (1:55:40.020)
you get out of March and April,
Lex Fridman (1:55:41.940)
and you get into December,
Lex Fridman (1:55:43.220)
you get into next March, if it's not changed, right?
Chris Lattner (1:55:46.580)
It's already terrifying.
Lex Fridman (1:55:47.820)
Well, you think about that,
Lex Fridman (1:55:49.140)
and you think about what is the nature of work?
Lex Fridman (1:55:51.420)
And how do we adapt?
Lex Fridman (1:55:52.740)
And humans are very adaptable species, right?
Lex Fridman (1:55:55.100)
We can learn things when we're forced to,
Lex Fridman (1:55:58.260)
and there's a catalyst to make that happen.
Lex Fridman (1:56:00.580)
And so what is it that comes out of this,
Lex Fridman (1:56:02.700)
and are we better or worse off?
Lex Fridman (1:56:04.740)
I think that you look at the Bay Area,
Chris Lattner (1:56:07.180)
housing prices are insane.
Lex Fridman (1:56:08.940)
Well, why?
Chris Lattner (1:56:09.900)
Well, there's a high incentive to be physically located,
Lex Fridman (1:56:12.460)
because if you don't have proximity,
Lex Fridman (1:56:14.980)
you end up paying for it and commute, right?
Lex Fridman (1:56:18.420)
And there has been huge social pressure
Lex Fridman (1:56:21.060)
in terms of you will be there for the meeting, right?
Lex Fridman (1:56:24.660)
Or whatever scenario it is.
Lex Fridman (1:56:26.940)
And I think that's gonna be way better.
Lex Fridman (1:56:28.260)
I think it's gonna be much more the norm
Chris Lattner (1:56:30.020)
to have remote employees,
Lex Fridman (1:56:31.660)
and I think this is gonna be really great.
Lex Fridman (1:56:33.260)
Do you have friends, or do you hear of people moving?
Lex Fridman (1:56:37.260)
I know one family friend that moved.
Chris Lattner (1:56:40.780)
They moved back to Michigan,
Lex Fridman (1:56:41.980)
and they were a family with three kids
Chris Lattner (1:56:45.620)
living in a small apartment,
Lex Fridman (1:56:46.820)
and we're going insane, right?
Lex Fridman (1:56:50.820)
And they're in tech, husband works for Google.
Lex Fridman (1:56:54.300)
So first of all, friends of mine
Chris Lattner (1:56:57.140)
are in the process of, or have already lost the business.
Lex Fridman (1:57:00.580)
The thing that represents their passion, their dream,
Chris Lattner (1:57:03.220)
it could be small entrepreneurial projects,
Lex Fridman (1:57:05.340)
but it could be large businesses,
Chris Lattner (1:57:06.820)
like people that run gyms.
Lex Fridman (1:57:08.500)
Restaurants, tons of things, yeah.
Lex Fridman (1:57:10.860)
But also, people look at themselves in the mirror
Lex Fridman (1:57:14.180)
and ask the question of, what do I wanna do in life?
Chris Lattner (1:57:17.660)
For some reason, they haven't done it until COVID.
Lex Fridman (1:57:20.980)
They really ask that question,
Lex Fridman (1:57:22.100)
and that results often in moving or leaving the company
Lex Fridman (1:57:26.660)
with starting your own business
Chris Lattner (1:57:28.140)
or transitioning to a different company.
Lex Fridman (1:57:30.660)
Do you think we're gonna see that a lot?
Chris Lattner (1:57:33.620)
Well, I can't speak to that.
Lex Fridman (1:57:36.220)
I mean, we're definitely gonna see it
Chris Lattner (1:57:37.300)
at a higher frequency than we did before,
Lex Fridman (1:57:39.380)
just because I think what you're trying to say
Chris Lattner (1:57:41.580)
is there are decisions that you make yourself,
Lex Fridman (1:57:45.220)
big life decisions that you make yourself,
Lex Fridman (1:57:47.260)
and I'm gonna quit my job and start a new thing.
Lex Fridman (1:57:49.860)
There's also decisions that get made for you.
Lex Fridman (1:57:52.740)
I got fired from my job, what am I gonna do, right?
Lex Fridman (1:57:55.420)
And that's not a decision that you think about,
Lex Fridman (1:57:58.060)
but you're forced to act, okay?
Lex Fridman (1:58:00.340)
And so I think that those, you're forced to act
Chris Lattner (1:58:02.900)
kind of moments where global pandemic
Lex Fridman (1:58:05.300)
comes and wipes out the economy,
Lex Fridman (1:58:06.660)
and now your business doesn't exist.
Lex Fridman (1:58:09.900)
I think that does lead to more reflection, right?
Chris Lattner (1:58:11.820)
Because you're less anchored on what you have,
Lex Fridman (1:58:14.460)
and it's not a, what do I have to lose
Chris Lattner (1:58:17.060)
versus what do I have to gain, A, B, comparison.
Lex Fridman (1:58:19.980)
It's more of a fresh slate.
Chris Lattner (1:58:21.940)
Cool, I could do anything now.
Lex Fridman (1:58:23.900)
Do I wanna do the same thing I was doing?
Lex Fridman (1:58:26.380)
Did that make me happy?
Lex Fridman (1:58:27.820)
Is this now time to go back to college
Lex Fridman (1:58:29.500)
and take a class and learn a new skill?
Lex Fridman (1:58:33.100)
Is this a time to spend time with family
Lex Fridman (1:58:36.580)
if you can afford to do that?
Lex Fridman (1:58:37.820)
Is this time to literally move in with parents, right?
Chris Lattner (1:58:40.980)
I mean, all these things that were not normative before
Lex Fridman (1:58:43.900)
suddenly become, I think, very, the value systems change.
Lex Fridman (1:58:49.140)
And I think that's actually a good thing
Lex Fridman (1:58:50.820)
in the short term, at least, because it leads to,
Chris Lattner (1:58:56.340)
there's kind of been an overoptimization
Lex Fridman (1:58:58.420)
along one set of priorities for the world,
Lex Fridman (1:59:01.540)
and now maybe we'll get to a more balanced
Lex Fridman (1:59:03.540)
and more interesting world
Chris Lattner (1:59:05.180)
where people are doing different things.
Lex Fridman (1:59:06.780)
I think it could be good.
Chris Lattner (1:59:07.660)
I think there could be more innovation
Lex Fridman (1:59:08.980)
that comes out of it, for example.
Lex Fridman (1:59:10.100)
What do you think about all the social chaos
Lex Fridman (1:59:12.780)
we're in the middle of?
Chris Lattner (1:59:13.980)
It sucks.
Lex Fridman (1:59:14.820)
You think it's, let me ask you a whole,
Lex Fridman (1:59:18.740)
you think it's all gonna be okay?
Lex Fridman (1:59:21.100)
Well, I think humanity will survive.
Chris Lattner (1:59:23.420)
The, from an existential,
Lex Fridman (1:59:25.420)
like we're not all gonna kill, yeah, well.
Chris Lattner (1:59:27.260)
Yeah, I don't think the virus is gonna kill all the humans.
Lex Fridman (1:59:30.380)
I don't think all the humans are gonna kill all the humans.
Chris Lattner (1:59:31.980)
I think that's unlikely.
Lex Fridman (1:59:32.900)
But I look at it as progress requires a catalyst, right?
Lex Fridman (1:59:42.460)
So you need a reason for people to be willing
Lex Fridman (1:59:45.580)
to do things that are uncomfortable.
Chris Lattner (1:59:47.740)
I think that the US, at least,
Lex Fridman (1:59:50.740)
but I think the world in general
Chris Lattner (1:59:51.740)
is a pretty unoptimal place to live in for a lot of people.
Lex Fridman (1:59:56.780)
And I think that what we're seeing right now
Chris Lattner (1:59:58.900)
is we're seeing a lot of unhappiness.
Lex Fridman (20:01.260)
It's great because you define away the bug,
Chris Lattner (20:04.220)
which is a big deal for productivity,
Lex Fridman (20:05.980)
the number one thing most people care about,
Lex Fridman (20:08.220)
but it's also good for performance
Lex Fridman (20:09.740)
because when you're doing a clone,
Lex Fridman (20:11.580)
so you pass the array down to the thing,
Lex Fridman (20:13.460)
it's like, I don't know if anybody else has it,
Chris Lattner (20:15.420)
I have to clone it.
Lex Fridman (20:16.620)
Well, you just did a copy of a bunch of data.
Chris Lattner (20:18.460)
It could be big.
Lex Fridman (20:19.940)
And then it could be that the thing that called you
Chris Lattner (20:21.980)
is not keeping track of the old thing.
Lex Fridman (20:24.020)
So you just made a copy of it,
Lex Fridman (20:25.700)
and you may not have had to.
Lex Fridman (20:27.800)
And so the way the value semantics work in Swift
Chris Lattner (20:30.180)
is it uses this thing called copy on write,
Lex Fridman (20:32.060)
which means that you get the benefit of safety
Lex Fridman (20:35.500)
and performance.
Lex Fridman (20:36.420)
And it has another special trick
Chris Lattner (20:38.340)
because if you think certain languages like Java,
Lex Fridman (20:41.180)
for example, they have immutable strings.
Lex Fridman (20:43.940)
And so what they're trying to do
Lex Fridman (20:44.940)
is they provide value semantics
Chris Lattner (20:46.400)
by having pure immutability.
Lex Fridman (20:48.980)
Functional languages have pure immutability
Chris Lattner (20:51.060)
in lots of different places,
Lex Fridman (20:52.300)
and this provides a much safer model
Lex Fridman (20:53.960)
and it provides value semantics.
Lex Fridman (20:56.140)
The problem with this is if you have immutability,
Chris Lattner (20:58.380)
everything is expensive.
Lex Fridman (20:59.500)
Everything requires a copy.
Chris Lattner (21:02.420)
For example, in Java, if you have a string X
Lex Fridman (21:05.420)
and a string Y, you append them together,
Chris Lattner (21:07.900)
we have to allocate a new string to hold X, Y.
Lex Fridman (21:12.180)
If they're immutable.
Chris Lattner (21:13.720)
Well, strings in Java are immutable.
Lex Fridman (21:16.900)
And if there's optimizations for short ones,
Chris Lattner (21:19.580)
it's complicated, but generally think about them
Lex Fridman (21:22.820)
as a separate allocation.
Lex Fridman (21:24.580)
And so when you append them together,
Lex Fridman (21:26.620)
you have to go allocate a third thing
Chris Lattner (21:28.580)
because somebody might have a pointer
Lex Fridman (21:29.660)
to either of the other ones, right?
Lex Fridman (21:31.060)
And you can't go change them.
Lex Fridman (21:32.060)
So you have to go allocate a third thing.
Chris Lattner (21:34.700)
Because of the beauty of how the Swift value semantics
Lex Fridman (21:36.740)
system works out, if you have a string in Swift
Lex Fridman (21:38.780)
and you say, hey, put in X, right?
Lex Fridman (21:40.980)
And they say, append on Y, Z, W,
Chris Lattner (21:44.980)
it knows that there's only one reference to that.
Lex Fridman (21:47.500)
And so it can do an in place update.
Lex Fridman (21:50.220)
And so you're not allocating tons of stuff on the side.
Lex Fridman (21:53.420)
You don't have all those problems.
Chris Lattner (21:54.620)
When you pass it off,
Lex Fridman (21:56.040)
you can know you have the only reference.
Chris Lattner (21:57.520)
If you pass it off to multiple different people,
Lex Fridman (21:59.340)
but nobody changes it, they can all share the same thing.
Lex Fridman (22:02.620)
So you get a lot of the benefit of purely immutable design.
Lex Fridman (22:05.780)
And so you get a really nice sweet spot
Chris Lattner (22:07.640)
that I haven't seen in other languages.
Lex Fridman (22:09.300)
Yeah, that's interesting.
Chris Lattner (22:10.540)
I thought there was going to be a philosophical narrative
Lex Fridman (22:15.780)
here that you're gonna have to pay a cost for it.
Chris Lattner (22:19.420)
Cause it sounds like, I think value semantics
Lex Fridman (22:24.500)
is beneficial for easing of debugging
Chris Lattner (22:27.440)
or minimizing the risk of errors,
Lex Fridman (22:30.980)
like bringing the errors closer to the source,
Chris Lattner (22:35.780)
bringing the symptom of the error closer
Lex Fridman (22:38.180)
to the source of the error, however you say that.
Lex Fridman (22:40.840)
But you're saying there's not a performance cost either
Lex Fridman (22:44.980)
if you implement it correctly.
Chris Lattner (22:46.300)
Well, so there's trade offs with everything.
Lex Fridman (22:48.300)
And so if you are doing very low level stuff,
Chris Lattner (22:51.860)
then sometimes you can notice a cost,
Lex Fridman (22:53.180)
but then what you're doing is you're saying,
Lex Fridman (22:54.880)
what is the right default?
Lex Fridman (22:56.540)
So coming back to user interface,
Chris Lattner (22:59.100)
when you talk about programming languages,
Lex Fridman (23:00.740)
one of the major things that Swift does
Chris Lattner (23:03.000)
that makes people love it,
Lex Fridman (23:04.500)
that is not obvious when it comes to designing a language
Chris Lattner (23:08.220)
is this UI principle of progressive disclosure
Lex Fridman (23:11.460)
of complexity.
Chris Lattner (23:12.980)
Okay, so Swift, like many languages is very powerful.
Lex Fridman (23:16.700)
The question is, when do you have to learn
Lex Fridman (23:18.420)
the power as a user?
Lex Fridman (23:20.820)
So Swift, like Python, allows you to start with like,
Lex Fridman (23:22.980)
print hello world, right?
Lex Fridman (23:24.940)
Certain other languages start with like,
Chris Lattner (23:26.920)
public static void main class,
Lex Fridman (23:30.580)
like all the ceremony, right?
Lex Fridman (23:32.140)
And so you go to teach a new person,
Lex Fridman (23:34.620)
hey, welcome to this new thing.
Chris Lattner (23:36.780)
Let's talk about public access control classes.
Lex Fridman (23:40.300)
Wait, what's that?
Chris Lattner (23:41.140)
String system.out.println, like packages,
Lex Fridman (23:44.740)
like, God, right?
Lex Fridman (23:46.700)
And so instead, if you take this and you say,
Lex Fridman (23:48.700)
hey, we need packages, modules,
Chris Lattner (23:51.740)
we need powerful things like classes,
Lex Fridman (23:54.220)
we need data structures, we need like all these things.
Lex Fridman (23:57.380)
The question is, how do you factor the complexity?
Lex Fridman (23:59.420)
And how do you make it so that the normal case scenario
Chris Lattner (24:02.820)
is you're dealing with things that work the right way
Lex Fridman (24:05.620)
in the right way, give you good performance
Chris Lattner (24:07.940)
by default, but then as a power user,
Lex Fridman (24:11.140)
if you want to dive down to it,
Chris Lattner (24:12.340)
you have full C performance, full control
Lex Fridman (24:15.140)
over low level pointers.
Chris Lattner (24:15.980)
You can call malloc if you want to call malloc.
Lex Fridman (24:18.340)
This is not recommended on the first page of every tutorial,
Lex Fridman (24:20.780)
but it's actually really important
Lex Fridman (24:22.300)
when you want to get work done, right?
Lex Fridman (24:23.780)
And so being able to have that is really the design
Lex Fridman (24:27.460)
in programming language design,
Lex Fridman (24:28.820)
and design is really, really hard.
Lex Fridman (24:31.300)
It's something that I think a lot of people kind of,
Chris Lattner (24:34.940)
outside of UI, again, a lot of people just think
Lex Fridman (24:37.900)
is subjective, like there's nothing,
Chris Lattner (24:40.940)
you know, it's just like curly braces or whatever.
Lex Fridman (24:43.620)
It's just like somebody's preference,
Lex Fridman (24:45.340)
but actually good design is something that you can feel.
Lex Fridman (24:48.740)
And how many people are involved with good design?
Lex Fridman (24:52.100)
So if we looked at Swift, but look at historically,
Lex Fridman (24:54.860)
I mean, this might touch like,
Chris Lattner (24:57.340)
it's almost like a Steve Jobs question too,
Lex Fridman (24:59.700)
like how much dictatorial decision making is required
Chris Lattner (25:04.100)
versus collaborative, and we'll talk about
Lex Fridman (25:09.100)
how all that can go wrong or right, but.
Chris Lattner (25:11.700)
Yeah, well, Swift, so I can't speak to in general,
Lex Fridman (25:14.340)
all design everywhere.
Lex Fridman (25:15.540)
So the way it works with Swift is that there's a core team,
Lex Fridman (25:19.740)
and so a core team is six or seven people ish,
Chris Lattner (25:22.460)
something like that, that is people that have been working
Lex Fridman (25:25.020)
with Swift since very early days, and so.
Lex Fridman (25:27.100)
And by early days is not that long ago.
Lex Fridman (25:30.060)
Okay, yeah, so it became public in 2014,
Lex Fridman (25:33.580)
so it's been six years public now,
Lex Fridman (25:35.500)
but so that's enough time that there's a story arc there.
Chris Lattner (25:38.820)
Okay, yeah, and there's mistakes have been made
Lex Fridman (25:41.940)
that then get fixed, and you learn something,
Lex Fridman (25:43.700)
and then you, you know, and so what the core team does
Lex Fridman (25:46.980)
is it provides continuity, and so you wanna have a,
Chris Lattner (25:50.420)
okay, well, there's a big hole that we wanna fill.
Lex Fridman (25:54.020)
We know we wanna fill it, so don't do other things
Lex Fridman (25:56.900)
that invade that space until we fill the hole, right?
Lex Fridman (25:59.940)
There's a boulder that's missing here,
Chris Lattner (26:01.100)
we wanna do, we will do that boulder,
Lex Fridman (26:03.060)
even though it's not today, keep out of that space.
Lex Fridman (26:06.100)
And the whole team remembers the myth of the boulder
Lex Fridman (26:10.340)
that's there.
Chris Lattner (26:11.180)
Yeah, yeah, there's a general sense
Lex Fridman (26:12.540)
of what the future looks like in broad strokes,
Lex Fridman (26:14.460)
and a shared understanding of that,
Lex Fridman (26:16.460)
combined with a shared understanding of what has happened
Chris Lattner (26:18.780)
in the past that worked out well and didn't work out well.
Lex Fridman (26:22.100)
The next level out is you have the,
Chris Lattner (26:24.260)
what's called the Swift evolution community,
Lex Fridman (26:25.820)
and you've got, in that case, hundreds of people
Chris Lattner (26:27.700)
that really care passionately about the way Swift evolves,
Lex Fridman (26:30.980)
and that's like an amazing thing to, again,
Chris Lattner (26:33.900)
the core team doesn't necessarily need to come up
Lex Fridman (26:35.540)
with all the good ideas.
Chris Lattner (26:36.820)
You got hundreds of people out there
Lex Fridman (26:38.060)
that care about something,
Lex Fridman (26:39.020)
and they come up with really good ideas too,
Lex Fridman (26:41.100)
and that provides this rock tumbler for ideas.
Lex Fridman (26:45.180)
And so the evolution process is,
Lex Fridman (26:48.780)
a lot of people in a discourse forum,
Chris Lattner (26:50.380)
they're like hashing it out and trying to talk about,
Lex Fridman (26:52.100)
okay, well, should we go left or right,
Lex Fridman (26:54.100)
or if we did this, what would be good?
Lex Fridman (26:55.700)
And here you're talking about hundreds of people,
Lex Fridman (26:57.700)
so you're not gonna get consensus, necessarily,
Lex Fridman (27:00.380)
not obvious consensus, and so there's a proposal process
Chris Lattner (27:04.620)
that then allows the core team and the community
Lex Fridman (27:07.660)
to work this out, and what the core team does
Chris Lattner (27:10.020)
is it aims to get consensus out of the community
Lex Fridman (27:12.820)
and provide guardrails, but also provide long term,
Chris Lattner (27:17.420)
make sure we're going the right direction kind of things.
Lex Fridman (27:20.380)
So does that group represent like the,
Lex Fridman (27:23.540)
how much people will love the user interface?
Lex Fridman (27:27.420)
Like, do you think they're able to capture that?
Chris Lattner (27:29.420)
Well, I mean, it's something we talk about a lot,
Lex Fridman (27:31.020)
it's something we care about.
Lex Fridman (27:32.340)
How well we do that's up for debate,
Lex Fridman (27:34.780)
but I think that we've done pretty well so far.
Lex Fridman (27:36.780)
Is the beginner in mind?
Lex Fridman (27:38.540)
Yeah. Like, because you said
Chris Lattner (27:39.380)
the progressive disclosure complexity.
Lex Fridman (27:40.780)
Yeah, so we care a lot about that,
Chris Lattner (27:44.180)
a lot about power, a lot about efficiency,
Lex Fridman (27:46.420)
a lot about, there are many factors to good design,
Lex Fridman (27:48.700)
and you have to figure out a way
Lex Fridman (27:50.100)
to kind of work your way through that, and.
Lex Fridman (27:53.300)
So if you think about, like the language I love is Lisp,
Lex Fridman (27:57.540)
probably still because I use Emacs,
Lex Fridman (27:59.340)
but I haven't done anything, any serious work in Lisp,
Lex Fridman (28:02.180)
but it has a ridiculous amount of parentheses.
Chris Lattner (28:05.020)
Yeah.
Lex Fridman (28:06.540)
I've also, you know, with Java and C++, the braces,
Chris Lattner (28:14.300)
you know, I like, I enjoyed the comfort
Lex Fridman (28:17.500)
of being between braces, you know?
Chris Lattner (28:20.580)
Yeah, yeah, well, let's talk.
Lex Fridman (28:21.420)
And then Python is, sorry to interrupt,
Chris Lattner (28:23.140)
just like, and last thing to me, as a designer,
Lex Fridman (28:25.740)
if I was a language designer, God forbid,
Chris Lattner (28:28.740)
is I would be very surprised that Python with no braces
Lex Fridman (28:34.020)
would nevertheless somehow be comforting also.
Lex Fridman (28:38.220)
So like, I could see arguments for all of this.
Lex Fridman (28:40.620)
But look at this, this is evidence
Chris Lattner (28:41.940)
that it's not about braces versus tabs.
Lex Fridman (28:44.260)
Right, exactly, you're good, that's a good point.
Chris Lattner (28:47.020)
Right, so like, you know, there's evidence that.
Lex Fridman (28:50.020)
But see, like, it's one of the most argued about things.
Chris Lattner (28:52.380)
Oh yeah, of course, just like tabs and spaces,
Lex Fridman (28:54.140)
which it doesn't, I mean, there's one obvious right answer,
Lex Fridman (28:57.180)
but it doesn't actually matter.
Lex Fridman (28:59.140)
What's that?
Chris Lattner (28:59.980)
Let's not, like, come on, we're friends.
Lex Fridman (29:01.780)
Like, come on, what are you trying to do to me here?
Chris Lattner (29:03.460)
People are gonna, yeah, half the people are gonna tune out.
Lex Fridman (29:05.500)
Yeah, so these, so you're able to identify things
Chris Lattner (29:09.420)
that don't really matter for the experience.
Lex Fridman (29:12.620)
Well, no, no, no, it's always a really hard,
Lex Fridman (29:14.780)
so the easy decisions are easy, right?
Lex Fridman (29:16.900)
I mean, fine, those are not the interesting ones.
Lex Fridman (29:19.540)
The hard ones are the ones that are most interesting, right?
Lex Fridman (29:21.780)
The hard ones are the places where,
Chris Lattner (29:23.580)
hey, we wanna do a thing, everybody agrees we should do it,
Lex Fridman (29:27.020)
there's one proposal on the table,
Lex Fridman (29:28.900)
but it has all these bad things associated with it.
Lex Fridman (29:31.580)
Well, okay, what are we gonna do about that?
Lex Fridman (29:33.740)
Do we just take it?
Lex Fridman (29:34.980)
Do we delay it?
Chris Lattner (29:36.260)
Do we say, hey, well, maybe there's this other feature
Lex Fridman (29:38.500)
that if we do that first, this will work out better.
Lex Fridman (29:41.500)
How does this, if we do this,
Lex Fridman (29:44.060)
are we paying ourselves into a corner, right?
Lex Fridman (29:46.180)
And so this is where, again,
Lex Fridman (29:47.340)
you're having that core team of people
Chris Lattner (29:48.580)
that has some continuity and has perspective,
Lex Fridman (29:51.660)
has some of the historical understanding,
Chris Lattner (29:53.620)
is really valuable because you get,
Lex Fridman (29:56.100)
it's not just like one brain,
Chris Lattner (29:57.180)
you get the power of multiple people coming together
Lex Fridman (29:59.220)
to make good decisions,
Lex Fridman (2:00:00.460)
And because of all the pressure,
Lex Fridman (2:00:03.580)
because of all the badness in the world
Chris Lattner (2:00:05.500)
that's coming together,
Lex Fridman (2:00:06.340)
it's really kind of igniting some of that debate
Lex Fridman (2:00:07.860)
that should have happened a long time ago, right?
Lex Fridman (2:00:10.140)
I mean, I think that we'll see more progress.
Chris Lattner (2:00:11.620)
You're asking about, offline you're asking about politics
Lex Fridman (2:00:14.220)
and wouldn't it be great if politics moved faster
Chris Lattner (2:00:15.740)
because there's all these problems in the world
Lex Fridman (2:00:16.940)
and we can move it.
Chris Lattner (2:00:18.140)
Well, people are intentionally, are inherently conservative.
Lex Fridman (2:00:22.300)
And so if you're talking about conservative people,
Chris Lattner (2:00:25.020)
particularly if they have heavy burdens on their shoulders
Lex Fridman (2:00:27.460)
because they represent literally thousands of people,
Chris Lattner (2:00:31.700)
it makes sense to be conservative.
Lex Fridman (2:00:33.220)
But on the other hand, when you need change,
Lex Fridman (2:00:35.300)
how do you get it?
Lex Fridman (2:00:37.140)
The global pandemic will probably lead to some change.
Lex Fridman (2:00:40.500)
And it's not a directed, it's not a directed plan,
Lex Fridman (2:00:44.300)
but I think that it leads to people
Chris Lattner (2:00:45.900)
asking really interesting questions.
Lex Fridman (2:00:47.340)
And some of those questions
Chris Lattner (2:00:48.180)
should have been asked a long time ago.
Lex Fridman (2:00:50.100)
Well, let me know if you've observed this as well.
Chris Lattner (2:00:53.260)
Something that's bothered me in the machine learning
Lex Fridman (2:00:55.620)
community, I'm guessing it might be prevalent
Chris Lattner (2:00:58.260)
in other places, is something that feels like in 2020
Lex Fridman (2:01:02.500)
increase the level of toxicity.
Chris Lattner (2:01:05.260)
Like people are just quicker to pile on,
Lex Fridman (2:01:09.700)
to just be, they're just harsh on each other,
Chris Lattner (2:01:13.260)
to like mob, pick a person that screwed up
Lex Fridman (2:01:19.300)
and like make it a big thing.
Lex Fridman (2:01:22.020)
And is there something that we can like,
Lex Fridman (2:01:26.340)
yeah, have you observed that in other places?
Lex Fridman (2:01:28.180)
Is there some way out of this?
Lex Fridman (2:01:30.180)
I think there's an inherent thing in humanity
Chris Lattner (2:01:32.140)
that's kind of an us versus them thing,
Lex Fridman (2:01:34.420)
which is that you wanna succeed and how do you succeed?
Chris Lattner (2:01:37.100)
Well, it's relative to somebody else.
Lex Fridman (2:01:39.580)
And so what's happening in, at least in some part
Chris Lattner (2:01:43.100)
is that with the internet and with online communication,
Lex Fridman (2:01:47.100)
the world's getting smaller, right?
Lex Fridman (2:01:49.580)
And so we're having some of the social ties
Lex Fridman (2:01:53.020)
of like my town versus your town's football team, right?
Chris Lattner (2:01:57.540)
Turn into much larger and yet shallower problems.
Lex Fridman (2:02:02.980)
And people don't have time, the incentives,
Chris Lattner (2:02:06.580)
the clickbait and like all these things
Lex Fridman (2:02:08.060)
kind of really feed into this machine.
Lex Fridman (2:02:10.500)
And I don't know where that goes.
Lex Fridman (2:02:12.460)
Yeah, I mean, the reason I think about that,
Chris Lattner (2:02:15.060)
I mentioned to you this offline a little bit,
Lex Fridman (2:02:17.500)
but I have a few difficult conversations scheduled,
Chris Lattner (2:02:23.060)
some of them political related,
Lex Fridman (2:02:25.100)
some of them within the community,
Chris Lattner (2:02:28.180)
difficult personalities that went through some stuff.
Lex Fridman (2:02:30.620)
I mean, one of them I've talked before,
Chris Lattner (2:02:32.140)
I will talk again is Yann LeCun.
Lex Fridman (2:02:34.340)
He got a little bit of crap on Twitter
Chris Lattner (2:02:38.380)
for talking about a particular paper
Lex Fridman (2:02:40.940)
and the bias within a data set.
Lex Fridman (2:02:42.740)
And then there's been a huge, in my view,
Lex Fridman (2:02:45.940)
and I'm willing, comfortable saying it,
Chris Lattner (2:02:49.700)
irrational, over exaggerated pile on his comments
Lex Fridman (2:02:54.380)
because he made pretty basic comments about the fact that
Chris Lattner (2:02:58.460)
if there's bias in the data,
Lex Fridman (2:02:59.860)
there's going to be bias in the results.
Lex Fridman (2:03:02.380)
So we should not have bias in the data,
Lex Fridman (2:03:04.540)
but people piled on to him because he said
Chris Lattner (2:03:07.300)
he trivialized the problem of bias.
Lex Fridman (2:03:10.020)
Like it's a lot more than just bias in the data,
Lex Fridman (2:03:13.180)
but like, yes, that's a very good point,
Lex Fridman (2:03:16.500)
but that's not what he was saying.
Chris Lattner (2:03:18.900)
That's not what he was saying.
Lex Fridman (2:03:19.740)
And the response, like the implied response
Chris Lattner (2:03:23.100)
that he's basically sexist and racist
Lex Fridman (2:03:27.620)
is something that completely drives away
Chris Lattner (2:03:30.420)
the possibility of nuanced discussion.
Lex Fridman (2:03:32.860)
One nice thing about like a pocket long form of conversation
Chris Lattner (2:03:37.940)
is you can talk it out.
Lex Fridman (2:03:40.300)
You can lay your reasoning out.
Lex Fridman (2:03:42.860)
And even if you're wrong,
Lex Fridman (2:03:44.500)
you can still show that you're a good human being
Chris Lattner (2:03:47.140)
underneath it.
Lex Fridman (2:03:48.220)
You know, your point about
Chris Lattner (2:03:49.100)
you can't have a productive discussion.
Lex Fridman (2:03:50.980)
Well, how do you get to the point where people can turn?
Chris Lattner (2:03:53.860)
They can learn, they can listen, they can think,
Lex Fridman (2:03:56.260)
they can engage versus just being a shallow like,
Lex Fridman (2:04:00.660)
and then keep moving, right?
Lex Fridman (2:04:02.500)
And I don't think that progress really comes from that,
Lex Fridman (2:04:06.620)
right?
Lex Fridman (2:04:07.460)
And I don't think that one should expect that.
Chris Lattner (2:04:09.820)
I think that you'd see that as reinforcing
Lex Fridman (2:04:12.260)
individual circles and the us versus them thing.
Lex Fridman (2:04:14.460)
And I think that's fairly divisive.
Lex Fridman (2:04:17.500)
Yeah, I think there's a big role in,
Chris Lattner (2:04:20.900)
like the people that bother me most on Twitter
Lex Fridman (2:04:24.020)
when I observe things is not the people
Chris Lattner (2:04:26.580)
who get very emotional, angry, like over the top.
Lex Fridman (2:04:30.060)
It's the people who like prop them up.
Chris Lattner (2:04:33.820)
It's all the, it's this,
Lex Fridman (2:04:36.100)
I think what should be the,
Chris Lattner (2:04:37.940)
we should teach each other is to be sort of empathetic.
Lex Fridman (2:04:42.300)
The thing that it's really easy to forget,
Chris Lattner (2:04:44.660)
particularly on like Twitter or the internet or an email,
Lex Fridman (2:04:47.740)
is that sometimes people just have a bad day, right?
Chris Lattner (2:04:50.740)
You have a bad day or you're like,
Lex Fridman (2:04:53.100)
I've been in the situation where it's like between meetings,
Chris Lattner (2:04:55.500)
like fire off a quick response to an email
Lex Fridman (2:04:57.260)
because I want to like help get something unblocked,
Chris Lattner (2:05:00.660)
phrase it really objectively wrong.
Lex Fridman (2:05:03.620)
I screwed up.
Lex Fridman (2:05:04.980)
And suddenly this is now something that sticks with people.
Lex Fridman (2:05:08.660)
And it's not because they're bad.
Chris Lattner (2:05:10.540)
It's not because you're bad.
Lex Fridman (2:05:11.820)
Just psychology of like, you said a thing,
Chris Lattner (2:05:15.180)
it sticks with you.
Lex Fridman (2:05:16.020)
You didn't mean it that way,
Lex Fridman (2:05:16.940)
but it really impacted somebody
Lex Fridman (2:05:18.460)
because the way they interpret it.
Lex Fridman (2:05:20.860)
And this is just an aspect of working together as humans.
Lex Fridman (2:05:23.340)
And I have a lot of optimism in the long term,
Chris Lattner (2:05:26.140)
the very long term about what we as humanity can do.
Lex Fridman (2:05:29.060)
But I think that's going to be,
Chris Lattner (2:05:29.980)
it's just always a rough ride.
Lex Fridman (2:05:31.100)
And you came into this by saying like,
Lex Fridman (2:05:33.100)
what does COVID and all the social strife
Lex Fridman (2:05:36.180)
that's happening right now mean?
Lex Fridman (2:05:38.060)
And I think that it's really bad in the short term,
Lex Fridman (2:05:40.900)
but I think it'll lead to progress.
Lex Fridman (2:05:42.540)
And for that, I'm very thankful.
Lex Fridman (2:05:45.940)
Yeah, painful in the short term though.
Chris Lattner (2:05:47.980)
Well, yeah.
Lex Fridman (2:05:48.820)
I mean, people are out of jobs.
Chris Lattner (2:05:49.740)
Like some people can't eat.
Lex Fridman (2:05:50.860)
Like it's horrible.
Chris Lattner (2:05:52.500)
And, but you know, it's progress.
Lex Fridman (2:05:56.940)
So we'll see what happens.
Chris Lattner (2:05:58.500)
I mean, the real question is when you look back 10 years,
Lex Fridman (2:06:01.900)
20 years, a hundred years from now,
Lex Fridman (2:06:03.580)
how do we evaluate the decisions are being made right now?
Lex Fridman (2:06:06.860)
I think that's really the way you can frame that
Lex Fridman (2:06:09.780)
and look at it.
Lex Fridman (2:06:10.620)
And you say, you know,
Chris Lattner (2:06:11.660)
you integrate across all the short term horribleness
Lex Fridman (2:06:14.260)
that's happening and you look at what that means
Lex Fridman (2:06:17.220)
and is the improvement across the world
Lex Fridman (2:06:19.660)
or the regression across the world significant enough
Lex Fridman (2:06:22.980)
to make it a good or a bad thing?
Lex Fridman (2:06:24.700)
I think that's the question.
Chris Lattner (2:06:26.820)
Yeah.
Lex Fridman (2:06:27.660)
And for that, it's good to study history.
Chris Lattner (2:06:29.460)
I mean, one of the big problems for me right now
Lex Fridman (2:06:32.020)
is I'm reading The Rise and Fall of the Third Reich.
Lex Fridman (2:06:36.060)
Light reading?
Lex Fridman (2:06:37.340)
So it's everything is just,
Chris Lattner (2:06:39.660)
I just see parallels and it means it's,
Lex Fridman (2:06:42.140)
you have to be really careful not to overstep it,
Lex Fridman (2:06:45.300)
but just the thing that worries me the most
Lex Fridman (2:06:48.700)
is the pain that people feel when a few things combine,
Chris Lattner (2:06:54.380)
which is like economic depression,
Lex Fridman (2:06:55.940)
which is quite possible in this country.
Lex Fridman (2:06:57.820)
And then just being disrespected in some kind of way,
Lex Fridman (2:07:02.540)
which the German people were really disrespected
Chris Lattner (2:07:05.100)
by most of the world, like in a way that's over the top,
Lex Fridman (2:07:10.220)
that something can build up
Lex Fridman (2:07:12.100)
and then all you need is a charismatic leader
Lex Fridman (2:07:15.940)
to go either positive or negative and both work
Chris Lattner (2:07:19.460)
as long as they're charismatic.
Lex Fridman (2:07:21.060)
And there's...
Chris Lattner (2:07:22.140)
It's taking advantage of, again,
Lex Fridman (2:07:23.740)
that inflection point that the world's in
Lex Fridman (2:07:26.340)
and what they do with it could be good or bad.
Lex Fridman (2:07:29.700)
And so it's a good way to think about times now,
Chris Lattner (2:07:32.700)
like on an individual level,
Lex Fridman (2:07:34.740)
what we decide to do is when history is written,
Chris Lattner (2:07:38.260)
30 years from now, what happened in 2020,
Lex Fridman (2:07:40.940)
probably history is gonna remember 2020.
Chris Lattner (2:07:43.140)
Yeah, I think so.
Lex Fridman (2:07:43.980)
Either for good or bad,
Lex Fridman (2:07:46.820)
and it's up to us to write it so it's good.
Lex Fridman (2:07:49.540)
Well, one of the things I've observed
Chris Lattner (2:07:50.900)
that I find fascinating is most people act
Lex Fridman (2:07:54.180)
as though the world doesn't change.
Lex Fridman (2:07:56.460)
You make decision knowingly, right?
Lex Fridman (2:07:59.980)
You make a decision where you're predicting the future
Chris Lattner (2:08:02.620)
based on what you've seen in the recent past.
Lex Fridman (2:08:04.780)
And so if something's always been,
Chris Lattner (2:08:06.100)
it's rained every single day,
Lex Fridman (2:08:07.300)
then of course you expect it to rain today too, right?
Chris Lattner (2:08:10.060)
On the other hand, the world changes all the time.
Lex Fridman (2:08:13.420)
Yeah.
Chris Lattner (2:08:14.260)
Constantly, like for better and for worse.
Lex Fridman (2:08:16.780)
And so the question is,
Chris Lattner (2:08:17.700)
if you're interested in something that's not right,
Lex Fridman (2:08:20.900)
what is the inflection point that led to a change?
Lex Fridman (2:08:22.900)
And you can look to history for this.
Lex Fridman (2:08:24.380)
Like what is the catalyst that led to that explosion
Chris Lattner (2:08:27.980)
that led to that bill that led to the,
Lex Fridman (2:08:30.220)
like you can kind of work your way backwards from that.
Lex Fridman (2:08:33.220)
And maybe if you pull together the right people
Lex Fridman (2:08:35.740)
and you get the right ideas together,
Chris Lattner (2:08:36.940)
you can actually start driving that change
Lex Fridman (2:08:38.980)
and doing it in a way that's productive
Lex Fridman (2:08:40.340)
and hurts fewer people.
Lex Fridman (2:08:41.780)
Yeah, like a single person,
Chris Lattner (2:08:43.020)
a single event can turn all of history.
Lex Fridman (2:08:44.820)
Absolutely, everything starts somewhere.
Lex Fridman (2:08:46.420)
And often it's a combination of multiple factors,
Lex Fridman (2:08:48.500)
but yeah, these things can be engineered.
Chris Lattner (2:08:52.500)
That's actually the optimistic view that.
Lex Fridman (2:08:54.980)
I'm a longterm optimist on pretty much everything
Lex Fridman (2:08:57.580)
and human nature.
Lex Fridman (2:08:59.340)
We can look to all the negative things
Chris Lattner (2:09:00.700)
that humanity has, all the pettiness
Lex Fridman (2:09:03.300)
and all the like self servingness
Lex Fridman (2:09:05.860)
and the just the cruelty, right?
Lex Fridman (2:09:09.780)
The biases, the just humans can be very horrible.
Lex Fridman (2:09:13.380)
But on the other hand, we're capable of amazing things.
Lex Fridman (2:09:17.140)
And the progress across 100 year chunks
Chris Lattner (2:09:21.540)
is striking.
Lex Fridman (2:09:23.300)
And even across decades, we've come a long ways
Lex Fridman (2:09:26.700)
and there's still a long ways to go,
Lex Fridman (2:09:27.820)
but that doesn't mean that we've stopped.
Chris Lattner (2:09:29.980)
Yeah, the kind of stuff we've done in the last 100 years
Lex Fridman (2:09:33.060)
is unbelievable.
Chris Lattner (2:09:34.900)
It's kind of scary to think what's gonna happen
Lex Fridman (2:09:36.740)
in this 100 years.
Chris Lattner (2:09:37.580)
It's scary, like exciting.
Lex Fridman (2:09:39.020)
Like scary in a sense that it's kind of sad
Chris Lattner (2:09:41.700)
that the kind of technology is gonna come out
Lex Fridman (2:09:43.780)
in 10, 20, 30 years.
Chris Lattner (2:09:45.740)
We're probably too old to really appreciate
Lex Fridman (2:09:47.820)
if you don't grow up with it.
Chris Lattner (2:09:49.100)
It'll be like kids these days with their virtual reality
Lex Fridman (2:09:51.700)
and their TikToks and stuff like this.
Chris Lattner (2:09:54.500)
Like, how does this thing and like,
Lex Fridman (2:09:56.820)
come on, give me my static photo.
Chris Lattner (2:10:00.860)
My Commodore 64.
Lex Fridman (2:10:02.300)
Yeah, exactly.
Chris Lattner (2:10:03.740)
Okay, sorry, we kind of skipped over.
Lex Fridman (2:10:05.820)
Let me ask on, the machine learning world
Chris Lattner (2:10:11.060)
has been kind of inspired, their imagination captivated
Lex Fridman (2:10:15.740)
with GPT3 and these language models.
Chris Lattner (2:10:18.740)
I thought it'd be cool to get your opinion on it.
Lex Fridman (2:10:21.820)
What's your thoughts on this exciting world of,
Chris Lattner (2:10:27.300)
it connects to computation actually,
Lex Fridman (2:10:29.940)
is of language models that are huge
Lex Fridman (2:10:33.020)
and take many, many computers, not just to train,
Lex Fridman (2:10:37.420)
but to also do inference on.
Chris Lattner (2:10:39.420)
Sure.
Lex Fridman (2:10:40.460)
Well, I mean, it depends on what you're speaking to there.
Lex Fridman (2:10:43.420)
But I mean, I think that there's been
Lex Fridman (2:10:45.300)
a pretty well understood maximum in deep learning
Chris Lattner (2:10:48.380)
that if you make the model bigger
Lex Fridman (2:10:49.660)
and you shove more data into it,
Chris Lattner (2:10:51.380)
assuming you train it right
Lex Fridman (2:10:52.420)
and you have a good model architecture,
Chris Lattner (2:10:54.020)
that you'll get a better model out.
Lex Fridman (2:10:55.820)
And so on one hand, GPT3 was not that surprising.
Chris Lattner (2:10:59.740)
On the other hand, a tremendous amount of engineering
Lex Fridman (2:11:02.060)
went into making it possible.
Chris Lattner (2:11:04.740)
The implications of it are pretty huge.
Lex Fridman (2:11:07.060)
I think that when GPT2 came out,
Chris Lattner (2:11:08.980)
there was a very provocative blog post from OpenAI
Lex Fridman (2:11:11.380)
talking about, we're not gonna release it
Chris Lattner (2:11:13.660)
because of the social damage it could cause
Lex Fridman (2:11:15.460)
if it's misused.
Chris Lattner (2:11:18.660)
I think that's still a concern.
Lex Fridman (2:11:20.140)
I think that we need to look at how technology is applied
Lex Fridman (2:11:23.300)
and well meaning tools can be applied in very horrible ways
Lex Fridman (2:11:26.900)
and they can have very profound impact on that.
Chris Lattner (2:11:30.620)
I think that GPT3 is a huge technical achievement.
Lex Fridman (2:11:34.020)
And what will GPT4 be?
Chris Lattner (2:11:35.780)
Well, it'll probably be bigger, more expensive to train.
Lex Fridman (2:11:38.540)
Really cool architectural tricks.
Lex Fridman (2:11:42.020)
What do you think, is there,
Lex Fridman (2:11:43.980)
I don't know how much thought you've done
Chris Lattner (2:11:46.460)
on distributed computing.
Lex Fridman (2:11:48.700)
Is there some technical challenges that are interesting
Chris Lattner (2:11:52.940)
that you're hopeful about exploring
Lex Fridman (2:11:54.620)
in terms of a system that,
Chris Lattner (2:11:57.660)
like a piece of code that with GPT4 that might have,
Lex Fridman (2:12:05.260)
I don't know, hundreds of trillions of parameters
Chris Lattner (2:12:09.340)
which have to run on thousands of computers.
Lex Fridman (2:12:11.580)
Is there some hope that we can make that happen?
Chris Lattner (2:12:15.340)
Yeah, well, I mean, today you can write a check
Lex Fridman (2:12:18.940)
and get access to a thousand TPU cores
Lex Fridman (2:12:21.780)
and do really interesting large scale training
Lex Fridman (2:12:23.940)
and inference and things like that in Google Cloud,
Lex Fridman (2:12:26.540)
for example, right?
Lex Fridman (2:12:27.420)
And so I don't think it's a question about scale,
Chris Lattner (2:12:31.340)
it's a question about utility.
Lex Fridman (2:12:33.220)
And when I look at the transformer series of architectures
Chris Lattner (2:12:36.220)
that the GPT series is based on,
Lex Fridman (2:12:38.780)
it's really interesting to look at that
Chris Lattner (2:12:39.900)
because they're actually very simple designs.
Lex Fridman (2:12:42.940)
They're not recurrent.
Chris Lattner (2:12:44.740)
The training regimens are pretty simple.
Lex Fridman (2:12:47.460)
And so they don't really reflect like human brains, right?
Lex Fridman (2:12:51.700)
But they're really good at learning language models
Lex Fridman (2:12:54.620)
and they're unrolled enough
Lex Fridman (2:12:55.740)
that you can simulate some recurrence, right?
Lex Fridman (2:12:59.020)
And so the question I think about is,
Lex Fridman (2:13:02.100)
where does this take us?
Lex Fridman (2:13:03.260)
Like, so we can just keep scaling it,
Chris Lattner (2:13:05.140)
have more parameters, more data, more things,
Lex Fridman (2:13:07.700)
we'll get a better result for sure.
Lex Fridman (2:13:09.460)
But are there architectural techniques
Lex Fridman (2:13:11.820)
that can lead to progress at a faster pace, right?
Chris Lattner (2:13:15.300)
This is when, you know, how do you get,
Lex Fridman (2:13:17.740)
instead of just like making it a constant time bigger,
Lex Fridman (2:13:20.660)
how do you get like an algorithmic improvement
Lex Fridman (2:13:23.380)
out of this, right?
Lex Fridman (2:13:24.220)
And whether it be a new training regimen,
Lex Fridman (2:13:25.780)
if it becomes sparse networks, for example,
Chris Lattner (2:13:30.380)
the human brain is sparse, all these networks are dense.
Lex Fridman (2:13:33.660)
The connectivity patterns can be very different.
Chris Lattner (2:13:36.140)
I think this is where I get very interested
Lex Fridman (2:13:38.260)
and I'm way out of my league
Chris Lattner (2:13:39.500)
on the deep learning side of this.
Lex Fridman (2:13:41.580)
But I think that could lead to big breakthroughs.
Chris Lattner (2:13:43.700)
When you talk about large scale networks,
Lex Fridman (2:13:46.140)
one of the things that Jeff Dean likes to talk about
Lex Fridman (2:13:47.980)
and he's given a few talks on
Lex Fridman (2:13:50.940)
is this idea of having a sparsely gated mixture of experts
Chris Lattner (2:13:54.220)
kind of a model where you have, you know,
Lex Fridman (2:13:57.420)
different nets that are trained
Lex Fridman (2:13:59.460)
and are really good at certain kinds of tasks.
Lex Fridman (2:14:02.060)
And so you have this distributor across a cluster
Lex Fridman (2:14:04.820)
and so you have a lot of different computers
Lex Fridman (2:14:06.420)
that end up being kind of locally specialized
Chris Lattner (2:14:08.580)
in different demands.
Lex Fridman (2:14:09.740)
And then when a query comes in,
Chris Lattner (2:14:11.060)
you gate it and you use learn techniques
Lex Fridman (2:14:13.740)
to route to different parts of the network.
Lex Fridman (2:14:15.460)
And then you utilize the compute resources
Lex Fridman (2:14:18.020)
of the entire cluster by having specialization within it.
Lex Fridman (2:14:20.660)
And I don't know where that goes
Lex Fridman (2:14:23.700)
or when it starts to work,
Lex Fridman (2:14:25.540)
but I think things like that
Lex Fridman (2:14:26.700)
could be really interesting as well.
Lex Fridman (2:14:28.380)
And on the data side too,
Lex Fridman (2:14:30.060)
if you can think of data selection
Chris Lattner (2:14:33.700)
as a kind of programming.
Lex Fridman (2:14:35.820)
Yeah.
Chris Lattner (2:14:36.660)
I mean, essentially, if you look at it,
Lex Fridman (2:14:37.980)
like Karpathy talked about software 2.0,
Chris Lattner (2:14:40.580)
I mean, in a sense, data is the program.
Lex Fridman (2:14:44.020)
Yeah, yeah.
Lex Fridman (2:14:44.860)
So let me try to summarize Andre's position really quick
Lex Fridman (2:14:48.340)
before I disagree with it.
Chris Lattner (2:14:50.020)
Yeah.
Lex Fridman (2:14:51.140)
So Andre Karpathy is amazing.
Lex Fridman (2:14:53.420)
So this is nothing personal with him.
Lex Fridman (2:14:55.180)
He's an amazing engineer.
Lex Fridman (2:14:57.380)
And also a good blog post writer.
Lex Fridman (2:14:59.220)
Yeah, well, he's a great communicator.
Chris Lattner (2:15:01.100)
You know, he's just an amazing person.
Lex Fridman (2:15:02.420)
He's also really sweet.
Lex Fridman (2:15:03.720)
So his basic premise is that software is suboptimal.
Lex Fridman (2:15:09.360)
I think we can all agree to that.
Chris Lattner (2:15:11.040)
He also points out that deep learning
Lex Fridman (2:15:14.480)
and other learning based techniques are really great
Chris Lattner (2:15:16.360)
because you can solve problems
Lex Fridman (2:15:17.520)
in more structured ways with less like ad hoc code
Chris Lattner (2:15:22.120)
that people write out and don't write test cases for
Lex Fridman (2:15:24.440)
in some cases.
Lex Fridman (2:15:25.280)
And so they don't even know if it works in the first place.
Lex Fridman (2:15:27.800)
And so if you start replacing systems of imperative code
Chris Lattner (2:15:32.320)
with deep learning models, then you get a better result.
Lex Fridman (2:15:37.400)
Okay.
Lex Fridman (2:15:38.380)
And I think that he argues that software 2.0
Lex Fridman (2:15:40.680)
is a pervasively learned set of models
Lex Fridman (2:15:44.120)
and you get away from writing code.
Lex Fridman (2:15:45.920)
And he's given talks where he talks about, you know,
Chris Lattner (2:15:49.040)
swapping over more and more and more parts of the code
Lex Fridman (2:15:50.960)
to being learned and driven that way.
Chris Lattner (2:15:54.840)
I think that works.
Lex Fridman (2:15:56.640)
And if you're predisposed to liking machine learning,
Chris Lattner (2:15:59.280)
then I think that that's definitely a good thing.
Lex Fridman (2:16:01.760)
I think this is also good for accessibility in many ways
Chris Lattner (2:16:04.720)
because certain people are not gonna write C code
Lex Fridman (2:16:06.800)
or something.
Lex Fridman (2:16:07.720)
And so having a data driven approach to do
Lex Fridman (2:16:10.200)
this kind of stuff, I think can be very valuable.
Chris Lattner (2:16:12.720)
On the other hand, there are huge trade offs.
Lex Fridman (2:16:14.200)
It's not clear to me that software 2.0 is the answer.
Lex Fridman (2:16:19.200)
And probably Andre wouldn't argue that it's the answer
Lex Fridman (2:16:21.440)
for every problem either.
Lex Fridman (2:16:22.960)
But I look at machine learning as not a replacement
Lex Fridman (2:16:26.760)
for software 1.0.
Chris Lattner (2:16:27.920)
I look at it as a new programming paradigm.
Lex Fridman (2:16:30.120)
And so programming paradigms, when you look across demands,
Chris Lattner (2:16:35.140)
is structured programming where you go from go tos
Lex Fridman (2:16:38.500)
to if, then, else, or functional programming from Lisp.
Lex Fridman (2:16:42.300)
And you start talking about higher order functions
Lex Fridman (2:16:44.440)
and values and things like this.
Chris Lattner (2:16:45.900)
Or you talk about object oriented programming.
Lex Fridman (2:16:48.060)
You're talking about encapsulation,
Chris Lattner (2:16:49.440)
subclassing, inheritance.
Lex Fridman (2:16:50.460)
You start talking about generic programming
Chris Lattner (2:16:52.640)
where you start talking about code reuse
Lex Fridman (2:16:54.460)
through specialization and different type instantiations.
Chris Lattner (2:16:59.240)
When you start talking about differentiable programming,
Lex Fridman (2:17:01.740)
something that I am very excited about
Chris Lattner (2:17:04.500)
in the context of machine learning,
Lex Fridman (2:17:05.940)
talking about taking functions and generating variants,
Chris Lattner (2:17:09.500)
like the derivative of another function.
Lex Fridman (2:17:11.140)
Like that's a programming paradigm that's very useful
Chris Lattner (2:17:13.780)
for solving certain classes of problems.
Lex Fridman (2:17:16.220)
Machine learning is amazing
Chris Lattner (2:17:17.660)
at solving certain classes of problems.
Lex Fridman (2:17:19.180)
Like you're not gonna write a cat detector
Chris Lattner (2:17:21.940)
or even a language translation system by writing C code.
Lex Fridman (2:17:25.900)
That's not a very productive way to do things anymore.
Lex Fridman (2:17:28.920)
And so machine learning is absolutely
Lex Fridman (2:17:31.060)
the right way to do that.
Chris Lattner (2:17:31.980)
In fact, I would say that learned models
Lex Fridman (2:17:34.120)
are really one of the best ways to work
Chris Lattner (2:17:35.980)
with the human world in general.
Lex Fridman (2:17:38.220)
And so anytime you're talking about sensory input
Chris Lattner (2:17:40.320)
of different modalities,
Lex Fridman (2:17:41.360)
anytime that you're talking about generating things
Chris Lattner (2:17:44.300)
in a way that makes sense to a human,
Lex Fridman (2:17:45.780)
I think that learned models are really, really useful.
Lex Fridman (2:17:48.900)
And that's because humans are very difficult
Lex Fridman (2:17:50.580)
to characterize, okay?
Lex Fridman (2:17:52.620)
And so this is a very powerful paradigm
Lex Fridman (2:17:55.220)
for solving classes of problems.
Lex Fridman (2:17:57.120)
But on the other hand, imperative code is too.
Lex Fridman (2:17:59.700)
You're not gonna write a bootloader for your computer
Chris Lattner (2:18:03.020)
with a deep learning model.
Lex Fridman (2:18:04.060)
Deep learning models are very hardware intensive.
Chris Lattner (2:18:07.060)
They're very energy intensive
Lex Fridman (2:18:08.980)
because you have a lot of parameters
Lex Fridman (2:18:11.060)
and you can provably implement any function
Lex Fridman (2:18:14.520)
with a learned model, like this has been shown,
Lex Fridman (2:18:17.700)
but that doesn't make it efficient.
Lex Fridman (2:18:20.060)
And so if you're talking about caring about a few orders
Chris Lattner (2:18:22.300)
of magnitudes worth of energy usage,
Lex Fridman (2:18:24.100)
then it's useful to have other tools in the toolbox.
Chris Lattner (2:18:26.940)
There's also robustness too.
Lex Fridman (2:18:28.420)
I mean, as a...
Chris Lattner (2:18:29.260)
Yeah, exactly.
Lex Fridman (2:18:30.100)
All the problems of dealing with data and bias in data,
Chris Lattner (2:18:32.500)
all the problems of software 2.0.
Lex Fridman (2:18:35.100)
And one of the great things that Andre is arguing towards,
Chris Lattner (2:18:39.340)
which I completely agree with him,
Lex Fridman (2:18:40.940)
is that when you start implementing things
Chris Lattner (2:18:43.820)
with deep learning, you need to learn from software 1.0
Lex Fridman (2:18:46.220)
in terms of testing, continuous integration,
Lex Fridman (2:18:49.060)
how you deploy, how do you validate,
Lex Fridman (2:18:51.220)
all these things and building systems around that
Lex Fridman (2:18:53.980)
so that you're not just saying like,
Lex Fridman (2:18:55.020)
oh, it seems like it's good, ship it, right?
Lex Fridman (2:18:58.460)
Well, what happens when I regress something?
Lex Fridman (2:18:59.840)
What happens when I make a classification that's wrong
Lex Fridman (2:19:02.500)
and now I hurt somebody, right?
Lex Fridman (2:19:05.540)
I mean, all these things you have to reason about.
Chris Lattner (2:19:07.380)
Yeah, but at the same time,
Lex Fridman (2:19:08.380)
the bootloader that works for us humans
Lex Fridman (2:19:12.980)
looks awfully a lot like a neural network, right?
Lex Fridman (2:19:15.700)
Yeah.
Chris Lattner (2:19:16.540)
It's messy and you can cut out
Lex Fridman (2:19:18.520)
different parts of the brain.
Chris Lattner (2:19:19.780)
There's a lot of this neuroplasticity work
Lex Fridman (2:19:22.400)
that shows that it's gonna adjust.
Chris Lattner (2:19:24.100)
It's a really interesting question,
Lex Fridman (2:19:26.900)
how much of the world's programming
Lex Fridman (2:19:29.700)
could be replaced by software 2.0?
Lex Fridman (2:19:31.780)
Like with...
Chris Lattner (2:19:32.620)
Oh, well, I mean, it's provably true
Lex Fridman (2:19:35.180)
that you could replace all of it.
Chris Lattner (2:19:37.540)
Right, so then it's a question of the trade offs.
Lex Fridman (2:19:39.220)
Anything that's a function, you can.
Lex Fridman (2:19:40.940)
So it's not a question about if.
Lex Fridman (2:19:42.940)
I think it's a economic question.
Lex Fridman (2:19:44.900)
It's a, what kind of talent can you get?
Lex Fridman (2:19:47.740)
What kind of trade offs in terms of maintenance, right?
Chris Lattner (2:19:50.460)
Those kinds of questions, I think.
Lex Fridman (2:19:51.680)
What kind of data can you collect?
Chris Lattner (2:19:53.280)
I think one of the reasons that I'm most interested
Lex Fridman (2:19:55.120)
in machine learning as a programming paradigm
Chris Lattner (2:19:58.580)
is that one of the things that we've seen
Lex Fridman (2:20:00.340)
across computing in general is that
Chris Lattner (2:20:02.340)
being laser focused on one paradigm
Lex Fridman (2:20:04.680)
often puts you in a box that's not super great.
Lex Fridman (2:20:07.540)
And so you look at object oriented programming,
Lex Fridman (2:20:10.420)
like it was all the rage in the early 80s
Lex Fridman (2:20:12.060)
and like everything has to be objects.
Lex Fridman (2:20:13.520)
And people forgot about functional programming
Chris Lattner (2:20:15.620)
even though it came first.
Lex Fridman (2:20:17.380)
And then people rediscovered that,
Chris Lattner (2:20:19.820)
hey, if you mix functional and object oriented
Lex Fridman (2:20:21.820)
in structure, like you mix these things together,
Chris Lattner (2:20:24.300)
you can provide very interesting tools
Lex Fridman (2:20:25.820)
that are good at solving different problems.
Lex Fridman (2:20:28.460)
And so the question there is how do you get
Lex Fridman (2:20:30.800)
the best way to solve the problems?
Lex Fridman (2:20:32.660)
It's not about whose tribe should win, right?
Lex Fridman (2:20:36.020)
It's not about, you know, that shouldn't be the question.
Chris Lattner (2:20:38.780)
The question is how do you make it
Lex Fridman (2:20:40.060)
so that people can solve those problems the fastest
Lex Fridman (2:20:42.180)
and they have the right tools in their box
Lex Fridman (2:20:44.340)
to build good libraries and they can solve these problems.
Lex Fridman (2:20:47.180)
And when you look at that, that's like, you know,
Lex Fridman (2:20:49.100)
you look at reinforcement learning
Chris Lattner (2:20:50.340)
as one really interesting subdomain of this.
Lex Fridman (2:20:52.660)
Reinforcement learning, often you have to have
Chris Lattner (2:20:55.080)
the integration of a learned model
Lex Fridman (2:20:57.660)
combined with your Atari or whatever the other scenario
Chris Lattner (2:21:00.820)
it is that you're working in.
Lex Fridman (2:21:02.880)
You have to combine that thing with the robot control
Lex Fridman (2:21:05.700)
for the arm, right?
Lex Fridman (2:21:07.660)
And so now it's not just about that one paradigm,
Chris Lattner (2:21:11.960)
it's about integrating that with all the other systems
Lex Fridman (2:21:14.600)
that you have, including often legacy systems
Lex Fridman (2:21:17.100)
and things like this, right?
Lex Fridman (2:21:18.160)
And so to me, I think that the interesting thing to say
Chris Lattner (2:21:21.500)
is like, how do you get the best out of this domain
Lex Fridman (2:21:23.820)
and how do you enable people to achieve things
Chris Lattner (2:21:25.820)
that they otherwise couldn't do
Lex Fridman (2:21:27.340)
without excluding all the good things
Lex Fridman (2:21:29.720)
we already know how to do?
Lex Fridman (2:21:31.300)
Right, but okay, this is a crazy question,
Lex Fridman (2:21:35.340)
but we talked a little bit about GPT3,
Lex Fridman (2:21:38.860)
but do you think it's possible that these language models
Chris Lattner (2:21:42.340)
that in essence, in the language domain,
Lex Fridman (2:21:47.340)
Software 2.0 could replace some aspect of compilation,
Chris Lattner (2:21:51.820)
for example, or do program synthesis,
Lex Fridman (2:21:54.260)
replace some aspect of programming?
Chris Lattner (2:21:56.900)
Yeah, absolutely.
Lex Fridman (2:21:57.740)
So I think that learned models in general
Chris Lattner (2:22:00.380)
are extremely powerful,
Lex Fridman (2:22:01.580)
and I think that people underestimate them.
Chris Lattner (2:22:04.740)
Maybe you can suggest what I should do.
Lex Fridman (2:22:07.180)
So if I have access to the GPT3 API,
Lex Fridman (2:22:11.380)
would I be able to generate Swift code, for example?
Lex Fridman (2:22:14.260)
Do you think that could do something interesting
Lex Fridman (2:22:16.060)
and would work?
Lex Fridman (2:22:17.420)
So GPT3 is probably not trained on the right corpus,
Lex Fridman (2:22:21.140)
so it probably has the ability to generate some Swift.
Lex Fridman (2:22:23.700)
I bet it does.
Chris Lattner (2:22:25.220)
It's probably not gonna generate a large enough body
Lex Fridman (2:22:27.280)
of Swift to be useful,
Lex Fridman (2:22:28.340)
but take it a next step further.
Lex Fridman (2:22:30.580)
Like if you had the goal of training something like GPT3
Lex Fridman (2:22:33.980)
and you wanted to train it to generate source code, right?
Lex Fridman (2:22:38.020)
It could definitely do that.
Chris Lattner (2:22:39.780)
Now the question is, how do you express the intent
Lex Fridman (2:22:42.640)
of what you want filled in?
Chris Lattner (2:22:44.300)
You can definitely write scaffolding of code
Lex Fridman (2:22:47.060)
and say, fill in the hole,
Lex Fridman (2:22:48.940)
and sort of put in some for loops,
Lex Fridman (2:22:50.300)
or put in some classes or whatever.
Lex Fridman (2:22:51.540)
And the power of these models is impressive,
Lex Fridman (2:22:53.700)
but there's an unsolved question, at least unsolved to me,
Lex Fridman (2:22:56.940)
which is, how do I express the intent of what to fill in?
Lex Fridman (2:22:59.700)
Right?
Lex Fridman (2:23:01.000)
And kind of what you'd really want to have,
Lex Fridman (2:23:03.180)
and I don't know that these models are up to the task,
Chris Lattner (2:23:06.340)
is you wanna be able to say,
Lex Fridman (2:23:08.300)
here's the scaffolding,
Lex Fridman (2:23:09.660)
and here are the assertions at the end.
Lex Fridman (2:23:12.460)
And the assertions always pass.
Lex Fridman (2:23:14.020)
And so you want a generative model on the one hand, yes.
Lex Fridman (2:23:16.460)
Oh, that's fascinating, yeah.
Chris Lattner (2:23:17.580)
Right, but you also want some loop back,
Lex Fridman (2:23:20.420)
some reinforcement learning system or something,
Chris Lattner (2:23:23.180)
where you're actually saying like,
Lex Fridman (2:23:24.660)
I need to hill climb towards something
Chris Lattner (2:23:26.580)
that is more correct.
Lex Fridman (2:23:28.500)
And I don't know that we have that.
Lex Fridman (2:23:29.720)
So it would generate not only a bunch of the code,
Lex Fridman (2:23:33.640)
but like the checks that do the testing.
Chris Lattner (2:23:35.940)
It would generate the tests.
Lex Fridman (2:23:37.100)
I think the humans would generate the tests, right?
Chris Lattner (2:23:38.860)
Oh, okay.
Lex Fridman (2:23:39.700)
But it would be fascinating if...
Chris Lattner (2:23:41.340)
Well, the tests are the requirements.
Lex Fridman (2:23:43.060)
Yes, but the, okay, so...
Chris Lattner (2:23:44.260)
Because you have to express to the model
Lex Fridman (2:23:45.940)
what you want to...
Chris Lattner (2:23:47.100)
You don't just want gibberish code.
Lex Fridman (2:23:49.020)
Look at how compelling this code looks.
Chris Lattner (2:23:51.340)
You want a story about four horned unicorns or something.
Lex Fridman (2:23:54.800)
Well, okay, so exactly.
Lex Fridman (2:23:55.980)
But that's human requirements.
Lex Fridman (2:23:57.720)
But then I thought it's a compelling idea
Chris Lattner (2:24:00.220)
that the GPT4 model could generate checks
Lex Fridman (2:24:06.980)
that are more high fidelity that check for correctness.
Chris Lattner (2:24:11.980)
Because the code it generates,
Lex Fridman (2:24:14.680)
like say I ask it to generate a function
Chris Lattner (2:24:18.400)
that gives me the Fibonacci sequence.
Lex Fridman (2:24:21.620)
Sure.
Chris Lattner (2:24:22.460)
I don't like...
Lex Fridman (2:24:24.340)
So decompose the problem, right?
Lex Fridman (2:24:25.640)
So you have two things.
Lex Fridman (2:24:26.980)
You have, you need the ability to generate
Lex Fridman (2:24:29.360)
syntactically correct Swift code that's interesting, right?
Lex Fridman (2:24:33.080)
I think GPT series of model architectures can do that.
Lex Fridman (2:24:37.560)
But then you need the ability to add the requirements.
Lex Fridman (2:24:41.320)
So generate Fibonacci.
Chris Lattner (2:24:43.360)
The human needs to express that goal.
Lex Fridman (2:24:46.040)
We don't have that language that I know of.
Chris Lattner (2:24:49.160)
No, I mean, it can generate stuff.
Lex Fridman (2:24:50.840)
Have you seen what GPT3 can generate?
Chris Lattner (2:24:52.840)
You can say, I mean, there's a interface stuff
Lex Fridman (2:24:55.760)
like it can generate HTML.
Chris Lattner (2:24:58.360)
It can generate basic for loops that give you like...
Lex Fridman (2:25:02.000)
Right, but pick HTML.
Lex Fridman (2:25:02.880)
How do I say I want google.com?
Lex Fridman (2:25:06.080)
Well, no, you could say...
Chris Lattner (2:25:07.800)
Or not literally google.com.
Lex Fridman (2:25:09.360)
How do I say I want a webpage
Lex Fridman (2:25:10.520)
that's got a shopping cart and this and that?
Lex Fridman (2:25:13.160)
It does that.
Chris Lattner (2:25:14.000)
I mean, so, okay.
Lex Fridman (2:25:14.840)
So just, I don't know if you've seen these demonstrations
Lex Fridman (2:25:17.680)
but you type in, I want a red button
Lex Fridman (2:25:20.340)
with the text that says hello.
Lex Fridman (2:25:22.440)
And you type that in natural language
Lex Fridman (2:25:24.160)
and it generates the correct HTML.
Chris Lattner (2:25:26.120)
I've done this demo.
Lex Fridman (2:25:27.600)
It's kind of compelling.
Lex Fridman (2:25:29.000)
So you have to prompt it with similar kinds of mappings.
Lex Fridman (2:25:33.280)
Of course, it's probably handpicked.
Chris Lattner (2:25:35.640)
I got to experiment that probably,
Lex Fridman (2:25:37.940)
but the fact that you can do that once
Chris Lattner (2:25:39.520)
even out of like 20 is quite impressive.
Lex Fridman (2:25:43.200)
Again, that's very basic.
Chris Lattner (2:25:45.200)
Like the HTML is kind of messy and bad.
Lex Fridman (2:25:48.440)
But yes, the intent is...
Chris Lattner (2:25:49.960)
The idea is the intent is specified in natural language.
Lex Fridman (2:25:53.320)
Yeah, so I have not seen that.
Chris Lattner (2:25:54.400)
That's really cool.
Lex Fridman (2:25:55.240)
Yeah.
Chris Lattner (2:25:56.560)
Yeah, but the question is the correctness of that.
Lex Fridman (2:25:59.840)
Like visually you can check, oh, the button is red,
Lex Fridman (2:26:02.840)
but for more complicated functions
Lex Fridman (2:26:10.160)
where the intent is harder to check.
Chris Lattner (2:26:12.080)
This goes into like NP completeness kind of things.
Lex Fridman (2:26:15.460)
Like I want to know that this code is correct
Lex Fridman (2:26:18.120)
and generates a giant thing
Lex Fridman (2:26:20.720)
that does some kind of calculation.
Chris Lattner (2:26:23.680)
It seems to be working.
Lex Fridman (2:26:25.400)
It's interesting to think like,
Chris Lattner (2:26:27.880)
should the system also try to generate checks
Lex Fridman (2:26:30.720)
for itself for correctness?
Chris Lattner (2:26:32.080)
Yeah, I don't know.
Lex Fridman (2:26:33.000)
And this is way beyond my experience.
Chris Lattner (2:26:35.960)
The thing that I think about is that
Lex Fridman (2:26:39.200)
there doesn't seem to be a lot of
Chris Lattner (2:26:41.120)
equational reasoning going on.
Lex Fridman (2:26:43.480)
There's a lot of pattern matching and filling in
Lex Fridman (2:26:45.280)
and kind of propagating patterns
Lex Fridman (2:26:47.560)
that have been seen before into the future
Lex Fridman (2:26:49.220)
and into the generator result.
Lex Fridman (2:26:50.680)
And so if you want to get correctness,
Chris Lattner (2:26:53.240)
you kind of need theorem proving kind of things
Lex Fridman (2:26:55.180)
and like higher level logic.
Lex Fridman (2:26:57.320)
And I don't know that...
Lex Fridman (2:26:58.600)
You could talk to Jan about that
Lex Fridman (2:27:00.520)
and see what the bright minds
Lex Fridman (2:27:03.560)
are thinking about right now,
Lex Fridman (2:27:04.720)
but I don't think the GPT is in that vein.
Lex Fridman (2:27:08.180)
It's still really cool.
Chris Lattner (2:27:09.240)
Yeah, and surprisingly, who knows,
Lex Fridman (2:27:11.880)
maybe reasoning is...
Chris Lattner (2:27:13.960)
Is overrated.
Lex Fridman (2:27:14.780)
Yeah, is overrated.
Lex Fridman (2:27:15.620)
Right, I mean, do we reason?
Lex Fridman (2:27:17.320)
Yeah.
Lex Fridman (2:27:18.160)
How do you tell, right?
Lex Fridman (2:27:18.980)
Are we just pattern matching based on what we have
Lex Fridman (2:27:20.560)
and then reverse justifying to ourselves?
Lex Fridman (2:27:22.960)
Exactly, the reverse.
Lex Fridman (2:27:24.280)
So like I think what the neural networks are missing
Lex Fridman (2:27:26.920)
and I think GPT4 might have
Chris Lattner (2:27:29.820)
is to be able to tell stories to itself about what it did.
Lex Fridman (2:27:33.800)
Well, that's what humans do, right?
Lex Fridman (2:27:34.920)
I mean, you talk about like network explainability, right?
Lex Fridman (2:27:38.240)
And we give, no, that's a hard time about this,
Lex Fridman (2:27:40.720)
but humans don't know why we make decisions.
Lex Fridman (2:27:42.420)
We have this thing called intuition
Lex Fridman (2:27:43.800)
and then we try to like say,
Lex Fridman (2:27:45.240)
this feels like the right thing, but why, right?
Lex Fridman (2:27:47.360)
And you wrestle with that
Lex Fridman (2:27:49.120)
when you're making hard decisions
Lex Fridman (2:27:50.320)
and is that science?
Lex Fridman (2:27:52.200)
Not really.
Chris Lattner (2:27:54.400)
Let me ask you about a few high level questions, I guess.
Lex Fridman (2:27:57.400)
Because you've done a million things in your life
Lex Fridman (2:28:02.400)
and been very successful.
Lex Fridman (2:28:04.240)
A bunch of young folks listen to this,
Chris Lattner (2:28:07.000)
ask for advice from successful people like you.
Lex Fridman (2:28:11.720)
If you were to give advice to somebody,
Chris Lattner (2:28:15.680)
you know, another graduate student
Lex Fridman (2:28:17.080)
or some high school student
Chris Lattner (2:28:19.040)
about pursuing a career in computing
Lex Fridman (2:28:23.560)
or just advice about life in general,
Lex Fridman (2:28:25.600)
is there some words of wisdom you can give them?
Lex Fridman (2:28:28.880)
So I think you come back to change
Lex Fridman (2:28:30.860)
and profound leaps happen
Lex Fridman (2:28:34.160)
because people are willing to believe
Chris Lattner (2:28:35.420)
that change is possible
Lex Fridman (2:28:36.520)
and that the world does change
Lex Fridman (2:28:39.200)
and are willing to do the hard thing
Lex Fridman (2:28:41.040)
that it takes to make change happen.
Lex Fridman (2:28:42.720)
And whether it be implementing a new programming language
Lex Fridman (2:28:45.920)
or employing a new system
Chris Lattner (2:28:47.120)
or employing a new research paper,
Lex Fridman (2:28:49.240)
designing a new thing,
Chris Lattner (2:28:50.240)
moving the world forward in science
Lex Fridman (2:28:51.680)
and philosophy, whatever,
Chris Lattner (2:28:53.540)
it really comes down to somebody
Lex Fridman (2:28:54.560)
who's willing to put in the work, right?
Lex Fridman (2:28:57.120)
And you have, the work is hard
Lex Fridman (2:29:00.320)
for a whole bunch of different reasons.
Lex Fridman (2:29:01.560)
One of which is, it's work, right?
Lex Fridman (2:29:06.960)
And so you have to have the space in your life
Chris Lattner (2:29:08.840)
in which you can do that work,
Lex Fridman (2:29:09.880)
which is why going to grad school
Chris Lattner (2:29:11.020)
can be a beautiful thing for certain people.
Lex Fridman (2:29:14.720)
But also there's a self doubt that happens.
Chris Lattner (2:29:16.860)
Like you're two years into a project,
Lex Fridman (2:29:18.360)
is it going anywhere, right?
Lex Fridman (2:29:20.320)
Well, what do you do?
Lex Fridman (2:29:21.160)
Do you just give up because it's hard?
Chris Lattner (2:29:23.320)
No, no, I mean, some people like suffering.
Lex Fridman (2:29:26.720)
And so you plow through it.
Chris Lattner (2:29:29.280)
The secret to me is that you have to love what you're doing
Lex Fridman (2:29:31.960)
and follow that passion
Chris Lattner (2:29:35.000)
because when you get to the hard times,
Lex Fridman (2:29:37.080)
that's when, if you love what you're doing,
Chris Lattner (2:29:40.080)
you're willing to kind of push through.
Lex Fridman (2:29:41.680)
And this is really hard
Chris Lattner (2:29:45.420)
because it's hard to know what you will love doing
Lex Fridman (2:29:48.640)
until you start doing a lot of things.
Lex Fridman (2:29:50.200)
And so that's why I think that,
Lex Fridman (2:29:51.640)
particularly early in your career,
Chris Lattner (2:29:53.280)
it's good to experiment.
Lex Fridman (2:29:54.920)
Do a little bit of everything.
Chris Lattner (2:29:55.920)
Go take the survey class on
Lex Fridman (2:29:59.360)
the first half of every class in your upper division lessons
Lex Fridman (2:30:03.760)
and just get exposure to things
Lex Fridman (2:30:05.720)
because certain things will resonate with you
Lex Fridman (2:30:07.120)
and you'll find out, wow, I'm really good at this.
Lex Fridman (2:30:08.960)
I'm really smart at this.
Chris Lattner (2:30:10.080)
Well, it's just because it works with the way your brain.
Lex Fridman (2:30:13.040)
And when something jumps out,
Chris Lattner (2:30:14.340)
I mean, that's one of the things
Lex Fridman (2:30:15.620)
that people often ask about is like,
Chris Lattner (2:30:19.160)
well, I think there's a bunch of cool stuff out there.
Lex Fridman (2:30:21.400)
Like how do I pick the thing?
Chris Lattner (2:30:23.240)
Like how do you hook, in your life,
Lex Fridman (2:30:27.600)
how did you just hook yourself in and stuck with it?
Lex Fridman (2:30:30.440)
Well, I got lucky, right?
Lex Fridman (2:30:31.680)
I mean, I think that many people forget
Lex Fridman (2:30:34.800)
that a huge amount of it or most of it is luck, right?
Lex Fridman (2:30:38.760)
So let's not forget that.
Lex Fridman (2:30:41.880)
So for me, I fell in love with computers early on
Lex Fridman (2:30:44.800)
because they spoke to me, I guess.
Lex Fridman (2:30:49.280)
What language did they speak?
Lex Fridman (2:30:50.740)
Basic.
Chris Lattner (2:30:51.580)
Basic, yeah.
Lex Fridman (2:30:53.380)
But then it was just kind of following
Chris Lattner (2:30:56.960)
a set of logical progressions,
Lex Fridman (2:30:58.200)
but also deciding that something that was hard
Lex Fridman (2:31:01.400)
was worth doing and a lot of fun, right?
Lex Fridman (2:31:04.080)
And so I think that that is also something
Chris Lattner (2:31:06.240)
that's true for many other domains,
Lex Fridman (2:31:08.080)
which is if you find something that you love doing
Chris Lattner (2:31:10.360)
that's also hard, if you invest yourself in it
Lex Fridman (2:31:13.440)
and add value to the world,
Lex Fridman (2:31:14.960)
then it will mean something generally, right?
Lex Fridman (2:31:17.120)
And again, that can be a research paper,
Chris Lattner (2:31:19.120)
that can be a software system,
Lex Fridman (2:31:20.400)
that can be a new robot,
Chris Lattner (2:31:22.040)
that can be, there's many things that that can be,
Lex Fridman (2:31:24.760)
but a lot of it is like real value
Chris Lattner (2:31:27.100)
comes from doing things that are hard.
Lex Fridman (2:31:29.320)
And that doesn't mean you have to suffer, but.
Chris Lattner (2:31:33.840)
It's hard.
Lex Fridman (2:31:34.680)
I mean, you don't often hear that message.
Chris Lattner (2:31:36.360)
We talked about it last time a little bit,
Lex Fridman (2:31:38.000)
but it's one of my, not enough people talk about this.
Chris Lattner (2:31:43.840)
It's beautiful to hear a successful person.
Lex Fridman (2:31:47.400)
Well, and self doubt and imposter syndrome,
Chris Lattner (2:31:49.460)
these are all things that successful people
Lex Fridman (2:31:52.360)
suffer with as well,
Chris Lattner (2:31:53.960)
particularly when they put themselves
Lex Fridman (2:31:55.120)
in a point of being uncomfortable,
Chris Lattner (2:31:56.660)
which I like to do now and then
Lex Fridman (2:31:59.200)
just because it puts you in learning mode.
Chris Lattner (2:32:02.060)
Like if you wanna grow as a person,
Lex Fridman (2:32:04.080)
put yourself in a room with a bunch of people
Chris Lattner (2:32:07.000)
that know way more about whatever you're talking about
Lex Fridman (2:32:09.160)
than you do and ask dumb questions.
Lex Fridman (2:32:11.520)
And guess what?
Lex Fridman (2:32:13.040)
Smart people love to teach often,
Chris Lattner (2:32:15.280)
not always, but often.
Lex Fridman (2:32:16.800)
And if you listen, if you're prepared to listen,
Chris Lattner (2:32:18.320)
if you're prepared to grow,
Lex Fridman (2:32:19.180)
if you're prepared to make connections,
Chris Lattner (2:32:20.680)
you can do some really interesting things.
Lex Fridman (2:32:22.380)
And I think a lot of progress is made by people
Chris Lattner (2:32:25.360)
who kind of hop between domains now and then,
Lex Fridman (2:32:28.000)
because they bring a perspective into a field
Chris Lattner (2:32:32.480)
that nobody else has,
Lex Fridman (2:32:34.760)
if people have only been working in that field themselves.
Chris Lattner (2:32:38.280)
We mentioned that the universe is kind of like a compiler.
Lex Fridman (2:32:43.120)
The entirety of it, the whole evolution
Chris Lattner (2:32:44.880)
is kind of a kind of compilation.
Lex Fridman (2:32:46.720)
Maybe us human beings are kind of compilers.
Chris Lattner (2:32:51.640)
Let me ask the old sort of question
Lex Fridman (2:32:53.560)
that I didn't ask you last time,
Lex Fridman (2:32:54.920)
which is what's the meaning of it all?
Lex Fridman (2:32:57.740)
Is there a meaning?
Chris Lattner (2:32:58.720)
Like if you asked a compiler why,
Lex Fridman (2:33:01.680)
what would a compiler say?
Lex Fridman (2:33:03.380)
What's the meaning of life?
Lex Fridman (2:33:04.600)
What's the meaning of life?
Chris Lattner (2:33:06.800)
I'm prepared for it not to mean anything.
Lex Fridman (2:33:09.720)
Here we are, all biological things programmed to survive
Lex Fridman (2:33:14.200)
and propagate our DNA.
Lex Fridman (2:33:19.160)
And maybe the universe is just a computer
Lex Fridman (2:33:21.400)
and you just go until entropy takes over the universe
Lex Fridman (2:33:25.320)
and then you're done.
Chris Lattner (2:33:27.440)
I don't think that's a very productive way
Lex Fridman (2:33:29.640)
to live your life, if so.
Lex Fridman (2:33:33.000)
And so I prefer to bias towards the other way,
Lex Fridman (2:33:34.760)
which is saying the universe has a lot of value.
Lex Fridman (2:33:37.960)
And I take happiness out of other people.
Lex Fridman (2:33:41.840)
And a lot of times part of that's having kids,
Lex Fridman (2:33:43.840)
but also the relationships you build with other people.
Lex Fridman (2:33:46.920)
And so the way I try to live my life is like,
Lex Fridman (2:33:49.680)
what can I do that has value?
Lex Fridman (2:33:51.240)
How can I move the world forward?
Lex Fridman (2:33:52.480)
How can I take what I'm good at
Lex Fridman (2:33:54.540)
and bring it into the world?
Lex Fridman (2:33:57.600)
And I'm one of these people that likes to work really hard
Lex Fridman (2:34:00.640)
and be very focused on the things that I do.
Lex Fridman (2:34:03.140)
And so if I'm gonna do that,
Lex Fridman (2:34:05.040)
how can it be in a domain that actually will matter?
Chris Lattner (2:34:08.040)
Because a lot of things that we do,
Lex Fridman (2:34:10.020)
we find ourselves in the cycle of like,
Chris Lattner (2:34:11.680)
okay, I'm doing a thing.
Lex Fridman (2:34:12.880)
I'm very familiar with it.
Chris Lattner (2:34:13.760)
I've done it for a long time.
Lex Fridman (2:34:15.400)
I've never done anything else,
Lex Fridman (2:34:16.680)
but I'm not really learning, right?
Lex Fridman (2:34:19.760)
I'm not really, I'm keeping things going,
Lex Fridman (2:34:21.720)
but there's a younger generation
Lex Fridman (2:34:23.680)
that can do the same thing,
Lex Fridman (2:34:24.640)
maybe even better than me, right?
Lex Fridman (2:34:26.480)
Maybe if I actually step out of this
Lex Fridman (2:34:28.000)
and jump into something I'm less comfortable with,
Lex Fridman (2:34:31.280)
it's scary.
Lex Fridman (2:34:32.280)
But on the other hand,
Lex Fridman (2:34:33.440)
it gives somebody else a new opportunity.
Chris Lattner (2:34:34.920)
It also then puts you back in learning mode,
Lex Fridman (2:34:37.480)
and that can be really interesting.
Lex Fridman (2:34:38.960)
And one of the things I've learned is that
Lex Fridman (2:34:41.280)
when you go through that,
Chris Lattner (2:34:42.360)
that first you're deep into imposter syndrome,
Lex Fridman (2:34:45.040)
but when you start working your way out,
Chris Lattner (2:34:46.940)
you start to realize,
Lex Fridman (2:34:47.780)
hey, well, there's actually a method to this.
Lex Fridman (2:34:49.980)
And now I'm able to add new things
Lex Fridman (2:34:53.280)
because I bring different perspective.
Lex Fridman (2:34:54.680)
And this is one of the good things
Lex Fridman (2:34:57.280)
about bringing different kinds of people together.
Chris Lattner (2:34:59.800)
Diversity of thought is really important.
Lex Fridman (2:35:01.880)
And if you can pull together people
Chris Lattner (2:35:04.440)
that are coming at things from different directions,
Lex Fridman (2:35:06.460)
you often get innovation.
Lex Fridman (2:35:07.760)
And I love to see that, that aha moment
Lex Fridman (2:35:10.540)
where you're like, oh, we've really cracked this.
Chris Lattner (2:35:12.760)
This is something nobody's ever done before.
Lex Fridman (2:35:15.200)
And then if you can do it in the context
Chris Lattner (2:35:16.760)
where it adds value, other people can build on it,
Lex Fridman (2:35:18.960)
it helps move the world,
Chris Lattner (2:35:20.280)
then that's what really excites me.
Lex Fridman (2:35:22.720)
So that kind of description
Chris Lattner (2:35:24.480)
of the magic of the human experience,
Lex Fridman (2:35:26.480)
do you think we'll ever create that in an AGI system?
Lex Fridman (2:35:29.880)
Do you think we'll be able to create,
Lex Fridman (2:35:34.440)
give AI systems a sense of meaning
Chris Lattner (2:35:38.040)
where they operate in this kind of world
Lex Fridman (2:35:39.640)
exactly in the way you've described,
Chris Lattner (2:35:41.800)
which is they interact with each other,
Lex Fridman (2:35:43.240)
they interact with us humans.
Chris Lattner (2:35:44.800)
Sure, sure.
Lex Fridman (2:35:45.640)
Well, so, I mean, why are you being so a speciest, right?
Chris Lattner (2:35:50.840)
All right, so AGI versus Bionets,
Lex Fridman (2:35:54.640)
or something like that versus biology, right?
Lex Fridman (2:35:57.720)
You know, what are we but machines, right?
Lex Fridman (2:36:00.280)
We're just programmed to run our,
Chris Lattner (2:36:02.900)
we have our objective function
Lex Fridman (2:36:03.920)
that we were optimized for, right?
Lex Fridman (2:36:06.440)
And so we're doing our thing,
Lex Fridman (2:36:07.640)
we think we have purpose, but do we really, right?
Chris Lattner (2:36:10.000)
I'm not prepared to say that those newfangled AGI's
Lex Fridman (2:36:13.760)
have no soul just because we don't understand them, right?
Lex Fridman (2:36:16.840)
And I think that would be, when they exist,
Lex Fridman (2:36:20.120)
that would be very premature to look at a new thing
Chris Lattner (2:36:24.180)
through your own lens without fully understanding it.
Lex Fridman (2:36:28.200)
You might be just saying that
Chris Lattner (2:36:29.400)
because AI systems in the future
Lex Fridman (2:36:31.680)
will be listening to this and then.
Chris Lattner (2:36:33.080)
Oh yeah, exactly.
Lex Fridman (2:36:34.080)
You don't wanna say anything.
Chris Lattner (2:36:34.920)
Please be nice to me, you know,
Lex Fridman (2:36:36.280)
when Skynet kills everybody, please spare me.
Lex Fridman (2:36:39.160)
So wise look ahead thinking.
Lex Fridman (2:36:42.600)
Yeah, but I mean, I think that people will spend
Chris Lattner (2:36:44.760)
a lot of time worrying about this kind of stuff,
Lex Fridman (2:36:46.320)
and I think that what we should be worrying about
Lex Fridman (2:36:48.140)
is how do we make the world better?
Lex Fridman (2:36:49.840)
And the thing that I'm most scared about with AGI's
Chris Lattner (2:36:52.840)
is not that necessarily the Skynet
Lex Fridman (2:36:57.400)
will start shooting everybody with lasers
Lex Fridman (2:36:58.920)
and stuff like that to use us for our calories.
Lex Fridman (2:37:03.040)
The thing that I'm worried about is that
Chris Lattner (2:37:05.360)
humanity, I think, needs a challenge.
Lex Fridman (2:37:08.280)
And if we get into a mode of not having a personal challenge,
Chris Lattner (2:37:11.600)
not having a personal contribution,
Lex Fridman (2:37:13.560)
whether that be like, you know, your kids
Lex Fridman (2:37:15.880)
and seeing what they grow into and helping guide them,
Lex Fridman (2:37:18.800)
whether it be your community that you're engaged in,
Chris Lattner (2:37:21.920)
you're driving forward, whether it be your work
Lex Fridman (2:37:23.920)
and the things that you're doing
Lex Fridman (2:37:25.040)
and the people you're working with
Lex Fridman (2:37:25.880)
and the products you're building and the contribution there,
Chris Lattner (2:37:28.880)
if people don't have a objective,
Lex Fridman (2:37:31.960)
I'm afraid what that means.
Lex Fridman (2:37:33.360)
And I think that this would lead to a rise
Lex Fridman (2:37:37.840)
of the worst part of people, right?
Chris Lattner (2:37:39.920)
Instead of people striving together
Lex Fridman (2:37:42.240)
and trying to make the world better,
Chris Lattner (2:37:45.120)
it could degrade into a very unpleasant world.
Lex Fridman (2:37:49.720)
But I don't know.
Chris Lattner (2:37:51.140)
I mean, we hopefully have a long ways to go
Lex Fridman (2:37:53.600)
before we discover that.
Lex Fridman (2:37:54.800)
And fortunately, we have pretty on the ground problems
Lex Fridman (2:37:57.680)
with the pandemic right now,
Lex Fridman (2:37:58.660)
and so I think we should be focused on that as well.
Lex Fridman (2:38:01.520)
Yeah, ultimately, just as you said, you're optimistic.
Chris Lattner (2:38:04.700)
I think it helps for us to be optimistic.
Lex Fridman (2:38:07.360)
So that's fake it until you make it.
Lex Fridman (2:38:10.400)
Yeah, well, and why not?
Lex Fridman (2:38:11.520)
What's the other side, right?
Chris Lattner (2:38:12.800)
So, I mean, I'm not personally a very religious person,
Lex Fridman (2:38:17.500)
but I've heard people say like,
Chris Lattner (2:38:19.160)
oh yeah, of course I believe in God.
Lex Fridman (2:38:20.480)
Of course I go to church, because if God's real,
Chris Lattner (2:38:23.360)
you know, I wanna be on the right side of that.
Lex Fridman (2:38:25.960)
If it's not real, it doesn't matter.
Chris Lattner (2:38:27.120)
Yeah, it doesn't matter.
Lex Fridman (2:38:27.960)
And so, you know, that's a fair way to do it.
Chris Lattner (2:38:32.200)
Yeah, I mean, the same thing with nuclear deterrence,
Lex Fridman (2:38:35.640)
all, you know, global warming, all these things,
Chris Lattner (2:38:38.440)
all these threats, natural engineer pandemics,
Lex Fridman (2:38:41.380)
all these threats we face.
Chris Lattner (2:38:42.720)
I think it's paralyzing to be terrified
Lex Fridman (2:38:49.700)
of all the possible ways we could destroy ourselves.
Chris Lattner (2:38:52.540)
I think it's much better or at least productive
Lex Fridman (2:38:56.560)
to be hopeful and to engineer defenses
Chris Lattner (2:38:59.840)
against these things, to engineer a future
Lex Fridman (2:39:03.000)
where like, you know, see like a positive future
Lex Fridman (2:39:06.640)
and engineer that future.
Lex Fridman (2:39:07.960)
Yeah, well, and I think that's another thing
Chris Lattner (2:39:10.220)
to think about as, you know, a human,
Lex Fridman (2:39:12.680)
particularly if you're young and trying to figure out
Lex Fridman (2:39:14.560)
what it is that you wanna be when you grow up, like I am.
Lex Fridman (2:39:18.120)
I'm always looking for that.
Lex Fridman (2:39:19.840)
The question then is, how do you wanna spend your time?
Lex Fridman (2:39:24.300)
And right now there seems to be a norm
Chris Lattner (2:39:26.020)
of being a consumption culture.
Lex Fridman (2:39:28.800)
Like I'm gonna watch the news and revel
Chris Lattner (2:39:31.520)
in how horrible everything is right now.
Lex Fridman (2:39:33.520)
I'm going to go find out about the latest atrocity
Lex Fridman (2:39:36.560)
and find out all the details of like the terrible thing
Lex Fridman (2:39:38.840)
that happened and be outraged by it.
Chris Lattner (2:39:41.960)
You can spend a lot of time watching TV
Lex Fridman (2:39:44.000)
and watching the news at home or whatever
Chris Lattner (2:39:46.640)
people watch these days, I don't know.
Lex Fridman (2:39:49.340)
But that's a lot of hours, right?
Lex Fridman (2:39:51.160)
And those are hours that if you're turned
Lex Fridman (2:39:53.400)
to being productive, learning, growing,
Chris Lattner (2:39:56.960)
experiencing, you know, when the pandemic's over,
Lex Fridman (2:39:59.600)
going exploring, right, it leads to more growth.
Lex Fridman (2:40:03.600)
And I think it leads to more optimism and happiness
Lex Fridman (2:40:06.400)
because you're building, right?
Chris Lattner (2:40:08.620)
You're building yourself, you're building your capabilities,
Lex Fridman (2:40:10.960)
you're building your viewpoints,
Chris Lattner (2:40:12.200)
you're building your perspective.
Lex Fridman (2:40:13.400)
And I think that a lot of the consuming
Chris Lattner (2:40:18.360)
of other people's messages leads
Lex Fridman (2:40:19.960)
to kind of a negative viewpoint,
Chris Lattner (2:40:21.720)
which you need to be aware of what's happening
Lex Fridman (2:40:24.360)
because that's also important,
Lex Fridman (2:40:25.680)
but there's a balance that I think focusing
Lex Fridman (2:40:28.120)
on creation is a very valuable thing to do.
Chris Lattner (2:40:32.000)
Yeah, so what you're saying is people should focus
Lex Fridman (2:40:33.840)
on working on the sexiest field of them all,
Chris Lattner (2:40:37.320)
which is compiler design.
Lex Fridman (2:40:38.440)
Exactly.
Chris Lattner (2:40:39.680)
Hey, you could go work on machine learning
Lex Fridman (2:40:41.160)
and be crowded out by the thousands of graduates
Chris Lattner (2:40:43.640)
popping out of school that all want to do the same thing,
Lex Fridman (2:40:45.620)
or you could work in the place that people overpay you
Chris Lattner (2:40:48.560)
because there's not enough smart people working in it.
Lex Fridman (2:40:51.260)
And here at the end of Moore's Law,
Chris Lattner (2:40:53.480)
according to some people,
Lex Fridman (2:40:55.480)
actually the software is the hard part too.
Chris Lattner (2:40:58.480)
I mean, optimization is truly, truly beautiful.
Lex Fridman (2:41:02.280)
And also on the YouTube side or education side,
Chris Lattner (2:41:07.920)
it'd be nice to have some material
Lex Fridman (2:41:09.760)
that shows the beauty of compilers.
Chris Lattner (2:41:12.160)
Yeah, yeah.
Lex Fridman (2:41:13.160)
That's something.
Lex Fridman (2:41:14.480)
So that's a call for people to create
Lex Fridman (2:41:17.440)
that kind of content as well.
Chris Lattner (2:41:18.920)
Chris, you're one of my favorite people to talk to.
Lex Fridman (2:41:22.800)
It's such a huge honor that you would waste your time
Chris Lattner (2:41:25.560)
talking to me.
Lex Fridman (2:41:26.560)
I've always appreciated it.
Chris Lattner (2:41:27.760)
Thank you so much for talking to me.
Lex Fridman (2:41:29.600)
The truth of it is you spent a lot of time talking to me
Chris Lattner (2:41:32.320)
just on walks and other things like that,
Lex Fridman (2:41:34.440)
so it's great to catch up with.
Chris Lattner (2:41:35.640)
Thanks, man.
Lex Fridman (2:41:37.200)
Thanks for listening to this conversation
Chris Lattner (2:41:39.240)
with Chris Latner, and thank you to our sponsors.
Lex Fridman (2:41:42.360)
Blinkist, an app that summarizes key ideas
Chris Lattner (2:41:45.200)
from thousands of books.
Lex Fridman (2:41:46.600)
Neuro, which is a maker of functional gum and mints
Chris Lattner (2:41:49.640)
that supercharge my mind.
Lex Fridman (2:41:51.440)
Masterclass, which are online courses from world experts.
Lex Fridman (2:41:55.480)
And finally, Cash App, which is an app
Lex Fridman (2:41:57.840)
for sending money to friends.
Chris Lattner (2:42:00.200)
Please check out these sponsors in the description
Lex Fridman (2:42:02.360)
to get a discount and to support this podcast.
Chris Lattner (2:42:06.120)
If you enjoy this thing, subscribe on YouTube,
Lex Fridman (2:42:08.440)
review it with five stars on Apple Podcast,
Chris Lattner (2:42:10.600)
follow on Spotify, support it on Patreon,
Lex Fridman (2:42:13.280)
connect with me on Twitter at Lex Friedman.
Lex Fridman (2:42:16.320)
And now, let me leave you with some words from Chris Latner.
Lex Fridman (2:42:19.080)
So much of language design is about tradeoffs,
Lex Fridman (2:42:21.760)
and you can't see those tradeoffs
Lex Fridman (2:42:23.720)
unless you have a community of people
Chris Lattner (2:42:25.640)
that really represent those different points.
Lex Fridman (2:42:28.560)
Thank you for listening, and hope to see you next time.
Lex Fridman (30:00.100)
and then you get the best out of all these people,
Lex Fridman (30:02.500)
and you also can harness the community around it.
Lex Fridman (30:06.300)
And what about the decision of whether in Python
Lex Fridman (30:09.380)
having one type or having strict typing?
Chris Lattner (30:14.140)
Yeah, okay.
Lex Fridman (30:15.100)
Yeah, let's talk about this.
Lex Fridman (30:16.060)
So I like how you put that, by the way.
Lex Fridman (30:19.580)
So many people would say that Python doesn't have types.
Chris Lattner (30:21.900)
Doesn't have types, yeah.
Lex Fridman (30:22.900)
But you're right.
Chris Lattner (30:23.740)
I haven't listened to you enough to where,
Lex Fridman (30:26.660)
I'm a fan of yours and I've listened to way too many
Chris Lattner (30:29.940)
podcasts and videos of you talking about this stuff.
Lex Fridman (30:32.460)
Oh yeah, so I would argue that Python has one type.
Lex Fridman (30:34.780)
And so like when you import Python into Swift,
Lex Fridman (30:38.180)
which by the way works really well,
Chris Lattner (30:39.780)
you have everything comes in as a Python object.
Lex Fridman (30:41.820)
Now here are their trade offs because
Chris Lattner (30:46.220)
it depends on what you're optimizing for.
Lex Fridman (30:47.460)
And Python is a super successful language
Chris Lattner (30:49.300)
for a really good reason.
Lex Fridman (30:51.060)
Because it has one type,
Chris Lattner (30:52.740)
you get duck typing for free and things like this.
Lex Fridman (30:55.340)
But also you're pushing,
Chris Lattner (30:56.940)
you're making it very easy to pound out code on one hand,
Lex Fridman (31:00.580)
but you're also making it very easy to introduce
Chris Lattner (31:03.580)
complicated bugs that you have to debug.
Lex Fridman (31:05.260)
And you pass a string into something that expects an integer
Lex Fridman (31:08.180)
and it doesn't immediately die.
Lex Fridman (31:10.220)
It goes all the way down the stack trace
Lex Fridman (31:12.060)
and you find yourself in the middle of some code
Lex Fridman (31:13.460)
that you really didn't want to know anything about.
Lex Fridman (31:14.900)
And it blows up and you're just saying,
Lex Fridman (31:16.420)
well, what did I do wrong, right?
Lex Fridman (31:18.180)
And so types are good and bad and they have trade offs.
Lex Fridman (31:21.740)
They're good for performance and certain other things
Chris Lattner (31:23.620)
depending on where you're coming from,
Lex Fridman (31:24.740)
but it's all about trade offs.
Lex Fridman (31:26.340)
And so this is what design is, right?
Lex Fridman (31:28.580)
Design is about weighing trade offs
Lex Fridman (31:30.220)
and trying to understand the ramifications
Lex Fridman (31:32.580)
of the things that you're weighing,
Chris Lattner (31:34.300)
like types or not, or one type or many types.
Lex Fridman (31:38.660)
But also within many types,
Lex Fridman (31:39.820)
how powerful do you make that type system
Lex Fridman (31:41.740)
is another very complicated question
Chris Lattner (31:44.500)
with lots of trade offs.
Lex Fridman (31:45.380)
It's very interesting by the way,
Lex Fridman (31:47.540)
but that's like one dimension and there's a bunch
Lex Fridman (31:52.620)
of other dimensions, JIT compiled versus static compiled,
Chris Lattner (31:55.260)
garbage collected versus reference counted
Lex Fridman (31:57.820)
versus manual memory management versus,
Chris Lattner (32:00.900)
like in like all these different trade offs
Lex Fridman (32:02.980)
and how you balance them
Chris Lattner (32:03.820)
are what make a program language good.
Lex Fridman (32:05.580)
Concurrency.
Chris Lattner (32:06.820)
Yeah.
Lex Fridman (32:07.660)
So in all those things, I guess,
Chris Lattner (32:09.860)
when you're designing the language,
Lex Fridman (32:11.300)
you also have to think of how that's gonna get
Chris Lattner (32:13.020)
all compiled down to.
Lex Fridman (32:15.220)
If you care about performance, yeah.
Lex Fridman (32:17.420)
Well, and go back to Lisp, right?
Lex Fridman (32:18.780)
So Lisp also, I would say JavaScript
Lex Fridman (32:20.940)
is another example of a very simple language, right?
Lex Fridman (32:24.100)
And so one of the, so I also love Lisp.
Chris Lattner (32:27.220)
I don't use it as much as maybe you do or you did.
Lex Fridman (32:29.780)
No, I think we're both, everyone who loves Lisp,
Chris Lattner (32:32.500)
it's like, you love, it's like, I don't know,
Lex Fridman (32:35.140)
I love Frank Sinatra,
Lex Fridman (32:36.260)
but like how often do I seriously listen to Frank Sinatra?
Lex Fridman (32:39.180)
Sure, sure.
Lex Fridman (32:40.020)
But you look at that or you look at JavaScript,
Lex Fridman (32:42.780)
which is another very different,
Lex Fridman (32:44.100)
but relatively simple language.
Lex Fridman (32:45.980)
And there are certain things that don't exist
Chris Lattner (32:47.740)
in the language,
Lex Fridman (32:49.140)
but there is inherent complexity to the problems
Chris Lattner (32:51.900)
that we're trying to model.
Lex Fridman (32:53.140)
And so what happens to the complexity?
Chris Lattner (32:54.660)
In the case of both of them, for example,
Lex Fridman (32:57.220)
you say, well, what about large scale software development?
Chris Lattner (33:00.100)
Okay, well, you need something like packages.
Lex Fridman (33:02.380)
Neither language has a language affordance for packages.
Lex Fridman (33:05.780)
And so what you get is patterns.
Lex Fridman (33:07.420)
You get things like NPN.
Chris Lattner (33:08.700)
You get things like these ecosystems that get built around.
Lex Fridman (33:12.060)
And I'm a believer that if you don't model
Chris Lattner (33:15.140)
at least the most important inherent complexity
Lex Fridman (33:17.740)
in the language,
Chris Lattner (33:18.740)
then what ends up happening
Lex Fridman (33:19.620)
is that complexity gets pushed elsewhere.
Lex Fridman (33:22.740)
And when it gets pushed elsewhere,
Lex Fridman (33:24.100)
sometimes that's great because often building things
Chris Lattner (33:26.580)
as libraries is very flexible and very powerful
Lex Fridman (33:28.900)
and allows you to evolve and things like that.
Lex Fridman (33:30.700)
But often it leads to a lot of unnecessary divergence
Lex Fridman (33:34.020)
in the force and fragmentation.
Lex Fridman (33:35.580)
And when that happens, you just get kind of a mess.
Lex Fridman (33:39.540)
And so the question is, how do you balance that?
Chris Lattner (33:42.940)
Don't put too much stuff in the language
Lex Fridman (33:44.260)
because that's really expensive
Lex Fridman (33:45.220)
and it makes things complicated.
Lex Fridman (33:46.740)
But how do you model enough of the inherent complexity
Chris Lattner (33:49.620)
of the problem that you provide the framework
Lex Fridman (33:52.380)
and the structure for people to think about?
Chris Lattner (33:54.860)
Well, so the key thing to think about
Lex Fridman (33:57.220)
with programming languages,
Lex Fridman (33:59.060)
and you think about what a programming language is therefore,
Lex Fridman (34:01.340)
is it's about making a human more productive, right?
Lex Fridman (34:04.220)
And so there's an old, I think it's Steve Jobs quote
Lex Fridman (34:07.100)
about it's a bicycle for the mind, right?
Chris Lattner (34:10.420)
You can definitely walk,
Lex Fridman (34:13.020)
but you'll get there a lot faster
Chris Lattner (34:15.260)
if you can bicycle on your way.
Lex Fridman (34:17.540)
And...
Lex Fridman (34:18.380)
A programming language is a bicycle for the mind?
Lex Fridman (34:20.140)
Yeah.
Chris Lattner (34:20.980)
Is it basically, wow,
Lex Fridman (34:22.140)
that's really interesting way to think about it.
Chris Lattner (34:23.900)
By raising the level of abstraction,
Lex Fridman (34:25.540)
now you can fit more things in your head.
Chris Lattner (34:27.380)
By being able to just directly leverage somebody's library,
Lex Fridman (34:30.100)
you can now get something done quickly.
Chris Lattner (34:33.420)
In the case of Swift, Swift UI is this new framework
Lex Fridman (34:36.180)
that Apple has released recently for doing UI programming.
Lex Fridman (34:39.780)
And it has this declarative programming model,
Lex Fridman (34:42.980)
which defines away entire classes of bugs.
Chris Lattner (34:45.660)
It builds on value semantics and many other nice Swift things.
Lex Fridman (34:48.820)
And what this does is it allows you to just get way more done
Chris Lattner (34:51.620)
with way less code.
Lex Fridman (34:53.260)
And now your productivity as a developer is much higher,
Lex Fridman (34:56.580)
right?
Lex Fridman (34:57.420)
And so that's really what programming languages
Chris Lattner (34:59.420)
should be about,
Lex Fridman (35:00.260)
is it's not about tabs versus spaces
Chris Lattner (35:01.780)
or curly braces or whatever.
Lex Fridman (35:03.300)
It's about how productive do you make the person.
Lex Fridman (35:05.380)
And you can only see that when you have libraries
Lex Fridman (35:08.980)
that were built with the right intention
Chris Lattner (35:11.060)
that the language was designed for.
Lex Fridman (35:13.740)
And with Swift, I think we're still a little bit early,
Lex Fridman (35:16.620)
but Swift UI and many other things that are coming out now
Lex Fridman (35:19.460)
are really showing that.
Lex Fridman (35:20.300)
And I think that they're opening people's eyes.
Lex Fridman (35:22.500)
It's kind of interesting to think about like how
Chris Lattner (35:26.300)
that the knowledge of something,
Lex Fridman (35:29.620)
of how good the bicycle is,
Lex Fridman (35:31.620)
how people learn about that.
Lex Fridman (35:33.740)
So I've used C++,
Chris Lattner (35:36.020)
now this is not going to be a trash talking session
Lex Fridman (35:38.980)
about C++, but I used C++ for a really long time.
Chris Lattner (35:41.820)
You can go there if you want, I have the scars.
Lex Fridman (35:45.100)
I feel like I spent many years without realizing
Chris Lattner (35:49.580)
like there's languages that could,
Lex Fridman (35:51.500)
for my particular lifestyle, brain style, thinking style,
Chris Lattner (35:56.820)
there's languages that could make me a lot more productive
Lex Fridman (36:00.340)
in the debugging stage, in just the development stage
Lex Fridman (36:04.340)
and thinking like the bicycle for the mind
Lex Fridman (36:05.940)
that I can fit more stuff into my...
Lex Fridman (36:07.740)
Python's a great example of that, right?
Lex Fridman (36:09.220)
I mean, a machine learning framework in Python
Chris Lattner (36:10.940)
is a great example of that.
Lex Fridman (36:12.260)
It's just very high abstraction level.
Lex Fridman (36:14.660)
And so you can be thinking about things
Lex Fridman (36:15.860)
on a like very high level algorithmic level
Chris Lattner (36:19.020)
instead of thinking about, okay, well,
Lex Fridman (36:20.420)
am I copying this tensor to a GPU or not, right?
Chris Lattner (36:23.780)
It's not what you want to be thinking about.
Lex Fridman (36:25.500)
And as I was telling you, I mean,
Chris Lattner (36:26.940)
I guess the question I had is,
Lex Fridman (36:29.740)
how does a person like me or in general people
Lex Fridman (36:31.740)
discover more productive languages?
Lex Fridman (36:36.860)
Like how I was, as I've been telling you offline,
Chris Lattner (36:39.940)
I've been looking for like a project to work on in Swift
Lex Fridman (36:43.220)
so I can really try it out.
Chris Lattner (36:45.540)
I mean, my intuition was like doing a hello world
Lex Fridman (36:48.580)
is not going to get me there
Chris Lattner (36:50.420)
to get me to experience the power of language.
Lex Fridman (36:54.100)
You need a few weeks to change your metabolism.
Chris Lattner (36:55.900)
Exactly, beautifully put.
Lex Fridman (36:59.500)
That's one of the problems with people with diets,
Chris Lattner (37:01.500)
like I'm actually currently, to go in parallel,
Lex Fridman (37:05.260)
but in a small tangent is I've been recently
Lex Fridman (37:07.780)
eating only meat, okay?
Lex Fridman (37:10.260)
And okay, so most people are like,
Chris Lattner (37:14.940)
they think that's horribly unhealthy or whatever,
Lex Fridman (37:16.820)
you have like a million, whatever the science is,
Chris Lattner (37:20.540)
it just doesn't sound right.
Lex Fridman (37:22.460)
Well, so back when I was in college,
Chris Lattner (37:24.060)
we did the Atkins diet, that was a thing.
Lex Fridman (37:26.460)
Similar, but you have to always give these things a chance.
Chris Lattner (37:30.620)
I mean, with dieting, I was not dieting,
Lex Fridman (37:33.220)
but it's just the things that you like.
Chris Lattner (37:35.660)
If I eat, personally, if I eat meat,
Lex Fridman (37:38.060)
just everything, I can be super focused
Chris Lattner (37:40.140)
or more focused than usual.
Lex Fridman (37:42.900)
I just feel great.
Chris Lattner (37:43.940)
I mean, I've been running a lot,
Lex Fridman (37:46.260)
doing pushups and pull ups and so on.
Chris Lattner (37:47.940)
I mean, Python is similar in that sense for me.
Lex Fridman (37:50.620)
Where are you going with this?
Chris Lattner (37:53.540)
I mean, literally, I just felt I had like a stupid smile
Lex Fridman (37:57.300)
on my face when I first started using Python.
Chris Lattner (38:00.740)
I could code up really quick things.
Lex Fridman (38:02.900)
Like I would see the world,
Chris Lattner (38:05.700)
I would be empowered to write a script
Lex Fridman (38:07.740)
to do some basic data processing,
Chris Lattner (38:11.780)
to rename files on my computer.
Lex Fridman (38:14.180)
Like Perl didn't do that for me,
Chris Lattner (38:18.780)
a little bit.
Lex Fridman (38:19.620)
And again, none of these are about which is best
Chris Lattner (38:22.540)
or something like that,
Lex Fridman (38:23.380)
but there's definitely better and worse here.
Lex Fridman (38:25.020)
But it clicks, right?
Lex Fridman (38:26.060)
Well, yeah.
Chris Lattner (38:27.580)
If you look at Perl, for example,
Lex Fridman (38:29.300)
you get bogged down in scalars versus arrays
Chris Lattner (38:32.660)
versus hashes versus type globs
Lex Fridman (38:34.340)
and like all that kind of stuff.
Lex Fridman (38:35.700)
And Python's like, yeah, let's not do this.
Lex Fridman (38:38.780)
And some of it is debugging.
Chris Lattner (38:39.940)
Like everyone has different priorities.
Lex Fridman (38:41.500)
But for me, it's, can I create systems for myself
Lex Fridman (38:44.940)
that empower me to debug quickly?
Lex Fridman (38:47.820)
Like I've always been a big fan,
Chris Lattner (38:50.380)
even just crude like asserts,
Lex Fridman (38:52.060)
like always stating things that should be true,
Chris Lattner (38:57.220)
which in Python, I found in myself doing more
Lex Fridman (38:59.780)
because of type, all these kinds of stuff.
Chris Lattner (39:02.340)
Well, you could think of types in a programming language
Lex Fridman (39:04.540)
as being kind of assert.
Chris Lattner (39:05.860)
Yeah.
Lex Fridman (39:06.700)
They could check the compile time, right?
Lex Fridman (39:08.900)
So how do you learn a new thing?
Lex Fridman (39:10.980)
Well, so this, or how do people learn new things, right?
Chris Lattner (39:13.900)
This is hard.
Lex Fridman (39:15.260)
People don't like to change.
Chris Lattner (39:17.140)
People generally don't like change around them either.
Lex Fridman (39:19.300)
And so we're all very slow to adapt and change.
Lex Fridman (39:22.860)
And usually there's a catalyst that's required
Lex Fridman (39:25.460)
to force yourself over this.
Lex Fridman (39:27.980)
So for learning a programming language,
Lex Fridman (39:29.980)
it really comes down to finding an excuse,
Chris Lattner (39:32.700)
like build a thing that the language is actually good for,
Lex Fridman (39:36.300)
that the ecosystem's ready for.
Lex Fridman (39:38.820)
And so if you were to write an iOS app, for example,
Lex Fridman (39:42.980)
that'd be the easy case.
Lex Fridman (39:44.220)
Obviously you would use Swift for that, right?
Lex Fridman (39:46.700)
There are other...
Chris Lattner (39:47.540)
Android.
Lex Fridman (39:48.380)
So Swift runs on Android.
Lex Fridman (39:50.540)
Oh, does it?
Lex Fridman (39:51.380)
Oh yeah.
Chris Lattner (39:52.220)
Yeah, Swift runs in lots of places.
Lex Fridman (39:53.060)
How does that work?
Chris Lattner (39:54.700)
So...
Lex Fridman (39:55.540)
Okay, so Swift is built on top of LLVM.
Chris Lattner (39:58.580)
LLVM runs everywhere.
Lex Fridman (40:00.420)
LLVM, for example, builds the Android kernel.
Chris Lattner (40:03.180)
Oh, okay.
Lex Fridman (40:04.180)
So yeah.
Chris Lattner (40:05.020)
I didn't realize this.
Lex Fridman (40:06.780)
Yeah, so Swift is very portable, runs on Windows.
Chris Lattner (40:09.900)
There's, it runs on lots of different things.
Lex Fridman (40:12.580)
And Swift, sorry to interrupt, Swift UI,
Lex Fridman (40:15.540)
and then there's a thing called UI Kit.
Lex Fridman (40:17.900)
So can I build an app with Swift?
Chris Lattner (40:21.060)
Well, so that's the thing,
Lex Fridman (40:22.180)
is the ecosystem is what matters there.
Lex Fridman (40:23.860)
So Swift UI and UI Kit are Apple technologies.
Lex Fridman (40:27.060)
Okay, got it.
Lex Fridman (40:27.900)
And so they happen to,
Lex Fridman (40:28.740)
like Swift UI happens to be written in Swift,
Lex Fridman (40:30.540)
but it's an Apple proprietary framework
Lex Fridman (40:32.900)
that Apple loves and wants to keep on its platform,
Chris Lattner (40:35.580)
which makes total sense.
Lex Fridman (40:36.900)
You go to Android and you don't have that library, right?
Lex Fridman (40:39.660)
And so Android has a different ecosystem of things
Lex Fridman (40:42.900)
that hasn't been built out
Lex Fridman (40:44.100)
and doesn't work as well with Swift.
Lex Fridman (40:45.420)
And so you can totally use Swift to do like arithmetic
Lex Fridman (40:48.900)
and things like this,
Lex Fridman (40:49.740)
but building UI with Swift on Android
Chris Lattner (40:51.740)
is not a great experience right now.
Lex Fridman (40:54.620)
So if I wanted to learn Swift, what's the,
Chris Lattner (40:58.780)
I mean, the one practical different version of that
Lex Fridman (41:01.860)
is Swift for TensorFlow, for example.
Lex Fridman (41:05.580)
And one of the inspiring things for me
Lex Fridman (41:08.420)
with both TensorFlow and PyTorch
Chris Lattner (41:10.460)
is how quickly the community can like switch
Lex Fridman (41:13.100)
from different libraries, like you could see
Chris Lattner (41:16.820)
some of the communities switching to PyTorch now,
Lex Fridman (41:19.700)
but it's very easy to see.
Lex Fridman (41:21.940)
And then TensorFlow is really stepping up its game.
Lex Fridman (41:24.500)
And then there's no reason why,
Chris Lattner (41:26.140)
I think the way it works is basically
Lex Fridman (41:27.860)
it has to be one GitHub repo,
Chris Lattner (41:29.580)
like one paper steps up.
Lex Fridman (41:31.500)
It gets people excited.
Chris Lattner (41:32.340)
It gets people excited and they're like,
Lex Fridman (41:33.740)
ah, I have to learn this Swift for,
Lex Fridman (41:37.180)
what's Swift again?
Lex Fridman (41:39.500)
And then they learn and they fall in love with it.
Chris Lattner (41:41.220)
I mean, that's what happened, PyTorch has it.
Lex Fridman (41:43.100)
There has to be a reason, a catalyst.
Chris Lattner (41:44.420)
Yeah.
Lex Fridman (41:45.260)
And so, and there, I mean, people don't like change,
Lex Fridman (41:48.700)
but it turns out that once you've worked
Lex Fridman (41:50.420)
with one or two programming languages,
Chris Lattner (41:52.660)
the basics are pretty similar.
Lex Fridman (41:54.100)
And so one of the fun things
Chris Lattner (41:55.740)
about learning programming languages,
Lex Fridman (41:57.340)
even maybe Lisp, I don't know if you agree with this,
Chris Lattner (41:59.860)
is that when you start doing that,
Lex Fridman (42:01.380)
you start learning new things.
Chris Lattner (42:04.020)
Cause you have a new way to do things
Lex Fridman (42:05.660)
and you're forced to do them.
Lex Fridman (42:06.820)
And that forces you to explore
Lex Fridman (42:09.260)
and it puts you in learning mode.
Lex Fridman (42:10.300)
And when you get in learning mode,
Lex Fridman (42:11.340)
your mind kind of opens a little bit
Lex Fridman (42:12.740)
and you can see things in a new way,
Lex Fridman (42:15.260)
even when you go back to the old place.
Chris Lattner (42:17.020)
Right.
Lex Fridman (42:17.860)
Yeah, it's totally, well Lisp is functional.
Chris Lattner (42:19.900)
Yeah.
Lex Fridman (42:21.140)
But yeah, I wish there was a kind of window,
Chris Lattner (42:23.580)
maybe you can tell me if there is, there you go.
Lex Fridman (42:26.060)
This is a question to ask,
Lex Fridman (42:28.260)
what is the most beautiful feature
Lex Fridman (42:29.660)
in a programming language?
Chris Lattner (42:30.940)
Before I ask it, let me say like with Python,
Lex Fridman (42:33.260)
I remember I saw Lisp comprehensions.
Chris Lattner (42:36.660)
Yeah.
Lex Fridman (42:37.500)
Was like, when I like really took it in.
Chris Lattner (42:40.820)
Yeah.
Lex Fridman (42:41.940)
I don't know, I just loved it.
Chris Lattner (42:43.700)
It was like fun to do, like it was fun to do that kind of,
Lex Fridman (42:49.820)
something about it to be able to filter through a list
Lex Fridman (42:52.820)
and to create a new list all in a single line was elegant.
Lex Fridman (42:56.260)
I could all get into my head
Lex Fridman (42:58.220)
and it just made me fall in love with the language.
Lex Fridman (43:01.900)
Yeah.
Lex Fridman (43:02.740)
So is there, let me ask you a question.
Lex Fridman (43:04.860)
Is there, what do you use the most beautiful feature
Chris Lattner (43:07.620)
in a programming languages that you've ever encountered
Lex Fridman (43:11.780)
in Swift maybe and then outside of Swift?
Chris Lattner (43:15.140)
I think the thing that I like the most
Lex Fridman (43:17.460)
from a programming language.
Lex Fridman (43:18.860)
So I think the thing you have to think about
Lex Fridman (43:21.260)
with the programming language, again, what is the goal?
Chris Lattner (43:23.580)
You're trying to get people to get things done quickly.
Lex Fridman (43:27.140)
And so you need libraries, you need high quality libraries
Lex Fridman (43:30.500)
and then you need a user base around them
Lex Fridman (43:32.620)
that can assemble them and do cool things with them, right?
Lex Fridman (43:35.020)
And so to me, the question is
Lex Fridman (43:36.180)
what enables high quality libraries?
Chris Lattner (43:39.860)
Okay.
Lex Fridman (43:40.700)
Yeah.
Lex Fridman (43:41.540)
And there's a huge divide in the world
Lex Fridman (43:43.380)
between libraries who enable high quality libraries
Chris Lattner (43:48.300)
versus the ones that put special stuff in the language.
Lex Fridman (43:52.780)
So programming languages that enable high quality libraries?
Chris Lattner (43:56.740)
Got it.
Lex Fridman (43:57.580)
So, and what I mean by that is expressive libraries
Chris Lattner (44:00.860)
that then feel like a natural integrated part
Lex Fridman (44:03.700)
of the language itself.
Lex Fridman (44:05.580)
So an example of this in Swift is that int and float
Lex Fridman (44:09.860)
and also array and string, things like this,
Chris Lattner (44:12.100)
these are all part of the library.
Lex Fridman (44:13.740)
Like int is not hard coded into Swift.
Lex Fridman (44:17.420)
And so what that means is that
Lex Fridman (44:19.060)
because int is just a library thing
Chris Lattner (44:21.340)
defined in the standard library,
Lex Fridman (44:22.580)
along with strings and arrays and all the other things
Chris Lattner (44:24.660)
that come with the standard library.
Lex Fridman (44:27.180)
Well, hopefully you do like int,
Lex Fridman (44:29.220)
but anything that any language features
Lex Fridman (44:31.980)
that you needed to define int,
Chris Lattner (44:33.940)
you can also use in your own types.
Lex Fridman (44:36.100)
So if you wanted to find a quaternion
Lex Fridman (44:39.540)
or something like this, right?
Lex Fridman (44:41.420)
Well, it doesn't come in the standard library.
Chris Lattner (44:43.580)
There's a very special set of people
Lex Fridman (44:45.660)
that care a lot about this,
Lex Fridman (44:47.180)
but those people are also important.
Lex Fridman (44:49.420)
It's not about classism, right?
Chris Lattner (44:51.100)
It's not about the people who care about ints and floats
Lex Fridman (44:53.500)
are more important than the people who care about quaternions.
Lex Fridman (44:55.780)
And so to me, the beautiful things
Lex Fridman (44:56.940)
about programming languages is when you allow
Chris Lattner (44:58.980)
those communities to build high quality libraries,
Lex Fridman (45:02.300)
they feel native.
Chris Lattner (45:03.140)
They feel like they're built into the compiler
Lex Fridman (45:05.820)
without having to be.
Lex Fridman (45:08.020)
What does it mean for the int to be part
Lex Fridman (45:11.140)
of not hard coded in?
Lex Fridman (45:13.220)
So is it like how, so what is an int?
Lex Fridman (45:18.580)
Okay, int is just a integer.
Chris Lattner (45:20.820)
In this case, it's like a 64 bit integer
Lex Fridman (45:23.540)
or something like this.
Lex Fridman (45:24.620)
But so like the 64 bit is hard coded or no?
Lex Fridman (45:28.100)
No, none of that's hard coded.
Lex Fridman (45:29.380)
So int, if you go look at how it's implemented,
Lex Fridman (45:32.140)
it's just a struct in Swift.
Lex Fridman (45:34.740)
And so it's a struct.
Lex Fridman (45:35.860)
And then how do you add two structs?
Chris Lattner (45:37.460)
Well, you define plus.
Lex Fridman (45:39.620)
And so you can define plus on int.
Chris Lattner (45:41.780)
Well, you can define plus on your thing too.
Lex Fridman (45:43.540)
You can define, int is an odd method
Chris Lattner (45:46.660)
or something like that on it.
Lex Fridman (45:47.780)
And so yeah, you can add methods on the things.
Chris Lattner (45:50.420)
Yeah.
Lex Fridman (45:51.340)
So you can define operators, like how it behaves.
Chris Lattner (45:55.380)
That's just beautiful when there's something
Lex Fridman (45:57.500)
about the language which enables others
Chris Lattner (46:00.420)
to create libraries which are not hacky.
Lex Fridman (46:05.340)
Yeah, they feel native.
Lex Fridman (46:07.180)
And so one of the best examples of this is Lisp, right?
Lex Fridman (46:10.860)
Because in Lisp, like all the libraries
Lex Fridman (46:13.780)
are basically part of the language, right?
Lex Fridman (46:15.420)
You write, turn, rewrite systems and things like this.
Lex Fridman (46:17.500)
And so.
Lex Fridman (46:18.340)
Can you as a counter example provide
Lex Fridman (46:20.780)
what makes it difficult to write a library that's native?
Lex Fridman (46:23.820)
Is it the Python C?
Chris Lattner (46:25.500)
Well, so one example, I'll give you two examples.
Lex Fridman (46:29.020)
Java and C++, there's Java and C.
Chris Lattner (46:33.060)
They both allow you to define your own types,
Lex Fridman (46:35.780)
but int is hard code in the language.
Lex Fridman (46:38.420)
Okay, well, why?
Lex Fridman (46:39.340)
Well, in Java, for example, coming back
Chris Lattner (46:41.620)
to this whole reference semantic value semantic thing,
Lex Fridman (46:45.180)
int gets passed around by value.
Chris Lattner (46:48.860)
Yeah.
Lex Fridman (46:49.700)
But if you make like a pair or something like that,
Chris Lattner (46:53.740)
a complex number, right, it's a class in Java.
Lex Fridman (46:56.860)
And now it gets passed around by reference, by pointer.
Lex Fridman (46:59.900)
And so now you lose value semantics, right?
Lex Fridman (47:02.620)
You lost math, okay.
Lex Fridman (47:04.980)
Well, that's not great, right?
Lex Fridman (47:06.860)
If you can do something with int,
Lex Fridman (47:08.140)
why can't I do it with my type, right?
Lex Fridman (47:10.460)
So that's the negative side of the thing I find beautiful
Chris Lattner (47:15.340)
is when you can solve that,
Lex Fridman (47:17.340)
when you can have full expressivity,
Chris Lattner (47:19.260)
where you as a user of the language
Lex Fridman (47:21.700)
have as much or almost as much power
Chris Lattner (47:24.180)
as the people who implemented
Lex Fridman (47:25.500)
all the standard built in stuff,
Chris Lattner (47:27.300)
because what that enables
Lex Fridman (47:28.500)
is that enables truly beautiful libraries.
Chris Lattner (47:31.460)
You know, it's kind of weird
Lex Fridman (47:32.620)
because I've gotten used to that.
Chris Lattner (47:36.260)
That's one, I guess, other aspect
Lex Fridman (47:37.860)
of program language design.
Chris Lattner (47:39.100)
You have to think, you know,
Lex Fridman (47:41.140)
the old first principles thinking,
Lex Fridman (47:43.500)
like, why are we doing it this way?
Lex Fridman (47:45.580)
By the way, I mean, I remember,
Chris Lattner (47:47.900)
because I was thinking about the walrus operator
Lex Fridman (47:50.900)
and I'll ask you about it later,
Lex Fridman (47:53.260)
but it hit me that like the equal sign for assignment.
Lex Fridman (47:57.700)
Yeah.
Lex Fridman (47:58.780)
Like, why are we using the equal sign for assignment?
Lex Fridman (48:00.820)
It's wrong, yeah.
Lex Fridman (48:02.500)
And that's not the only solution, right?
Lex Fridman (48:04.500)
So if you look at Pascal,
Chris Lattner (48:05.420)
they use colon equals for assignment
Lex Fridman (48:07.740)
and equals for equality.
Lex Fridman (48:11.420)
And they use like less than greater than
Lex Fridman (48:12.980)
instead of the not equal thing.
Chris Lattner (48:14.580)
Yeah.
Lex Fridman (48:15.420)
Like, there are other answers here.
Chris Lattner (48:16.380)
So, but like, and yeah, like I ask you all,
Lex Fridman (48:19.900)
but how do you then decide to break convention
Chris Lattner (48:24.860)
to say, you know what, everybody's doing it wrong.
Lex Fridman (48:29.700)
We're gonna do it right.
Chris Lattner (48:30.980)
Yeah.
Lex Fridman (48:31.980)
So it's like an ROI,
Lex Fridman (48:33.740)
like return on investment trade off, right?
Lex Fridman (48:35.460)
So if you do something weird,
Chris Lattner (48:37.340)
let's just say like not like colon equal
Lex Fridman (48:39.820)
instead of equal for assignment,
Lex Fridman (48:40.940)
that would be weird with today's aesthetic, right?
Lex Fridman (48:44.900)
And so you'd say, cool, this is theoretically better,
Lex Fridman (48:47.460)
but is it better in which ways?
Lex Fridman (48:49.620)
Like, what do I get out of that?
Lex Fridman (48:50.700)
Do I define away class of bugs?
Lex Fridman (48:52.340)
Well, one of the class of bugs that C has
Chris Lattner (48:54.260)
is that you can use like, you know,
Lex Fridman (48:55.860)
if X equals without equals equals X equals Y, right?
Chris Lattner (49:01.740)
Well, turns out you can solve that problem in lots of ways.
Lex Fridman (49:05.220)
Clang, for example, GCC, all these compilers
Chris Lattner (49:07.620)
will detect that as a likely bug, produce a warning.
Lex Fridman (49:10.740)
Do they?
Chris Lattner (49:11.580)
Yeah.
Lex Fridman (49:12.420)
I feel like they didn't.
Chris Lattner (49:13.260)
Oh, Clang does.
Lex Fridman (49:14.100)
They didn't.
Chris Lattner (49:14.940)
GCC didn't.
Lex Fridman (49:15.900)
It's like one of the important things
Chris Lattner (49:17.820)
about programming language design is like,
Lex Fridman (49:19.820)
you're literally creating suffering in the world.
Chris Lattner (49:22.940)
Okay.
Lex Fridman (49:23.780)
Like, I feel like, I mean, one way to see it
Chris Lattner (49:27.820)
is the bicycle for the mind,
Lex Fridman (49:29.140)
but the other way is to like minimizing suffering.
Lex Fridman (49:32.140)
Well, you have to decide if it's worth it, right?
Lex Fridman (49:33.580)
And so let's come back to that.
Chris Lattner (49:35.460)
Okay.
Lex Fridman (49:36.300)
But if you look at this, and again,
Chris Lattner (49:38.300)
this is where there's a lot of detail
Lex Fridman (49:40.020)
that goes into each of these things.
Chris Lattner (49:42.700)
Equal and C returns a value.
Lex Fridman (49:46.700)
Yep.
Lex Fridman (49:47.540)
Is it messed up?
Lex Fridman (49:48.860)
That allows you to say X equals Y equals Z,
Chris Lattner (49:51.060)
like that works in C.
Lex Fridman (49:52.380)
Yeah.
Lex Fridman (49:53.380)
Is it messed up?
Lex Fridman (49:54.580)
You know, most people think it's messed up, I think.
Chris Lattner (49:57.460)
It is very, by messed up, what I mean is
Lex Fridman (50:00.700)
it is very rarely used for good,
Lex Fridman (50:03.460)
and it's often used for bugs.
Lex Fridman (50:05.460)
Yeah.
Chris Lattner (50:06.300)
Right, and so.
Lex Fridman (50:07.140)
That's a good definition of messed up, yeah.
Chris Lattner (50:09.340)
You could use, you know, in hindsight,
Lex Fridman (50:12.060)
this was not such a great idea, right?
Chris Lattner (50:13.500)
No.
Lex Fridman (50:14.340)
One of the things with Swift that is really powerful
Lex Fridman (50:16.100)
and one of the reasons it's actually good
Lex Fridman (50:18.420)
versus it being full of good ideas
Chris Lattner (50:20.260)
is that when we launched Swift 1,
Lex Fridman (50:23.420)
we announced that it was public,
Chris Lattner (50:26.020)
people could use it, people could build apps,
Lex Fridman (50:27.900)
but it was gonna change and break, okay?
Chris Lattner (50:30.940)
When Swift 2 came out, we said, hey, it's open source,
Lex Fridman (50:33.220)
and there's this open process
Chris Lattner (50:34.460)
which people can help evolve and direct the language.
Lex Fridman (50:37.900)
So the community at large, like Swift users,
Chris Lattner (50:40.140)
can now help shape the language as it is.
Lex Fridman (50:43.100)
And what happened as part of that process is
Chris Lattner (50:46.140)
a lot of really bad mistakes got taken out.
Lex Fridman (50:49.540)
So for example, Swift used to have the C style plus plus
Lex Fridman (50:53.180)
and minus minus operators.
Lex Fridman (50:55.020)
Like, what does it mean when you put it before
Lex Fridman (50:56.580)
versus after, right?
Lex Fridman (50:59.340)
Well, that got cargo culted from C into Swift early on.
Lex Fridman (51:02.620)
What's cargo culted?
Lex Fridman (51:03.740)
Cargo culted means brought forward
Chris Lattner (51:05.300)
without really considering it.
Lex Fridman (51:07.740)
Okay.
Chris Lattner (51:08.580)
This is maybe not the most PC term, but.
Lex Fridman (51:12.060)
You have to look it up in Urban Dictionary, yeah.
Chris Lattner (51:13.580)
Yeah, so it got pulled into C without,
Lex Fridman (51:17.500)
or it got pulled into Swift
Chris Lattner (51:18.580)
without very good consideration.
Lex Fridman (51:20.500)
And we went through this process,
Lex Fridman (51:22.180)
and one of the first things got ripped out
Lex Fridman (51:23.700)
was plus plus and minus minus,
Chris Lattner (51:25.580)
because they lead to confusion.
Lex Fridman (51:27.740)
They have very low value over saying x plus equals one,
Lex Fridman (51:31.580)
and x plus equals one is way more clear.
Lex Fridman (51:34.180)
And so when you're optimizing for teachability and clarity
Lex Fridman (51:36.980)
and bugs and this multidimensional space
Lex Fridman (51:39.540)
that you're looking at,
Chris Lattner (51:40.980)
things like that really matter.
Lex Fridman (51:42.300)
And so being first principles on where you're coming from
Lex Fridman (51:45.500)
and what you're trying to achieve
Lex Fridman (51:46.460)
and being anchored on the objective is really important.
Chris Lattner (51:50.100)
Well, let me ask you about the most,
Lex Fridman (51:54.460)
sort of this podcast isn't about information,
Chris Lattner (51:58.140)
it's about drama.
Lex Fridman (51:59.300)
Okay.
Chris Lattner (52:00.140)
Let me talk to you about some drama.
Lex Fridman (52:01.300)
So you mentioned Pascal and colon equals,
Chris Lattner (52:06.300)
there's something that's called the Walrus operator.
Lex Fridman (52:08.980)
Okay.
Lex Fridman (52:09.820)
And Python in Python 3.8 added the Walrus operator.
Lex Fridman (52:15.580)
And the reason I think it's interesting
Chris Lattner (52:19.100)
is not just because of the feature,
Lex Fridman (52:21.340)
it has the same kind of expression feature
Chris Lattner (52:23.420)
you can mention to see that it returns
Lex Fridman (52:25.180)
the value of the assignment.
Lex Fridman (52:27.060)
And then maybe you can comment on that in general,
Lex Fridman (52:29.620)
but on the other side of it,
Chris Lattner (52:31.180)
it's also the thing that toppled the dictator.
Lex Fridman (52:36.540)
So it finally drove Guido
Chris Lattner (52:39.220)
to step down from BDFL, the toxicity of the community.
Lex Fridman (52:42.820)
So maybe what do you think about the Walrus operator
Lex Fridman (52:46.300)
in Python?
Lex Fridman (52:47.140)
Is there an equivalent thing in Swift
Lex Fridman (52:50.020)
that really stress tested the community?
Lex Fridman (52:54.180)
And then on the flip side,
Lex Fridman (52:56.620)
what do you think about Guido stepping down over it?
Lex Fridman (52:58.700)
Yeah, well, if I look past the details
Chris Lattner (53:01.180)
of the Walrus operator,
Lex Fridman (53:02.380)
one of the things that makes it most polarizing
Chris Lattner (53:04.180)
is that it's syntactic sugar.
Lex Fridman (53:06.900)
Okay.
Lex Fridman (53:07.740)
What do you mean by syntactic sugar?
Lex Fridman (53:09.140)
It means you can take something
Chris Lattner (53:10.540)
that already exists in the language
Lex Fridman (53:11.780)
and you can express it in a more concise way.
Chris Lattner (53:14.420)
So, okay, I'm going to play dollars advocate.
Lex Fridman (53:15.980)
So this is great.
Lex Fridman (53:18.780)
Is that a objective or subjective statement?
Lex Fridman (53:21.580)
Like, can you argue that basically anything
Lex Fridman (53:24.420)
isn't syntactic sugar or not?
Lex Fridman (53:27.460)
No, not everything is syntactic sugar.
Lex Fridman (53:30.340)
So for example, the type system,
Lex Fridman (53:32.740)
like can you have classes versus,
Lex Fridman (53:35.680)
like, do you have types or not, right?
Lex Fridman (53:37.960)
So one type versus many types
Chris Lattner (53:40.040)
is not something that affects syntactic sugar.
Lex Fridman (53:42.600)
And so if you say,
Chris Lattner (53:43.760)
I want to have the ability to define types,
Lex Fridman (53:46.040)
I have to have all this like language mechanics
Chris Lattner (53:47.840)
to define classes.
Lex Fridman (53:49.080)
And oh, now I have to have inheritance.
Lex Fridman (53:51.200)
And I have like, I have all this stuff
Lex Fridman (53:52.880)
that's just making the language more complicated.
Chris Lattner (53:54.920)
That's not about sugaring it.
Lex Fridman (53:58.440)
Swift has the sugar.
Lex Fridman (54:00.080)
So like Swift has this thing called if let,
Lex Fridman (54:02.040)
and it has a lot of different types
Lex Fridman (54:04.640)
and it has various operators
Lex Fridman (54:06.520)
that are used to concisify specific use cases.
Lex Fridman (54:10.480)
So the problem with syntactic sugar,
Lex Fridman (54:12.840)
when you're talking about,
Chris Lattner (54:14.000)
hey, I have a thing that takes a lot to write
Lex Fridman (54:16.240)
and I have a new way to write it.
Chris Lattner (54:17.660)
You have this like horrible trade off,
Lex Fridman (54:19.880)
which becomes almost completely subjective,
Lex Fridman (54:22.340)
which is how often does this happen and does it matter?
Lex Fridman (54:26.280)
And one of the things that is true about human psychology,
Chris Lattner (54:28.480)
particularly when you're talking about introducing
Lex Fridman (54:29.780)
a new thing is that people overestimate
Chris Lattner (54:34.240)
the burden of learning something.
Lex Fridman (54:36.120)
And so it looks foreign when you haven't gotten used to it.
Lex Fridman (54:38.840)
But if it was there from the beginning,
Lex Fridman (54:40.360)
of course it's just part of Python.
Chris Lattner (54:42.000)
Like unquestionably, like this is just the thing I know.
Lex Fridman (54:45.080)
And it's not a new thing that you're worried about learning.
Chris Lattner (54:47.640)
It's just part of the deal.
Lex Fridman (54:49.400)
Now with Guido, I don't know Guido well.
Lex Fridman (54:55.480)
Yeah, have you passed cross much?
Lex Fridman (54:56.920)
Yeah, I've met him a couple of times,
Lex Fridman (54:58.180)
but I don't know Guido well.
Lex Fridman (55:00.000)
But the sense that I got out of that whole dynamic
Chris Lattner (55:03.280)
was that he had put the,
Lex Fridman (55:04.720)
not just the decision maker weight on his shoulders,
Lex Fridman (55:08.840)
but it was so tied to his personal identity
Lex Fridman (55:11.960)
that he took it personally and he felt the need
Lex Fridman (55:15.060)
and he kind of put himself in the situation
Lex Fridman (55:16.560)
of being the person,
Chris Lattner (55:18.200)
instead of building a base of support around him.
Lex Fridman (55:20.960)
I mean, this is probably not quite literally true,
Lex Fridman (55:23.960)
but by too much concentrated on him, right?
Lex Fridman (55:29.560)
And that can wear you down.
Chris Lattner (55:31.360)
Well, yeah, particularly because people then say,
Lex Fridman (55:33.800)
Guido, you're a horrible person.
Chris Lattner (55:35.400)
I hate this thing, blah, blah, blah, blah, blah, blah, blah.
Lex Fridman (55:37.600)
And sure, it's like maybe 1% of the community
Chris Lattner (55:40.040)
that's doing that, but Python's got a big community.
Lex Fridman (55:43.560)
And 1% of millions of people is a lot of hate mail.
Lex Fridman (55:46.640)
And that just from human factor will just wear on you.
Lex Fridman (55:49.480)
Well, to clarify, it looked from just what I saw
Chris Lattner (55:52.600)
in the messaging for the,
Lex Fridman (55:53.920)
let's not look at the million Python users,
Lex Fridman (55:55.800)
but at the Python core developers,
Lex Fridman (55:58.380)
it feels like the majority, the big majority
Chris Lattner (56:01.880)
on a vote were opposed to it.
Lex Fridman (56:03.680)
Okay, I'm not that close to it, so I don't know.
Chris Lattner (56:06.840)
Okay, so the situation is like literally,
Lex Fridman (56:10.920)
yeah, I mean, the majority of the core developers
Chris Lattner (56:13.120)
are against it.
Lex Fridman (56:13.960)
Were opposed to it.
Chris Lattner (56:14.780)
So, and they weren't even like against it.
Lex Fridman (56:20.920)
It was, there was a few, well, they were against it,
Lex Fridman (56:23.080)
but the against it wasn't like, this is a bad idea.
Lex Fridman (56:27.800)
They were more like, we don't see why this is a good idea.
Lex Fridman (56:31.240)
And what that results in is there's a stalling feeling,
Lex Fridman (56:35.160)
like you just slow things down.
Chris Lattner (56:37.980)
Now, from my perspective, that you could argue this,
Lex Fridman (56:41.600)
and I think it's very interesting
Chris Lattner (56:44.600)
if we look at politics today and the way Congress works,
Lex Fridman (56:47.600)
it's slowed down everything.
Chris Lattner (56:49.560)
It's a dampener.
Lex Fridman (56:50.400)
Yeah, it's a dampener, but like,
Chris Lattner (56:51.880)
that's a dangerous thing too,
Lex Fridman (56:53.660)
because if it dampens things like, you know,
Chris Lattner (56:57.480)
if the dampening results.
Lex Fridman (56:58.680)
What are you talking about?
Chris Lattner (56:59.520)
Like, it's a low pass filter,
Lex Fridman (57:00.520)
but if you need billions of dollars
Chris Lattner (57:02.320)
injected into the economy or trillions of dollars,
Lex Fridman (57:05.080)
then suddenly stuff happens, right?
Lex Fridman (57:06.840)
And so.
Lex Fridman (57:07.680)
For sure.
Lex Fridman (57:09.360)
So you're talking about.
Lex Fridman (57:10.460)
I'm not defending our political situation,
Chris Lattner (57:11.980)
just to be clear.
Lex Fridman (57:13.320)
But you're talking about like a global pandemic.
Chris Lattner (57:16.480)
Well.
Lex Fridman (57:17.320)
I was hoping we could fix like the healthcare system
Lex Fridman (57:20.560)
and the education system, like, you know.
Lex Fridman (57:22.960)
I'm not a politics person.
Chris Lattner (57:24.720)
I don't know.
Lex Fridman (57:26.240)
When it comes to languages,
Chris Lattner (57:28.120)
the community's kind of right in terms of,
Lex Fridman (57:30.760)
it's a very high burden to add something to a language.
Lex Fridman (57:33.200)
So as soon as you add something,
Lex Fridman (57:34.400)
you have a community of people building on it
Lex Fridman (57:35.720)
and you can't remove it, okay?
Lex Fridman (57:38.080)
And if there's a community of people
Chris Lattner (57:39.620)
that feel really uncomfortable with it,
Lex Fridman (57:41.620)
then taking it slow, I think, is an important thing to do.
Lex Fridman (57:45.600)
And there's no rush, particularly if it was something
Lex Fridman (57:48.080)
that's 25 years old and is very established.
Chris Lattner (57:50.360)
And, you know, it's not like coming into its own.
Lex Fridman (57:54.360)
What about features?
Chris Lattner (57:55.840)
Well, so I think that the issue with Guido
Lex Fridman (57:58.800)
is that maybe this is a case
Chris Lattner (58:00.360)
where he realized it had outgrown him
Lex Fridman (58:03.600)
and it went from being the language.
Lex Fridman (58:06.240)
So Python, I mean, Guido's amazing,
Lex Fridman (58:09.660)
but Python isn't about Guido anymore.
Chris Lattner (58:12.260)
It's about the users.
Lex Fridman (58:13.520)
And to a certain extent, the users own it.
Chris Lattner (58:15.320)
And, you know, Guido spent years of his life,
Lex Fridman (58:19.720)
a significant fraction of his career on Python.
Lex Fridman (58:22.880)
And from his perspective, I imagine he's like,
Lex Fridman (58:24.720)
well, this is my thing.
Chris Lattner (58:25.800)
I should be able to do the thing I think is right.
Lex Fridman (58:28.240)
But you can also understand the users
Chris Lattner (58:30.360)
where they feel like, you know, this is my thing.
Lex Fridman (58:33.140)
I use this, like, and I don't know, it's a hard thing.
Lex Fridman (58:38.320)
But what, if we could talk about leadership in this,
Lex Fridman (58:41.400)
because it's so interesting to me.
Chris Lattner (58:42.540)
I'm gonna make, I'm gonna work.
Lex Fridman (58:44.440)
Hopefully somebody makes it.
Chris Lattner (58:45.520)
If not, I'll make it a Walrus Operator shirt,
Lex Fridman (58:47.720)
because I think it represents, to me,
Chris Lattner (58:50.380)
maybe it's my Russian roots or something.
Lex Fridman (58:52.480)
But, you know, it's the burden of leadership.
Chris Lattner (58:56.100)
Like, I feel like to push back,
Lex Fridman (59:01.000)
I feel like progress can only,
Chris Lattner (59:02.980)
like most difficult decisions, just like you said,
Lex Fridman (59:06.220)
there'll be a lot of divisiveness over,
Chris Lattner (59:09.100)
especially in a passionate community.
Lex Fridman (59:12.180)
It just feels like leaders need to take
Chris Lattner (59:14.540)
those risky decisions that if you like listen,
Lex Fridman (59:19.540)
that with some nonzero probability,
Chris Lattner (59:23.020)
maybe even a high probability would be the wrong decision.
Lex Fridman (59:26.100)
But they have to use their gut and make that decision.
Chris Lattner (59:29.260)
Well, this is like one of the things
Lex Fridman (59:30.940)
where you see amazing founders.
Chris Lattner (59:34.180)
The founders understand exactly what's happened
Lex Fridman (59:36.220)
and how the company got there and are willing to say,
Chris Lattner (59:39.100)
we have been doing thing X the last 20 years,
Lex Fridman (59:42.780)
but today we're gonna do thing Y.
Lex Fridman (59:45.460)
And they make a major pivot for the whole company.
Lex Fridman (59:47.380)
The company lines up behind them,
Chris Lattner (59:48.580)
they move and it's the right thing.
Lex Fridman (59:50.540)
But then when the founder dies,
Chris Lattner (59:52.380)
the successor doesn't always feel that agency
Lex Fridman (59:57.060)
to be able to make those kinds of decisions.
Chris Lattner (59:59.140)
Even though they're a CEO,
🔗 相关节目