Gilbert Strang: Linear Algebra, Deep Learning, Teaching, and MIT OpenCourseWare
数学AI 与机器学习音乐与艺术技术与编程太空与探索
📋 章节目录
暂无章节信息
🔑 关键词
mathlinearalgebramatrixgotdataspacenumberscalculuslearningclassflatdondimensionsappstudentsvectordeepmathematicscourse
💬 精彩语录
暂无语录
🎙️ 完整对话（945 条）
Lex Fridman (00:00.000)
The following is a conversation with Gilbert Strang.
以下是与吉尔伯特·斯特朗的对话。
Lex Fridman (00:03.320)
He's a professor of mathematics at MIT
他是麻省理工学院的数学教授
Lex Fridman (00:05.760)
and perhaps one of the most famous
也许是最著名的之一
Lex Fridman (00:07.480)
and impactful teachers of math in the world.
以及世界上有影响力的数学教师。
Lex Fridman (00:10.600)
His MIT OpenCourseWare lectures on linear algebra
他的麻省理工学院开放式课程线性代数讲座
Gilbert Strang (00:13.640)
have been viewed millions of times.
已被浏览数百万次。
Lex Fridman (00:15.920)
As an undergraduate student,
作为一名本科生，
Gilbert Strang (00:17.480)
I was one of those millions of students.
我是那数百万学生中的一员。
Lex Fridman (00:19.960)
There's something inspiring about the way he teaches.
他的教学方式有一些鼓舞人心的地方。
Gilbert Strang (00:22.760)
There's at once calm, simple, and yet full of passion
既平静、简单，又充满激情
Lex Fridman (00:26.120)
for the elegance inherent to mathematics.
数学固有的优雅。
Gilbert Strang (00:29.080)
I remember doing the exercise in his book,
我记得做了他书中的练习，
Lex Fridman (00:31.200)
Introduction to Linear Algebra,
线性代数导论，
Lex Fridman (00:33.000)
and slowly realizing that the world of matrices,
并慢慢意识到矩阵的世界，
Lex Fridman (00:35.880)
of vector spaces, of determinants and eigenvalues,
向量空间、行列式和特征值、
Gilbert Strang (00:39.360)
of geometric transformations and matrix decompositions
几何变换和矩阵分解
Lex Fridman (00:43.040)
reveal a set of powerful tools
揭示一组强大的工具
Gilbert Strang (00:45.120)
in the toolbox of artificial intelligence.
在人工智能的工具箱里。
Lex Fridman (00:47.920)
From signals to images,
从信号到图像，
Gilbert Strang (00:49.720)
from numerical optimization to robotics,
从数值优化到机器人技术，
Lex Fridman (00:51.920)
computer vision, deep learning, computer graphics,
Lex Fridman (00:54.880)
and everywhere outside AI,
Lex Fridman (00:56.600)
including, of course, a quantum mechanical study
Gilbert Strang (01:00.080)
of our universe.
Lex Fridman (01:01.520)
This is the Artificial Intelligence Podcast.
Gilbert Strang (01:04.440)
If you enjoy it, subscribe on YouTube,
Lex Fridman (01:06.840)
give it five stars on Apple Podcast,
Gilbert Strang (01:08.880)
support on Patreon,
Lex Fridman (01:10.240)
or simply connect with me on Twitter
Gilbert Strang (01:12.360)
at Lex Friedman, spelled F R I D M A N.
Lex Fridman (01:16.440)
This podcast is supported by ZipRecruiter.
Gilbert Strang (01:19.040)
Hiring great people is hard,
Lex Fridman (01:21.200)
and to me is the most important element
Gilbert Strang (01:23.280)
of a successful mission driven team.
Lex Fridman (01:26.080)
I've been fortunate to be a part of
Lex Fridman (01:28.120)
and to lead several great engineering teams.
Lex Fridman (01:30.720)
The hiring I've done in the past
Gilbert Strang (01:32.680)
was mostly through tools that we built ourselves,
Lex Fridman (01:35.400)
but reinventing the wheel was painful.
Gilbert Strang (01:38.400)
ZipRecruiter is a tool that's already available for you.
Lex Fridman (01:41.240)
It seeks to make hiring simple, fast, and smart.
Gilbert Strang (01:45.000)
For example, Codable cofounder Gretchen Huebner
Lex Fridman (01:47.960)
used ZipRecruiter to find a new game artist
Gilbert Strang (01:50.400)
to join her education tech company.
Lex Fridman (01:52.720)
By using ZipRecruiter's screening questions
Gilbert Strang (01:54.840)
to filter candidates,
Lex Fridman (01:56.280)
Gretchen found it easier to focus on the best candidates
Lex Fridman (01:59.080)
and finally hiring the perfect person for the role
Lex Fridman (02:02.440)
in less than two weeks from start to finish.
Gilbert Strang (02:05.440)
ZipRecruiter, the smartest way to hire.
Lex Fridman (02:08.560)
See why ZipRecruiter is effective
Gilbert Strang (02:10.200)
for businesses of all sizes by signing up,
Lex Fridman (02:12.680)
as I did, for free at ziprecruiter.com slash lexpod.
Gilbert Strang (02:17.600)
That's ziprecruiter.com slash lexpod.
Lex Fridman (02:22.280)
This show is presented by Cash App,
Gilbert Strang (02:24.360)
the number one finance app in the App Store.
Lex Fridman (02:26.880)
I personally use Cash App to send money to friends,
Lex Fridman (02:29.640)
but you can also use it to buy, sell, and deposit Bitcoin.
Lex Fridman (02:33.080)
Most Bitcoin exchanges take days
Gilbert Strang (02:34.880)
for a bank transfer to become investable.
Lex Fridman (02:37.120)
Through Cash App, it takes seconds.
Gilbert Strang (02:39.480)
Cash App also has a new investing feature.
Lex Fridman (02:42.840)
You can buy fractions of a stock,
Gilbert Strang (02:44.600)
which to me is a really interesting concept.
Lex Fridman (02:47.320)
So you can buy $1 worth no matter what the stock price is.
Gilbert Strang (02:51.600)
Brokerage services are provided by Cash App Investing,
Lex Fridman (02:54.520)
a subsidiary of Square, and member SIPC.
Gilbert Strang (02:58.320)
I'm excited to be working with Cash App
Lex Fridman (03:00.120)
to support one of my favorite organizations
Gilbert Strang (03:02.800)
that many of you may know and have benefited from,
Lex Fridman (03:05.720)
called First, best known for their first robotics
Lex Fridman (03:09.200)
and Lego competitions.
Lex Fridman (03:10.840)
They educate and inspire hundreds of thousands of students
Gilbert Strang (03:14.560)
in over 110 countries,
Lex Fridman (03:16.520)
and have a perfect rating on Charity Navigator,
Gilbert Strang (03:19.000)
which means the donated money is used
Lex Fridman (03:21.600)
to maximum effectiveness.
Gilbert Strang (03:23.920)
When you get Cash App from the App Store or Google Play,
Lex Fridman (03:26.920)
and use code LexPodcast,
Gilbert Strang (03:29.400)
you get $10, and Cash App will also donate $10 to First,
Lex Fridman (03:33.200)
which again, is an organization
Gilbert Strang (03:35.200)
that I've personally seen inspire girls and boys
Lex Fridman (03:38.000)
to dream of engineering a better world.
Lex Fridman (03:41.000)
And now, here's my conversation with Gilbert Strang.
Lex Fridman (03:44.740)
How does it feel to be one of the modern day rock stars
Lex Fridman (03:50.500)
of mathematics?
Lex Fridman (03:51.760)
I don't feel like a rock star.
Gilbert Strang (03:53.900)
That's kind of crazy for an old math person.
Lex Fridman (03:57.340)
But it's true that the videos in linear algebra
Gilbert Strang (04:03.540)
that I made way back in 2000, I think,
Lex Fridman (04:08.540)
have been watched a lot.
Lex Fridman (04:09.880)
And well, partly the importance of linear algebra,
Lex Fridman (04:14.420)
which I'm sure you'll ask me,
Lex Fridman (04:15.940)
and give me a chance to say that linear algebra
Lex Fridman (04:19.220)
as a subject has just surged in importance.
Lex Fridman (04:22.580)
But also, it was a class that I taught a bunch of times,
Lex Fridman (04:26.460)
so I kind of got it organized and enjoyed doing it.
Gilbert Strang (04:32.220)
The videos were just the class.
Lex Fridman (04:34.180)
So they're on OpenCourseWare and on YouTube
Lex Fridman (04:36.740)
and translated, and it's fun.
Lex Fridman (04:38.900)
But there's something about that chalkboard
Lex Fridman (04:41.100)
and the simplicity of the way you explain
Lex Fridman (04:44.460)
the basic concepts in the beginning.
Gilbert Strang (04:46.820)
To be honest, when I went to undergrad.
Lex Fridman (04:50.440)
You didn't do linear algebra, probably.
Gilbert Strang (04:52.660)
Of course I didn't do linear algebra.
Lex Fridman (04:53.980)
You did.
Gilbert Strang (04:54.820)
Yeah, yeah, yeah, of course.
Lex Fridman (04:55.640)
But before going through the course at my university,
Gilbert Strang (05:00.940)
there was going through OpenCourseWare.
Lex Fridman (05:02.540)
You were my instructor for linear algebra.
Gilbert Strang (05:04.620)
Right, yeah.
Lex Fridman (05:05.500)
And that, I mean, we're using your book.
Lex Fridman (05:07.560)
And I mean, the fact that there is thousands,
Lex Fridman (05:13.100)
hundreds of thousands, millions of people
Gilbert Strang (05:14.860)
that watch that video, I think that's really powerful.
Lex Fridman (05:18.260)
So how do you think the idea of putting lectures online,
Lex Fridman (05:23.700)
what really MIT OpenCourseWare has innovated?
Lex Fridman (05:27.260)
That was a wonderful idea.
Gilbert Strang (05:29.140)
I think the story that I've heard is the committee
Lex Fridman (05:34.140)
was appointed by the president, President Vest,
Gilbert Strang (05:37.740)
at that time, a wonderful guy.
Lex Fridman (05:40.020)
And the idea of the committee was to figure out
Lex Fridman (05:43.500)
how MIT could be like other universities,
Lex Fridman (05:48.660)
market the work we were doing.
Lex Fridman (05:52.620)
And then they didn't see a way.
Lex Fridman (05:54.500)
And after a weekend, and they had an inspiration,
Gilbert Strang (05:57.300)
came back to President Vest and said,
Lex Fridman (06:00.020)
what if we just gave it away?
Lex Fridman (06:02.260)
And he decided that was okay, good idea.
Lex Fridman (06:07.700)
So.
Gilbert Strang (06:08.540)
You know, that's a crazy idea.
Lex Fridman (06:10.660)
If we think of a university as a thing
Gilbert Strang (06:12.740)
that creates a product, isn't knowledge,
Lex Fridman (06:17.780)
the kind of educational knowledge,
Gilbert Strang (06:19.580)
isn't the product and giving that away,
Lex Fridman (06:22.660)
are you surprised that it went through?
Gilbert Strang (06:26.140)
The result that he did it,
Lex Fridman (06:28.300)
well, knowing a little bit President Vest, it was like him,
Gilbert Strang (06:32.300)
I think, and it was really the right idea.
Lex Fridman (06:38.780)
MIT is a kind of, it's known for being high level,
Gilbert Strang (06:43.220)
technical things, and this is the best way we can say,
Lex Fridman (06:48.300)
tell, we can show what MIT really is like,
Gilbert Strang (06:52.580)
because in my case, those 1806 videos
Lex Fridman (06:57.380)
are just teaching the class.
Gilbert Strang (06:59.940)
They were there in 26, 100.
Lex Fridman (07:03.660)
They're kind of fun to look at.
Gilbert Strang (07:04.860)
People write to me and say, oh, you've got a sense of humor,
Lex Fridman (07:08.300)
but I don't know where that comes through.
Gilbert Strang (07:10.940)
Somehow I'm friendly with the class, I like students.
Lex Fridman (07:15.140)
And then your algebra, the subject,
Gilbert Strang (07:19.260)
we gotta give the subject most of the credit.
Lex Fridman (07:21.700)
It really has come forward in importance in these years.
Lex Fridman (07:29.980)
So let's talk about linear algebra a little bit,
Lex Fridman (07:32.740)
because it is such a, it's both a powerful
Lex Fridman (07:34.580)
and a beautiful subfield of mathematics.
Lex Fridman (07:39.100)
So what's your favorite specific topic in linear algebra,
Gilbert Strang (07:44.300)
or even math in general to give a lecture on,
Lex Fridman (07:46.620)
to convey, to tell a story, to teach students?
Gilbert Strang (07:50.220)
Okay, well, on the teaching side,
Lex Fridman (07:54.500)
so it's not deep mathematics at all,
Lex Fridman (07:56.820)
but I'm kind of proud of the idea of the four subspaces,
Lex Fridman (08:02.380)
the four fundamental subspaces,
Gilbert Strang (08:06.460)
which are of course known before,
Lex Fridman (08:09.660)
long before my name for them, but.
Lex Fridman (08:13.660)
Can you go through them?
Lex Fridman (08:14.700)
Can you go through the four subspaces?
Gilbert Strang (08:15.540)
Sure I can, yeah.
Lex Fridman (08:17.020)
So the first one to understand is,
Lex Fridman (08:19.660)
so the matrix is, maybe I should say the matrix is.
Lex Fridman (08:22.900)
What is a matrix?
Lex Fridman (08:23.780)
What's a matrix?
Lex Fridman (08:24.860)
Well, so we have like a rectangle of numbers.
Lex Fridman (08:28.500)
So it's got n columns, got a bunch of columns,
Lex Fridman (08:32.220)
and also got an m rows, let's say,
Lex Fridman (08:36.500)
and the relation between,
Lex Fridman (08:37.940)
so of course the columns and the rows,
Gilbert Strang (08:39.860)
it's the same numbers.
Lex Fridman (08:41.580)
So there's gotta be connections there,
Lex Fridman (08:44.340)
but they're not simple.
Lex Fridman (08:45.660)
The columns might be longer than the rows,
Lex Fridman (08:50.060)
and they're all different, the numbers are mixed up.
Lex Fridman (08:53.700)
First space to think about is take the columns,
Lex Fridman (08:57.740)
so those are vectors, those are points in n dimensions.
Lex Fridman (09:01.860)
What's a vector?
Lex Fridman (09:02.700)
So a physicist would imagine a vector
Lex Fridman (09:05.660)
or might imagine a vector as a arrow in space
Gilbert Strang (09:10.860)
or the point it ends at in space.
Lex Fridman (09:14.820)
For me, it's a column of numbers.
Gilbert Strang (09:18.300)
You often think of, this is very interesting
Lex Fridman (09:20.940)
in terms of linear algebra, in terms of a vector,
Gilbert Strang (09:23.940)
you think a little bit more abstract
Lex Fridman (09:26.660)
than how it's very commonly used, perhaps.
Gilbert Strang (09:30.660)
You think this arbitrary multidimensional space.
Lex Fridman (09:34.700)
Right away, I'm in high dimensions.
Gilbert Strang (09:38.180)
Dreamland.
Lex Fridman (09:39.020)
Yeah, that's right.
Gilbert Strang (09:40.060)
In the lecture, I try to,
Lex Fridman (09:42.420)
so if you think of two vectors in 10 dimensions,
Gilbert Strang (09:46.780)
I'll do this in class, and I'll readily admit
Lex Fridman (09:50.420)
that I have no good image in my mind
Gilbert Strang (09:54.380)
of a vector of an arrow in 10 dimensional space,
Lex Fridman (09:58.260)
but whatever.
Gilbert Strang (10:00.700)
You can add one bunch of 10 numbers
Lex Fridman (10:03.780)
to another bunch of 10 numbers,
Lex Fridman (10:05.300)
so you can add a vector to a vector,
Lex Fridman (10:08.060)
and you can multiply a vector by three,
Lex Fridman (10:10.300)
and that's, if you know how to do those,
Lex Fridman (10:12.180)
you've got linear algebra.
Gilbert Strang (10:14.380)
10 dimensions, there's this beautiful thing about math,
Lex Fridman (10:18.860)
if we look at string theory and all these theories,
Gilbert Strang (10:21.700)
which are really fundamentally derived through math,
Lex Fridman (10:24.460)
but are very difficult to visualize.
Lex Fridman (10:26.500)
How do you think about the things,
Lex Fridman (10:28.900)
like a 10 dimensional vector,
Lex Fridman (10:31.140)
that we can't really visualize?
Lex Fridman (10:34.780)
And yet, math reveals some beauty underlying our world
Gilbert Strang (10:39.780)
in that weird thing we can't visualize.
Lex Fridman (10:43.820)
How do you think about that difference?
Gilbert Strang (10:46.020)
Well, probably, I'm not a very geometric person,
Lex Fridman (10:48.900)
so I'm probably thinking in three dimensions,
Lex Fridman (10:51.700)
and the beauty of linear algebra is that
Lex Fridman (10:55.500)
it goes on to 10 dimensions with no problem.
Gilbert Strang (10:58.180)
I mean, that if you're just seeing what happens
Lex Fridman (11:01.220)
if you add two vectors in 3D,
Gilbert Strang (11:04.580)
yeah, then you can add them in 10D.
Lex Fridman (11:06.460)
You're just adding the 10 components.
Gilbert Strang (11:10.900)
So, I can't say that I have a picture,
Lex Fridman (11:14.860)
but yet I try to push the class
Gilbert Strang (11:16.860)
to think of a flat surface in 10 dimensions.
Lex Fridman (11:21.220)
So a plane in 10 dimensions,
Lex Fridman (11:23.420)
and so that's one of the spaces.
Lex Fridman (11:27.180)
Take all the columns of the matrix,
Gilbert Strang (11:29.900)
take all their combinations,
Lex Fridman (11:31.820)
so much of this column, so much of this one,
Gilbert Strang (11:35.180)
then if you put all those together,
Lex Fridman (11:36.820)
you get some kind of a flat surface
Gilbert Strang (11:39.940)
that I call a vector space, space of vectors.
Lex Fridman (11:44.700)
And my imagination is just seeing
Gilbert Strang (11:47.260)
like a piece of paper in 3D, but anyway,
Lex Fridman (11:52.500)
so that's one of the spaces, that's space number one,
Gilbert Strang (11:55.940)
the column space of the matrix.
Lex Fridman (11:58.460)
And then there's the row space, which is, as I said,
Gilbert Strang (12:01.580)
different, but came from the same numbers.
Lex Fridman (12:04.740)
So we got the column space,
Gilbert Strang (12:07.100)
all combinations of the columns,
Lex Fridman (12:10.040)
and then we've got the row space,
Gilbert Strang (12:11.960)
all combinations of the rows.
Lex Fridman (12:14.580)
So those words are easy for me to say,
Lex Fridman (12:17.520)
and I can't really draw them on a blackboard,
Lex Fridman (12:20.240)
but I try with my thick chalk.
Gilbert Strang (12:22.540)
Everybody likes that railroad chalk, and me too.
Lex Fridman (12:27.380)
I wouldn't use anything else now.
Lex Fridman (12:30.220)
And then the other two spaces are perpendicular to those.
Lex Fridman (12:35.060)
So like if you have a plane in 3D,
Gilbert Strang (12:39.100)
just a plane is just a flat surface in 3D,
Lex Fridman (12:43.500)
then perpendicular to that plane would be a line.
Lex Fridman (12:47.060)
So that would be the null space.
Lex Fridman (12:50.200)
So we've got two, we've got a column space, a row space,
Lex Fridman (12:54.000)
and there are two perpendicular spaces.
Lex Fridman (12:56.940)
So those four fit together in a beautiful picture
Gilbert Strang (13:01.540)
of a matrix, yeah, yeah.
Lex Fridman (13:03.740)
It's sort of a fundamental, it's not a difficult idea.
Gilbert Strang (13:06.740)
It comes pretty early in 1806, and it's basic.
Lex Fridman (13:12.740)
Planes in these multidimensional spaces,
Lex Fridman (13:16.380)
how difficult of an idea is that to come to, do you think?
Lex Fridman (13:20.140)
If you look back in time,
Gilbert Strang (13:23.620)
I think mathematically it makes sense,
Lex Fridman (13:26.820)
but I don't know if it's intuitive for us to imagine,
Gilbert Strang (13:29.600)
just as we were talking about.
Lex Fridman (13:31.100)
It feels like calculus is easier to intuit.
Gilbert Strang (13:34.800)
Well, I have to admit, calculus came earlier,
Lex Fridman (13:38.400)
earlier than linear algebra.
Lex Fridman (13:39.860)
So Newton and Leibniz were the great men
Lex Fridman (13:42.180)
to understand the key ideas of calculus.
Lex Fridman (13:47.100)
But linear algebra to me is like, okay,
Lex Fridman (13:50.100)
it's the starting point,
Gilbert Strang (13:51.620)
because it's all about flat things.
Lex Fridman (13:54.180)
Calculus has got, all the complications of calculus
Gilbert Strang (13:57.160)
come from the curves, the bending, the curved surfaces.
Lex Fridman (14:03.060)
Linear algebra, the surfaces are all flat.
Gilbert Strang (14:05.900)
Nothing bends in linear algebra.
Lex Fridman (14:08.060)
So it should have come first, but it didn't.
Lex Fridman (14:11.660)
And calculus also comes first in high school classes,
Lex Fridman (14:17.440)
in college class, it'll be freshman math,
Gilbert Strang (14:20.900)
it'll be calculus, and then I say, enough of it.
Lex Fridman (14:24.340)
Like, okay, get to the good stuff.
Lex Fridman (14:27.820)
And that's...
Lex Fridman (14:28.660)
Do you think linear algebra should come first?
Gilbert Strang (14:30.900)
Well, it really, I'm okay with it not coming first,
Lex Fridman (14:34.700)
but it should, yeah, it should.
Gilbert Strang (14:37.100)
It's simpler.
Lex Fridman (14:39.060)
Because everything is flat.
Gilbert Strang (14:40.180)
Yeah, everything's flat.
Lex Fridman (14:41.660)
Well, of course, for that reason,
Gilbert Strang (14:43.580)
calculus sort of sticks to one dimension,
Lex Fridman (14:46.620)
or eventually you do multivariate,
Lex Fridman (14:49.500)
but that basically means two dimensions.
Lex Fridman (14:52.260)
Linear algebra, you take off into 10 dimensions, no problem.
Gilbert Strang (14:55.500)
It just feels scary and dangerous
Lex Fridman (14:57.620)
to go beyond two dimensions, that's all.
Gilbert Strang (15:01.100)
If everything's flat, you can't go wrong.
Lex Fridman (15:03.700)
So what concept or theorem in linear algebra or in math
Gilbert Strang (15:09.780)
you find most beautiful,
Lex Fridman (15:12.700)
that gives you pause that leaves you in awe?
Gilbert Strang (15:15.420)
Well, I'll stick with linear algebra here.
Lex Fridman (15:18.060)
I hope the viewer knows that really,
Gilbert Strang (15:20.700)
mathematics is amazing, amazing subject
Lex Fridman (15:23.420)
and deep, deep connections between ideas
Gilbert Strang (15:28.740)
that didn't look connected, they turned out they were.
Lex Fridman (15:32.620)
But if we stick with linear algebra...
Lex Fridman (15:35.580)
So we have a matrix.
Lex Fridman (15:37.060)
That's like the basic thing, a rectangle of numbers.
Lex Fridman (15:40.740)
And it might be a rectangle of data.
Lex Fridman (15:42.660)
You're probably gonna ask me later about data science,
Gilbert Strang (15:46.740)
where often data comes in a matrix.
Lex Fridman (15:50.380)
You have maybe every column corresponds to a drug
Lex Fridman (15:57.420)
and every row corresponds to a patient.
Lex Fridman (16:00.380)
And if the patient reacted favorably to the drug,
Gilbert Strang (16:06.580)
then you put up some positive number in there.
Lex Fridman (16:09.180)
Anyway, rectangle of numbers, a matrix is basic.
Lex Fridman (16:14.180)
So the big problem is to understand all those numbers.
Lex Fridman (16:18.140)
You got a big, big set of numbers.
Lex Fridman (16:20.980)
And what are the patterns, what's going on?
Lex Fridman (16:23.660)
And so one of the ways to break down that matrix
Gilbert Strang (16:29.780)
into simple pieces is uses something called singular values.
Lex Fridman (16:36.100)
And that's come on as fundamental in the last,
Gilbert Strang (16:41.100)
certainly in my lifetime.
Lex Fridman (16:44.540)
Eigenvalues, if you have viewers who've done engineering,
Gilbert Strang (16:48.500)
math, or basic linear algebra, eigenvalues were in there.
Lex Fridman (16:55.620)
But those are restricted to square matrices.
Lex Fridman (16:58.780)
And data comes in rectangular matrices.
Lex Fridman (17:01.340)
So you gotta take that next step.
Gilbert Strang (17:04.380)
I'm always pushing math faculty, get on, do it, do it.
Lex Fridman (17:09.380)
Singular values.
Lex Fridman (17:11.180)
So those are a way to break, to find the important pieces
Lex Fridman (17:18.540)
of the matrix, which add up to the whole matrix.
Lex Fridman (17:22.140)
So you're breaking a matrix into simple pieces.
Lex Fridman (17:26.100)
And the first piece is the most important part of the data.
Gilbert Strang (17:30.300)
The second piece is the second most important part.
Lex Fridman (17:33.100)
And then often, so a data set is a matrix.
Lex Fridman (17:38.100)
And often, so a data scientist will like,
Lex Fridman (17:41.540)
if a data scientist can find those first and second pieces,
Gilbert Strang (17:46.820)
stop there, the rest of the data is probably round off,
Lex Fridman (17:55.660)
experimental error maybe.
Lex Fridman (17:57.660)
So you're looking for the important part.
Lex Fridman (18:00.180)
So what do you find beautiful about singular values?
Gilbert Strang (18:03.020)
Well, yeah, I didn't give the theorem.
Lex Fridman (18:06.260)
So here's the idea of singular values.
Gilbert Strang (18:09.420)
Every matrix, every matrix, rectangular, square, whatever,
Lex Fridman (18:15.180)
can be written as a product
Gilbert Strang (18:16.980)
of three very simple special matrices.
Lex Fridman (18:20.020)
So that's the theorem.
Gilbert Strang (18:21.340)
Every matrix can be written as a rotation times a stretch,
Lex Fridman (18:26.220)
which is just a diagonal matrix,
Gilbert Strang (18:30.340)
otherwise all zeros except on the one diagonal.
Lex Fridman (18:34.220)
And then the third factor is another rotation.
Lex Fridman (18:37.980)
So rotation, stretch, rotation
Lex Fridman (18:41.420)
is the breakup of any matrix.
Gilbert Strang (18:45.940)
The structure of that, the ability that you can do that,
Lex Fridman (18:48.420)
what do you find appealing?
Lex Fridman (18:49.860)
What do you find beautiful about it?
Lex Fridman (18:51.060)
Well, geometrically, as I freely admit,
Gilbert Strang (18:54.220)
the action of a matrix is not so easy to visualize,
Lex Fridman (18:59.620)
but everybody can visualize a rotation.
Gilbert Strang (19:02.380)
Take two dimensional space and just turn it
Lex Fridman (19:07.060)
around the center.
Gilbert Strang (19:09.260)
Take three dimensional space.
Lex Fridman (19:10.740)
So a pilot has to know about,
Gilbert Strang (19:13.220)
well, what are the three, the yaw is one of them.
Lex Fridman (19:16.860)
I've forgotten all the three turns that a pilot makes.
Gilbert Strang (19:22.260)
Up to 10 dimensions, you've got 10 ways to turn,
Lex Fridman (19:25.460)
but you can visualize a rotation.
Gilbert Strang (19:28.540)
Take the space and turn it.
Lex Fridman (19:30.300)
And you can visualize a stretch.
Lex Fridman (19:32.100)
So to break a matrix with all those numbers in it
Lex Fridman (19:38.660)
into something you can visualize,
Gilbert Strang (19:41.100)
rotate, stretch, rotate is pretty neat.
Lex Fridman (19:44.860)
It's pretty neat.
Gilbert Strang (19:45.740)
That's pretty powerful.
Lex Fridman (19:47.660)
On YouTube, just consuming a bunch of videos
Lex Fridman (19:51.980)
and just watching what people connect with
Lex Fridman (19:53.980)
and what they really enjoy and are inspired by,
Gilbert Strang (19:57.300)
math seems to come up again and again.
Lex Fridman (19:59.580)
I'm trying to understand why that is.
Gilbert Strang (20:03.940)
Perhaps you can help give me clues.
Lex Fridman (20:06.500)
So it's not just the kinds of lectures that you give,
Lex Fridman (20:10.740)
but it's also just other folks like with Numberphile,
Lex Fridman (20:14.180)
there's a channel where they just chat about things
Gilbert Strang (20:16.940)
that are extremely complicated, actually.
Lex Fridman (20:19.860)
People nevertheless connect with them.
Lex Fridman (20:22.820)
What do you think that is?
Lex Fridman (20:24.580)
It's wonderful, isn't it?
Gilbert Strang (20:25.820)
I mean, I wasn't really aware of it.
Lex Fridman (20:28.500)
We're conditioned to think math is hard,
Gilbert Strang (20:32.020)
math is abstract, math is just for a few people,
Lex Fridman (20:35.300)
but it isn't that way.
Gilbert Strang (20:36.620)
A lot of people quite like math and they liked it.
Lex Fridman (20:41.380)
I get messages from people saying,
Gilbert Strang (20:44.100)
now I'm retired, I'm gonna learn some more math.
Lex Fridman (20:46.780)
I get a lot of those.
Gilbert Strang (20:47.980)
It's really encouraging.
Lex Fridman (20:49.940)
And I think what people like is that there's some order,
Gilbert Strang (20:53.460)
a lot of order and things are not obvious, but they're true.
Lex Fridman (21:00.380)
So it's really cheering to think that so many people
Gilbert Strang (21:06.180)
really wanna learn more about math.
Lex Fridman (21:08.100)
Yeah.
Lex Fridman (21:08.940)
And in terms of truth, again,
Lex Fridman (21:11.820)
sorry to slide into philosophy at times,
Lex Fridman (21:15.500)
but math does reveal pretty strongly what things are true.
Lex Fridman (21:20.500)
I mean, that's the whole point of proving things.
Gilbert Strang (21:23.500)
It is, yeah.
Lex Fridman (21:24.340)
And yet, sort of our real world is messy and complicated.
Gilbert Strang (21:29.420)
It is.
Lex Fridman (21:30.260)
What do you think about the nature of truth
Lex Fridman (21:33.220)
that math reveals?
Lex Fridman (21:34.540)
Oh, wow.
Gilbert Strang (21:35.500)
Because it is a source of comfort like you've mentioned.
Lex Fridman (21:37.980)
Yeah, that's right.
Gilbert Strang (21:39.540)
Well, I have to say, I'm not much of a philosopher.
Lex Fridman (21:43.020)
I just like numbers.
Gilbert Strang (21:44.740)
As a kid, this was before you had to go in,
Lex Fridman (21:52.100)
when you had a filly in your teeth,
Gilbert Strang (21:54.020)
you had to kind of just take it.
Lex Fridman (21:56.060)
So what I did was think about math,
Gilbert Strang (21:59.260)
like take powers of two, two, four, eight, 16,
Lex Fridman (22:03.100)
up until the time the tooth stopped hurting
Lex Fridman (22:05.900)
and the dentist said you're through.
Lex Fridman (22:08.700)
Or counting.
Gilbert Strang (22:09.900)
Yeah.
Lex Fridman (22:10.740)
So that was a source of just, source of peace almost.
Gilbert Strang (22:14.700)
Yeah.
Lex Fridman (22:15.540)
What is it about math do you think that brings that?
Gilbert Strang (22:19.660)
Yeah.
Lex Fridman (22:20.500)
What is that?
Gilbert Strang (22:21.340)
Well, you know where you are.
Lex Fridman (22:22.380)
Yeah, it's symmetry, it's certainty.
Gilbert Strang (22:25.820)
The fact that, you know, if you multiply two by itself
Lex Fridman (22:29.820)
10 times, you get 1,024 period.
Gilbert Strang (22:33.220)
Everybody's gonna get that.
Lex Fridman (22:34.900)
Do you see math as a powerful tool or as an art form?
Lex Fridman (22:39.020)
So it's both.
Lex Fridman (22:40.300)
That's really one of the neat things.
Gilbert Strang (22:42.420)
You can be an artist and like math,
Lex Fridman (22:46.380)
you can be an engineer and use math.
Lex Fridman (22:50.940)
Which are you?
Lex Fridman (22:51.940)
Which am I?
Lex Fridman (22:53.500)
What did you connect with most?
Lex Fridman (22:54.820)
Yeah, I'm somewhere between.
Gilbert Strang (22:57.300)
I'm certainly not a artist type, philosopher type person.
Lex Fridman (23:01.540)
Might sound that way this morning, but I'm not.
Gilbert Strang (23:04.060)
Yeah, I really enjoy teaching engineers
Lex Fridman (23:09.060)
because they go for an answer.
Lex Fridman (23:13.260)
And yeah, so probably within the MIT math department,
Lex Fridman (23:20.380)
most people enjoy teaching people,
Gilbert Strang (23:23.620)
teaching students who get the abstract idea.
Lex Fridman (23:26.940)
I'm okay with, I'm good with engineers
Gilbert Strang (23:32.220)
who are looking for a way to find answers.
Lex Fridman (23:34.780)
Yeah.
Gilbert Strang (23:35.620)
Actually, that's an interesting question.
Lex Fridman (23:37.700)
Do you think for teaching and in general,
Gilbert Strang (23:41.340)
thinking about new concepts,
Lex Fridman (23:42.740)
do you think it's better to plug in the numbers
Lex Fridman (23:46.820)
or to think more abstractly?
Lex Fridman (23:49.060)
So looking at theorems and proving the theorems
Gilbert Strang (23:53.620)
or actually building up a basic intuition of the theorem
Lex Fridman (23:58.220)
or the method, the approach,
Lex Fridman (23:59.900)
and then just plugging in numbers and seeing it work.
Lex Fridman (24:02.940)
Yeah, well, certainly many of us like to see examples.
Gilbert Strang (24:09.220)
First, we understand,
Lex Fridman (24:11.060)
it might be a pretty abstract sounding example,
Gilbert Strang (24:13.980)
like a three dimensional rotation.
Lex Fridman (24:16.820)
How are you gonna understand a rotation in 3D?
Lex Fridman (24:22.780)
Or in 10D?
Lex Fridman (24:28.100)
And then some of us like to keep going with it
Gilbert Strang (24:30.860)
to the point where you got numbers,
Lex Fridman (24:32.740)
where you got 10 angles, 10 axes, 10 angles.
Lex Fridman (24:38.100)
But the best, the great mathematicians probably,
Lex Fridman (24:43.620)
I don't know if they do that,
Gilbert Strang (24:44.740)
because for them, an example would be a highly abstract thing
Lex Fridman (24:53.980)
to the rest of it.
Gilbert Strang (24:54.820)
Right, but nevertheless, working in the space of examples.
Lex Fridman (24:57.540)
Yeah, examples.
Gilbert Strang (24:58.380)
It seems to.
Lex Fridman (24:59.220)
Examples of structure.
Gilbert Strang (25:01.820)
Our brains seem to connect with that.
Lex Fridman (25:03.620)
Yeah, yeah.
Lex Fridman (25:04.620)
So I'm not sure if you're familiar with him,
Lex Fridman (25:07.180)
but Andrew Yang is a presidential candidate
Gilbert Strang (25:11.820)
currently running with math in all capital letters
Lex Fridman (25:17.060)
and his hats as a slogan.
Gilbert Strang (25:18.820)
I see.
Lex Fridman (25:19.660)
Stands for Make America Think Hard.
Gilbert Strang (25:21.700)
Okay, I'll vote for him.
Lex Fridman (25:25.180)
So, and his name rhymes with yours, Yang, Strang.
Lex Fridman (25:28.660)
But he also loves math and he comes from that world
Lex Fridman (25:31.260)
of, but he also, looking at it,
Gilbert Strang (25:35.500)
makes me realize that math, science, and engineering
Lex Fridman (25:38.580)
are not really part of our politics, political discourse,
Gilbert Strang (25:43.300)
about political government in general.
Lex Fridman (25:46.140)
Why do you think that is?
Gilbert Strang (25:48.620)
Well.
Lex Fridman (25:49.460)
What are your thoughts on that in general?
Gilbert Strang (25:51.180)
Well, certainly somewhere in the system,
Lex Fridman (25:52.740)
we need people who are comfortable with numbers,
Gilbert Strang (25:56.860)
comfortable with quantities.
Lex Fridman (25:58.340)
You know, if you say this leads to that,
Gilbert Strang (26:02.460)
they see it and it's undeniable.
Lex Fridman (26:05.940)
But isn't that strange to you that we have almost no,
Gilbert Strang (26:10.180)
I mean, I'm pretty sure we have no elected officials
Lex Fridman (26:14.420)
in Congress or obviously the president
Gilbert Strang (26:18.380)
that either has an engineering degree or a math degree.
Lex Fridman (26:22.900)
Yeah, well, that's too bad.
Gilbert Strang (26:25.820)
A few could, a few who could make the connection.
Lex Fridman (26:30.660)
Yeah, it would have to be people who understand
Gilbert Strang (26:35.420)
engineering or science and at the same time
Lex Fridman (26:38.580)
can make speeches and lead, yeah.
Gilbert Strang (26:44.420)
Yeah, inspire people.
Lex Fridman (26:45.540)
Yeah, inspire, yeah.
Gilbert Strang (26:46.580)
You were, speaking of inspiration,
Lex Fridman (26:49.260)
the president of the Society
Gilbert Strang (26:50.580)
for Industrial and Applied Mathematics.
Lex Fridman (26:52.860)
Oh, yes.
Gilbert Strang (26:53.700)
It's a major organization in math, applied math.
Lex Fridman (26:57.940)
What do you see as a role of that society,
Lex Fridman (27:01.180)
you know, in our public discourse?
Lex Fridman (27:02.860)
Right.
Gilbert Strang (27:03.700)
In public.
Lex Fridman (27:04.540)
Yeah, so, well, it was fun to be president at the time.
Gilbert Strang (27:08.380)
A couple years, a few years.
Lex Fridman (27:09.420)
Two years, around 2000.
Gilbert Strang (27:13.660)
I just hope that's president of a pretty small society.
Lex Fridman (27:16.820)
But nevertheless, it was a time when math
Gilbert Strang (27:19.620)
was getting some more attention in Washington.
Lex Fridman (27:24.380)
But yeah, I got to give a little 10 minutes
Gilbert Strang (27:29.220)
to a committee of the House of Representatives
Lex Fridman (27:33.900)
talking about who I met.
Lex Fridman (27:35.300)
And then, actually, it was fun
Lex Fridman (27:36.980)
because one of the members of the House
Gilbert Strang (27:42.460)
had been a student, had been in my class.
Lex Fridman (27:44.860)
What do you think of that?
Gilbert Strang (27:46.060)
Yeah, as you say, pretty rare, most members of the House
Lex Fridman (27:49.980)
have had a different training, different background.
Lex Fridman (27:52.860)
But there was one from New Hampshire
Lex Fridman (27:56.340)
who was my friend, really, by being in the class.
Gilbert Strang (28:02.460)
Yeah, so those years were good.
Lex Fridman (28:05.780)
Then, of course, other things take over in importance
Gilbert Strang (28:10.780)
in Washington, and math just, at this point,
Lex Fridman (28:16.980)
is not so visible.
Lex Fridman (28:18.260)
But for a little moment, it was.
Lex Fridman (28:20.220)
There's some excitement, some concern
Gilbert Strang (28:23.780)
about artificial intelligence in Washington now.
Lex Fridman (28:26.300)
Yes, sure. About the future.
Gilbert Strang (28:27.460)
Yeah. And I think at the core
Lex Fridman (28:28.820)
of that is math.
Gilbert Strang (28:30.020)
Well, it is, yeah.
Lex Fridman (28:32.020)
Maybe it's hidden.
Gilbert Strang (28:32.860)
Maybe it's wearing a different hat.
Lex Fridman (28:34.380)
Well, artificial intelligence, and particularly,
Lex Fridman (28:39.220)
can I use the words deep learning?
Lex Fridman (28:41.980)
Deep learning is a particular approach
Gilbert Strang (28:44.260)
to understanding data.
Lex Fridman (28:47.580)
Again, you've got a big, whole lot of data
Gilbert Strang (28:51.060)
where data is just swamping the computers of the world.
Lex Fridman (28:56.140)
And to understand it, out of all those numbers,
Gilbert Strang (29:00.660)
to find what's important in climate, in everything.
Lex Fridman (29:05.180)
And artificial intelligence is two words
Gilbert Strang (29:08.500)
for one approach to data.
Lex Fridman (29:11.700)
Deep learning is a specific approach there,
Gilbert Strang (29:15.540)
which uses a lot of linear algebra.
Lex Fridman (29:17.420)
So I got into it.
Gilbert Strang (29:19.300)
I thought, okay, I've gotta learn about this.
Lex Fridman (29:21.580)
So maybe from your perspective,
Gilbert Strang (29:24.140)
let me ask the most basic question.
Lex Fridman (29:27.900)
How do you think of a neural network?
Lex Fridman (29:30.340)
What is a neural network?
Lex Fridman (29:31.700)
Yeah, okay.
Lex Fridman (29:32.660)
So can I start with the idea about deep learning?
Lex Fridman (29:37.220)
What does that mean?
Lex Fridman (29:38.860)
What is deep learning?
Lex Fridman (29:39.940)
What is deep learning, yeah.
Lex Fridman (29:41.980)
So we're trying to learn, from all this data,
Lex Fridman (29:46.300)
we're trying to learn what's important.
Lex Fridman (29:47.900)
What's it telling us?
Lex Fridman (29:50.260)
So you've got data, you've got some inputs
Gilbert Strang (29:55.300)
for which you know the right outputs.
Lex Fridman (29:57.620)
The question is, can you see the pattern there?
Lex Fridman (30:02.100)
Can you figure out a way for a new input,
Lex Fridman (30:04.660)
which we haven't seen, to understand
Lex Fridman (30:09.740)
what the output will be from that new input?
Lex Fridman (30:12.220)
So we've got a million inputs with their outputs.
Lex Fridman (30:15.940)
So we're trying to create some pattern,
Lex Fridman (30:19.260)
some rule that'll take those inputs,
Gilbert Strang (30:22.180)
those million training inputs, which we know about,
Lex Fridman (30:25.580)
to the correct million outputs.
Lex Fridman (30:28.180)
And this idea of a neural net
Lex Fridman (30:32.780)
is part of the structure of our new way to create a rule.
Gilbert Strang (30:40.700)
We're looking for a rule that will take
Lex Fridman (30:43.900)
these training inputs to the known outputs.
Lex Fridman (30:48.460)
And then we're gonna use that rule on new inputs
Lex Fridman (30:51.660)
that we don't know the output and see what comes.
Gilbert Strang (30:56.100)
Linear algebra is a big part of finding that rule.
Lex Fridman (30:59.140)
That's right, linear algebra is a big part.
Gilbert Strang (31:01.860)
Not all the part.
Lex Fridman (31:03.500)
People were leaning on matrices, that's good, still do.
Gilbert Strang (31:08.300)
Linear is something special.
Lex Fridman (31:10.300)
It's all about straight lines and flat planes.
Lex Fridman (31:13.980)
And data isn't quite like that.
Lex Fridman (31:18.860)
It's more complicated.
Lex Fridman (31:21.220)
So you gotta introduce some complication.
Lex Fridman (31:23.700)
So you have to have some function
Gilbert Strang (31:25.460)
that's not a straight line.
Lex Fridman (31:27.460)
And it turned out, nonlinear, nonlinear, not linear.
Lex Fridman (31:31.620)
And it turned out that it was enough to use the function
Lex Fridman (31:35.860)
that's one straight line and then a different one.
Gilbert Strang (31:38.340)
Halfway, so piecewise linear.
Lex Fridman (31:40.900)
One piece has one slope,
Gilbert Strang (31:44.420)
one piece, the other piece has the second slope.
Lex Fridman (31:47.340)
And so that, getting that nonlinear,
Gilbert Strang (31:52.380)
simple nonlinearity in blew the problem open.
Lex Fridman (31:56.700)
That little piece makes it sufficiently complicated
Gilbert Strang (31:58.980)
to make things interesting.
Lex Fridman (32:00.460)
Because you're gonna use that piece
Gilbert Strang (32:02.020)
over and over a million times.
Lex Fridman (32:03.820)
So it has a fold in the graph, the graph, two pieces.
Lex Fridman (32:10.740)
But when you fold something a million times,
Lex Fridman (32:13.700)
you've got a pretty complicated function
Gilbert Strang (32:17.860)
that's pretty realistic.
Lex Fridman (32:19.260)
So that's the thing about neural networks
Gilbert Strang (32:21.140)
is they have a lot of these.
Lex Fridman (32:23.900)
A lot of these, that's right.
Lex Fridman (32:25.220)
So why do you think neural networks,
Lex Fridman (32:29.660)
by using sort of formulating an objective function,
Gilbert Strang (32:34.940)
very not a plain function of the folds,
Lex Fridman (32:39.380)
lots of folds of the inputs, the outputs,
Lex Fridman (32:42.340)
why do you think they work to be able to find a rule
Lex Fridman (32:47.300)
that we don't know is optimal,
Lex Fridman (32:48.780)
but it just seems to be pretty good in a lot of cases?
Lex Fridman (32:53.300)
What's your intuition?
Lex Fridman (32:54.580)
Is it surprising to you as it is to many people?
Lex Fridman (32:58.180)
Do you have an intuition of why this works at all?
Gilbert Strang (33:01.140)
Well, I'm beginning to have a better intuition.
Lex Fridman (33:04.300)
This idea of things that are piecewise linear,
Gilbert Strang (33:08.500)
flat pieces but with folds between them.
Lex Fridman (33:12.140)
Like think of a roof of a complicated,
Gilbert Strang (33:14.980)
infinitely complicated house or something.
Lex Fridman (33:17.780)
That curve, it almost curved, but every piece is flat.
Gilbert Strang (33:24.700)
That's been used by engineers,
Lex Fridman (33:26.820)
that idea has been used by engineers,
Gilbert Strang (33:29.660)
is used by engineers, big time.
Lex Fridman (33:32.140)
Something called the finite element method.
Gilbert Strang (33:34.220)
If you want to design a bridge,
Lex Fridman (33:36.980)
design a building, design an airplane,
Gilbert Strang (33:40.860)
you're using this idea of piecewise flat
Lex Fridman (33:47.300)
as a good, simple, computable approximation.
Lex Fridman (33:52.260)
But you have a sense that there's a lot of expressive power
Lex Fridman (33:57.260)
in this kind of piecewise linear.
Gilbert Strang (33:58.580)
Yeah, you used the right word.
Lex Fridman (34:01.820)
If you measure the expressivity,
Lex Fridman (34:04.460)
how complicated a thing can this piecewise flat guys express?
Lex Fridman (34:12.300)
The answer is very complicated, yeah.
Lex Fridman (34:15.500)
What do you think are the limits of such piecewise linear
Lex Fridman (34:20.380)
or just of neural networks?
Gilbert Strang (34:22.660)
The expressivity of neural networks.
Lex Fridman (34:24.100)
Well, you would have said a while ago
Gilbert Strang (34:26.660)
that they're just computational limits.
Lex Fridman (34:28.700)
It's a problem beyond a certain size.
Gilbert Strang (34:33.740)
A supercomputer isn't gonna do it.
Lex Fridman (34:36.060)
But those keep getting more powerful.
Lex Fridman (34:39.420)
So that limit has been moved
Lex Fridman (34:44.260)
to allow more and more complicated surfaces.
Lex Fridman (34:47.420)
So in terms of just mapping from inputs to outputs,
Lex Fridman (34:52.940)
looking at data, what do you think of,
Gilbert Strang (34:58.460)
in the context of neural networks in general,
Lex Fridman (35:00.500)
data is just tensor, vectors, matrices, tensors.
Gilbert Strang (35:04.180)
Right.
Lex Fridman (35:05.820)
How do you think about learning from data?
Lex Fridman (35:09.380)
How much of our world can be expressed in this way?
Lex Fridman (35:12.780)
How useful is this process?
Gilbert Strang (35:16.540)
I guess that's another way to ask you,
Lex Fridman (35:17.980)
what are the limits of this approach?
Gilbert Strang (35:19.340)
Well, that's a good question, yeah.
Lex Fridman (35:21.380)
So I guess the whole idea of deep learning
Gilbert Strang (35:24.220)
is that there's something there to learn.
Lex Fridman (35:26.220)
If the data is totally random,
Gilbert Strang (35:28.500)
just produced by random number generators,
Lex Fridman (35:31.380)
then we're not gonna find a useful rule
Gilbert Strang (35:36.220)
because there isn't one.
Lex Fridman (35:38.620)
So the extreme of having a rule is like knowing Newton's law.
Gilbert Strang (35:43.620)
If you hit a ball, it moves.
Lex Fridman (35:46.220)
So that's where you had laws of physics.
Gilbert Strang (35:48.940)
Newton and Einstein and other great, great people
Lex Fridman (35:54.140)
have found those laws and laws of the distribution
Gilbert Strang (36:02.940)
of oil in an underground thing.
Lex Fridman (36:05.900)
I mean, so engineers, petroleum engineers understand
Lex Fridman (36:10.900)
how oil will sit in an underground basin.
Lex Fridman (36:18.180)
So there were rules.
Gilbert Strang (36:20.060)
Now, the new idea of artificial intelligence is
Lex Fridman (36:25.620)
learn the rules instead of figuring out the rules
Gilbert Strang (36:29.940)
with help from Newton or Einstein.
Lex Fridman (36:32.740)
The computer is looking for the rules.
Lex Fridman (36:35.660)
So that's another step.
Lex Fridman (36:36.900)
But if there are no rules at all
Gilbert Strang (36:39.860)
that the computer could find,
Lex Fridman (36:41.220)
if it's totally random data, well, you've got nothing.
Gilbert Strang (36:45.300)
You've got no science to discover.
Lex Fridman (36:48.300)
It's an automated search for the underlying rules.
Gilbert Strang (36:51.380)
Yeah, search for the rules.
Lex Fridman (36:53.380)
Yeah, exactly.
Lex Fridman (36:54.780)
And there will be a lot of random parts.
Lex Fridman (36:57.820)
A lot of, I mean, I'm not knocking random
Gilbert Strang (36:59.860)
because that's there.
Lex Fridman (37:05.580)
There's a lot of randomness built in,
Lex Fridman (37:07.340)
but there's gotta be some basic.
Lex Fridman (37:09.380)
It's almost always signal, right?
Gilbert Strang (37:10.900)
In most things.
Lex Fridman (37:11.740)
There's gotta be some signal, yeah.
Gilbert Strang (37:12.820)
If it's all noise, then you're not gonna get anywhere.
Lex Fridman (37:17.420)
Well, this world around us does seem to be,
Gilbert Strang (37:19.900)
does seem to always have a signal of some kind.
Lex Fridman (37:22.420)
Yeah, yeah, that's right.
Gilbert Strang (37:23.340)
To be discovered.
Lex Fridman (37:24.220)
Right, that's it.
Lex Fridman (37:25.900)
So what excites you more?
Lex Fridman (37:30.580)
We just talked about a little bit of application.
Lex Fridman (37:32.860)
What excites you more, theory
Lex Fridman (37:35.580)
or the application of mathematics?
Gilbert Strang (37:38.380)
Well, for myself, I'm probably a theory person.
Lex Fridman (37:43.260)
I'm not, I'm speaking here pretty freely about applications,
Lex Fridman (37:49.700)
but I'm not the person who really,
Lex Fridman (37:53.220)
I'm not a physicist or a chemist or a neuroscientist.
Lex Fridman (37:58.100)
So for myself, I like the structure
Lex Fridman (38:03.220)
and the flat subspaces
Lex Fridman (38:06.460)
and the relation of matrices, columns to rows.
Lex Fridman (38:12.260)
That's my part in the spectrum.
Lex Fridman (38:17.860)
So really, science is a big spectrum of people
Lex Fridman (38:22.420)
from asking practical questions
Lex Fridman (38:25.740)
and answering them using some math,
Lex Fridman (38:28.740)
then some math guys like myself who are in the middle of it
Lex Fridman (38:33.740)
and then the geniuses of math and physics and chemistry
Lex Fridman (38:40.620)
who are finding fundamental rules
Lex Fridman (38:43.300)
and then doing the really understanding nature.
Lex Fridman (38:50.060)
That's incredible.
Gilbert Strang (38:51.820)
At its lowest, simplest level,
Lex Fridman (38:54.980)
maybe just a quick in broad strokes from your perspective,
Lex Fridman (38:58.980)
where does linear algebra sit as a subfield of mathematics?
Lex Fridman (39:04.740)
What are the various subfields that you think about
Lex Fridman (39:10.300)
in relation to linear algebra?
Lex Fridman (39:12.180)
So the big fields of math are algebra as a whole
Lex Fridman (39:18.020)
and problems like calculus and differential equations.
Lex Fridman (39:21.340)
So that's a second, quite different field.
Gilbert Strang (39:24.340)
Then maybe geometry deserves to be thought of
Lex Fridman (39:28.780)
as a different field to understand the geometry
Gilbert Strang (39:31.540)
of high dimensional surfaces.
Lex Fridman (39:35.700)
So I think, am I allowed to say this here?
Gilbert Strang (39:39.700)
I think this is where personal view comes in.
Lex Fridman (39:46.180)
I think math, we're thinking about undergraduate math,
Lex Fridman (39:51.980)
what millions of students study.
Lex Fridman (39:54.260)
I think we overdo the calculus at the cost of the algebra,
Gilbert Strang (40:00.780)
at the cost of linear.
Lex Fridman (40:02.660)
So you have this talk titled Calculus Versus Linear Algebra.
Gilbert Strang (40:05.300)
That's right, that's right.
Lex Fridman (40:07.380)
And you say that linear algebra wins.
Lex Fridman (40:09.420)
So can you dig into that a little bit?
Lex Fridman (40:13.780)
Why does linear algebra win?
Gilbert Strang (40:17.020)
Right, well, okay, the viewer is gonna think
Lex Fridman (40:21.100)
this guy is biased.
Gilbert Strang (40:22.700)
Not true, I'm just telling the truth as it is.
Lex Fridman (40:27.060)
Yeah, so I feel linear algebra is just a nice part of math
Gilbert Strang (40:31.940)
that people can get the idea of.
Lex Fridman (40:34.420)
They can understand something that's a little bit abstract
Gilbert Strang (40:37.780)
because once you get to 10 or 100 dimensions
Lex Fridman (40:42.140)
and very, very, very useful,
Gilbert Strang (40:44.380)
that's what's happened in my lifetime
Lex Fridman (40:47.860)
is the importance of data,
Gilbert Strang (40:52.540)
which does come in matrix form.
Lex Fridman (40:54.660)
So it's really set up for algebra.
Gilbert Strang (40:56.660)
It's not set up for differential equation.
Lex Fridman (40:59.260)
And let me fairly add probability,
Gilbert Strang (41:03.300)
the ideas of probability and statistics
Lex Fridman (41:06.860)
have become very, very important, have also jumped forward.
Gilbert Strang (41:11.260)
So, and that's different from linear algebra,
Lex Fridman (41:14.060)
quite different.
Lex Fridman (41:15.220)
So now we really have three major areas to me,
Lex Fridman (41:20.220)
calculus, linear algebra, matrices,
Lex Fridman (41:26.180)
and probability statistics.
Lex Fridman (41:28.980)
And they all deserve an important place.
Lex Fridman (41:33.980)
And calculus has traditionally had a lion's share
Lex Fridman (41:40.020)
of the time.
Gilbert Strang (41:40.860)
A disproportionate share.
Lex Fridman (41:41.900)
It is, thank you, disproportionate, that's a good word.
Gilbert Strang (41:45.700)
Of the love and attention from the excited young minds.
Lex Fridman (41:50.020)
Yeah.
Gilbert Strang (41:52.900)
I know it's hard to pick favorites,
Lex Fridman (41:55.500)
but what is your favorite matrix?
Lex Fridman (41:57.700)
What's my favorite matrix?
Lex Fridman (41:59.380)
Okay, so my favorite matrix is square, I admit it.
Gilbert Strang (42:03.220)
It's a square bunch of numbers
Lex Fridman (42:05.460)
and it has twos running down the main diagonal.
Lex Fridman (42:10.180)
And on the next diagonal,
Lex Fridman (42:13.020)
so think of top left to bottom right,
Gilbert Strang (42:15.380)
twos down the middle of the matrix
Lex Fridman (42:18.900)
and minus ones just above those twos
Lex Fridman (42:22.140)
and minus ones just below those twos
Lex Fridman (42:25.020)
and otherwise all zeros.
Lex Fridman (42:26.620)
So mostly zeros, just three nonzero diagonals coming down.
Lex Fridman (42:32.900)
What is interesting about it?
Gilbert Strang (42:34.380)
Well, all the different ways it comes up.
Lex Fridman (42:37.180)
You see it in engineering,
Gilbert Strang (42:39.260)
you see it as analogous in calculus to second derivative.
Lex Fridman (42:44.100)
So calculus learns about taking the derivative,
Gilbert Strang (42:47.180)
the figuring out how much, how fast something's changing.
Lex Fridman (42:51.500)
But second derivative, now that's also important.
Gilbert Strang (42:55.740)
That's how fast the change is changing,
Lex Fridman (42:58.740)
how fast the graph is bending, how fast it's curving.
Lex Fridman (43:06.460)
And Einstein showed that that's fundamental
Lex Fridman (43:10.140)
to understand space.
Lex Fridman (43:11.540)
So second derivatives should have a bigger place in calculus.
Lex Fridman (43:17.380)
Second, my matrices,
Gilbert Strang (43:21.020)
which are like the linear algebra version
Lex Fridman (43:24.980)
of second derivatives are neat in linear algebra.
Gilbert Strang (43:30.020)
Yeah, just everything comes out right with those guys.
Lex Fridman (43:34.020)
Beautiful.
Lex Fridman (43:35.220)
What did you learn about the process of learning
Lex Fridman (43:38.380)
by having taught so many students math over the years?
Gilbert Strang (43:42.820)
Ooh, that is hard.
Lex Fridman (43:45.700)
I'll have to admit here that I'm not really a good teacher
Gilbert Strang (43:51.260)
because I don't get into the exam part.
Lex Fridman (43:55.540)
The exam is the part of my life that I don't like
Lex Fridman (43:59.020)
and grading them and giving the students A or B or whatever.
Lex Fridman (44:04.380)
I do it because I'm supposed to do it,
Lex Fridman (44:08.260)
but I tell the class at the beginning,
Lex Fridman (44:11.900)
I don't know if they believe me.
Gilbert Strang (44:13.180)
Probably they don't.
Lex Fridman (44:14.580)
I tell the class, I'm here to teach you.
Gilbert Strang (44:18.020)
I'm here to teach you math and not to grade you.
Lex Fridman (44:22.700)
But they're thinking, okay, this guy is gonna,
Lex Fridman (44:26.300)
when is he gonna give me an A minus?
Lex Fridman (44:28.820)
Is he gonna give me a B plus?
Lex Fridman (44:30.580)
What?
Lex Fridman (44:31.420)
What have you learned about the process of learning?
Gilbert Strang (44:34.060)
Of learning.
Lex Fridman (44:34.940)
Yeah, well, maybe to give you a legitimate answer
Gilbert Strang (44:40.220)
about learning, I should have paid more attention
Lex Fridman (44:43.900)
to the assessment, the evaluation part at the end.
Lex Fridman (44:47.660)
But I like the teaching part at the start.
Lex Fridman (44:49.980)
That's the sexy part.
Gilbert Strang (44:52.060)
To tell somebody for the first time about a matrix, wow.
Lex Fridman (44:56.060)
Is there, are there moments,
Lex Fridman (44:58.700)
so you are teaching a concept,
Lex Fridman (45:01.900)
are there moments of learning that you just see
Lex Fridman (45:05.500)
in the student's eyes?
Lex Fridman (45:06.460)
You don't need to look at the grades.
Lex Fridman (45:08.220)
But you see in their eyes that you hook them,
Lex Fridman (45:11.540)
that you connect with them in a way where,
Gilbert Strang (45:16.260)
you know what, they fall in love
Lex Fridman (45:18.620)
with this beautiful world of math.
Gilbert Strang (45:21.180)
They see that it's got some beauty there.
Lex Fridman (45:24.460)
Or conversely, that they give up at that point
Gilbert Strang (45:28.060)
is the opposite.
Lex Fridman (45:29.140)
The dark could say that math, I'm just not good at math.
Gilbert Strang (45:32.420)
I don't wanna walk away.
Lex Fridman (45:33.260)
Yeah, yeah, yeah.
Gilbert Strang (45:34.300)
Maybe because of the approach in the past,
Lex Fridman (45:37.700)
they were discouraged, but don't be discouraged.
Gilbert Strang (45:40.500)
It's too good to miss.
Lex Fridman (45:44.420)
Yeah, well, if I'm teaching a big class,
Gilbert Strang (45:48.420)
do I know when, I think maybe I do.
Lex Fridman (45:51.900)
Sort of, I mentioned at the very start,
Gilbert Strang (45:55.460)
the four fundamental subspaces
Lex Fridman (45:59.340)
and the structure of the fundamental theorem
Gilbert Strang (46:03.100)
of linear algebra.
Lex Fridman (46:04.740)
The fundamental theorem of linear algebra.
Gilbert Strang (46:06.780)
That is the relation of those four subspaces,
Lex Fridman (46:11.740)
those four spaces.
Gilbert Strang (46:13.420)
Yeah, so I think that, I feel that the class gets it.
Lex Fridman (46:17.740)
At length.
Gilbert Strang (46:18.580)
Yeah.
Lex Fridman (46:19.940)
What advice do you have to a student
Lex Fridman (46:22.420)
just starting their journey in mathematics today?
Lex Fridman (46:25.140)
How do they get started?
Gilbert Strang (46:27.060)
Oh, yeah, that's hard.
Lex Fridman (46:30.100)
Well, I hope you have a teacher, professor,
Gilbert Strang (46:34.780)
who is still enjoying what he's doing,
Lex Fridman (46:39.860)
what he's teaching.
Gilbert Strang (46:41.380)
They're still looking for new ways to teach
Lex Fridman (46:44.020)
and to understand math.
Gilbert Strang (46:47.940)
Cause that's the pleasure,
Lex Fridman (46:51.140)
the moment when you see, oh yeah, that works.
Lex Fridman (46:54.980)
So it's less about the material you study,
Lex Fridman (46:58.500)
it's more about the source of the teacher
Gilbert Strang (47:02.460)
being full of passion.
Lex Fridman (47:03.900)
Yeah, more about the fun.
Gilbert Strang (47:05.740)
Yeah, the moment of getting it.
Lex Fridman (47:10.500)
But in terms of topics, linear algebra?
Gilbert Strang (47:14.140)
Well, that's my topic,
Lex Fridman (47:16.940)
but oh, there's beautiful things in geometry to understand.
Gilbert Strang (47:21.220)
What's wonderful is that in the end,
Lex Fridman (47:24.140)
there's a pattern, there are rules
Gilbert Strang (47:28.620)
that are followed in biology as there are in every field.
Lex Fridman (47:37.260)
You describe the life of a mathematician
Gilbert Strang (47:41.420)
as 100% wonderful.
Lex Fridman (47:44.260)
No.
Gilbert Strang (47:45.620)
Except for the grade stuff.
Lex Fridman (47:47.140)
Yeah.
Lex Fridman (47:47.980)
And the grades.
Lex Fridman (47:48.820)
Except for grades.
Gilbert Strang (47:49.660)
Yeah, when you look back at your life,
Lex Fridman (47:52.140)
what memories bring you the most joy and pride?
Gilbert Strang (47:55.980)
Well, that's a good question.
Lex Fridman (47:59.500)
I certainly feel good when I,
Gilbert Strang (48:01.620)
maybe I'm giving a class in 1806,
Lex Fridman (48:06.140)
that's MIT's linear algebra course that I started.
Lex Fridman (48:09.380)
So sort of, there's a good feeling that,
Lex Fridman (48:11.620)
okay, I started this course,
Gilbert Strang (48:13.740)
a lot of students take it, quite a few like it.
Lex Fridman (48:17.340)
Yeah, so I'm sort of happy
Gilbert Strang (48:21.380)
when I feel I'm helping make a connection
Lex Fridman (48:25.060)
between ideas and students,
Gilbert Strang (48:27.740)
between theory and the reader.
Lex Fridman (48:32.980)
Yeah, it's, I get a lot of very nice messages
Gilbert Strang (48:38.540)
from people who've watched the videos and it's inspiring.
Lex Fridman (48:43.460)
I just, I'll maybe take this chance to say thank you.
Gilbert Strang (48:48.060)
Well, there's millions of students
Lex Fridman (48:50.380)
who you've taught and I am grateful to be one of them.
Lex Fridman (48:54.220)
So Gilbert, thank you so much, it's been an honor.
Lex Fridman (48:56.540)
Thank you for talking today.
Gilbert Strang (48:58.140)
It was a pleasure, thanks.
Lex Fridman (49:00.700)
Thank you for listening to this conversation
Gilbert Strang (49:02.500)
with Gilbert Strang.
Lex Fridman (49:04.220)
And thank you to our presenting sponsor, Cash App.
Gilbert Strang (49:07.380)
Download it, use code LexPodcast,
Lex Fridman (49:09.940)
you'll get $10 and $10 will go to FIRST,
Gilbert Strang (49:12.780)
a STEM education nonprofit
Lex Fridman (49:14.500)
that inspires hundreds of thousands of young minds
Gilbert Strang (49:17.500)
to learn and to dream of engineering our future.
Lex Fridman (49:20.660)
If you enjoy this podcast, subscribe on YouTube.
Gilbert Strang (49:23.780)
We have five stars in Apple Podcast,
Lex Fridman (49:25.900)
support on Patreon or connect with me on Twitter.
Gilbert Strang (49:29.300)
Finally, some closing words of advice
Lex Fridman (49:31.860)
from the great Richard Feynman.
Gilbert Strang (49:33.940)
Study hard what interests you the most
Lex Fridman (49:36.300)
in the most undisciplined, irreverent
Lex Fridman (49:38.980)
and original manner possible.
Lex Fridman (49:41.220)
Thank you for listening and hope to see you next time.
🔗 相关节目