Tomaso Poggio: Brains, Minds, and Machines
生物与进化AI 与机器学习心理与人性音乐与艺术技术与编程
📋 章节目录
暂无章节信息
🔑 关键词
intelligencebraindonlearningunderstandingnetworksdeepsaidneuraltermsinterestinghumanartificialscienceneurosciencevisioncortexinstanceconsciousnesscomputer
💬 精彩语录
暂无语录
🎙️ 完整对话（1359 条）
Lex Fridman (00:00.000)
The following is a conversation with Tommaso Poggio.
Lex Fridman (00:02.920)
He's a professor at MIT and is a director of the Center
Lex Fridman (00:06.200)
for Brains, Minds, and Machines.
Lex Fridman (00:08.360)
Cited over 100,000 times, his work
Lex Fridman (00:11.640)
has had a profound impact on our understanding
Tomaso Poggio (00:14.560)
of the nature of intelligence in both biological and artificial
Lex Fridman (00:18.600)
neural networks.
Tomaso Poggio (00:19.920)
He has been an advisor to many highly impactful researchers
Lex Fridman (00:23.840)
and entrepreneurs in AI, including
Tomaso Poggio (00:26.120)
Demis Hassabis of DeepMind, Amnon Shashua of Mobileye,
Lex Fridman (00:29.880)
and Christoph Koch of the Allen Institute for Brain Science.
Tomaso Poggio (00:34.120)
This conversation is part of the MIT course
Lex Fridman (00:36.400)
on artificial general intelligence
Lex Fridman (00:38.120)
and the artificial intelligence podcast.
Lex Fridman (00:40.240)
If you enjoy it, subscribe on YouTube, iTunes,
Tomaso Poggio (00:42.760)
or simply connect with me on Twitter
Lex Fridman (00:44.640)
at Lex Friedman, spelled F R I D.
Lex Fridman (00:48.000)
And now, here's my conversation with Tommaso Poggio.
Lex Fridman (00:52.480)
You've mentioned that in your childhood,
Tomaso Poggio (00:54.520)
you've developed a fascination with physics, especially
Lex Fridman (00:57.560)
the theory of relativity.
Lex Fridman (00:59.720)
And that Einstein was also a childhood hero to you.
Lex Fridman (01:04.520)
What aspect of Einstein's genius, the nature of his genius,
Lex Fridman (01:09.000)
do you think was essential for discovering
Lex Fridman (01:11.280)
the theory of relativity?
Tomaso Poggio (01:12.960)
You know, Einstein was a hero to me,
Lex Fridman (01:15.920)
and I'm sure to many people, because he
Tomaso Poggio (01:18.680)
was able to make, of course, a major, major contribution
Lex Fridman (01:23.440)
to physics with simplifying a bit just a gedanken experiment,
Tomaso Poggio (01:31.920)
a thought experiment, you know, imagining communication
Lex Fridman (01:37.960)
with lights between a stationary observer
Lex Fridman (01:41.560)
and somebody on a train.
Lex Fridman (01:43.240)
And I thought, you know, the fact
Tomaso Poggio (01:47.960)
that just with the force of his thought, of his thinking,
Lex Fridman (01:51.320)
of his mind, he could get to something so deep
Tomaso Poggio (01:55.640)
in terms of physical reality, how time
Lex Fridman (01:58.760)
depend on space and speed, it was something
Tomaso Poggio (02:02.680)
absolutely fascinating.
Lex Fridman (02:04.120)
It was the power of intelligence,
Tomaso Poggio (02:06.720)
the power of the mind.
Lex Fridman (02:08.440)
Do you think the ability to imagine,
Tomaso Poggio (02:11.120)
to visualize as he did, as a lot of great physicists do,
Lex Fridman (02:15.200)
do you think that's in all of us human beings?
Tomaso Poggio (02:18.640)
Or is there something special to that one particular human
Lex Fridman (02:21.840)
being?
Tomaso Poggio (02:22.880)
I think, you know, all of us can learn and have, in principle,
Lex Fridman (02:30.480)
similar breakthroughs.
Tomaso Poggio (02:33.200)
There are lessons to be learned from Einstein.
Lex Fridman (02:37.200)
He was one of five PhD students at ETA,
Tomaso Poggio (02:42.720)
the Eidgenössische Technische Hochschule in Zurich,
Lex Fridman (02:47.560)
in physics.
Lex Fridman (02:48.520)
And he was the worst of the five,
Lex Fridman (02:50.800)
the only one who did not get an academic position when
Tomaso Poggio (02:55.360)
he graduated, when he finished his PhD.
Lex Fridman (02:57.960)
And he went to work, as everybody knows,
Tomaso Poggio (03:01.080)
for the patent office.
Lex Fridman (03:02.480)
And so it's not so much that he worked for the patent office,
Lex Fridman (03:05.840)
but the fact that obviously he was smart,
Lex Fridman (03:08.720)
but he was not a top student, obviously
Tomaso Poggio (03:11.760)
was the anti conformist.
Lex Fridman (03:13.560)
He was not thinking in the traditional way that probably
Tomaso Poggio (03:17.480)
his teachers and the other students were doing.
Lex Fridman (03:20.040)
So there is a lot to be said about trying
Tomaso Poggio (03:23.880)
to do the opposite or something quite different from what
Lex Fridman (03:29.800)
other people are doing.
Tomaso Poggio (03:31.080)
That's certainly true for the stock market.
Lex Fridman (03:32.960)
Never buy if everybody's buying.
Lex Fridman (03:36.800)
And also true for science.
Lex Fridman (03:38.600)
Yes.
Lex Fridman (03:39.680)
So you've also mentioned, staying
Lex Fridman (03:42.520)
on the theme of physics, that you were excited at a young age
Tomaso Poggio (03:47.600)
by the mysteries of the universe that physics could uncover.
Lex Fridman (03:51.800)
Such, as I saw mentioned, the possibility of time travel.
Lex Fridman (03:56.760)
So the most out of the box question,
Lex Fridman (03:58.960)
I think I'll get to ask today, do you
Lex Fridman (04:00.600)
think time travel is possible?
Lex Fridman (04:03.440)
Well, it would be nice if it were possible right now.
Tomaso Poggio (04:07.800)
In science, you never say no.
Lex Fridman (04:12.800)
But your understanding of the nature of time.
Tomaso Poggio (04:15.040)
Yeah.
Lex Fridman (04:15.920)
It's very likely that it's not possible to travel in time.
Tomaso Poggio (04:22.360)
We may be able to travel forward in time
Lex Fridman (04:26.000)
if we can, for instance, freeze ourselves or go
Tomaso Poggio (04:31.880)
on some spacecraft traveling close to the speed of light.
Lex Fridman (04:37.640)
But in terms of actively traveling, for instance,
Tomaso Poggio (04:40.440)
back in time, I find probably very unlikely.
Lex Fridman (04:45.320)
So do you still hold the underlying dream
Tomaso Poggio (04:49.160)
of the engineering intelligence that
Lex Fridman (04:52.320)
will build systems that are able to do such huge leaps,
Tomaso Poggio (04:56.800)
like discovering the kind of mechanism that would be
Lex Fridman (05:01.120)
required to travel through time?
Lex Fridman (05:02.600)
Do you still hold that dream or echoes of it
Lex Fridman (05:05.600)
from your childhood?
Tomaso Poggio (05:07.080)
Yeah.
Lex Fridman (05:08.640)
I don't think whether there are certain problems that probably
Tomaso Poggio (05:12.400)
cannot be solved, depending what you believe
Lex Fridman (05:16.840)
about the physical reality, like maybe totally impossible
Tomaso Poggio (05:21.440)
to create energy from nothing or to travel back in time,
Lex Fridman (05:27.720)
but about making machines that can think as well as we do
Tomaso Poggio (05:35.920)
or better, or more likely, especially
Lex Fridman (05:38.680)
in the short and midterm, help us think better,
Tomaso Poggio (05:42.400)
which is, in a sense, is happening already
Lex Fridman (05:44.280)
with the computers we have.
Lex Fridman (05:46.480)
And it will happen more and more.
Lex Fridman (05:48.440)
But that I certainly believe.
Lex Fridman (05:50.000)
And I don't see, in principle, why computers at some point
Lex Fridman (05:55.480)
could not become more intelligent than we are,
Tomaso Poggio (05:59.440)
although the word intelligence is a tricky one
Lex Fridman (06:03.640)
and one we should discuss.
Lex Fridman (06:05.840)
What I mean with that.
Lex Fridman (06:08.040)
Intelligence, consciousness, words like love,
Tomaso Poggio (06:13.200)
all these need to be disentangled.
Lex Fridman (06:16.800)
So you've mentioned also that you believe
Tomaso Poggio (06:18.760)
the problem of intelligence is the greatest problem
Lex Fridman (06:22.240)
in science, greater than the origin of life
Lex Fridman (06:24.480)
and the origin of the universe.
Lex Fridman (06:27.200)
You've also, in the talk I've listened to,
Tomaso Poggio (06:30.760)
said that you're open to arguments against you.
Lex Fridman (06:34.880)
So what do you think is the most captivating aspect
Lex Fridman (06:40.400)
of this problem of understanding the nature of intelligence?
Lex Fridman (06:43.320)
Why does it captivate you as it does?
Tomaso Poggio (06:47.440)
Well, originally, I think one of the motivation
Lex Fridman (06:51.600)
that I had as, I guess, a teenager when I was infatuated
Tomaso Poggio (06:56.440)
with theory of relativity was really
Lex Fridman (06:59.280)
that I found that there was the problem of time and space
Lex Fridman (07:05.960)
and general relativity.
Lex Fridman (07:07.960)
But there were so many other problems
Tomaso Poggio (07:10.000)
of the same level of difficulty and importance
Lex Fridman (07:13.720)
that I could, even if I were Einstein,
Tomaso Poggio (07:16.640)
it was difficult to hope to solve all of them.
Lex Fridman (07:19.600)
So what about solving a problem whose solution allowed
Lex Fridman (07:24.800)
me to solve all the problems?
Lex Fridman (07:26.640)
And this was, what if we could find the key to an intelligence
Lex Fridman (07:33.280)
10 times better or faster than Einstein?
Lex Fridman (07:37.040)
So that's sort of seeing artificial intelligence
Tomaso Poggio (07:40.160)
as a tool to expand our capabilities.
Lex Fridman (07:43.200)
But is there just an inherent curiosity in you
Tomaso Poggio (07:47.920)
in just understanding what it is in here
Lex Fridman (07:52.120)
that makes it all work?
Tomaso Poggio (07:54.360)
Yes, absolutely, you're right.
Lex Fridman (07:55.760)
So I started saying this was the motivation when
Tomaso Poggio (07:59.320)
I was a teenager.
Lex Fridman (08:00.560)
But soon after, I think the problem of human intelligence
Tomaso Poggio (08:07.160)
became a real focus of my science and my research
Lex Fridman (08:15.160)
because I think for me, the most interesting problem
Tomaso Poggio (08:22.520)
is really asking who we are.
Lex Fridman (08:28.120)
It's asking not only a question about science,
Lex Fridman (08:31.720)
but even about the very tool we are using to do science, which
Lex Fridman (08:36.000)
is our brain.
Lex Fridman (08:37.920)
How does our brain work?
Lex Fridman (08:39.880)
From where does it come from?
Lex Fridman (08:42.120)
What are its limitations?
Lex Fridman (08:43.720)
Can we make it better?
Lex Fridman (08:46.160)
And that, in many ways, is the ultimate question
Lex Fridman (08:50.000)
that underlies this whole effort of science.
Lex Fridman (08:54.400)
So you've made significant contributions
Lex Fridman (08:56.360)
in both the science of intelligence
Lex Fridman (08:58.240)
and the engineering of intelligence.
Lex Fridman (09:02.160)
In a hypothetical way, let me ask,
Lex Fridman (09:05.000)
how far do you think we can get in creating intelligence
Lex Fridman (09:07.960)
systems without understanding the biological,
Lex Fridman (09:12.080)
the understanding how the human brain creates intelligence?
Lex Fridman (09:15.800)
Put another way, do you think we can
Tomaso Poggio (09:17.200)
build a strong AI system without really getting at the core
Lex Fridman (09:22.080)
understanding the functional nature of the brain?
Tomaso Poggio (09:25.240)
Well, this is a real difficult question.
Lex Fridman (09:29.920)
We did solve problems like flying
Tomaso Poggio (09:35.280)
without really using too much our knowledge
Lex Fridman (09:40.680)
about how birds fly.
Tomaso Poggio (09:44.720)
It was important, I guess, to know that you could have
Lex Fridman (09:48.680)
things heavier than air being able to fly, like birds.
Lex Fridman (09:56.760)
But beyond that, probably we did not learn very much, some.
Lex Fridman (10:02.560)
The Brothers Wright did learn a lot of observation
Tomaso Poggio (10:06.800)
about birds and designing their aircraft.
Lex Fridman (10:12.120)
But you can argue we did not use much of biology
Tomaso Poggio (10:16.000)
in that particular case.
Lex Fridman (10:17.920)
Now, in the case of intelligence,
Tomaso Poggio (10:20.720)
I think that it's a bit of a bet right now.
Lex Fridman (10:28.920)
If you ask, OK, we all agree we'll get at some point,
Tomaso Poggio (10:36.280)
maybe soon, maybe later, to a machine that
Lex Fridman (10:39.480)
is indistinguishable from my secretary,
Tomaso Poggio (10:42.400)
say, in terms of what I can ask the machine to do.
Lex Fridman (10:47.600)
I think we'll get there.
Lex Fridman (10:49.000)
And now the question is, you can ask people,
Lex Fridman (10:51.960)
do you think we'll get there without any knowledge
Lex Fridman (10:54.240)
about the human brain?
Lex Fridman (10:57.000)
Or that the best way to get there
Lex Fridman (10:59.760)
is to understand better the human brain?
Lex Fridman (11:02.560)
OK, this is, I think, an educated bet
Tomaso Poggio (11:05.960)
that different people with different backgrounds
Lex Fridman (11:09.080)
will decide in different ways.
Tomaso Poggio (11:11.720)
The recent history of the progress
Lex Fridman (11:14.440)
in AI in the last, I would say, five years or 10 years
Tomaso Poggio (11:18.920)
has been that the main breakthroughs,
Lex Fridman (11:23.760)
the main recent breakthroughs, really start from neuroscience.
Tomaso Poggio (11:32.160)
I can mention reinforcement learning as one.
Lex Fridman (11:35.800)
It's one of the algorithms at the core of AlphaGo,
Tomaso Poggio (11:41.040)
which is the system that beat the kind of an official world
Lex Fridman (11:45.200)
champion of Go, Lee Sedol, two, three years ago in Seoul.
Tomaso Poggio (11:52.760)
That's one.
Lex Fridman (11:53.760)
And that started really with the work of Pavlov in 1900,
Tomaso Poggio (12:00.920)
Marvin Minsky in the 60s, and many other neuroscientists
Lex Fridman (12:05.800)
later on.
Lex Fridman (12:07.720)
And deep learning started, which is at the core, again,
Lex Fridman (12:12.560)
of AlphaGo and systems like autonomous driving
Tomaso Poggio (12:17.720)
systems for cars, like the systems that Mobileye,
Lex Fridman (12:22.520)
which is a company started by one of my ex postdocs,
Tomaso Poggio (12:25.600)
Amnon Shashua, did.
Lex Fridman (12:28.480)
So that is at the core of those things.
Lex Fridman (12:30.720)
And deep learning, really, the initial ideas
Lex Fridman (12:34.520)
in terms of the architecture of these layered
Tomaso Poggio (12:37.120)
hierarchical networks started with work of Torsten Wiesel
Lex Fridman (12:43.160)
and David Hubel at Harvard up the river in the 60s.
Lex Fridman (12:47.800)
So recent history suggests that neuroscience played a big role
Lex Fridman (12:53.240)
in these breakthroughs.
Tomaso Poggio (12:54.320)
My personal bet is that there is a good chance they continue
Lex Fridman (12:58.720)
to play a big role.
Tomaso Poggio (12:59.880)
Maybe not in all the future breakthroughs,
Lex Fridman (13:01.840)
but in some of them.
Tomaso Poggio (13:03.280)
At least in inspiration.
Lex Fridman (13:05.000)
At least in inspiration, absolutely, yes.
Lex Fridman (13:07.320)
So you studied both artificial and biological neural networks.
Lex Fridman (13:12.160)
You said these mechanisms that underlie deep learning
Lex Fridman (13:17.080)
and reinforcement learning.
Lex Fridman (13:19.760)
But there is nevertheless significant differences
Tomaso Poggio (13:23.880)
between biological and artificial neural networks
Lex Fridman (13:26.080)
as they stand now.
Lex Fridman (13:27.280)
So between the two, what do you find
Lex Fridman (13:30.800)
is the most interesting, mysterious, maybe even
Tomaso Poggio (13:33.080)
beautiful difference as it currently
Lex Fridman (13:35.560)
stands in our understanding?
Tomaso Poggio (13:37.800)
I must confess that until recently, I
Lex Fridman (13:41.560)
found that the artificial networks, too simplistic
Tomaso Poggio (13:46.040)
relative to real neural networks.
Lex Fridman (13:49.720)
But recently, I've been starting to think that, yes,
Tomaso Poggio (13:53.360)
there is a very big simplification of what
Lex Fridman (13:57.440)
you find in the brain.
Lex Fridman (13:59.040)
But on the other hand, they are much closer
Lex Fridman (14:03.000)
in terms of the architecture to the brain
Tomaso Poggio (14:07.080)
than other models that we had, that computer science used
Lex Fridman (14:11.280)
as model of thinking, which were mathematical logics, LISP,
Tomaso Poggio (14:16.560)
Prologue, and those kind of things.
Lex Fridman (14:19.480)
So in comparison to those, they're
Tomaso Poggio (14:21.520)
much closer to the brain.
Lex Fridman (14:23.320)
You have networks of neurons, which
Tomaso Poggio (14:26.360)
is what the brain is about.
Lex Fridman (14:27.840)
And the artificial neurons in the models, as I said,
Tomaso Poggio (14:32.200)
caricature of the biological neurons.
Lex Fridman (14:35.480)
But they're still neurons, single units communicating
Tomaso Poggio (14:38.520)
with other units, something that is absent
Lex Fridman (14:41.400)
in the traditional computer type models of mathematics,
Tomaso Poggio (14:48.520)
reasoning, and so on.
Lex Fridman (14:50.840)
So what aspect would you like to see
Tomaso Poggio (14:53.120)
in artificial neural networks added over time
Lex Fridman (14:57.280)
as we try to figure out ways to improve them?
Lex Fridman (14:59.920)
So one of the main differences and problems
Lex Fridman (15:07.320)
in terms of deep learning today, and it's not only
Tomaso Poggio (15:11.840)
deep learning, and the brain, is the need for deep learning
Lex Fridman (15:16.760)
techniques to have a lot of labeled examples.
Tomaso Poggio (15:23.160)
For instance, for ImageNet, you have
Lex Fridman (15:24.920)
like a training set, which is 1 million images, each one
Tomaso Poggio (15:29.160)
labeled by some human in terms of which object is there.
Lex Fridman (15:34.600)
And it's clear that in biology, a baby
Tomaso Poggio (15:42.720)
may be able to see millions of images
Lex Fridman (15:44.840)
in the first years of life, but will not
Tomaso Poggio (15:47.560)
have millions of labels given to him or her by parents
Lex Fridman (15:52.720)
or caretakers.
Lex Fridman (15:56.360)
So how do you solve that?
Lex Fridman (15:59.560)
I think there is this interesting challenge
Tomaso Poggio (16:03.880)
that today, deep learning and related techniques
Lex Fridman (16:08.080)
are all about big data, big data meaning
Tomaso Poggio (16:11.200)
a lot of examples labeled by humans,
Lex Fridman (16:18.760)
whereas in nature, you have this big data
Tomaso Poggio (16:24.400)
is n going to infinity.
Lex Fridman (16:26.280)
That's the best, n meaning labeled data.
Lex Fridman (16:30.200)
But I think the biological world is more n going to 1.
Lex Fridman (16:34.920)
A child can learn from a very small number
Tomaso Poggio (16:38.920)
of labeled examples.
Lex Fridman (16:42.720)
Like you tell a child, this is a car.
Tomaso Poggio (16:44.920)
You don't need to say, like in ImageNet, this is a car,
Lex Fridman (16:48.880)
this is a car, this is not a car, this is not a car,
Tomaso Poggio (16:51.120)
1 million times.
Lex Fridman (16:54.360)
And of course, with AlphaGo, or at least the AlphaZero
Tomaso Poggio (16:57.720)
variants, because the world of Go
Lex Fridman (17:01.360)
is so simplistic that you can actually
Tomaso Poggio (17:05.040)
learn by yourself through self play,
Lex Fridman (17:06.720)
you can play against each other.
Tomaso Poggio (17:08.520)
In the real world, the visual system
Lex Fridman (17:10.600)
that you've studied extensively is a lot more complicated
Tomaso Poggio (17:14.200)
than the game of Go.
Lex Fridman (17:16.720)
On the comment about children, which
Tomaso Poggio (17:18.400)
are fascinatingly good at learning new stuff,
Lex Fridman (17:23.000)
how much of it do you think is hardware,
Lex Fridman (17:24.680)
and how much of it is software?
Lex Fridman (17:26.640)
Yeah, that's a good, deep question.
Tomaso Poggio (17:29.800)
In a sense, it's the old question of nurture and nature,
Lex Fridman (17:32.960)
how much is in the gene, and how much
Tomaso Poggio (17:36.440)
is in the experience of an individual.
Lex Fridman (17:41.280)
Obviously, it's both that play a role.
Lex Fridman (17:44.720)
And I believe that the way evolution gives,
Lex Fridman (17:53.040)
puts prior information, so to speak, hardwired,
Tomaso Poggio (17:55.760)
is not really hardwired.
Lex Fridman (17:58.400)
But that's essentially an hypothesis.
Tomaso Poggio (18:02.720)
I think what's going on is that evolution has almost
Lex Fridman (18:10.000)
necessarily, if you believe in Darwin, is very opportunistic.
Lex Fridman (18:14.960)
And think about our DNA and the DNA of Drosophila.
Lex Fridman (18:24.320)
Our DNA does not have many more genes than Drosophila.
Tomaso Poggio (18:28.800)
The fly.
Lex Fridman (18:29.720)
The fly, the fruit fly.
Tomaso Poggio (18:32.560)
Now, we know that the fruit fly does not
Lex Fridman (18:35.520)
learn very much during its individual existence.
Tomaso Poggio (18:39.680)
It looks like one of these machinery
Lex Fridman (18:42.360)
that it's really mostly, not 100%, but 95%,
Tomaso Poggio (18:47.240)
hardcoded by the genes.
Lex Fridman (18:51.720)
But since we don't have many more genes than Drosophila,
Tomaso Poggio (18:55.040)
evolution could encode in as a general learning machinery,
Lex Fridman (19:02.640)
and then had to give very weak priors.
Tomaso Poggio (19:09.840)
Like, for instance, let me give a specific example,
Lex Fridman (19:15.000)
which is recent work by a member of our Center for Brains,
Tomaso Poggio (19:18.160)
Minds, and Machines.
Lex Fridman (19:20.680)
We know because of work of other people in our group
Lex Fridman (19:24.440)
and other groups, that there are cells
Lex Fridman (19:26.720)
in a part of our brain, neurons, that are tuned to faces.
Tomaso Poggio (19:31.160)
They seem to be involved in face recognition.
Lex Fridman (19:33.840)
Now, this face area seems to be present in young children
Lex Fridman (19:41.600)
and adults.
Lex Fridman (19:44.600)
And one question is, is there from the beginning?
Lex Fridman (19:48.400)
Is hardwired by evolution?
Lex Fridman (19:51.760)
Or somehow it's learned very quickly.
Lex Fridman (19:55.000)
So what's your, by the way, a lot of the questions I'm asking,
Lex Fridman (19:58.960)
the answer is we don't really know.
Tomaso Poggio (1:00:04.120)
or linear classifier, we really don't understand
Lex Fridman (1:00:08.120)
the individual units or so.
Lex Fridman (1:00:11.520)
But we understand what the computation and the limitations
Lex Fridman (1:00:17.280)
and the properties of it are.
Tomaso Poggio (1:00:20.440)
It's similar to many things.
Lex Fridman (1:00:24.040)
What does it mean to understand how a fusion bomb works?
Lex Fridman (1:00:29.600)
How many of us understand the basic principle?
Lex Fridman (1:00:36.360)
And some of us may understand deeper details.
Tomaso Poggio (1:00:40.600)
In that sense, understanding is, as a community,
Lex Fridman (1:00:43.440)
as a civilization, can we build another copy of it?
Lex Fridman (1:00:47.360)
And in that sense, do you think there
Lex Fridman (1:00:50.880)
will need to be some evolutionary component where
Lex Fridman (1:00:53.960)
it runs away from our understanding?
Lex Fridman (1:00:56.200)
Or do you think it could be engineered from the ground up,
Lex Fridman (1:00:59.440)
the same way you go from the transistor to PowerPoint?
Lex Fridman (1:01:02.640)
So many years ago, this was actually 40, 41 years ago,
Tomaso Poggio (1:01:09.160)
I wrote a paper with David Marr, who
Lex Fridman (1:01:13.560)
was one of the founding fathers of computer vision,
Tomaso Poggio (1:01:18.000)
computational vision.
Lex Fridman (1:01:20.440)
I wrote a paper about levels of understanding,
Tomaso Poggio (1:01:23.840)
which is related to the question we discussed earlier
Lex Fridman (1:01:26.160)
about understanding PowerPoint, understanding transistors,
Lex Fridman (1:01:30.200)
and so on.
Lex Fridman (1:01:31.840)
And in that kind of framework, we
Tomaso Poggio (1:01:36.560)
had the level of the hardware and the top level
Lex Fridman (1:01:39.760)
of the algorithms.
Tomaso Poggio (1:01:42.240)
We did not have learning.
Lex Fridman (1:01:45.040)
Recently, I updated adding levels.
Lex Fridman (1:01:48.280)
And one level I added to those three was learning.
Lex Fridman (1:01:55.160)
And you can imagine, you could have a good understanding
Tomaso Poggio (1:01:59.520)
of how you construct a learning machine, like we do.
Lex Fridman (1:02:04.960)
But being unable to describe in detail what the learning
Lex Fridman (1:02:09.720)
machines will discover, right?
Lex Fridman (1:02:13.680)
Now, that would be still a powerful understanding,
Tomaso Poggio (1:02:17.120)
if I can build a learning machine,
Lex Fridman (1:02:19.400)
even if I don't understand in detail every time it
Tomaso Poggio (1:02:24.480)
learns something.
Lex Fridman (1:02:26.160)
Just like our children, if they start
Tomaso Poggio (1:02:28.440)
listening to a certain type of music,
Lex Fridman (1:02:31.320)
I don't know, Miley Cyrus or something,
Tomaso Poggio (1:02:33.680)
you don't understand why they came
Lex Fridman (1:02:36.240)
to that particular preference.
Lex Fridman (1:02:37.640)
But you understand the learning process.
Lex Fridman (1:02:39.400)
That's very interesting.
Lex Fridman (1:02:41.440)
So on learning for systems to be part of our world,
Lex Fridman (1:02:50.360)
it has a certain, one of the challenging things
Tomaso Poggio (1:02:53.480)
that you've spoken about is learning ethics, learning
Lex Fridman (1:02:57.920)
morals.
Lex Fridman (1:02:59.400)
And how hard do you think is the problem of, first of all,
Lex Fridman (1:03:04.560)
humans understanding our ethics?
Lex Fridman (1:03:06.800)
What is the origin on the neural on the low level of ethics?
Lex Fridman (1:03:10.600)
What is it at the higher level?
Tomaso Poggio (1:03:12.400)
Is it something that's learnable from machines
Lex Fridman (1:03:15.160)
in your intuition?
Tomaso Poggio (1:03:17.840)
I think, yeah, ethics is learnable, very likely.
Lex Fridman (1:03:23.960)
I think it's one of these problems where
Tomaso Poggio (1:03:29.720)
I think understanding the neuroscience of ethics,
Lex Fridman (1:03:36.680)
people discuss there is an ethics of neuroscience.
Tomaso Poggio (1:03:41.480)
Yeah, yes.
Lex Fridman (1:03:42.800)
How a neuroscientist should or should not behave.
Lex Fridman (1:03:46.560)
Can you think of a neurosurgeon and the ethics
Lex Fridman (1:03:50.480)
rule he has to be or he, she has to be.
Lex Fridman (1:03:53.960)
But I'm more interested on the neuroscience of ethics.
Lex Fridman (1:03:57.560)
You're blowing my mind right now.
Tomaso Poggio (1:03:58.840)
The neuroscience of ethics is very meta.
Lex Fridman (1:04:01.080)
Yeah, and I think that would be important to understand also
Tomaso Poggio (1:04:05.080)
for being able to design machines that
Lex Fridman (1:04:10.880)
are ethical machines in our sense of ethics.
Lex Fridman (1:04:15.160)
And you think there is something in neuroscience,
Lex Fridman (1:04:18.520)
there's patterns, tools in neuroscience
Lex Fridman (1:04:21.520)
that could help us shed some light on ethics?
Lex Fridman (1:04:25.320)
Or is it mostly on the psychologists of sociology
Lex Fridman (1:04:28.920)
in which higher level?
Lex Fridman (1:04:29.840)
No, there is psychology.
Lex Fridman (1:04:30.960)
But there is also, in the meantime,
Lex Fridman (1:04:35.160)
there is evidence, fMRI, of specific areas of the brain
Tomaso Poggio (1:04:41.080)
that are involved in certain ethical judgment.
Lex Fridman (1:04:44.520)
And not only this, you can stimulate those area
Tomaso Poggio (1:04:47.640)
with magnetic fields and change the ethical decisions.
Lex Fridman (1:04:53.920)
Yeah, wow.
Lex Fridman (1:04:56.360)
So that's work by a colleague of mine, Rebecca Sachs.
Lex Fridman (1:05:00.800)
And there is other researchers doing similar work.
Lex Fridman (1:05:05.320)
And I think this is the beginning.
Lex Fridman (1:05:08.280)
But ideally, at some point, we'll
Tomaso Poggio (1:05:11.680)
have an understanding of how this works.
Lex Fridman (1:05:15.560)
And why it evolved, right?
Tomaso Poggio (1:05:18.520)
The big why question.
Lex Fridman (1:05:19.720)
Yeah, it must have some purpose.
Tomaso Poggio (1:05:22.000)
Yeah, obviously it has some social purposes, probably.
Lex Fridman (1:05:30.120)
If neuroscience holds the key to at least illuminate
Tomaso Poggio (1:05:33.600)
some aspect of ethics, that means
Lex Fridman (1:05:35.240)
it could be a learnable problem.
Tomaso Poggio (1:05:37.120)
Yeah, exactly.
Lex Fridman (1:05:38.880)
And as we're getting into harder and harder questions,
Tomaso Poggio (1:05:42.040)
let's go to the hard problem of consciousness.
Lex Fridman (1:05:45.440)
Is this an important problem for us
Tomaso Poggio (1:05:48.080)
to think about and solve on the engineering of intelligence
Lex Fridman (1:05:52.240)
side of your work, of our dream?
Tomaso Poggio (1:05:56.240)
It's unclear.
Lex Fridman (1:05:57.440)
So again, this is a deep problem,
Tomaso Poggio (1:06:02.680)
partly because it's very difficult to define
Lex Fridman (1:06:05.720)
consciousness.
Lex Fridman (1:06:06.760)
And there is a debate among neuroscientists
Lex Fridman (1:06:17.800)
about whether consciousness and philosophers, of course,
Tomaso Poggio (1:06:23.040)
whether consciousness is something that requires
Lex Fridman (1:06:28.280)
flesh and blood, so to speak.
Tomaso Poggio (1:06:31.360)
Or could be that we could have silicon devices that
Lex Fridman (1:06:38.680)
are conscious, or up to statement
Tomaso Poggio (1:06:42.840)
like everything has some degree of consciousness
Lex Fridman (1:06:45.800)
and some more than others.
Tomaso Poggio (1:06:48.480)
This is like Giulio Tonioni and phi.
Lex Fridman (1:06:53.960)
We just recently talked to Christoph Koch.
Tomaso Poggio (1:06:56.280)
OK.
Lex Fridman (1:06:57.600)
Christoph was my first graduate student.
Lex Fridman (1:07:00.680)
Do you think it's important to illuminate
Lex Fridman (1:07:04.480)
aspects of consciousness in order
Lex Fridman (1:07:07.480)
to engineer intelligence systems?
Lex Fridman (1:07:10.320)
Do you think an intelligent system would ultimately
Lex Fridman (1:07:13.080)
have consciousness?
Lex Fridman (1:07:14.480)
Are they interlinked?
Tomaso Poggio (1:07:18.800)
Most of the people working in artificial intelligence,
Lex Fridman (1:07:22.800)
I think, would answer, we don't strictly
Tomaso Poggio (1:07:25.800)
need consciousness to have an intelligent system.
Lex Fridman (1:07:30.040)
That's sort of the easier question,
Tomaso Poggio (1:07:31.800)
because it's a very engineering answer to the question.
Lex Fridman (1:07:36.000)
Pass the Turing test, we don't need consciousness.
Lex Fridman (1:07:38.120)
But if you were to go, do you think
Lex Fridman (1:07:41.360)
it's possible that we need to have
Lex Fridman (1:07:46.200)
that kind of self awareness?
Lex Fridman (1:07:48.280)
We may, yes.
Lex Fridman (1:07:49.920)
So for instance, I personally think
Lex Fridman (1:07:53.800)
that when test a machine or a person in a Turing test,
Tomaso Poggio (1:08:00.440)
in an extended Turing test, I think
Lex Fridman (1:08:05.200)
consciousness is part of what we require in that test,
Tomaso Poggio (1:08:11.520)
implicitly, to say that this is intelligent.
Lex Fridman (1:08:15.000)
Christoph disagrees.
Tomaso Poggio (1:08:17.440)
Yes, he does.
Lex Fridman (1:08:20.240)
Despite many other romantic notions he holds,
Tomaso Poggio (1:08:23.440)
he disagrees with that one.
Lex Fridman (1:08:24.800)
Yes, that's right.
Lex Fridman (1:08:26.520)
So we'll see.
Lex Fridman (1:08:29.880)
Do you think, as a quick question,
Tomaso Poggio (1:08:34.640)
Ernest Becker's fear of death, do you
Lex Fridman (1:08:38.520)
think mortality and those kinds of things
Lex Fridman (1:08:41.920)
are important for consciousness and for intelligence?
Lex Fridman (1:08:49.200)
The finiteness of life, finiteness of existence,
Tomaso Poggio (1:08:54.040)
or is that just a side effect of evolution,
Lex Fridman (1:08:56.600)
evolutionary side effect that's useful for natural selection?
Lex Fridman (1:09:01.120)
Do you think this kind of thing that this interview is
Lex Fridman (1:09:03.840)
going to run out of time soon, our life
Tomaso Poggio (1:09:06.160)
will run out of time soon, do you
Lex Fridman (1:09:08.200)
think that's needed to make this conversation good and life
Lex Fridman (1:09:11.720)
good?
Lex Fridman (1:09:12.240)
I never thought about it.
Tomaso Poggio (1:09:13.480)
It's a very interesting question.
Lex Fridman (1:09:15.920)
I think Steve Jobs, in his commencement speech
Tomaso Poggio (1:09:21.200)
at Stanford, argued that having a finite life
Lex Fridman (1:09:26.840)
was important for stimulating achievements.
Lex Fridman (1:09:30.280)
So it was different.
Lex Fridman (1:09:31.640)
Yeah, live every day like it's your last, right?
Tomaso Poggio (1:09:33.680)
Yeah.
Lex Fridman (1:09:34.840)
So rationally, I don't think strictly you need mortality
Tomaso Poggio (1:09:41.840)
for consciousness.
Lex Fridman (1:09:43.200)
But who knows?
Lex Fridman (1:09:45.960)
They seem to go together in our biological system, right?
Lex Fridman (1:09:48.760)
Yeah, yeah.
Tomaso Poggio (1:09:51.320)
You've mentioned before, and students are associated with,
Lex Fridman (1:09:57.880)
AlphaGo immobilized the big recent success stories in AI.
Lex Fridman (1:10:01.280)
And I think it's captivated the entire world of what AI can do.
Lex Fridman (1:10:06.040)
So what do you think will be the next breakthrough?
Lex Fridman (1:10:10.360)
And what's your intuition about the next breakthrough?
Lex Fridman (1:10:13.680)
Of course, I don't know where the next breakthrough is.
Tomaso Poggio (1:10:16.760)
I think that there is a good chance, as I said before,
Lex Fridman (1:10:21.440)
that the next breakthrough will also
Tomaso Poggio (1:10:23.200)
be inspired by neuroscience.
Lex Fridman (1:10:27.920)
But which one, I don't know.
Lex Fridman (1:10:32.320)
And there's, so MIT has this quest for intelligence.
Lex Fridman (1:10:35.880)
And there's a few moon shots, which in that spirit,
Lex Fridman (1:10:39.240)
which ones are you excited about?
Lex Fridman (1:10:41.800)
Which projects kind of?
Tomaso Poggio (1:10:44.080)
Well, of course, I'm excited about one
Lex Fridman (1:10:47.400)
of the moon shots, which is our Center for Brains, Minds,
Lex Fridman (1:10:51.040)
and Machines, which is the one which is fully funded by NSF.
Lex Fridman (1:10:58.560)
And it is about visual intelligence.
Lex Fridman (1:11:02.760)
And that one is particularly about understanding.
Lex Fridman (1:11:06.240)
Visual intelligence, so the visual cortex,
Lex Fridman (1:11:09.240)
and visual intelligence in the sense
Lex Fridman (1:11:13.400)
of how we look around ourselves and understand
Tomaso Poggio (1:11:20.000)
the world around ourselves, meaning what is going on,
Lex Fridman (1:11:25.440)
how we could go from here to there without hitting
Tomaso Poggio (1:11:29.880)
obstacles, whether there are other agents,
Lex Fridman (1:11:34.360)
people in the environment.
Tomaso Poggio (1:11:36.720)
These are all things that we perceive very quickly.
Lex Fridman (1:11:41.160)
And it's something actually quite close to being conscious,
Tomaso Poggio (1:11:46.920)
not quite.
Lex Fridman (1:11:47.640)
But there is this interesting experiment
Tomaso Poggio (1:11:50.360)
that was run at Google X, which is in a sense
Lex Fridman (1:11:54.800)
is just a virtual reality experiment,
Lex Fridman (1:11:58.840)
but in which they had a subject sitting, say,
Lex Fridman (1:12:02.760)
in a chair with goggles, like Oculus and so on, earphones.
Lex Fridman (1:12:11.800)
And they were seeing through the eyes of a robot
Lex Fridman (1:12:15.040)
nearby to cameras, microphones for receiving.
Lex Fridman (1:12:19.920)
So their sensory system was there.
Lex Fridman (1:12:23.840)
And the impression of all the subject, very strong,
Tomaso Poggio (1:12:28.120)
they could not shake it off, was that they
Lex Fridman (1:12:31.520)
were where the robot was.
Tomaso Poggio (1:12:35.240)
They could look at themselves from the robot
Lex Fridman (1:12:38.640)
and still feel they were where the robot is.
Tomaso Poggio (1:12:42.880)
They were looking at their body.
Lex Fridman (1:12:46.000)
Theirself had moved.
Lex Fridman (1:12:48.480)
So some aspect of scene understanding
Lex Fridman (1:12:50.440)
has to have ability to place yourself,
Tomaso Poggio (1:12:54.880)
have a self awareness about your position in the world
Lex Fridman (1:12:57.680)
and what the world is.
Lex Fridman (1:12:59.600)
So we may have to solve the hard problem of consciousness
Lex Fridman (1:13:04.080)
to solve it.
Tomaso Poggio (1:13:04.840)
On their way, yes.
Lex Fridman (1:13:05.920)
It's quite a moonshine.
Lex Fridman (1:13:07.760)
So you've been an advisor to some incredible minds,
Lex Fridman (1:13:12.440)
including Demis Hassabis, Krzysztof Koch, Amna Shashua,
Tomaso Poggio (1:13:15.680)
like you said.
Lex Fridman (1:13:17.360)
All went on to become seminal figures
Tomaso Poggio (1:13:20.120)
in their respective fields.
Lex Fridman (1:13:22.000)
From your own success as a researcher
Lex Fridman (1:13:24.240)
and from perspective as a mentor of these researchers,
Lex Fridman (1:13:29.320)
having guided them in the way of advice,
Lex Fridman (1:13:34.160)
what does it take to be successful in science
Lex Fridman (1:13:36.360)
and engineering careers?
Tomaso Poggio (1:13:39.800)
Whether you're talking to somebody in their teens,
Lex Fridman (1:13:43.280)
20s, and 30s, what does that path look like?
Tomaso Poggio (1:13:48.160)
It's curiosity and having fun.
Lex Fridman (1:13:53.200)
And I think it's important also having
Tomaso Poggio (1:13:57.400)
fun with other curious minds.
Lex Fridman (1:14:02.440)
It's the people you surround with too,
Lex Fridman (1:14:04.520)
so fun and curiosity.
Lex Fridman (1:14:06.640)
Is there, you mentioned Steve Jobs,
Tomaso Poggio (1:14:09.960)
is there also an underlying ambition
Lex Fridman (1:14:13.160)
that's unique that you saw?
Tomaso Poggio (1:14:14.720)
Or does it really does boil down
Lex Fridman (1:14:16.440)
to insatiable curiosity and fun?
Tomaso Poggio (1:14:18.800)
Well of course, it's being curious
Lex Fridman (1:14:22.240)
in an active and ambitious way, yes.
Tomaso Poggio (1:14:26.080)
Definitely.
Lex Fridman (1:14:29.640)
But I think sometime in science,
Tomaso Poggio (1:14:33.840)
there are friends of mine who are like this.
Lex Fridman (1:14:39.000)
There are some of the scientists
Tomaso Poggio (1:14:40.680)
like to work by themselves
Lex Fridman (1:14:44.080)
and kind of communicate only when they complete their work
Tomaso Poggio (1:14:50.920)
or discover something.
Lex Fridman (1:14:52.840)
I think I always found the actual process
Tomaso Poggio (1:14:58.720)
of discovering something is more fun
Lex Fridman (1:15:03.720)
if it's together with other intelligent
Lex Fridman (1:15:07.280)
and curious and fun people.
Lex Fridman (1:15:09.240)
So if you see the fun in that process,
Tomaso Poggio (1:15:11.320)
the side effect of that process
Lex Fridman (1:15:13.200)
will be that you'll actually end up
Tomaso Poggio (1:15:14.360)
discovering some interesting things.
Lex Fridman (1:15:16.320)
So as you've led many incredible efforts here,
Tomaso Poggio (1:15:23.320)
what's the secret to being a good advisor,
Lex Fridman (1:15:25.520)
mentor, leader in a research setting?
Lex Fridman (1:15:28.360)
Is it a similar spirit?
Lex Fridman (1:15:30.240)
Or yeah, what advice could you give
Lex Fridman (1:15:32.600)
to people, young faculty and so on?
Lex Fridman (1:15:35.960)
It's partly repeating what I said
Tomaso Poggio (1:15:38.320)
about an environment that should be friendly
Lex Fridman (1:15:41.280)
and fun and ambitious.
Lex Fridman (1:15:44.440)
And I think I learned a lot
Lex Fridman (1:15:49.280)
from some of my advisors and friends
Lex Fridman (1:15:52.880)
and some who are physicists.
Lex Fridman (1:15:55.280)
And there was, for instance,
Tomaso Poggio (1:15:57.480)
this behavior that was encouraged
Lex Fridman (1:16:02.800)
of when somebody comes with a new idea in the group,
Tomaso Poggio (1:16:06.720)
you are, unless it's really stupid,
Lex Fridman (1:16:09.080)
but you are always enthusiastic.
Lex Fridman (1:16:11.880)
And then, and you're enthusiastic for a few minutes,
Lex Fridman (1:16:14.280)
for a few hours.
Tomaso Poggio (1:16:15.120)
Then you start asking critically a few questions,
Lex Fridman (1:16:21.400)
testing this.
Lex Fridman (1:16:23.040)
But this is a process that is,
Lex Fridman (1:16:26.280)
I think it's very good.
Tomaso Poggio (1:16:29.360)
You have to be enthusiastic.
Lex Fridman (1:16:30.480)
Sometimes people are very critical from the beginning.
Tomaso Poggio (1:16:33.680)
That's not...
Lex Fridman (1:16:36.280)
Yes, you have to give it a chance
Tomaso Poggio (1:16:37.600)
for that seed to grow.
Lex Fridman (1:16:39.400)
That said, with some of your ideas,
Tomaso Poggio (1:16:41.600)
which are quite revolutionary,
Lex Fridman (1:16:42.800)
so there's a witness, especially in the human vision side
Lex Fridman (1:16:45.840)
and neuroscience side,
Lex Fridman (1:16:47.320)
there could be some pretty heated arguments.
Lex Fridman (1:16:50.000)
Do you enjoy these?
Lex Fridman (1:16:51.160)
Is that a part of science and academic pursuits
Lex Fridman (1:16:54.520)
that you enjoy?
Lex Fridman (1:16:55.360)
Yeah.
Lex Fridman (1:16:56.200)
Is that something that happens in your group as well?
Lex Fridman (1:17:01.040)
Yeah, absolutely.
Tomaso Poggio (1:17:02.440)
I also spent some time in Germany.
Lex Fridman (1:17:04.360)
Again, there is this tradition
Tomaso Poggio (1:17:05.880)
in which people are more forthright,
Lex Fridman (1:17:10.880)
less kind than here.
Lex Fridman (1:17:14.160)
So in the U.S., when you write a bad letter,
Lex Fridman (1:17:20.120)
you still say, this guy's nice.
Tomaso Poggio (1:17:23.080)
Yes, yes.
Lex Fridman (1:17:25.600)
So...
Tomaso Poggio (1:17:26.440)
Yeah, here in America, it's degrees of nice.
Lex Fridman (1:17:28.840)
Yes.
Tomaso Poggio (1:17:29.680)
It's all just degrees of nice, yeah.
Lex Fridman (1:17:31.040)
Right, right.
Lex Fridman (1:17:31.880)
So as long as this does not become personal,
Lex Fridman (1:17:36.400)
and it's really like a football game
Tomaso Poggio (1:17:40.680)
with these rules, that's great.
Lex Fridman (1:17:43.520)
That's fun.
Lex Fridman (1:17:46.600)
So if you somehow found yourself in a position
Lex Fridman (1:17:49.280)
to ask one question of an oracle,
Tomaso Poggio (1:17:51.840)
like a genie, maybe a god,
Lex Fridman (1:17:55.520)
and you're guaranteed to get a clear answer,
Lex Fridman (1:17:58.760)
what kind of question would you ask?
Lex Fridman (1:18:01.320)
What would be the question you would ask?
Tomaso Poggio (1:18:04.520)
In the spirit of our discussion,
Lex Fridman (1:18:06.040)
it could be, how could I become 10 times more intelligent?
Lex Fridman (1:18:10.080)
And so, but see, you only get a clear short answer.
Lex Fridman (1:18:16.240)
So do you think there's a clear short answer to that?
Tomaso Poggio (1:18:18.720)
No.
Lex Fridman (1:18:20.720)
And that's the answer you'll get.
Tomaso Poggio (1:18:22.760)
Okay, so you've mentioned Flowers of Algernon.
Lex Fridman (1:18:26.920)
Oh, yeah.
Tomaso Poggio (1:18:27.960)
As a story that inspires you in your childhood,
Lex Fridman (1:18:32.800)
as this story of a mouse,
Tomaso Poggio (1:18:37.200)
human achieving genius level intelligence,
Lex Fridman (1:18:39.360)
and then understanding what was happening
Tomaso Poggio (1:18:41.520)
while slowly becoming not intelligent again,
Lex Fridman (1:18:44.200)
and this tragedy of gaining intelligence
Lex Fridman (1:18:46.600)
and losing intelligence,
Lex Fridman (1:18:48.600)
do you think in that spirit, in that story,
Lex Fridman (1:18:51.440)
do you think intelligence is a gift or a curse
Lex Fridman (1:18:55.360)
from the perspective of happiness and meaning of life?
Tomaso Poggio (1:19:00.160)
You try to create an intelligent system
Lex Fridman (1:19:02.200)
that understands the universe,
Lex Fridman (1:19:03.880)
but on an individual level, the meaning of life,
Lex Fridman (1:19:06.480)
do you think intelligence is a gift?
Tomaso Poggio (1:19:10.840)
It's a good question.
Lex Fridman (1:19:17.120)
I don't know.
Tomaso Poggio (1:19:22.840)
As one of the, as one people consider
Lex Fridman (1:19:26.520)
the smartest people in the world,
Lex Fridman (1:19:29.280)
in some dimension, at the very least, what do you think?
Lex Fridman (1:19:33.320)
I don't know, it may be invariant to intelligence,
Tomaso Poggio (1:19:37.560)
that degree of happiness.
Lex Fridman (1:19:39.640)
It would be nice if it were.
Tomaso Poggio (1:19:43.680)
That's the hope.
Lex Fridman (1:19:44.680)
Yeah.
Tomaso Poggio (1:19:46.120)
You could be smart and happy and clueless and happy.
Lex Fridman (1:19:50.160)
Yeah.
Tomaso Poggio (1:19:51.800)
As always, on the discussion of the meaning of life,
Lex Fridman (1:19:54.480)
it's probably a good place to end.
Tomaso Poggio (1:19:57.320)
Tommaso, thank you so much for talking today.
Lex Fridman (1:19:59.240)
Thank you, this was great.
Lex Fridman (20:00.920)
But as a person who has contributed
Lex Fridman (20:04.520)
some profound ideas in these fields,
Tomaso Poggio (20:06.440)
you're a good person to guess at some of these.
Lex Fridman (20:08.360)
So of course, there's a caveat before a lot of the stuff
Tomaso Poggio (20:11.200)
we talk about.
Lex Fridman (20:11.760)
But what is your hunch?
Tomaso Poggio (20:14.680)
Is the face, the part of the brain
Lex Fridman (20:16.400)
that seems to be concentrated on face recognition,
Lex Fridman (20:20.120)
are you born with that?
Lex Fridman (20:21.240)
Or you just is designed to learn that quickly,
Lex Fridman (20:25.160)
like the face of the mother and so on?
Lex Fridman (20:26.920)
My hunch, my bias was the second one, learned very quickly.
Lex Fridman (20:32.280)
And it turns out that Marge Livingstone at Harvard
Lex Fridman (20:37.240)
has done some amazing experiments in which she raised
Tomaso Poggio (20:41.480)
baby monkeys, depriving them of faces
Lex Fridman (20:45.200)
during the first weeks of life.
Lex Fridman (20:48.560)
So they see technicians, but the technician have a mask.
Lex Fridman (20:53.000)
Yes.
Lex Fridman (20:55.080)
And so when they looked at the area
Lex Fridman (21:02.000)
in the brain of these monkeys that were usually
Tomaso Poggio (21:05.720)
defined faces, they found no face preference.
Lex Fridman (21:10.840)
So my guess is that what evolution does in this case
Tomaso Poggio (21:16.800)
is there is a plastic area, which
Lex Fridman (21:19.760)
is plastic, which is kind of predetermined
Tomaso Poggio (21:22.760)
to be imprinted very easily.
Lex Fridman (21:26.520)
But the command from the gene is not a detailed circuitry
Tomaso Poggio (21:30.160)
for a face template.
Lex Fridman (21:32.280)
Could be, but this will require probably a lot of bits.
Tomaso Poggio (21:36.280)
You had to specify a lot of connection of a lot of neurons.
Lex Fridman (21:39.720)
Instead, the command from the gene
Tomaso Poggio (21:42.320)
is something like imprint, memorize what you see most
Lex Fridman (21:47.400)
often in the first two weeks of life,
Tomaso Poggio (21:49.480)
especially in connection with food and maybe nipples.
Lex Fridman (21:53.440)
I don't know.
Tomaso Poggio (21:54.640)
Well, source of food.
Lex Fridman (21:55.960)
And so that area is very plastic at first and then solidifies.
Tomaso Poggio (22:00.320)
It'd be interesting if a variant of that experiment
Lex Fridman (22:03.600)
would show a different kind of pattern associated
Tomaso Poggio (22:06.800)
with food than a face pattern, whether that could stick.
Lex Fridman (22:10.200)
There are indications that during that experiment,
Lex Fridman (22:14.960)
what the monkeys saw quite often were
Lex Fridman (22:19.560)
the blue gloves of the technicians that were giving
Tomaso Poggio (22:23.200)
to the baby monkeys the milk.
Lex Fridman (22:25.560)
And some of the cells, instead of being face sensitive
Tomaso Poggio (22:29.400)
in that area, are hand sensitive.
Lex Fridman (22:33.680)
That's fascinating.
Lex Fridman (22:35.960)
Can you talk about what are the different parts of the brain
Lex Fridman (22:40.600)
and, in your view, sort of loosely,
Lex Fridman (22:43.920)
and how do they contribute to intelligence?
Lex Fridman (22:45.760)
Do you see the brain as a bunch of different modules,
Lex Fridman (22:49.520)
and they together come in the human brain
Lex Fridman (22:52.440)
to create intelligence?
Tomaso Poggio (22:53.800)
Or is it all one mush of the same kind
Lex Fridman (22:59.320)
of fundamental architecture?
Tomaso Poggio (23:04.600)
Yeah, that's an important question.
Lex Fridman (23:08.840)
And there was a phase in neuroscience back in the 1950
Tomaso Poggio (23:15.200)
or so in which it was believed for a while
Lex Fridman (23:19.360)
that the brain was equipotential.
Tomaso Poggio (23:21.920)
This was the term.
Lex Fridman (23:22.960)
You could cut out a piece, and nothing special
Tomaso Poggio (23:28.000)
happened apart a little bit less performance.
Lex Fridman (23:32.360)
There was a surgeon, Lashley, who
Tomaso Poggio (23:37.120)
did a lot of experiments of this type with mice and rats
Lex Fridman (23:41.800)
and concluded that every part of the brain
Tomaso Poggio (23:45.640)
was essentially equivalent to any other one.
Lex Fridman (23:51.360)
It turns out that that's really not true.
Tomaso Poggio (23:56.080)
There are very specific modules in the brain, as you said.
Lex Fridman (24:00.480)
And people may lose the ability to speak
Tomaso Poggio (24:05.280)
if you have a stroke in a certain region,
Lex Fridman (24:07.520)
or may lose control of their legs in another region.
Lex Fridman (24:12.840)
So they're very specific.
Lex Fridman (24:14.520)
The brain is also quite flexible and redundant,
Lex Fridman (24:17.920)
so often it can correct things and take over functions
Lex Fridman (24:27.360)
from one part of the brain to the other.
Lex Fridman (24:29.840)
But really, there are specific modules.
Lex Fridman (24:33.760)
So the answer that we know from this old work, which
Tomaso Poggio (24:40.000)
was basically based on lesions, either on animals,
Lex Fridman (24:44.840)
or very often there was a mine of very interesting data
Tomaso Poggio (24:52.960)
coming from the war, from different types of injuries
Lex Fridman (25:00.600)
that soldiers had in the brain.
Lex Fridman (25:03.800)
And more recently, functional MRI,
Lex Fridman (25:09.120)
which allow you to check which part of the brain
Tomaso Poggio (25:13.840)
are active when you are doing different tasks,
Lex Fridman (25:21.640)
can replace some of this.
Tomaso Poggio (25:23.720)
You can see that certain parts of the brain are involved,
Lex Fridman (25:27.560)
are active in certain tasks.
Tomaso Poggio (25:29.480)
Vision, language, yeah, that's right.
Lex Fridman (25:32.320)
But sort of taking a step back to that part of the brain
Tomaso Poggio (25:36.520)
that discovers that specializes in the face
Lex Fridman (25:39.320)
and how that might be learned, what's your intuition behind?
Tomaso Poggio (25:45.320)
Is it possible that from a physicist perspective,
Lex Fridman (25:48.880)
when you get lower and lower, that it's all the same stuff
Lex Fridman (25:51.920)
and it just, when you're born, it's plastic
Lex Fridman (25:54.800)
and quickly figures out this part is going to be about vision,
Tomaso Poggio (25:58.040)
this is going to be about language,
Lex Fridman (25:59.440)
this is about common sense reasoning.
Lex Fridman (26:02.000)
Do you have an intuition that that kind of learning
Lex Fridman (26:05.120)
is going on really quickly, or is it really
Lex Fridman (26:07.080)
kind of solidified in hardware?
Lex Fridman (26:09.760)
That's a great question.
Lex Fridman (26:11.440)
So there are parts of the brain like the cerebellum
Lex Fridman (26:16.920)
or the hippocampus that are quite different from each other.
Tomaso Poggio (26:21.560)
They clearly have different anatomy,
Lex Fridman (26:23.840)
different connectivity.
Tomaso Poggio (26:26.880)
Then there is the cortex, which is the most developed part
Lex Fridman (26:33.400)
of the brain in humans.
Lex Fridman (26:36.080)
And in the cortex, you have different regions
Lex Fridman (26:39.560)
of the cortex that are responsible for vision,
Tomaso Poggio (26:43.360)
for audition, for motor control, for language.
Lex Fridman (26:47.880)
Now, one of the big puzzles of this
Tomaso Poggio (26:50.760)
is that in the cortex is the cortex is the cortex.
Lex Fridman (26:55.240)
Looks like it is the same in terms of hardware,
Tomaso Poggio (27:00.920)
in terms of type of neurons and connectivity
Lex Fridman (27:05.040)
across these different modalities.
Lex Fridman (27:08.360)
So for the cortex, I think aside these other parts
Lex Fridman (27:13.680)
of the brain like spinal cord, hippocampus,
Tomaso Poggio (27:15.800)
cerebellum, and so on, for the cortex,
Lex Fridman (27:18.840)
I think your question about hardware and software
Lex Fridman (27:21.920)
and learning and so on, I think is rather open.
Lex Fridman (27:28.400)
And I find it very interesting for Risa
Tomaso Poggio (27:33.720)
to think about an architecture, computer architecture, that
Lex Fridman (27:36.960)
is good for vision and at the same time is good for language.
Tomaso Poggio (27:41.360)
Seems to be so different problem areas that you have to solve.
Lex Fridman (27:49.320)
But the underlying mechanism might be the same.
Lex Fridman (27:51.280)
And that's really instructive for artificial neural networks.
Lex Fridman (27:55.200)
So we've done a lot of great work in vision,
Tomaso Poggio (27:58.000)
in human vision, computer vision.
Lex Fridman (28:01.640)
And you mentioned the problem of human vision
Tomaso Poggio (28:03.800)
is really as difficult as the problem of general intelligence.
Lex Fridman (28:07.440)
And maybe that connects to the cortex discussion.
Lex Fridman (28:11.480)
Can you describe the human visual cortex
Lex Fridman (28:15.320)
and how the humans begin to understand the world
Lex Fridman (28:20.320)
through the raw sensory information?
Lex Fridman (28:22.480)
What's, for folks who are not familiar,
Tomaso Poggio (28:27.760)
especially on the computer vision side,
Lex Fridman (28:30.120)
we don't often actually take a step back except saying
Tomaso Poggio (28:33.400)
with a sentence or two that one is inspired by the other.
Lex Fridman (28:36.560)
What is it that we know about the human visual cortex?
Tomaso Poggio (28:40.000)
That's interesting.
Lex Fridman (28:40.760)
We know quite a bit.
Tomaso Poggio (28:41.880)
At the same time, we don't know a lot.
Lex Fridman (28:43.440)
But the bit we know, in a sense, we know a lot of the details.
Lex Fridman (28:50.080)
And many we don't know.
Lex Fridman (28:53.440)
And we know a lot of the top level,
Tomaso Poggio (28:58.520)
the answer to the top level question.
Lex Fridman (29:00.080)
But we don't know some basic ones,
Tomaso Poggio (29:02.200)
even in terms of general neuroscience, forgetting vision.
Lex Fridman (29:06.200)
Why do we sleep?
Tomaso Poggio (29:08.960)
It's such a basic question.
Lex Fridman (29:11.960)
And we really don't have an answer to that.
Lex Fridman (29:15.360)
So taking a step back on that.
Lex Fridman (29:17.160)
So sleep, for example, is fascinating.
Lex Fridman (29:18.760)
Do you think that's a neuroscience question?
Lex Fridman (29:22.040)
Or if we talk about abstractions, what do you
Tomaso Poggio (29:25.360)
think is an interesting way to study intelligence
Lex Fridman (29:28.160)
or most effective on the levels of abstraction?
Tomaso Poggio (29:30.680)
Is it chemical, is it biological,
Lex Fridman (29:33.120)
is it electrophysical, mathematical,
Lex Fridman (29:35.560)
as you've done a lot of excellent work on that side?
Lex Fridman (29:37.880)
Which psychology, at which level of abstraction do you think?
Tomaso Poggio (29:43.280)
Well, in terms of levels of abstraction,
Lex Fridman (29:46.880)
I think we need all of them.
Tomaso Poggio (29:50.160)
It's like if you ask me, what does it
Lex Fridman (29:54.360)
mean to understand a computer?
Tomaso Poggio (29:57.560)
That's much simpler.
Lex Fridman (29:58.640)
But in a computer, I could say, well,
Tomaso Poggio (30:01.080)
I understand how to use PowerPoint.
Lex Fridman (30:04.800)
That's my level of understanding a computer.
Tomaso Poggio (30:08.080)
It is reasonable.
Lex Fridman (30:09.400)
It gives me some power to produce slides
Lex Fridman (30:11.760)
and beautiful slides.
Lex Fridman (30:14.480)
Now, you can ask somebody else.
Tomaso Poggio (30:17.320)
He says, well, I know how the transistors work
Lex Fridman (30:19.840)
that are inside the computer.
Tomaso Poggio (30:21.360)
I can write the equation for transistor and diodes
Lex Fridman (30:25.920)
and circuits, logical circuits.
Lex Fridman (30:29.360)
And I can ask this guy, do you know how to operate PowerPoint?
Lex Fridman (30:32.440)
No idea.
Lex Fridman (30:34.040)
So do you think if we discovered computers walking amongst us
Lex Fridman (30:39.800)
full of these transistors that are also operating
Tomaso Poggio (30:43.400)
under windows and have PowerPoint,
Lex Fridman (30:45.560)
do you think it's digging in a little bit more?
Lex Fridman (30:49.960)
How useful is it to understand the transistor in order
Lex Fridman (30:53.280)
to be able to understand PowerPoint
Lex Fridman (30:58.040)
and these higher level intelligent processes?
Lex Fridman (31:00.320)
So I think in the case of computers,
Tomaso Poggio (31:03.720)
because they were made by engineers, by us,
Lex Fridman (31:06.960)
this different level of understanding
Tomaso Poggio (31:09.280)
are rather separate on purpose.
Lex Fridman (31:13.280)
They are separate modules so that the engineer that
Tomaso Poggio (31:17.240)
designed the circuit for the chips does not
Lex Fridman (31:19.640)
need to know what is inside PowerPoint.
Lex Fridman (31:23.600)
And somebody can write the software translating
Lex Fridman (31:27.440)
from one to the other.
Lex Fridman (31:30.360)
So in that case, I don't think understanding the transistor
Lex Fridman (31:36.960)
helps you understand PowerPoint, or very little.
Tomaso Poggio (31:41.120)
If you want to understand the computer, this question,
Lex Fridman (31:43.960)
I would say you have to understand it
Tomaso Poggio (31:45.960)
at different levels.
Lex Fridman (31:46.800)
If you really want to build one, right?
Lex Fridman (31:51.520)
But for the brain, I think these levels of understanding,
Lex Fridman (31:57.320)
so the algorithms, which kind of computation,
Tomaso Poggio (32:00.840)
the equivalent of PowerPoint, and the circuits,
Lex Fridman (32:04.640)
the transistors, I think they are much more
Tomaso Poggio (32:07.560)
intertwined with each other.
Lex Fridman (32:09.560)
There is not a neatly level of the software separate
Tomaso Poggio (32:14.480)
from the hardware.
Lex Fridman (32:15.840)
And so that's why I think in the case of the brain,
Tomaso Poggio (32:20.080)
the problem is more difficult and more than for computers
Lex Fridman (32:23.640)
requires the interaction, the collaboration
Tomaso Poggio (32:26.560)
between different types of expertise.
Lex Fridman (32:30.080)
The brain is a big hierarchical mess.
Tomaso Poggio (32:32.320)
You can't just disentangle levels.
Lex Fridman (32:35.120)
I think you can, but it's much more difficult.
Lex Fridman (32:37.880)
And it's not completely obvious.
Lex Fridman (32:40.840)
And as I said, I think it's one of the, personally,
Tomaso Poggio (32:44.720)
I think is the greatest problem in science.
Lex Fridman (32:47.240)
So I think it's fair that it's difficult.
Tomaso Poggio (32:51.880)
That's a difficult one.
Lex Fridman (32:53.320)
That said, you do talk about compositionality
Lex Fridman (32:56.920)
and why it might be useful.
Lex Fridman (32:58.280)
And when you discuss why these neural networks,
Tomaso Poggio (33:01.720)
in artificial or biological sense, learn anything,
Lex Fridman (33:05.200)
you talk about compositionality.
Tomaso Poggio (33:07.560)
See, there's a sense that nature can be disentangled.
Lex Fridman (33:13.480)
Or, well, all aspects of our cognition
Tomaso Poggio (33:19.840)
could be disentangled to some degree.
Lex Fridman (33:22.640)
So why do you think, first of all,
Lex Fridman (33:25.920)
how do you see compositionality?
Lex Fridman (33:27.720)
And why do you think it exists at all in nature?
Tomaso Poggio (33:31.640)
I spoke about, I use the term compositionality
Lex Fridman (33:39.800)
when we looked at deep neural networks, multilayers,
Lex Fridman (33:45.320)
and trying to understand when and why they are more powerful
Lex Fridman (33:50.560)
than more classical one layer networks,
Tomaso Poggio (33:54.800)
like linear classifier, kernel machines, so called.
Lex Fridman (34:01.600)
And what we found is that in terms
Tomaso Poggio (34:05.360)
of approximating or learning or representing
Lex Fridman (34:08.360)
a function, a mapping from an input to an output,
Tomaso Poggio (34:12.200)
like from an image to the label in the image,
Lex Fridman (34:16.760)
if this function has a particular structure,
Tomaso Poggio (34:20.840)
then deep networks are much more powerful than shallow networks
Lex Fridman (34:26.120)
to approximate the underlying function.
Lex Fridman (34:28.880)
And the particular structure is a structure of compositionality.
Lex Fridman (34:33.920)
If the function is made up of functions of function,
Lex Fridman (34:38.960)
so that you need to look on when you are interpreting an image,
Lex Fridman (34:45.800)
classifying an image, you don't need
Tomaso Poggio (34:47.720)
to look at all pixels at once.
Lex Fridman (34:51.040)
But you can compute something from small groups of pixels.
Lex Fridman (34:57.120)
And then you can compute something
Lex Fridman (34:59.920)
on the output of this local computation and so on,
Tomaso Poggio (35:04.760)
which is similar to what you do when you read a sentence.
Lex Fridman (35:07.320)
You don't need to read the first and the last letter.
Lex Fridman (35:11.360)
But you can read syllables, combine them in words,
Lex Fridman (35:16.000)
combine the words in sentences.
Lex Fridman (35:18.120)
So this is this kind of structure.
Lex Fridman (35:21.040)
So that's as part of a discussion
Tomaso Poggio (35:22.600)
of why deep neural networks may be more
Lex Fridman (35:26.120)
effective than the shallow methods.
Lex Fridman (35:27.880)
And is your sense, for most things
Lex Fridman (35:31.320)
we can use neural networks for, those problems
Tomaso Poggio (35:37.400)
are going to be compositional in nature, like language,
Lex Fridman (35:42.440)
like vision?
Lex Fridman (35:44.240)
How far can we get in this kind of way?
Lex Fridman (35:47.840)
So here is almost philosophy.
Tomaso Poggio (35:51.560)
Well, let's go there.
Lex Fridman (35:53.120)
Yeah, let's go there.
Lex Fridman (35:54.240)
So a friend of mine, Max Tegmark, who is a physicist at MIT.
Lex Fridman (36:00.200)
I've talked to him on this thing.
Lex Fridman (36:01.560)
Yeah, and he disagrees with you, right?
Lex Fridman (36:03.800)
A little bit.
Tomaso Poggio (36:04.440)
Yeah, we agree on most.
Lex Fridman (36:07.040)
But the conclusion is a bit different.
Tomaso Poggio (36:10.160)
His conclusion is that for images, for instance,
Lex Fridman (36:14.640)
the compositional structure of this function
Tomaso Poggio (36:19.440)
that we have to learn or to solve these problems
Lex Fridman (36:23.360)
comes from physics, comes from the fact
Tomaso Poggio (36:27.760)
that you have local interactions in physics
Lex Fridman (36:31.920)
between atoms and other atoms, between particle
Tomaso Poggio (36:37.440)
of matter and other particles, between planets
Lex Fridman (36:41.120)
and other planets, between stars and other.
Tomaso Poggio (36:44.400)
It's all local.
Lex Fridman (36:48.320)
And that's true.
Lex Fridman (36:51.160)
But you could push this argument a bit further.
Lex Fridman (36:56.280)
Not this argument, actually.
Tomaso Poggio (36:57.600)
You could argue that maybe that's part of the truth.
Lex Fridman (37:02.800)
But maybe what happens is kind of the opposite,
Tomaso Poggio (37:06.800)
is that our brain is wired up as a deep network.
Lex Fridman (37:11.840)
So it can learn, understand, solve
Tomaso Poggio (37:18.240)
problems that have this compositional structure
Lex Fridman (37:22.800)
and it cannot solve problems that don't have
Tomaso Poggio (37:27.520)
this compositional structure.
Lex Fridman (37:29.400)
So the problems we are accustomed to, we think about,
Tomaso Poggio (37:34.920)
we test our algorithms on, are this compositional structure
Lex Fridman (37:40.160)
because our brain is made up.
Lex Fridman (37:42.600)
And that's, in a sense, an evolutionary perspective
Lex Fridman (37:45.400)
that we've.
Lex Fridman (37:46.400)
So the ones that didn't have, that weren't
Lex Fridman (37:50.120)
dealing with the compositional nature of reality died off?
Tomaso Poggio (37:55.200)
Yes, but also could be maybe the reason
Lex Fridman (38:00.320)
why we have this local connectivity in the brain,
Tomaso Poggio (38:05.480)
like simple cells in cortex looking
Lex Fridman (38:08.840)
only at the small part of the image, each one of them,
Lex Fridman (38:11.920)
and then other cells looking at the small number
Lex Fridman (38:14.680)
of these simple cells and so on.
Tomaso Poggio (38:16.360)
The reason for this may be purely
Lex Fridman (38:19.960)
that it was difficult to grow long range connectivity.
Lex Fridman (38:25.080)
So suppose it's for biology.
Lex Fridman (38:28.640)
It's possible to grow short range connectivity but not
Tomaso Poggio (38:34.280)
long range also because there is a limited number of long range
Lex Fridman (38:38.560)
that you can.
Lex Fridman (38:39.720)
And so you have this limitation from the biology.
Lex Fridman (38:45.000)
And this means you build a deep convolutional network.
Tomaso Poggio (38:50.160)
This would be something like a deep convolutional network.
Lex Fridman (38:53.600)
And this is great for solving certain class of problems.
Tomaso Poggio (38:57.800)
These are the ones we find easy and important for our life.
Lex Fridman (39:02.880)
And yes, they were enough for us to survive.
Lex Fridman (39:07.320)
And you can start a successful business
Lex Fridman (39:10.800)
on solving those problems with Mobileye.
Tomaso Poggio (39:14.600)
Driving is a compositional problem.
Lex Fridman (39:17.360)
So on the learning task, we don't
Tomaso Poggio (39:21.080)
know much about how the brain learns
Lex Fridman (39:24.000)
in terms of optimization.
Lex Fridman (39:26.320)
So the thing that's stochastic gradient descent
Lex Fridman (39:29.040)
is what artificial neural networks use for the most part
Tomaso Poggio (39:33.760)
to adjust the parameters in such a way that it's
Lex Fridman (39:37.520)
able to deal based on the label data,
Tomaso Poggio (39:40.640)
it's able to solve the problem.
Lex Fridman (39:42.520)
So what's your intuition about why it works at all?
Lex Fridman (39:50.040)
How hard of a problem it is to optimize
Lex Fridman (39:53.360)
a neural network, artificial neural network?
Lex Fridman (39:56.320)
Is there other alternatives?
Lex Fridman (39:58.720)
Just in general, your intuition is
Tomaso Poggio (40:01.640)
behind this very simplistic algorithm
Lex Fridman (40:03.800)
that seems to do pretty good, surprisingly so.
Tomaso Poggio (40:06.640)
Yes.
Lex Fridman (40:07.840)
So I find neuroscience, the architecture of cortex,
Tomaso Poggio (40:13.840)
is really similar to the architecture of deep networks.
Lex Fridman (40:17.440)
So there is a nice correspondence there
Tomaso Poggio (40:20.360)
between the biology and this kind
Lex Fridman (40:23.160)
of local connectivity, hierarchical architecture.
Tomaso Poggio (40:28.200)
The stochastic gradient descent, as you said,
Lex Fridman (40:30.960)
is a very simple technique.
Tomaso Poggio (40:35.760)
It seems pretty unlikely that biology could do that
Lex Fridman (40:41.320)
from what we know right now about cortex and neurons
Lex Fridman (40:47.360)
and synapses.
Lex Fridman (40:50.200)
So it's a big question open whether there
Tomaso Poggio (40:53.080)
are other optimization learning algorithms that
Lex Fridman (40:59.040)
can replace stochastic gradient descent.
Lex Fridman (41:02.000)
And my guess is yes, but nobody has found yet a real answer.
Lex Fridman (41:11.760)
I mean, people are trying, still trying,
Lex Fridman (41:13.840)
and there are some interesting ideas.
Lex Fridman (41:18.280)
The fact that stochastic gradient descent
Tomaso Poggio (41:22.000)
is so successful, this has become clearly not so
Lex Fridman (41:26.160)
mysterious.
Lex Fridman (41:27.640)
And the reason is that it's an interesting fact.
Lex Fridman (41:33.840)
It's a change, in a sense, in how
Tomaso Poggio (41:36.840)
people think about statistics.
Lex Fridman (41:39.280)
And this is the following, is that typically when
Tomaso Poggio (41:45.160)
you had data and you had, say, a model with parameters,
Lex Fridman (41:51.800)
you are trying to fit the model to the data,
Tomaso Poggio (41:54.520)
to fit the parameter.
Lex Fridman (41:55.960)
Typically, the kind of crowd wisdom type idea
Tomaso Poggio (42:04.520)
was you should have at least twice the number of data
Lex Fridman (42:09.720)
than the number of parameters.
Tomaso Poggio (42:12.880)
Maybe 10 times is better.
Lex Fridman (42:15.480)
Now, the way you train neural networks these days
Tomaso Poggio (42:19.560)
is that they have 10 or 100 times more parameters
Lex Fridman (42:23.480)
than data, exactly the opposite.
Lex Fridman (42:26.760)
And it has been one of the puzzles about neural networks.
Lex Fridman (42:34.080)
How can you get something that really works
Lex Fridman (42:37.120)
when you have so much freedom?
Lex Fridman (42:40.640)
From that little data, it can generalize somehow.
Tomaso Poggio (42:43.000)
Right, exactly.
Lex Fridman (42:44.200)
Do you think the stochastic nature of it
Lex Fridman (42:46.400)
is essential, the randomness?
Lex Fridman (42:48.160)
So I think we have some initial understanding
Lex Fridman (42:50.640)
why this happens.
Lex Fridman (42:52.240)
But one nice side effect of having
Tomaso Poggio (42:56.480)
this overparameterization, more parameters than data,
Lex Fridman (43:00.920)
is that when you look for the minima of a loss function,
Tomaso Poggio (43:04.720)
like stochastic gradient descent is doing,
Lex Fridman (43:08.240)
you find I made some calculations based
Tomaso Poggio (43:12.120)
on some old basic theorem of algebra called the Bezu
Lex Fridman (43:19.040)
theorem that gives you an estimate of the number
Tomaso Poggio (43:23.240)
of solution of a system of polynomial equation.
Lex Fridman (43:25.960)
Anyway, the bottom line is that there are probably
Tomaso Poggio (43:30.520)
more minima for a typical deep networks
Lex Fridman (43:36.080)
than atoms in the universe.
Tomaso Poggio (43:39.480)
Just to say, there are a lot because
Lex Fridman (43:42.120)
of the overparameterization.
Tomaso Poggio (43:44.760)
A more global minimum, zero minimum, good minimum.
Lex Fridman (43:50.280)
A more global minima.
Tomaso Poggio (43:51.560)
Yeah, a lot of them.
Lex Fridman (43:53.200)
So you have a lot of solutions.
Lex Fridman (43:54.560)
So it's not so surprising that you can find them
Lex Fridman (43:57.920)
relatively easily.
Lex Fridman (44:00.400)
And this is because of the overparameterization.
Lex Fridman (44:04.200)
The overparameterization sprinkles that entire space
Tomaso Poggio (44:07.920)
with solutions that are pretty good.
Lex Fridman (44:09.720)
It's not so surprising, right?
Tomaso Poggio (44:11.240)
It's like if you have a system of linear equation
Lex Fridman (44:14.400)
and you have more unknowns than equations, then you have,
Tomaso Poggio (44:18.520)
we know, you have an infinite number of solutions.
Lex Fridman (44:22.040)
And the question is to pick one.
Tomaso Poggio (44:24.480)
That's another story.
Lex Fridman (44:25.440)
But you have an infinite number of solutions.
Lex Fridman (44:27.520)
So there are a lot of value of your unknowns
Lex Fridman (44:31.040)
that satisfy the equations.
Lex Fridman (44:33.160)
But it's possible that there's a lot of those solutions that
Lex Fridman (44:36.360)
aren't very good.
Tomaso Poggio (44:37.560)
What's surprising is that they're pretty good.
Lex Fridman (44:39.160)
So that's a good question.
Lex Fridman (44:40.160)
Why can you pick one that generalizes well?
Lex Fridman (44:42.840)
Yeah.
Tomaso Poggio (44:44.120)
That's a separate question with separate answers.
Lex Fridman (44:47.120)
One theorem that people like to talk about that kind of
Tomaso Poggio (44:51.160)
inspires imagination of the power of neural networks
Lex Fridman (44:53.800)
is the universality, universal approximation theorem,
Tomaso Poggio (44:57.840)
that you can approximate any computable function
Lex Fridman (45:00.960)
with just a finite number of neurons
Tomaso Poggio (45:02.840)
in a single hidden layer.
Lex Fridman (45:04.360)
Do you find this theorem one surprising?
Lex Fridman (45:07.680)
Do you find it useful, interesting, inspiring?
Lex Fridman (45:12.600)
No, this one, I never found it very surprising.
Tomaso Poggio (45:16.440)
It was known since the 80s, since I entered the field,
Lex Fridman (45:22.640)
because it's basically the same as Weierstrass theorem, which
Tomaso Poggio (45:27.560)
says that I can approximate any continuous function
Lex Fridman (45:32.000)
with a polynomial of sufficiently,
Tomaso Poggio (45:34.560)
with a sufficient number of terms, monomials.
Lex Fridman (45:38.120)
So basically the same.
Lex Fridman (45:39.360)
And the proofs are very similar.
Lex Fridman (45:41.680)
So your intuition was there was never
Tomaso Poggio (45:43.520)
any doubt that neural networks in theory
Lex Fridman (45:45.680)
could be very strong approximators.
Tomaso Poggio (45:48.000)
Right.
Lex Fridman (45:48.800)
The question, the interesting question,
Tomaso Poggio (45:50.760)
is that if this theorem says you can approximate, fine.
Lex Fridman (45:58.520)
But when you ask how many neurons, for instance,
Tomaso Poggio (46:03.200)
or in the case of polynomial, how many monomials,
Lex Fridman (46:06.400)
I need to get a good approximation.
Tomaso Poggio (46:11.360)
Then it turns out that that depends
Lex Fridman (46:15.960)
on the dimensionality of your function,
Lex Fridman (46:18.080)
how many variables you have.
Lex Fridman (46:20.520)
But it depends on the dimensionality
Tomaso Poggio (46:22.120)
of your function in a bad way.
Lex Fridman (46:25.080)
It's, for instance, suppose you want
Tomaso Poggio (46:28.000)
an error which is no worse than 10% in your approximation.
Lex Fridman (46:35.040)
You come up with a network that approximate your function
Tomaso Poggio (46:38.120)
within 10%.
Lex Fridman (46:40.440)
Then it turns out that the number of units you need
Tomaso Poggio (46:44.520)
are in the order of 10 to the dimensionality, d,
Lex Fridman (46:48.360)
how many variables.
Lex Fridman (46:50.080)
So if you have two variables, these two words,
Lex Fridman (46:54.840)
you have 100 units and OK.
Lex Fridman (46:57.240)
But if you have, say, 200 by 200 pixel images,
Lex Fridman (47:02.920)
now this is 40,000, whatever.
Tomaso Poggio (47:06.840)
We again go to the size of the universe pretty quickly.
Lex Fridman (47:09.800)
Exactly, 10 to the 40,000 or something.
Lex Fridman (47:14.120)
And so this is called the curse of dimensionality,
Lex Fridman (47:18.680)
not quite appropriately.
Lex Fridman (47:22.280)
And the hope is with the extra layers,
Lex Fridman (47:24.200)
you can remove the curse.
Lex Fridman (47:28.040)
What we proved is that if you have deep layers,
Lex Fridman (47:32.280)
hierarchical architecture with the local connectivity
Tomaso Poggio (47:36.200)
of the type of convolutional deep learning,
Lex Fridman (47:39.960)
and if you're dealing with a function that
Tomaso Poggio (47:42.000)
has this kind of hierarchical architecture,
Lex Fridman (47:46.680)
then you avoid completely the curse.
Tomaso Poggio (47:50.680)
You've spoken a lot about supervised deep learning.
Lex Fridman (47:54.520)
What are your thoughts, hopes, views
Tomaso Poggio (47:56.480)
on the challenges of unsupervised learning
Lex Fridman (47:59.640)
with GANs, with Generative Adversarial Networks?
Lex Fridman (48:05.800)
Do you see those as distinct?
Lex Fridman (48:08.120)
The power of GANs, do you see those
Tomaso Poggio (48:09.920)
as distinct from supervised methods in neural networks,
Lex Fridman (48:13.120)
or are they really all in the same representation ballpark?
Tomaso Poggio (48:16.640)
GANs is one way to get estimation of probability
Lex Fridman (48:24.040)
densities, which is a somewhat new way that people have not
Tomaso Poggio (48:28.760)
done before.
Lex Fridman (48:30.360)
I don't know whether this will really play an important role
Tomaso Poggio (48:36.080)
in intelligence.
Lex Fridman (48:39.000)
Or it's interesting.
Tomaso Poggio (48:43.080)
I'm less enthusiastic about it than many people in the field.
Lex Fridman (48:48.600)
I have the feeling that many people in the field
Tomaso Poggio (48:50.880)
are really impressed by the ability
Lex Fridman (48:54.320)
of producing realistic looking images in this generative way.
Tomaso Poggio (49:01.160)
Which describes the popularity of the methods.
Lex Fridman (49:03.080)
But you're saying that while that's exciting and cool
Tomaso Poggio (49:06.320)
to look at, it may not be the tool that's useful for it.
Lex Fridman (49:11.200)
So you describe it kind of beautifully.
Tomaso Poggio (49:13.560)
Current supervised methods go n to infinity
Lex Fridman (49:16.320)
in terms of number of labeled points.
Lex Fridman (49:18.200)
And we really have to figure out how to go to n to 1.
Lex Fridman (49:21.360)
And you're thinking GANs might help,
Lex Fridman (49:23.200)
but they might not be the right.
Lex Fridman (49:25.080)
I don't think for that problem, which I really think
Tomaso Poggio (49:28.480)
is important, I think they may help.
Lex Fridman (49:32.000)
They certainly have applications,
Tomaso Poggio (49:33.680)
for instance, in computer graphics.
Lex Fridman (49:35.760)
And I did work long ago, which was
Tomaso Poggio (49:41.560)
a little bit similar in terms of saying, OK, I have a network.
Lex Fridman (49:47.000)
And I present images.
Lex Fridman (49:49.760)
And I can input its images.
Lex Fridman (49:54.040)
And output is, for instance, the pose of the image.
Tomaso Poggio (49:57.520)
A face, how much is smiling, is rotated 45 degrees or not.
Lex Fridman (50:02.960)
What about having a network that I train with the same data
Tomaso Poggio (50:07.440)
set, but now I invert input and output.
Lex Fridman (50:10.600)
Now the input is the pose or the expression, a number,
Tomaso Poggio (50:15.920)
set of numbers.
Lex Fridman (50:16.920)
And the output is the image.
Lex Fridman (50:18.280)
And I train it.
Lex Fridman (50:20.240)
And we did pretty good, interesting results
Tomaso Poggio (50:22.520)
in terms of producing very realistic looking images.
Lex Fridman (50:27.840)
It was a less sophisticated mechanism.
Lex Fridman (50:31.920)
But the output was pretty less than GANs.
Lex Fridman (50:35.320)
But the output was pretty much of the same quality.
Lex Fridman (50:38.960)
So I think for a computer graphics type application,
Lex Fridman (50:43.400)
yeah, definitely GANs can be quite useful.
Lex Fridman (50:46.240)
And not only for that, but for helping,
Lex Fridman (50:52.880)
for instance, on this problem of unsupervised example
Tomaso Poggio (50:58.200)
of reducing the number of labeled examples.
Lex Fridman (51:02.400)
I think people, it's like they think they can get out
Tomaso Poggio (51:07.920)
more than they put in.
Lex Fridman (51:11.080)
There's no free lunch, as you said.
Lex Fridman (51:14.000)
What do you think, what's your intuition?
Lex Fridman (51:17.320)
How can we slow the growth of N to infinity in supervised,
Lex Fridman (51:22.720)
N to infinity in supervised learning?
Lex Fridman (51:25.080)
So for example, Mobileye has very successfully,
Tomaso Poggio (51:29.880)
I mean, essentially annotated large amounts of data
Lex Fridman (51:33.000)
to be able to drive a car.
Tomaso Poggio (51:34.680)
Now one thought is, so we're trying
Lex Fridman (51:37.440)
to teach machines, school of AI.
Lex Fridman (51:41.000)
And we're trying to, so how can we become better teachers,
Lex Fridman (51:45.560)
maybe?
Tomaso Poggio (51:46.040)
That's one way.
Lex Fridman (51:47.320)
No, I like that.
Tomaso Poggio (51:51.240)
Because again, one caricature of the history of computer
Lex Fridman (51:57.680)
science, you could say, begins with programmers, expensive.
Tomaso Poggio (52:05.360)
Continuous labelers, cheap.
Lex Fridman (52:09.640)
And the future will be schools, like we have for kids.
Tomaso Poggio (52:14.680)
Yeah.
Lex Fridman (52:16.360)
Currently, the labeling methods were not
Tomaso Poggio (52:20.280)
selective about which examples we teach networks with.
Lex Fridman (52:25.880)
So I think the focus of making networks that learn much faster
Tomaso Poggio (52:31.320)
is often on the architecture side.
Lex Fridman (52:33.680)
But how can we pick better examples with which to learn?
Lex Fridman (52:37.960)
Do you have intuitions about that?
Lex Fridman (52:39.440)
Well, that's part of the problem.
Lex Fridman (52:42.480)
But the other one is, if we look at biology,
Lex Fridman (52:50.360)
a reasonable assumption, I think,
Tomaso Poggio (52:52.960)
is in the same spirit that I said,
Lex Fridman (52:58.120)
evolution is opportunistic and has weak priors.
Tomaso Poggio (53:03.400)
The way I think the intelligence of a child,
Lex Fridman (53:08.280)
the baby may develop is by bootstrapping weak priors
Tomaso Poggio (53:16.240)
from evolution.
Lex Fridman (53:17.400)
For instance, you can assume that you
Tomaso Poggio (53:24.720)
have in most organisms, including human babies,
Lex Fridman (53:28.960)
built in some basic machinery to detect motion
Lex Fridman (53:35.400)
and relative motion.
Lex Fridman (53:38.200)
And in fact, we know all insects from fruit flies
Tomaso Poggio (53:42.920)
to other animals, they have this,
Lex Fridman (53:49.760)
even in the retinas, in the very peripheral part.
Tomaso Poggio (53:53.120)
It's very conserved across species, something
Lex Fridman (53:56.160)
that evolution discovered early.
Tomaso Poggio (53:59.040)
It may be the reason why babies tend
Lex Fridman (54:01.480)
to look in the first few days to moving objects
Lex Fridman (54:06.160)
and not to not moving objects.
Lex Fridman (54:08.320)
Now, moving objects means, OK, they're attracted by motion.
Lex Fridman (54:12.200)
But motion also means that motion
Lex Fridman (54:15.480)
gives automatic segmentation from the background.
Lex Fridman (54:20.560)
So because of motion boundaries, either the object
Lex Fridman (54:25.360)
is moving or the eye of the baby is tracking the moving object
Lex Fridman (54:30.600)
and the background is moving, right?
Lex Fridman (54:32.800)
Yeah, so just purely on the visual characteristics
Tomaso Poggio (54:36.040)
of the scene, that seems to be the most useful.
Lex Fridman (54:37.920)
Right, so it's like looking at an object without background.
Tomaso Poggio (54:43.960)
It's ideal for learning the object.
Lex Fridman (54:45.760)
Otherwise, it's really difficult because you
Tomaso Poggio (54:48.760)
have so much stuff.
Lex Fridman (54:50.440)
So suppose you do this at the beginning, first weeks.
Tomaso Poggio (54:55.120)
Then after that, you can recognize object.
Lex Fridman (54:58.560)
Now they are imprinted, the number one,
Tomaso Poggio (55:02.160)
even in the background, even without motion.
Lex Fridman (55:05.800)
So that's, by the way, I just want
Tomaso Poggio (55:08.160)
to ask on the object recognition problem.
Lex Fridman (55:10.920)
So there is this being responsive to movement
Lex Fridman (55:13.960)
and doing edge detection, essentially.
Lex Fridman (55:16.760)
What's the gap between being effective at visually
Tomaso Poggio (55:21.600)
recognizing stuff, detecting where it is,
Lex Fridman (55:24.560)
and understanding the scene?
Lex Fridman (55:27.640)
Is this a huge gap in many layers, or is it close?
Lex Fridman (55:32.960)
No, I think that's a huge gap.
Tomaso Poggio (55:35.120)
I think present algorithm with all the success that we have
Lex Fridman (55:42.040)
and the fact that there are a lot of very useful,
Tomaso Poggio (55:45.120)
I think we are in a golden age for applications
Lex Fridman (55:48.440)
of low level vision and low level speech recognition
Lex Fridman (55:53.720)
and so on, Alexa and so on.
Lex Fridman (55:56.800)
There are many more things of similar level
Tomaso Poggio (55:58.840)
to be done, including medical diagnosis and so on.
Lex Fridman (56:02.040)
But we are far from what we call understanding
Tomaso Poggio (56:05.600)
of a scene, of language, of actions, of people.
Lex Fridman (56:11.960)
That is, despite the claims, that's, I think, very far.
Tomaso Poggio (56:18.480)
We're a little bit off.
Lex Fridman (56:19.560)
So in popular culture and among many researchers,
Tomaso Poggio (56:23.160)
some of which I've spoken with, the Stuart Russell
Lex Fridman (56:25.640)
and Elon Musk, in and out of the AI field,
Tomaso Poggio (56:30.920)
there's a concern about the existential threat of AI.
Lex Fridman (56:34.520)
And how do you think about this concern?
Lex Fridman (56:40.000)
And is it valuable to think about large scale, long term,
Lex Fridman (56:45.560)
unintended consequences of intelligent systems
Lex Fridman (56:50.360)
we try to build?
Lex Fridman (56:51.440)
I always think it's better to worry first, early,
Tomaso Poggio (56:56.000)
rather than late.
Lex Fridman (56:58.640)
So worry is good.
Tomaso Poggio (56:59.640)
Yeah.
Lex Fridman (57:00.400)
I'm not against worrying at all.
Tomaso Poggio (57:03.000)
Personally, I think that it will take a long time
Lex Fridman (57:09.520)
before there is real reason to be worried.
Lex Fridman (57:15.920)
But as I said, I think it's good to put in place
Lex Fridman (57:19.440)
and think about possible safety against.
Lex Fridman (57:24.360)
What I find a bit misleading are things
Lex Fridman (57:27.360)
like that have been said by people I know, like Elon Musk,
Lex Fridman (57:31.480)
and what is Bostrom in particular,
Lex Fridman (57:35.240)
and what is his first name?
Tomaso Poggio (57:36.800)
Nick Bostrom.
Lex Fridman (57:37.400)
Nick Bostrom, right.
Lex Fridman (57:40.120)
And a couple of other people that, for instance, AI
Lex Fridman (57:44.080)
is more dangerous than nuclear weapons.
Tomaso Poggio (57:46.880)
I think that's really wrong.
Lex Fridman (57:50.400)
That can be misleading.
Tomaso Poggio (57:52.680)
Because in terms of priority, we should still
Lex Fridman (57:56.440)
be more worried about nuclear weapons
Lex Fridman (57:59.480)
and what people are doing about it and so on than AI.
Lex Fridman (58:05.600)
And you've spoken about Demis Hassabis
Lex Fridman (58:09.920)
and yourself saying that you think
Lex Fridman (58:12.840)
you'll be about 100 years out before we
Tomaso Poggio (58:16.440)
have a general intelligence system that's
Lex Fridman (58:18.920)
on par with a human being.
Lex Fridman (58:20.600)
Do you have any updates for those predictions?
Lex Fridman (58:22.520)
Well, I think he said.
Tomaso Poggio (58:24.080)
He said 20, I think.
Lex Fridman (58:25.080)
He said 20, right.
Tomaso Poggio (58:26.200)
This was a couple of years ago.
Lex Fridman (58:27.680)
I have not asked him again.
Lex Fridman (58:29.160)
So should I have?
Lex Fridman (58:31.480)
Your own prediction, what's your prediction
Lex Fridman (58:36.000)
about when you'll be truly surprised?
Lex Fridman (58:38.880)
And what's the confidence interval on that?
Tomaso Poggio (58:43.000)
It's so difficult to predict the future and even
Lex Fridman (58:45.760)
the present sometimes.
Tomaso Poggio (58:47.120)
It's pretty hard to predict.
Lex Fridman (58:48.480)
But I would be, as I said, this is completely,
Tomaso Poggio (58:53.360)
I would be more like Rod Brooks.
Lex Fridman (58:56.960)
I think he's about 200 years.
Tomaso Poggio (58:58.960)
200 years.
Lex Fridman (59:01.560)
When we have this kind of AGI system,
Tomaso Poggio (59:04.880)
artificial general intelligence system,
Lex Fridman (59:06.920)
you're sitting in a room with her, him, it.
Lex Fridman (59:12.840)
Do you think the underlying design of such a system
Lex Fridman (59:17.120)
is something we'll be able to understand?
Lex Fridman (59:19.080)
It will be simple?
Lex Fridman (59:20.480)
Do you think it'll be explainable,
Lex Fridman (59:25.800)
understandable by us?
Lex Fridman (59:27.560)
Your intuition, again, we're in the realm of philosophy
Tomaso Poggio (59:30.760)
a little bit.
Lex Fridman (59:32.080)
Well, probably no.
Lex Fridman (59:36.120)
But again, it depends what you really
Lex Fridman (59:40.280)
mean for understanding.
Lex Fridman (59:42.000)
So I think we don't understand how deep networks work.
Lex Fridman (59:53.280)
I think we are beginning to have a theory now.
Lex Fridman (59:56.520)
But in the case of deep networks,
Lex Fridman (59:59.240)
or even in the case of the simpler kernel machines
🔗 相关节目