Jeff Hawkins: Thousand Brains Theory of Intelligence
生物与进化AI 与机器学习心理与人性音乐与艺术技术与编程
📋 章节目录
暂无章节信息
🔑 关键词
braindoncupreferenceintelligenceneocortexneuronslearningcoffeemodeldoesnframehumanmodelspartstryingintelligenttheorysaidcells
💬 精彩语录
暂无语录
🎙️ 完整对话（3352 条）
Lex Fridman (00:00.000)
The following is a conversation with Jeff Hawkins.
以下是与杰夫·霍金斯的对话。
Lex Fridman (00:02.360)
He's the founder of the Redwood Center
他是红木中心的创始人
Lex Fridman (00:04.120)
for Theoretical Neuroscience in 2002, and NuMenta in 2005.
2002 年获得理论神经科学博士学位，2005 年获得 NuMenta 博士学位。
Lex Fridman (00:08.980)
In his 2004 book, titled On Intelligence,
在他 2004 年出版的《论智力》一书中，
Lex Fridman (00:11.920)
and in the research before and after,
并且在之前和之后的研究中，
Jeff Hawkins (00:13.840)
he and his team have worked to reverse engineer
他和他的团队致力于逆向工程
Lex Fridman (00:16.200)
the neural cortex, and propose artificial intelligence
神经皮层，并提出人工智能
Jeff Hawkins (00:19.160)
architectures, approaches, and ideas
架构、方法和想法
Lex Fridman (00:21.360)
that are inspired by the human brain.
受到人类大脑的启发。
Jeff Hawkins (00:23.640)
These ideas include Hierarchical Tupperware Memory,
这些想法包括分层特百惠内存、
Lex Fridman (00:25.960)
HTM, from 2004, and new work,
HTM，从2004年开始，还有新作品，
Jeff Hawkins (00:28.920)
the Thousand Brains Theory of Intelligence
智力的千脑理论
Lex Fridman (00:30.720)
from 2017, 18, and 19.
从 2017 年、18 日和 19 日开始。
Jeff Hawkins (00:33.760)
Jeff's ideas have been an inspiration
杰夫的想法一直是灵感
Lex Fridman (00:36.120)
to many who have looked for progress
对于许多寻求进步的人来说
Jeff Hawkins (00:38.200)
beyond the current machine learning approaches,
超越当前的机器学习方法，
Lex Fridman (00:40.480)
but they have also received criticism
但他们也受到了批评
Jeff Hawkins (00:42.720)
for lacking a body of empirical evidence
由于缺乏大量经验证据
Lex Fridman (00:44.680)
supporting the models.
支持模型。
Jeff Hawkins (00:46.240)
This is always a challenge when seeking more
当寻求更多时，这始终是一个挑战
Lex Fridman (00:48.440)
than small incremental steps forward in AI.
Jeff Hawkins (00:51.440)
Jeff is a brilliant mind, and many of the ideas
Lex Fridman (00:54.120)
he has developed and aggregated from neuroscience
Jeff Hawkins (00:56.500)
are worth understanding and thinking about.
Lex Fridman (00:59.120)
There are limits to deep learning,
Jeff Hawkins (01:00.920)
as it is currently defined.
Lex Fridman (01:02.920)
Forward progress in AI is shrouded in mystery.
Jeff Hawkins (01:05.760)
My hope is that conversations like this
Lex Fridman (01:07.760)
can help provide an inspiring spark for new ideas.
Jeff Hawkins (01:11.440)
This is the Artificial Intelligence Podcast.
Lex Fridman (01:14.020)
If you enjoy it, subscribe on YouTube, iTunes,
Jeff Hawkins (01:16.720)
or simply connect with me on Twitter
Lex Fridman (01:18.640)
at Lex Friedman, spelled F R I D.
Lex Fridman (01:21.520)
And now, here's my conversation with Jeff Hawkins.
Lex Fridman (01:26.780)
Are you more interested in understanding the human brain
Jeff Hawkins (01:29.860)
or in creating artificial systems
Lex Fridman (01:32.000)
that have many of the same qualities
Lex Fridman (01:34.640)
but don't necessarily require that you actually understand
Lex Fridman (01:38.560)
the underpinning workings of our mind?
Lex Fridman (01:41.480)
So there's a clear answer to that question.
Lex Fridman (01:44.000)
My primary interest is understanding the human brain.
Jeff Hawkins (01:46.760)
No question about it.
Lex Fridman (01:47.700)
But I also firmly believe that we will not be able
Jeff Hawkins (01:53.280)
to create fully intelligent machines
Lex Fridman (01:55.040)
until we understand how the human brain works.
Lex Fridman (01:57.280)
So I don't see those as separate problems.
Lex Fridman (02:00.120)
I think there's limits to what can be done
Jeff Hawkins (02:01.720)
with machine intelligence if you don't understand
Lex Fridman (02:03.540)
the principles by which the brain works.
Lex Fridman (02:05.680)
And so I actually believe that studying the brain
Lex Fridman (02:07.900)
is actually the fastest way to get to machine intelligence.
Lex Fridman (02:11.960)
And within that, let me ask the impossible question,
Lex Fridman (02:14.640)
how do you, not define, but at least think
Lex Fridman (02:17.160)
about what it means to be intelligent?
Lex Fridman (02:19.440)
So I didn't try to answer that question first.
Jeff Hawkins (02:22.280)
We said, let's just talk about how the brain works
Lex Fridman (02:24.520)
and let's figure out how certain parts of the brain,
Jeff Hawkins (02:26.720)
mostly the neocortex, but some other parts too.
Lex Fridman (02:29.920)
The parts of the brain most associated with intelligence.
Lex Fridman (02:32.340)
And let's discover the principles by how they work.
Lex Fridman (02:35.840)
Because intelligence isn't just like some mechanism
Lex Fridman (02:39.360)
and it's not just some capabilities.
Lex Fridman (02:40.680)
It's like, okay, we don't even know
Jeff Hawkins (02:42.520)
where to begin on this stuff.
Lex Fridman (02:44.040)
And so now that we've made a lot of progress on this,
Jeff Hawkins (02:49.320)
after we've made a lot of progress
Lex Fridman (02:50.480)
on how the neocortex works, and we can talk about that,
Jeff Hawkins (02:53.200)
I now have a very good idea what's gonna be required
Lex Fridman (02:55.840)
to make intelligent machines.
Jeff Hawkins (02:57.200)
I can tell you today, some of the things
Lex Fridman (02:59.600)
are gonna be necessary, I believe,
Jeff Hawkins (03:02.140)
to create intelligent machines.
Lex Fridman (03:03.480)
Well, so we'll get there.
Jeff Hawkins (03:04.600)
We'll get to the neocortex and some of the theories
Lex Fridman (03:07.440)
of how the whole thing works.
Lex Fridman (03:09.200)
And you're saying, as we understand more and more
Lex Fridman (03:11.760)
about the neocortex, about our own human mind,
Jeff Hawkins (03:14.760)
we'll be able to start to more specifically define
Lex Fridman (03:17.680)
what it means to be intelligent.
Jeff Hawkins (03:18.680)
It's not useful to really talk about that until.
Lex Fridman (03:21.840)
I don't know if it's not useful.
Jeff Hawkins (03:23.560)
Look, there's a long history of AI, as you know.
Lex Fridman (03:26.160)
And there's been different approaches taken to it.
Lex Fridman (03:28.900)
And who knows, maybe they're all useful.
Lex Fridman (03:32.240)
So the good old fashioned AI, the expert systems,
Jeff Hawkins (03:37.280)
the current convolutional neural networks,
Lex Fridman (03:38.920)
they all have their utility.
Jeff Hawkins (03:40.380)
They all have a value in the world.
Lex Fridman (03:43.780)
But I would think almost everyone agree
Jeff Hawkins (03:45.220)
that none of them are really intelligent
Lex Fridman (03:46.620)
in a sort of a deep way that humans are.
Lex Fridman (03:49.860)
And so it's just the question of how do you get
Lex Fridman (03:53.620)
from where those systems were or are today
Jeff Hawkins (03:56.420)
to where a lot of people think we're gonna go.
Lex Fridman (03:59.240)
And there's a big, big gap there, a huge gap.
Lex Fridman (04:02.340)
And I think the quickest way of bridging that gap
Lex Fridman (04:06.220)
is to figure out how the brain does that.
Lex Fridman (04:08.820)
And then we can sit back and look and say,
Lex Fridman (04:10.100)
oh, which of these principles that the brain works on
Lex Fridman (04:12.980)
are necessary and which ones are not?
Lex Fridman (04:15.140)
Clearly, we don't have to build this in,
Lex Fridman (04:16.620)
and intelligent machines aren't gonna be built
Lex Fridman (04:18.460)
out of organic living cells.
Lex Fridman (04:22.720)
But there's a lot of stuff that goes on the brain
Lex Fridman (04:24.700)
that's gonna be necessary.
Lex Fridman (04:25.900)
So let me ask maybe, before we get into the fun details,
Lex Fridman (04:30.260)
let me ask maybe a depressing or a difficult question.
Lex Fridman (04:33.060)
Do you think it's possible that we will never
Lex Fridman (04:36.220)
be able to understand how our brain works,
Jeff Hawkins (04:38.060)
that maybe there's aspects to the human mind,
Lex Fridman (04:41.820)
like we ourselves cannot introspectively get to the core,
Lex Fridman (04:46.140)
that there's a wall you eventually hit?
Lex Fridman (04:48.100)
Yeah, I don't believe that's the case.
Jeff Hawkins (04:52.020)
I have never believed that's the case.
Lex Fridman (04:53.240)
There's not been a single thing humans have ever put
Jeff Hawkins (04:56.620)
their minds to that we've said, oh, we reached the wall,
Lex Fridman (04:58.620)
we can't go any further.
Jeff Hawkins (04:59.700)
It's just, people keep saying that.
Lex Fridman (05:01.660)
People used to believe that about life.
Jeff Hawkins (05:03.380)
Alain Vital, right, there's like,
Lex Fridman (05:05.180)
what's the difference between living matter
Lex Fridman (05:06.380)
and nonliving matter, something special
Lex Fridman (05:07.980)
that we never understand.
Jeff Hawkins (05:09.100)
We no longer think that.
Lex Fridman (05:10.660)
So there's no historical evidence that suggests this
Jeff Hawkins (05:14.220)
is the case, and I just never even consider
Lex Fridman (05:16.340)
that's a possibility.
Jeff Hawkins (05:17.620)
I would also say, today, we understand so much
Lex Fridman (05:21.860)
about the neocortex.
Jeff Hawkins (05:22.820)
We've made tremendous progress in the last few years
Lex Fridman (05:25.480)
that I no longer think of it as an open question.
Jeff Hawkins (05:30.000)
The answers are very clear to me.
Lex Fridman (05:32.100)
The pieces we don't know are clear to me,
Lex Fridman (05:34.800)
but the framework is all there, and it's like,
Lex Fridman (05:36.740)
oh, okay, we're gonna be able to do this.
Jeff Hawkins (05:38.620)
This is not a problem anymore, just takes time and effort,
Lex Fridman (05:41.100)
but there's no mystery, a big mystery anymore.
Lex Fridman (05:44.060)
So then let's get into it for people like myself
Lex Fridman (05:47.800)
who are not very well versed in the human brain,
Jeff Hawkins (05:52.940)
except my own.
Lex Fridman (05:54.780)
Can you describe to me, at the highest level,
Lex Fridman (05:57.300)
what are the different parts of the human brain,
Lex Fridman (05:59.140)
and then zooming in on the neocortex,
Jeff Hawkins (06:02.060)
the parts of the neocortex, and so on,
Lex Fridman (06:04.120)
a quick overview.
Jeff Hawkins (06:05.500)
Yeah, sure.
Lex Fridman (06:06.620)
The human brain, we can divide it roughly into two parts.
Jeff Hawkins (06:10.780)
There's the old parts, lots of pieces,
Lex Fridman (06:14.220)
and then there's the new part.
Jeff Hawkins (06:15.700)
The new part is the neocortex.
Lex Fridman (06:18.020)
It's new because it didn't exist before mammals.
Jeff Hawkins (06:20.420)
The only mammals have a neocortex,
Lex Fridman (06:22.180)
and in humans, in primates, it's very large.
Jeff Hawkins (06:24.780)
In the human brain, the neocortex occupies
Lex Fridman (06:26.900)
about 70 to 75% of the volume of the brain.
Jeff Hawkins (06:30.660)
It's huge.
Lex Fridman (06:32.100)
And the old parts of the brain are,
Jeff Hawkins (06:34.860)
there's lots of pieces there.
Lex Fridman (06:36.020)
There's the spinal cord, and there's the brain stem,
Lex Fridman (06:38.740)
and the cerebellum, and the different parts
Lex Fridman (06:40.240)
of the basal ganglia, and so on.
Jeff Hawkins (06:42.020)
In the old parts of the brain,
Lex Fridman (06:42.960)
you have the autonomic regulation,
Jeff Hawkins (06:44.800)
like breathing and heart rate.
Lex Fridman (06:46.280)
You have basic behaviors, so like walking and running
Jeff Hawkins (06:49.460)
are controlled by the old parts of the brain.
Lex Fridman (06:51.380)
All the emotional centers of the brain
Jeff Hawkins (06:53.060)
are in the old part of the brain,
Lex Fridman (06:53.940)
so when you feel anger or hungry, lust,
Jeff Hawkins (06:55.380)
or things like that, those are all
Lex Fridman (06:56.500)
in the old parts of the brain.
Lex Fridman (06:57.940)
And we associate with the neocortex
Lex Fridman (07:02.180)
all the things we think about as sort of
Jeff Hawkins (07:04.060)
high level perception and cognitive functions,
Lex Fridman (07:08.100)
anything from seeing and hearing and touching things
Jeff Hawkins (07:12.240)
to language to mathematics and engineering
Lex Fridman (07:15.140)
and science and so on.
Jeff Hawkins (07:16.940)
Those are all associated with the neocortex,
Lex Fridman (07:19.760)
and they're certainly correlated.
Jeff Hawkins (07:21.760)
Our abilities in those regards are correlated
Lex Fridman (07:23.980)
with the relative size of our neocortex
Jeff Hawkins (07:25.820)
compared to other mammals.
Lex Fridman (07:27.940)
So that's like the rough division,
Lex Fridman (07:30.520)
and you obviously can't understand
Lex Fridman (07:32.740)
the neocortex completely isolated,
Lex Fridman (07:35.160)
but you can understand a lot of it
Lex Fridman (07:37.020)
with just a few interfaces to the old parts of the brain,
Lex Fridman (07:40.340)
and so it gives you a system to study.
Lex Fridman (07:44.980)
The other remarkable thing about the neocortex,
Jeff Hawkins (07:48.020)
compared to the old parts of the brain,
Lex Fridman (07:49.880)
is the neocortex is extremely uniform.
Jeff Hawkins (07:52.900)
It's not visibly or anatomically,
Lex Fridman (07:57.060)
it's very, I always like to say
Jeff Hawkins (07:59.460)
it's like the size of a dinner napkin,
Lex Fridman (08:01.300)
about two and a half millimeters thick,
Lex Fridman (08:03.740)
and it looks remarkably the same everywhere.
Lex Fridman (08:05.980)
Everywhere you look in that two and a half millimeters
Jeff Hawkins (08:07.900)
is this detailed architecture,
Lex Fridman (08:10.060)
and it looks remarkably the same everywhere,
Lex Fridman (08:11.580)
and that's across species.
Lex Fridman (08:12.620)
A mouse versus a cat and a dog and a human.
Jeff Hawkins (08:15.380)
Where if you look at the old parts of the brain,
Lex Fridman (08:17.060)
there's lots of little pieces do specific things.
Lex Fridman (08:19.620)
So it's like the old parts of our brain evolved,
Lex Fridman (08:22.060)
like this is the part that controls heart rate,
Lex Fridman (08:23.660)
and this is the part that controls this,
Lex Fridman (08:24.860)
and this is this kind of thing,
Lex Fridman (08:25.780)
and that's this kind of thing,
Lex Fridman (08:27.180)
and these evolved for eons a long, long time,
Lex Fridman (08:30.100)
and they have their specific functions,
Lex Fridman (08:31.580)
and all of a sudden mammals come along,
Lex Fridman (08:33.220)
and they got this thing called the neocortex,
Lex Fridman (08:35.180)
and it got large by just replicating the same thing
Jeff Hawkins (08:38.140)
over and over and over again.
Lex Fridman (08:39.420)
This is like, wow, this is incredible.
Lex Fridman (08:42.660)
So all the evidence we have,
Lex Fridman (08:46.260)
and this is an idea that was first articulated
Jeff Hawkins (08:50.020)
in a very cogent and beautiful argument
Lex Fridman (08:52.020)
by a guy named Vernon Malcastle in 1978, I think it was,
Jeff Hawkins (08:56.820)
that the neocortex all works on the same principle.
Lex Fridman (09:01.580)
So language, hearing, touch, vision, engineering,
Jeff Hawkins (09:05.260)
all these things are basically underlying,
Lex Fridman (09:06.980)
are all built on the same computational substrate.
Jeff Hawkins (09:10.340)
They're really all the same problem.
Lex Fridman (09:11.820)
So the low level of the building blocks all look similar.
Jeff Hawkins (09:14.860)
Yeah, and they're not even that low level.
Lex Fridman (09:16.300)
We're not talking about like neurons.
Jeff Hawkins (09:17.900)
We're talking about this very complex circuit
Lex Fridman (09:19.940)
that exists throughout the neocortex.
Jeff Hawkins (09:21.420)
It's remarkably similar.
Lex Fridman (09:23.500)
It's like, yes, you see variations of it here and there,
Jeff Hawkins (09:26.580)
more of the cell, less and less, and so on.
Lex Fridman (09:29.620)
But what Malcastle argued was, he says,
Jeff Hawkins (09:32.700)
you know, if you take a section of neocortex,
Lex Fridman (09:35.580)
why is one a visual area and one is a auditory area?
Jeff Hawkins (09:38.580)
Or why is, and his answer was,
Lex Fridman (09:41.180)
it's because one is connected to eyes
Lex Fridman (09:43.180)
and one is connected to ears.
Lex Fridman (09:45.380)
Literally, you mean just it's most closest
Jeff Hawkins (09:47.820)
in terms of number of connections
Lex Fridman (09:49.020)
to the sensor. Literally, literally,
Jeff Hawkins (09:50.900)
if you took the optic nerve and attached it
Lex Fridman (09:53.780)
to a different part of the neocortex,
Jeff Hawkins (09:55.300)
that part would become a visual region.
Lex Fridman (09:57.940)
This actually, this experiment was actually done
Jeff Hawkins (10:00.380)
by Merkankasur in developing, I think it was lemurs,
Lex Fridman (10:04.980)
I can't remember what it was, some animal.
Lex Fridman (10:06.700)
And there's a lot of evidence to this.
Lex Fridman (10:08.540)
You know, if you take a blind person,
Jeff Hawkins (10:09.940)
a person who's born blind at birth,
Lex Fridman (10:12.180)
they're born with a visual neocortex.
Jeff Hawkins (10:15.420)
It doesn't, may not get any input from the eyes
Lex Fridman (10:18.260)
because of some congenital defect or something.
Lex Fridman (10:21.260)
And that region becomes, does something else.
Lex Fridman (10:24.700)
It picks up another task.
Jeff Hawkins (10:27.020)
So, and it's, so it's this very complex thing.
Lex Fridman (10:32.300)
It's not like, oh, they're all built on neurons.
Jeff Hawkins (10:33.740)
No, they're all built in this very complex circuit
Lex Fridman (10:36.460)
and somehow that circuit underlies everything.
Lex Fridman (10:40.300)
And so this is the, it's called
Lex Fridman (10:43.580)
the common cortical algorithm, if you will.
Jeff Hawkins (10:45.900)
Some scientists just find it hard to believe
Lex Fridman (10:47.980)
and they just, I can't believe that's true,
Lex Fridman (10:50.060)
but the evidence is overwhelming in this case.
Lex Fridman (10:52.100)
And so a large part of what it means
Jeff Hawkins (10:54.340)
to figure out how the brain creates intelligence
Lex Fridman (10:56.420)
and what is intelligence in the brain
Jeff Hawkins (10:59.860)
is to understand what that circuit does.
Lex Fridman (11:02.020)
If you can figure out what that circuit does,
Jeff Hawkins (11:05.020)
as amazing as it is, then you can,
Lex Fridman (11:06.940)
then you understand what all these
Jeff Hawkins (11:08.620)
other cognitive functions are.
Lex Fridman (11:10.500)
So if you were to sort of put neocortex
Jeff Hawkins (11:13.300)
outside of your book on intelligence,
Lex Fridman (11:15.140)
you look, if you wrote a giant tome, a textbook
Jeff Hawkins (11:18.020)
on the neocortex, and you look maybe
Lex Fridman (11:21.980)
a couple of centuries from now,
Lex Fridman (11:23.740)
how much of what we know now would still be accurate
Lex Fridman (11:26.500)
two centuries from now?
Lex Fridman (11:27.660)
So how close are we in terms of understanding?
Lex Fridman (11:30.820)
I have to speak from my own particular experience here.
Lex Fridman (11:32.980)
So I run a small research lab here.
Lex Fridman (11:35.860)
It's like any other research lab.
Jeff Hawkins (11:38.020)
I'm sort of the principal investigator.
Lex Fridman (11:39.420)
There's actually two of us
Lex Fridman (11:40.260)
and there's a bunch of other people.
Lex Fridman (11:42.540)
And this is what we do.
Jeff Hawkins (11:43.820)
We study the neocortex and we publish our results
Lex Fridman (11:46.060)
and so on.
Lex Fridman (11:46.900)
So about three years ago,
Lex Fridman (11:49.820)
we had a real breakthrough in this field.
Jeff Hawkins (11:52.460)
Just tremendous breakthrough.
Lex Fridman (11:53.300)
We've now published, I think, three papers on it.
Lex Fridman (11:56.500)
And so I have a pretty good understanding
Lex Fridman (12:00.180)
of all the pieces and what we're missing.
Jeff Hawkins (12:02.300)
I would say that almost all the empirical data
Lex Fridman (12:06.260)
we've collected about the brain, which is enormous.
Jeff Hawkins (12:08.540)
If you don't know the neuroscience literature,
Lex Fridman (12:10.340)
it's just incredibly big.
Lex Fridman (12:13.980)
And it's, for the most part, all correct.
Lex Fridman (12:16.860)
It's facts and experimental results and measurements
Lex Fridman (12:21.660)
and all kinds of stuff.
Lex Fridman (12:22.980)
But none of that has been really assimilated
Jeff Hawkins (12:25.860)
into a theoretical framework.
Lex Fridman (12:27.900)
It's data without, in the language of Thomas Kuhn,
Jeff Hawkins (12:32.300)
the historian, would be a sort of a pre paradigm science.
Lex Fridman (12:35.340)
Lots of data, but no way to fit it together.
Jeff Hawkins (12:38.180)
I think almost all of that's correct.
Lex Fridman (12:39.540)
There's just gonna be some mistakes in there.
Lex Fridman (12:42.180)
And for the most part,
Lex Fridman (12:43.300)
there aren't really good cogent theories about it,
Lex Fridman (12:45.940)
how to put it together.
Lex Fridman (12:47.300)
It's not like we have two or three competing good theories,
Jeff Hawkins (12:50.060)
which ones are right and which ones are wrong.
Lex Fridman (12:51.540)
It's like, nah, people are just scratching their heads.
Jeff Hawkins (12:54.780)
Some people have given up
Lex Fridman (12:55.620)
on trying to figure out what the whole thing does.
Jeff Hawkins (12:57.620)
In fact, there's very, very few labs that we do
Lex Fridman (13:01.020)
that focus really on theory
Lex Fridman (13:03.340)
and all this unassimilated data and trying to explain it.
Lex Fridman (13:06.780)
So it's not like we've got it wrong.
Jeff Hawkins (13:08.940)
It's just that we haven't got it at all.
Lex Fridman (13:11.100)
So it's really, I would say, pretty early days
Jeff Hawkins (13:15.020)
in terms of understanding the fundamental theory's forces
Lex Fridman (13:19.060)
of the way our mind works.
Jeff Hawkins (13:20.220)
I don't think so.
Lex Fridman (13:21.500)
I would have said that's true five years ago.
Lex Fridman (13:25.340)
So as I said,
Lex Fridman (13:26.980)
we had some really big breakthroughs on this recently
Lex Fridman (13:29.300)
and we started publishing papers on this.
Lex Fridman (13:30.780)
So we'll get to that.
Lex Fridman (13:34.180)
But so I don't think it's,
Lex Fridman (13:35.940)
I'm an optimist and from where I sit today,
Jeff Hawkins (13:38.260)
most people would disagree with this,
Lex Fridman (13:39.420)
but from where I sit today, from what I know,
Jeff Hawkins (13:43.260)
it's not super early days anymore.
Lex Fridman (13:44.940)
We are, the way these things go
Lex Fridman (13:46.860)
is it's not a linear path, right?
Lex Fridman (13:48.180)
You don't just start accumulating
Lex Fridman (13:49.820)
and get better and better and better.
Lex Fridman (13:50.820)
No, all this stuff you've collected,
Jeff Hawkins (13:52.900)
none of it makes sense.
Lex Fridman (13:53.780)
All these different things are just sort of around.
Lex Fridman (13:55.580)
And then you're gonna have some breaking points
Lex Fridman (13:57.100)
where all of a sudden, oh my God, now we got it right.
Jeff Hawkins (13:59.420)
That's how it goes in science.
Lex Fridman (14:01.100)
And I personally feel like we passed that little thing
Jeff Hawkins (14:04.460)
about a couple of years ago,
Lex Fridman (14:06.300)
all that big thing a couple of years ago.
Lex Fridman (14:07.580)
So we can talk about that.
Lex Fridman (14:09.620)
Time will tell if I'm right,
Lex Fridman (14:11.020)
but I feel very confident about it.
Lex Fridman (14:12.660)
That's why I'm willing to say it on tape like this.
Jeff Hawkins (14:15.220)
At least very optimistic.
Lex Fridman (14:18.060)
So let's, before those few years ago,
Jeff Hawkins (14:20.220)
let's take a step back to HTM,
Lex Fridman (14:23.260)
the hierarchical temporal memory theory,
Jeff Hawkins (14:26.020)
which you first proposed on intelligence
Lex Fridman (14:27.580)
and went through a few different generations.
Lex Fridman (14:29.340)
Can you describe what it is,
Lex Fridman (14:31.300)
how it evolved through the three generations
Lex Fridman (14:33.740)
since you first put it on paper?
Lex Fridman (14:35.460)
Yeah, so one of the things that neuroscientists
Jeff Hawkins (14:39.340)
just sort of missed for many, many years,
Lex Fridman (14:42.980)
and especially people who were thinking about theory,
Jeff Hawkins (14:45.820)
was the nature of time in the brain.
Lex Fridman (14:49.100)
Brains process information through time.
Jeff Hawkins (14:51.700)
The information coming into the brain
Lex Fridman (14:52.900)
is constantly changing.
Jeff Hawkins (14:55.220)
The patterns from my speech right now,
Lex Fridman (14:57.620)
if you were listening to it at normal speed,
Jeff Hawkins (15:00.140)
would be changing on your ears
Lex Fridman (15:01.500)
about every 10 milliseconds or so, you'd have a change.
Jeff Hawkins (15:04.100)
This constant flow, when you look at the world,
Lex Fridman (15:06.740)
your eyes are moving constantly,
Jeff Hawkins (15:08.220)
three to five times a second,
Lex Fridman (15:09.700)
and the input's completely changing.
Jeff Hawkins (15:11.380)
If I were to touch something like a coffee cup,
Lex Fridman (15:13.500)
as I move my fingers, the input changes.
Lex Fridman (15:15.220)
So this idea that the brain works on time changing patterns
Lex Fridman (15:19.500)
is almost completely, or was almost completely missing
Jeff Hawkins (15:22.340)
from a lot of the basic theories,
Lex Fridman (15:23.620)
like fears of vision and so on.
Jeff Hawkins (15:25.020)
It's like, oh no, we're gonna put this image
Lex Fridman (15:26.860)
in front of you and flash it and say, what is it?
Lex Fridman (15:29.580)
Convolutional neural networks work that way today, right?
Lex Fridman (15:32.180)
Classify this picture.
Lex Fridman (15:34.220)
But that's not what vision is like.
Lex Fridman (15:35.980)
Vision is this sort of crazy time based pattern
Jeff Hawkins (15:38.740)
that's going all over the place,
Lex Fridman (15:40.060)
and so is touch and so is hearing.
Lex Fridman (15:41.820)
So the first part of hierarchical temporal memory
Lex Fridman (15:43.780)
was the temporal part.
Jeff Hawkins (15:45.060)
It's to say, you won't understand the brain,
Lex Fridman (15:48.260)
nor will you understand intelligent machines
Jeff Hawkins (15:50.020)
unless you're dealing with time based patterns.
Lex Fridman (15:52.460)
The second thing was, the memory component of it was,
Jeff Hawkins (15:55.460)
is to say that we aren't just processing input,
Lex Fridman (16:00.300)
we learn a model of the world.
Lex Fridman (16:02.820)
And the memory stands for that model.
Lex Fridman (16:05.500)
The point of the brain, the part of the neocortex,
Jeff Hawkins (16:07.340)
it learns a model of the world.
Lex Fridman (16:08.500)
We have to store things, our experiences,
Jeff Hawkins (16:11.580)
in a form that leads to a model of the world.
Lex Fridman (16:14.220)
So we can move around the world,
Jeff Hawkins (16:15.700)
we can pick things up and do things and navigate
Lex Fridman (16:17.380)
and know how it's going on.
Lex Fridman (16:18.220)
So that's what the memory referred to.
Lex Fridman (16:19.980)
And many people just, they were thinking about
Jeff Hawkins (16:22.100)
like certain processes without memory at all.
Lex Fridman (16:25.140)
They're just like processing things.
Lex Fridman (16:26.740)
And then finally, the hierarchical component
Lex Fridman (16:29.020)
was a reflection to that the neocortex,
Jeff Hawkins (16:32.260)
although it's this uniform sheet of cells,
Lex Fridman (16:35.820)
different parts of it project to other parts,
Jeff Hawkins (16:37.580)
which project to other parts.
Lex Fridman (16:39.340)
And there is a sort of rough hierarchy in terms of that.
Lex Fridman (16:43.060)
So the hierarchical temporal memory is just saying,
Lex Fridman (16:45.980)
look, we should be thinking about the brain
Jeff Hawkins (16:47.700)
as time based, model memory based,
Lex Fridman (16:52.020)
and hierarchical processing.
Lex Fridman (16:54.780)
And that was a placeholder for a bunch of components
Lex Fridman (16:58.180)
that we would then plug into that.
Jeff Hawkins (17:00.860)
We still believe all those things I just said,
Lex Fridman (17:02.620)
but we now know so much more that I'm stopping to use
Jeff Hawkins (17:06.980)
the word hierarchical temporal memory yet
Lex Fridman (17:08.180)
because it's insufficient to capture the stuff we know.
Lex Fridman (17:11.340)
So again, it's not incorrect, but it's,
Lex Fridman (17:13.660)
I now know more and I would rather describe it
Jeff Hawkins (17:15.820)
more accurately.
Lex Fridman (17:16.820)
Yeah, so you're basically, we could think of HTM
Jeff Hawkins (17:20.340)
as emphasizing that there's three aspects of intelligence
Lex Fridman (17:24.780)
that are important to think about
Jeff Hawkins (17:25.900)
whatever the eventual theory it converges to.
Lex Fridman (17:28.900)
So in terms of time, how do you think of nature of time
Lex Fridman (17:32.460)
across different time scales?
Lex Fridman (17:33.860)
So you mentioned things changing,
Jeff Hawkins (17:36.820)
sensory inputs changing every 10, 20 minutes.
Lex Fridman (17:39.140)
What about every few minutes, every few months and years?
Jeff Hawkins (17:42.100)
Well, if you think about a neuroscience problem,
Lex Fridman (17:44.820)
the brain problem, neurons themselves can stay active
Jeff Hawkins (17:49.620)
for certain periods of time, parts of the brain
Lex Fridman (17:52.780)
where they stay active for minutes.
Jeff Hawkins (17:54.260)
You could hold a certain perception or an activity
Lex Fridman (17:59.460)
for a certain period of time,
Lex Fridman (18:01.580)
but most of them don't last that long.
Lex Fridman (18:04.820)
And so if you think about your thoughts
Jeff Hawkins (18:07.180)
are the activity of neurons,
Lex Fridman (18:09.180)
if you're gonna wanna involve something
Jeff Hawkins (18:10.580)
that happened a long time ago,
Lex Fridman (18:11.980)
even just this morning, for example,
Jeff Hawkins (18:14.420)
the neurons haven't been active throughout that time.
Lex Fridman (18:16.420)
So you have to store that.
Lex Fridman (18:17.860)
So if I ask you, what did you have for breakfast today?
Lex Fridman (18:20.860)
That is memory, that is you've built into your model
Jeff Hawkins (18:23.660)
the world now, you remember that.
Lex Fridman (18:24.860)
And that memory is in the synapses,
Jeff Hawkins (18:27.780)
is basically in the formation of synapses.
Lex Fridman (18:29.980)
And so you're sliding into what,
Jeff Hawkins (18:34.700)
you know, it's the different timescales.
Lex Fridman (18:36.660)
There's timescales of which we are like understanding
Jeff Hawkins (18:39.060)
my language and moving about and seeing things rapidly
Lex Fridman (18:41.260)
and over time, that's the timescales
Jeff Hawkins (18:42.540)
of activities of neurons.
Lex Fridman (18:44.220)
But if you wanna get in longer timescales,
Jeff Hawkins (18:46.140)
then it's more memory.
Lex Fridman (18:47.100)
And we have to invoke those memories to say,
Jeff Hawkins (18:49.460)
oh yes, well now I can remember what I had for breakfast
Lex Fridman (18:51.740)
because I stored that someplace.
Jeff Hawkins (18:54.180)
I may forget it tomorrow, but I'd store it for now.
Lex Fridman (18:58.140)
So does memory also need to have,
Lex Fridman (19:02.820)
so the hierarchical aspect of reality
Lex Fridman (19:06.180)
is not just about concepts, it's also about time?
Lex Fridman (19:08.780)
Do you think of it that way?
Lex Fridman (19:10.260)
Yeah, time is infused in everything.
Jeff Hawkins (19:12.820)
It's like you really can't separate it out.
Lex Fridman (19:15.540)
If I ask you, what is your, you know,
Lex Fridman (19:18.700)
how's the brain learn a model of this coffee cup here?
Lex Fridman (19:21.340)
I have a coffee cup and I'm at the coffee cup.
Jeff Hawkins (19:23.220)
I say, well, time is not an inherent property
Lex Fridman (19:25.980)
of the model I have of this cup,
Jeff Hawkins (19:28.540)
whether it's a visual model or a tactile model.
Lex Fridman (19:31.460)
I can sense it through time,
Lex Fridman (19:32.580)
but the model itself doesn't really have much time.
Lex Fridman (19:34.900)
If I asked you, if I said,
Lex Fridman (19:36.420)
well, what is the model of my cell phone?
Lex Fridman (19:38.980)
My brain has learned a model of the cell phone.
Lex Fridman (19:40.740)
So if you have a smartphone like this,
Lex Fridman (19:43.380)
and I said, well, this has time aspects to it.
Jeff Hawkins (19:45.700)
I have expectations when I turn it on,
Lex Fridman (19:48.040)
what's gonna happen, what or how long it's gonna take
Jeff Hawkins (19:50.460)
to do certain things, if I bring up an app,
Lex Fridman (19:52.860)
what sequences, and so I have,
Lex Fridman (19:54.540)
and it's like melodies in the world, you know?
Lex Fridman (19:57.260)
Melody has a sense of time.
Lex Fridman (19:58.540)
So many things in the world move and act,
Lex Fridman (1:00:00.380)
one way to think about it,
Lex Fridman (1:00:01.740)
is we have all these models of the world, okay?
Lex Fridman (1:00:04.740)
And we model everything.
Lex Fridman (1:00:06.140)
And as I said earlier, I kind of snuck it in there,
Lex Fridman (1:00:08.860)
our models are actually, we build composite structure.
Lex Fridman (1:00:12.500)
So every object is composed of other objects,
Lex Fridman (1:00:15.260)
which are composed of other objects,
Lex Fridman (1:00:16.420)
and they become members of other objects.
Lex Fridman (1:00:18.700)
So this room has chairs and a table and a room
Lex Fridman (1:00:20.700)
and walls and so on.
Lex Fridman (1:00:21.620)
Now we can just arrange these things in a certain way
Lex Fridman (1:00:24.300)
and go, oh, that's the nomenclature conference room.
Lex Fridman (1:00:26.580)
So, and what we do is when we go around the world
Lex Fridman (1:00:31.260)
and we experience the world,
Lex Fridman (1:00:33.620)
by walking into a room, for example,
Jeff Hawkins (1:00:35.740)
the first thing I do is I can say,
Lex Fridman (1:00:36.780)
oh, I'm in this room, do I recognize the room?
Jeff Hawkins (1:00:38.660)
Then I can say, oh, look, there's a table here.
Lex Fridman (1:00:41.900)
And by attending to the table,
Jeff Hawkins (1:00:43.460)
I'm then assigning this table in the context of the room.
Lex Fridman (1:00:45.620)
Then I can say, oh, on the table, there's a coffee cup.
Jeff Hawkins (1:00:48.100)
Oh, and on the table, there's a logo.
Lex Fridman (1:00:49.740)
And in the logo, there's the word Nementa.
Jeff Hawkins (1:00:51.260)
Oh, and look in the logo, there's the letter E.
Lex Fridman (1:00:53.420)
Oh, and look, it has an unusual serif.
Lex Fridman (1:00:55.740)
And it doesn't actually, but I pretended to serif.
Lex Fridman (1:00:59.660)
So the point is your attention is kind of drilling
Jeff Hawkins (1:01:03.860)
deep in and out of these nested structures.
Lex Fridman (1:01:07.460)
And I can pop back up and I can pop back down.
Jeff Hawkins (1:01:09.340)
I can pop back up and I can pop back down.
Lex Fridman (1:01:10.900)
So when I attend to the coffee cup,
Jeff Hawkins (1:01:13.220)
I haven't lost the context of everything else,
Lex Fridman (1:01:15.660)
but it's sort of, there's this sort of nested structure.
Lex Fridman (1:01:18.900)
So the attention filters the reference frame information
Lex Fridman (1:01:22.980)
for that particular period of time?
Jeff Hawkins (1:01:24.420)
Yes, it basically, moment to moment,
Lex Fridman (1:01:26.620)
you attend the sub components,
Lex Fridman (1:01:28.420)
and then you can attend the sub components
Lex Fridman (1:01:29.740)
to sub components.
Lex Fridman (1:01:30.580)
And you can move up and down.
Lex Fridman (1:01:31.420)
You can move up and down.
Jeff Hawkins (1:01:32.340)
We do that all the time.
Lex Fridman (1:01:33.180)
You're not even, now that I'm aware of it,
Jeff Hawkins (1:01:35.580)
I'm very conscious of it.
Lex Fridman (1:01:36.700)
But until, but most people don't even think about this.
Jeff Hawkins (1:01:39.980)
You just walk in a room and you don't say,
Lex Fridman (1:01:41.700)
oh, I looked at the chair and I looked at the board
Lex Fridman (1:01:43.500)
and looked at that word on the board
Lex Fridman (1:01:44.620)
and I looked over here, what's going on, right?
Lex Fridman (1:01:47.100)
So what percent of your day are you deeply aware of this?
Lex Fridman (1:01:50.020)
And what part can you actually relax and just be Jeff?
Lex Fridman (1:01:52.860)
Me personally, like my personal day?
Lex Fridman (1:01:54.460)
Yeah.
Jeff Hawkins (1:01:55.540)
Unfortunately, I'm afflicted with too much of the former.
Lex Fridman (1:02:01.340)
Well, unfortunately or unfortunately.
Jeff Hawkins (1:02:02.820)
Yeah.
Lex Fridman (1:02:03.660)
You don't think it's useful?
Jeff Hawkins (1:02:04.580)
Oh, it is useful, totally useful.
Lex Fridman (1:02:06.820)
I think about this stuff almost all the time.
Lex Fridman (1:02:09.180)
And one of my primary ways of thinking
Lex Fridman (1:02:12.540)
is when I'm in sleep at night,
Jeff Hawkins (1:02:13.860)
I always wake up in the middle of the night.
Lex Fridman (1:02:15.860)
And then I stay awake for at least an hour
Jeff Hawkins (1:02:17.860)
with my eyes shut in sort of a half sleep state
Lex Fridman (1:02:20.700)
thinking about these things.
Jeff Hawkins (1:02:21.660)
I come up with answers to problems very often
Lex Fridman (1:02:23.700)
in that sort of half sleeping state.
Jeff Hawkins (1:02:25.660)
I think about it on my bike ride, I think about it on walks.
Lex Fridman (1:02:27.460)
I'm just constantly thinking about this.
Jeff Hawkins (1:02:28.780)
I have to almost schedule time
Lex Fridman (1:02:32.420)
to not think about this stuff
Jeff Hawkins (1:02:34.100)
because it's very, it's mentally taxing.
Lex Fridman (1:02:37.820)
Are you, when you're thinking about this stuff,
Jeff Hawkins (1:02:39.780)
are you thinking introspectively,
Lex Fridman (1:02:41.220)
like almost taking a step outside of yourself
Lex Fridman (1:02:43.700)
and trying to figure out what is your mind doing right now?
Lex Fridman (1:02:45.660)
I do that all the time, but that's not all I do.
Jeff Hawkins (1:02:49.060)
I'm constantly observing myself.
Lex Fridman (1:02:50.780)
So as soon as I started thinking about grid cells,
Jeff Hawkins (1:02:53.060)
for example, and getting into that,
Lex Fridman (1:02:55.260)
I started saying, oh, well, grid cells
Jeff Hawkins (1:02:56.780)
can have my place of sense in the world.
Lex Fridman (1:02:58.380)
That's where you know where you are.
Lex Fridman (1:02:59.660)
And it's interesting, we always have a sense
Lex Fridman (1:03:01.380)
of where we are unless we're lost.
Lex Fridman (1:03:03.020)
And so I started at night when I got up
Lex Fridman (1:03:04.740)
to go to the bathroom, I would start trying to do it
Jeff Hawkins (1:03:06.980)
completely with my eyes closed all the time.
Lex Fridman (1:03:08.500)
And I would test my sense of grid cells.
Jeff Hawkins (1:03:10.060)
I would walk five feet and say, okay, I think I'm here.
Lex Fridman (1:03:13.700)
Am I really there?
Lex Fridman (1:03:14.540)
What's my error?
Lex Fridman (1:03:15.460)
And then I would calculate my error again
Lex Fridman (1:03:16.780)
and see how the errors could accumulate.
Lex Fridman (1:03:17.940)
So even something as simple as getting up
Jeff Hawkins (1:03:19.460)
in the middle of the night to go to the bathroom,
Lex Fridman (1:03:20.420)
I'm testing these theories out.
Jeff Hawkins (1:03:22.620)
It's kind of fun.
Lex Fridman (1:03:23.460)
I mean, the coffee cup is an example of that too.
Lex Fridman (1:03:25.580)
So I find that these sort of everyday introspections
Lex Fridman (1:03:30.380)
are actually quite helpful.
Jeff Hawkins (1:03:32.820)
It doesn't mean you can ignore the science.
Lex Fridman (1:03:34.860)
I mean, I spend hours every day
Jeff Hawkins (1:03:37.060)
reading ridiculously complex papers.
Lex Fridman (1:03:40.180)
That's not nearly as much fun,
Lex Fridman (1:03:41.740)
but you have to sort of build up those constraints
Lex Fridman (1:03:44.580)
and the knowledge about the field and who's doing what
Lex Fridman (1:03:46.860)
and what exactly they think is happening here.
Lex Fridman (1:03:48.860)
And then you can sit back and say,
Jeff Hawkins (1:03:50.060)
okay, let's try to piece this all together.
Lex Fridman (1:03:53.380)
Let's come up with some, I'm very,
Jeff Hawkins (1:03:56.020)
in this group here, people, they know they do,
Lex Fridman (1:03:58.460)
I do this all the time.
Jeff Hawkins (1:03:59.300)
I come in with these introspective ideas and say,
Lex Fridman (1:04:01.220)
well, have you ever thought about this?
Jeff Hawkins (1:04:02.380)
Now watch, well, let's all do this together.
Lex Fridman (1:04:04.700)
And it's helpful.
Jeff Hawkins (1:04:05.940)
It's not, as long as you don't,
Lex Fridman (1:04:09.580)
all you did was that, then you're just making up stuff.
Lex Fridman (1:04:12.340)
But if you're constraining it by the reality
Lex Fridman (1:04:14.780)
of the neuroscience, then it's really helpful.
Lex Fridman (1:04:17.820)
So let's talk a little bit about deep learning
Lex Fridman (1:04:20.180)
and the successes in the applied space of neural networks,
Jeff Hawkins (1:04:26.860)
ideas of training model on data
Lex Fridman (1:04:29.020)
and these simple computational units,
Jeff Hawkins (1:04:31.420)
artificial neurons that with backpropagation,
Lex Fridman (1:04:36.580)
statistical ways of being able to generalize
Jeff Hawkins (1:04:40.460)
from the training set onto data
Lex Fridman (1:04:42.780)
that's similar to that training set.
Lex Fridman (1:04:44.300)
So where do you think are the limitations
Lex Fridman (1:04:47.420)
of those approaches?
Lex Fridman (1:04:48.460)
What do you think are its strengths
Lex Fridman (1:04:50.380)
relative to your major efforts
Lex Fridman (1:04:52.180)
of constructing a theory of human intelligence?
Lex Fridman (1:04:56.020)
Well, I'm not an expert in this field.
Jeff Hawkins (1:04:57.820)
I'm somewhat knowledgeable.
Lex Fridman (1:04:59.140)
So, but I'm not.
Jeff Hawkins (1:04:59.980)
Some of it is in just your intuition.
Lex Fridman (1:05:01.620)
What are your?
Jeff Hawkins (1:05:02.460)
Well, I have a little bit more than intuition,
Lex Fridman (1:05:03.860)
but I just want to say like,
Jeff Hawkins (1:05:05.420)
you know, one of the things that you asked me,
Lex Fridman (1:05:07.660)
do I spend all my time thinking about neuroscience?
Jeff Hawkins (1:05:09.220)
I do.
Lex Fridman (1:05:10.060)
That's to the exclusion of thinking about things
Jeff Hawkins (1:05:11.340)
like convolutional neural networks.
Lex Fridman (1:05:13.660)
But I try to stay current.
Lex Fridman (1:05:15.260)
So look, I think it's great, the progress they've made.
Lex Fridman (1:05:17.860)
It's fantastic.
Lex Fridman (1:05:18.780)
And as I mentioned earlier,
Lex Fridman (1:05:19.860)
it's very highly useful for many things.
Jeff Hawkins (1:05:22.940)
The models that we have today are actually derived
Lex Fridman (1:05:26.140)
from a lot of neuroscience principles.
Jeff Hawkins (1:05:28.220)
There are distributed processing systems
Lex Fridman (1:05:30.020)
and distributed memory systems,
Lex Fridman (1:05:31.260)
and that's how the brain works.
Lex Fridman (1:05:33.260)
They use things that we might call them neurons,
Lex Fridman (1:05:35.900)
but they're really not neurons at all.
Lex Fridman (1:05:37.020)
So we can just, they're not really neurons.
Lex Fridman (1:05:39.220)
So they're distributed processing systems.
Lex Fridman (1:05:41.220)
And that nature of hierarchy,
Jeff Hawkins (1:05:44.700)
that came also from neuroscience.
Lex Fridman (1:05:47.140)
And so there's a lot of things,
Jeff Hawkins (1:05:48.220)
the learning rules, basically,
Lex Fridman (1:05:49.780)
not back prop, but other, you know,
Jeff Hawkins (1:05:51.140)
sort of heavy on top of that.
Lex Fridman (1:05:52.540)
I'd be curious to say they're not neurons at all.
Lex Fridman (1:05:55.020)
Can you describe in which way?
Lex Fridman (1:05:56.180)
I mean, some of it is obvious,
Lex Fridman (1:05:57.700)
but I'd be curious if you have specific ways
Lex Fridman (1:06:00.380)
in which you think are the biggest differences.
Jeff Hawkins (1:06:02.820)
Yeah, we had a paper in 2016 called
Lex Fridman (1:06:04.940)
Why Neurons Have Thousands of Synapses.
Lex Fridman (1:06:06.940)
And if you read that paper,
Lex Fridman (1:06:09.460)
you'll know what I'm talking about here.
Jeff Hawkins (1:06:11.420)
A real neuron in the brain is a complex thing.
Lex Fridman (1:06:14.460)
And let's just start with the synapses on it,
Jeff Hawkins (1:06:17.180)
which is a connection between neurons.
Lex Fridman (1:06:19.020)
Real neurons can have everywhere
Jeff Hawkins (1:06:20.700)
from five to 30,000 synapses on them.
Lex Fridman (1:06:25.460)
The ones near the cell body,
Jeff Hawkins (1:06:27.220)
the ones that are close to the soma of the cell body,
Lex Fridman (1:06:30.420)
those are like the ones that people model
Jeff Hawkins (1:06:32.100)
in artificial neurons.
Lex Fridman (1:06:33.740)
There is a few hundred of those.
Jeff Hawkins (1:06:35.060)
Maybe they can affect the cell.
Lex Fridman (1:06:37.100)
They can make the cell become active.
Jeff Hawkins (1:06:39.700)
95% of the synapses can't do that.
Lex Fridman (1:06:43.540)
They're too far away.
Lex Fridman (1:06:44.580)
So if you activate one of those synapses,
Lex Fridman (1:06:45.980)
it just doesn't affect the cell body enough
Jeff Hawkins (1:06:47.860)
to make any difference.
Lex Fridman (1:06:48.860)
Any one of them individually.
Jeff Hawkins (1:06:50.100)
Any one of them individually,
Lex Fridman (1:06:50.940)
or even if you do a mass of them.
Lex Fridman (1:06:54.060)
What real neurons do is the following.
Lex Fridman (1:06:57.420)
If you activate or you get 10 to 20 of them
Jeff Hawkins (1:07:03.500)
active at the same time,
Lex Fridman (1:07:04.460)
meaning they're all receiving an input at the same time,
Lex Fridman (1:07:06.660)
and those 10 to 20 synapses or 40 synapses
Lex Fridman (1:07:09.100)
within a very short distance on the dendrite,
Jeff Hawkins (1:07:11.340)
like 40 microns, a very small area.
Lex Fridman (1:07:13.300)
So if you activate a bunch of these
Jeff Hawkins (1:07:14.580)
right next to each other at some distant place,
Lex Fridman (1:07:17.580)
what happens is it creates
Jeff Hawkins (1:07:19.300)
what's called the dendritic spike.
Lex Fridman (1:07:21.300)
And the dendritic spike travels through the dendrites
Lex Fridman (1:07:24.540)
and can reach the soma or the cell body.
Lex Fridman (1:07:27.820)
Now, when it gets there, it changes the voltage,
Jeff Hawkins (1:07:31.260)
which is sort of like gonna make the cell fire,
Lex Fridman (1:07:33.580)
but never enough to make the cell fire.
Jeff Hawkins (1:07:36.060)
It's sort of what we call, it says we depolarize the cell,
Lex Fridman (1:07:38.500)
you raise the voltage a little bit,
Lex Fridman (1:07:39.580)
but not enough to do anything.
Lex Fridman (1:07:41.620)
It's like, well, what good is that?
Lex Fridman (1:07:42.580)
And then it goes back down again.
Lex Fridman (1:07:44.460)
So we propose a theory,
Jeff Hawkins (1:07:47.780)
which I'm very confident in basics are,
Lex Fridman (1:07:50.500)
is that what's happening there is
Jeff Hawkins (1:07:52.780)
those 95% of the synapses are recognizing
Lex Fridman (1:07:55.860)
dozens to hundreds of unique patterns.
Jeff Hawkins (1:07:58.460)
They can write about 10, 20 synapses at a time,
Lex Fridman (1:08:02.060)
and they're acting like predictions.
Lex Fridman (1:08:04.460)
So the neuron actually is a predictive engine on its own.
Lex Fridman (1:08:07.620)
It can fire when it gets enough,
Lex Fridman (1:08:09.700)
what they call proximal input
Lex Fridman (1:08:10.900)
from those ones near the cell fire,
Lex Fridman (1:08:11.980)
but it can get ready to fire from dozens to hundreds
Lex Fridman (1:08:15.460)
of patterns that it recognizes from the other guys.
Lex Fridman (1:08:18.100)
And the advantage of this to the neuron
Lex Fridman (1:08:21.260)
is that when it actually does produce a spike
Jeff Hawkins (1:08:23.500)
in action potential,
Lex Fridman (1:08:24.780)
it does so slightly sooner than it would have otherwise.
Lex Fridman (1:08:27.700)
And so what could is slightly sooner?
Lex Fridman (1:08:29.740)
Well, the slightly sooner part is it,
Jeff Hawkins (1:08:31.820)
all the excitatory neurons in the brain
Lex Fridman (1:08:34.940)
are surrounded by these inhibitory neurons,
Lex Fridman (1:08:36.660)
and they're very fast, the inhibitory neurons,
Lex Fridman (1:08:38.980)
these basket cells.
Lex Fridman (1:08:40.420)
And if I get my spike out
Lex Fridman (1:08:42.580)
a little bit sooner than someone else,
Lex Fridman (1:08:44.220)
I inhibit all my neighbors around me, right?
Lex Fridman (1:08:47.020)
And what you end up with is a different representation.
Jeff Hawkins (1:08:49.740)
You end up with a reputation that matches your prediction.
Lex Fridman (1:08:52.060)
It's a sparser representation,
Jeff Hawkins (1:08:53.780)
meaning fewer neurons are active,
Lex Fridman (1:08:55.740)
but it's much more specific.
Lex Fridman (1:08:57.860)
And so we showed how networks of these neurons
Lex Fridman (1:09:00.300)
can do very sophisticated temporal prediction, basically.
Lex Fridman (1:09:04.180)
So this, summarize this,
Lex Fridman (1:09:07.020)
real neurons in the brain are time based prediction engines,
Lex Fridman (1:09:10.980)
and there's no concept of this at all
Lex Fridman (1:09:14.660)
in artificial, what we call point neurons.
Jeff Hawkins (1:09:18.100)
I don't think you can build a brain without them.
Lex Fridman (1:09:20.060)
I don't think you can build intelligence without them,
Jeff Hawkins (1:09:21.340)
because it's where a large part of the time comes from.
Lex Fridman (1:09:26.020)
These are predictive models, and the time is,
Jeff Hawkins (1:09:29.060)
there's a prior and a prediction and an action,
Lex Fridman (1:09:32.220)
and it's inherent through every neuron in the neocortex.
Lex Fridman (1:09:34.940)
So I would say that point neurons sort of model
Lex Fridman (1:09:37.740)
a piece of that, and not very well at that either.
Lex Fridman (1:09:40.620)
But like for example, synapses are very unreliable,
Lex Fridman (1:09:46.060)
and you cannot assign any precision to them.
Lex Fridman (1:09:49.900)
So even one digit of precision is not possible.
Lex Fridman (1:09:52.460)
So the way real neurons work is they don't add these,
Jeff Hawkins (1:09:55.540)
they don't change these weights accurately
Lex Fridman (1:09:57.420)
like artificial neural networks do.
Jeff Hawkins (1:09:59.340)
They basically form new synapses,
Lex Fridman (1:10:01.020)
and so what you're trying to always do is
Jeff Hawkins (1:10:03.780)
detect the presence of some 10 to 20
Lex Fridman (1:10:06.540)
active synapses at the same time,
Jeff Hawkins (1:10:08.780)
as opposed, and they're almost binary.
Lex Fridman (1:10:11.300)
It's like, because you can't really represent
Jeff Hawkins (1:10:12.820)
anything much finer than that.
Lex Fridman (1:10:14.620)
So these are the kind of,
Lex Fridman (1:10:16.220)
and I think that's actually another essential component,
Lex Fridman (1:10:18.060)
because the brain works on sparse patterns,
Lex Fridman (1:10:20.940)
and all that mechanism is based on sparse patterns,
Lex Fridman (1:10:24.180)
and I don't actually think you could build real brains
Jeff Hawkins (1:10:26.620)
or machine intelligence without
Lex Fridman (1:10:29.100)
incorporating some of those ideas.
Jeff Hawkins (1:10:30.900)
It's hard to even think about the complexity
Lex Fridman (1:10:32.660)
that emerges from the fact that
Jeff Hawkins (1:10:34.420)
the timing of the firing matters in the brain,
Lex Fridman (1:10:37.140)
the fact that you form new synapses,
Lex Fridman (1:10:40.980)
and I mean, everything you just mentioned
Lex Fridman (1:10:44.020)
in the past couple minutes.
Jeff Hawkins (1:10:44.940)
Trust me, if you spend time on it,
Lex Fridman (1:10:46.540)
you can get your mind around it.
Jeff Hawkins (1:10:47.940)
It's not like, it's no longer a mystery to me.
Lex Fridman (1:10:49.860)
No, but sorry, as a function, in a mathematical way,
Lex Fridman (1:10:53.820)
can you start getting an intuition about
Lex Fridman (1:10:56.940)
what gets it excited, what not,
Lex Fridman (1:10:58.540)
and what kind of representation?
Lex Fridman (1:10:59.380)
Yeah, it's not as easy as,
Jeff Hawkins (1:11:02.580)
there's many other types of neural networks
Lex Fridman (1:11:04.660)
that are more amenable to pure analysis,
Jeff Hawkins (1:11:09.220)
especially very simple networks.
Lex Fridman (1:11:10.780)
Oh, I have four neurons, and they're doing this.
Jeff Hawkins (1:11:12.580)
Can we describe to them mathematically
Lex Fridman (1:11:14.500)
what they're doing type of thing?
Jeff Hawkins (1:11:16.300)
Even the complexity of convolutional neural networks today,
Lex Fridman (1:11:19.340)
it's sort of a mystery.
Jeff Hawkins (1:11:20.300)
They can't really describe the whole system.
Lex Fridman (1:11:22.500)
And so it's different.
Jeff Hawkins (1:11:24.780)
My colleague Subitai Ahmad, he did a nice paper on this.
Lex Fridman (1:11:31.500)
You can get all this stuff on our website
Jeff Hawkins (1:11:32.740)
if you're interested,
Lex Fridman (1:11:34.100)
talking about sort of the mathematical properties
Jeff Hawkins (1:11:36.180)
of sparse representations.
Lex Fridman (1:11:37.660)
And so what we can do is we can show mathematically,
Jeff Hawkins (1:11:40.620)
for example, why 10 to 20 synapses to recognize a pattern
Lex Fridman (1:11:44.940)
is the correct number, is the right number you'd wanna use.
Lex Fridman (1:11:47.740)
And by the way, that matches biology.
Lex Fridman (1:11:49.980)
We can show mathematically some of these concepts
Jeff Hawkins (1:11:53.900)
about the show why the brain is so robust
Lex Fridman (1:11:58.620)
to noise and error and fallout and so on.
Jeff Hawkins (1:12:01.020)
We can show that mathematically
Lex Fridman (1:12:02.260)
as well as empirically in simulations.
Lex Fridman (1:12:05.020)
But the system can't be analyzed completely.
Lex Fridman (1:12:07.860)
Any complex system can't, and so that's out of the realm.
Lex Fridman (1:12:11.980)
But there is mathematical benefits and intuitions
Lex Fridman (1:12:17.660)
that can be derived from mathematics.
Lex Fridman (1:12:19.460)
And we try to do that as well.
Lex Fridman (1:12:20.860)
Most of our papers have a section about that.
Lex Fridman (1:12:23.300)
So I think it's refreshing and useful for me
Lex Fridman (1:12:25.900)
to be talking to you about deep neural networks,
Jeff Hawkins (1:12:29.060)
because your intuition basically says
Lex Fridman (1:12:30.900)
that we can't achieve anything like intelligence
Jeff Hawkins (1:12:34.540)
with artificial neural networks.
Lex Fridman (1:12:35.940)
Well, not in the current form.
Jeff Hawkins (1:12:36.940)
Not in the current form.
Lex Fridman (1:12:37.780)
I'm sure we can do it in the ultimate form, sure.
Lex Fridman (1:12:40.180)
So let me dig into it
Lex Fridman (1:12:41.260)
and see what your thoughts are there a little bit.
Lex Fridman (1:12:43.300)
So I'm not sure if you read this little blog post
Lex Fridman (1:12:45.980)
called Bitter Lesson by Rich Sutton recently.
Jeff Hawkins (1:12:49.460)
He's a reinforcement learning pioneer.
Lex Fridman (1:12:51.660)
I'm not sure if you're familiar with him.
Jeff Hawkins (1:12:53.260)
His basic idea is that all the stuff we've done in AI
Lex Fridman (1:12:56.780)
in the past 70 years, he's one of the old school guys.
Jeff Hawkins (1:13:02.980)
The biggest lesson learned is that all the tricky things
Lex Fridman (1:13:06.860)
we've done, they benefit in the short term,
Lex Fridman (1:13:10.420)
but in the long term, what wins out
Lex Fridman (1:13:12.100)
is a simple general method that just relies on Moore's law,
Jeff Hawkins (1:13:16.700)
on computation getting faster and faster.
Lex Fridman (1:13:19.820)
This is what he's saying.
Jeff Hawkins (1:13:21.260)
This is what has worked up to now.
Lex Fridman (1:13:23.220)
This is what has worked up to now.
Jeff Hawkins (1:13:25.380)
If you're trying to build a system,
Lex Fridman (1:13:29.060)
if we're talking about,
Jeff Hawkins (1:13:30.060)
he's not concerned about intelligence.
Lex Fridman (1:13:31.420)
He's concerned about a system that works
Jeff Hawkins (1:13:34.420)
in terms of making predictions
Lex Fridman (1:13:36.500)
on applied narrow AI problems, right?
Jeff Hawkins (1:13:38.780)
That's what this discussion is about.
Lex Fridman (1:13:40.620)
That you just try to go as general as possible
Lex Fridman (1:13:44.220)
and wait years or decades for the computation
Lex Fridman (1:13:48.500)
to make it actually.
Jeff Hawkins (1:13:50.220)
Is he saying that as a criticism
Lex Fridman (1:13:51.700)
or is he saying this is a prescription
Lex Fridman (1:13:53.260)
of what we ought to be doing?
Lex Fridman (1:13:54.340)
Well, it's very difficult.
Jeff Hawkins (1:13:55.860)
He's saying this is what has worked
Lex Fridman (1:13:57.980)
and yes, a prescription, but it's a difficult prescription
Jeff Hawkins (1:14:00.340)
because it says all the fun things
Lex Fridman (1:14:02.380)
you guys are trying to do, we are trying to do.
Jeff Hawkins (1:14:05.820)
He's part of the community.
Lex Fridman (1:14:07.340)
He's saying it's only going to be short term gains.
Lex Fridman (1:14:10.780)
So this all leads up to a question, I guess,
Lex Fridman (1:14:13.780)
on artificial neural networks
Lex Fridman (1:14:15.580)
and maybe our own biological neural networks
Lex Fridman (1:14:19.060)
is do you think if we just scale things up significantly,
Lex Fridman (1:14:23.780)
so take these dumb artificial neurons,
Lex Fridman (1:14:27.180)
the point neurons, I like that term.
Jeff Hawkins (1:14:30.420)
If we just have a lot more of them,
Lex Fridman (1:14:33.260)
do you think some of the elements
Lex Fridman (1:14:34.540)
that we see in the brain may start emerging?
Lex Fridman (1:14:38.060)
No, I don't think so.
Jeff Hawkins (1:14:39.540)
We can do bigger problems of the same type.
Lex Fridman (1:14:43.420)
I mean, it's been pointed out by many people
Jeff Hawkins (1:14:45.260)
that today's convolutional neural networks
Lex Fridman (1:14:46.860)
aren't really much different
Jeff Hawkins (1:14:47.860)
than the ones we had quite a while ago.
Lex Fridman (1:14:50.580)
They're bigger and train more
Lex Fridman (1:14:51.820)
and we have more labeled data and so on.
Lex Fridman (1:14:56.300)
But I don't think you can get to the kind of things
Jeff Hawkins (1:14:58.580)
I know the brain can do and that we think about
Lex Fridman (1:15:01.380)
as intelligence by just scaling it up.
Lex Fridman (1:15:03.700)
So that may be, it's a good description
Lex Fridman (1:15:06.580)
of what's happened in the past,
Jeff Hawkins (1:15:07.660)
what's happened recently with the reemergence
Lex Fridman (1:15:09.940)
of artificial neural networks.
Jeff Hawkins (1:15:12.500)
It may be a good prescription
Lex Fridman (1:15:14.380)
for what's gonna happen in the short term.
Lex Fridman (1:15:17.580)
But I don't think that's the path.
Lex Fridman (1:15:19.180)
I've said that earlier.
Jeff Hawkins (1:15:20.860)
There's an alternate path.
Lex Fridman (1:15:21.700)
I should mention to you, by the way,
Jeff Hawkins (1:15:22.900)
that we've made sufficient progress
Lex Fridman (1:15:25.900)
on the whole cortical theory in the last few years
Jeff Hawkins (1:15:28.900)
that last year we decided to start actively pursuing
Lex Fridman (1:15:35.660)
how do we get these ideas embedded into machine learning?
Jeff Hawkins (1:15:40.100)
Well, that's, again, being led by my colleague,
Lex Fridman (1:15:41.860)
Subed Tariman, and he's more of a machine learning guy.
Jeff Hawkins (1:15:45.140)
I'm more of a neuroscience guy.
Lex Fridman (1:15:46.740)
So this is now, I wouldn't say our focus,
Lex Fridman (1:15:51.180)
but it is now an equal focus here
Lex Fridman (1:15:54.140)
because we need to proselytize what we've learned
Lex Fridman (1:15:58.220)
and we need to show how it's beneficial
Lex Fridman (1:16:01.460)
to the machine learning layer.
Lex Fridman (1:16:03.740)
So we're putting, we have a plan in place right now.
Lex Fridman (1:16:05.580)
In fact, we just did our first paper on this.
Jeff Hawkins (1:16:07.700)
I can tell you about that.
Lex Fridman (1:16:09.700)
But one of the reasons I wanna talk to you
Jeff Hawkins (1:16:11.380)
is because I'm trying to get more people
Lex Fridman (1:16:14.100)
in the machine learning community to say,
Jeff Hawkins (1:16:15.980)
I need to learn about this stuff.
Lex Fridman (1:16:17.140)
And maybe we should just think about this a bit more
Jeff Hawkins (1:16:19.380)
about what we've learned about the brain
Lex Fridman (1:16:20.860)
and what are those team at Nimenta, what have they done?
Lex Fridman (1:16:23.860)
Is that useful for us?
Lex Fridman (1:16:25.220)
Yeah, so is there elements of all the cortical theory
Jeff Hawkins (1:16:28.500)
that things we've been talking about
Lex Fridman (1:16:29.820)
that may be useful in the short term?
Jeff Hawkins (1:16:31.900)
Yes, in the short term, yes.
Lex Fridman (1:16:33.420)
This is the, sorry to interrupt,
Lex Fridman (1:16:34.780)
but the open question is,
Lex Fridman (1:16:37.740)
it certainly feels from my perspective
Jeff Hawkins (1:16:39.260)
that in the long term,
Lex Fridman (1:16:41.060)
some of the ideas we've been talking about
Jeff Hawkins (1:16:42.820)
will be extremely useful.
Lex Fridman (1:16:44.260)
The question is whether in the short term.
Jeff Hawkins (1:16:46.020)
Well, this is always what I would call
Lex Fridman (1:16:48.340)
the entrepreneur's dilemma.
Lex Fridman (1:16:50.620)
So you have this long term vision,
Lex Fridman (1:16:53.060)
oh, we're gonna all be driving electric cars
Jeff Hawkins (1:16:55.300)
or we're all gonna have computers
Lex Fridman (1:16:56.780)
or we're all gonna, whatever.
Lex Fridman (1:16:59.020)
And you're at some point in time and you say,
Lex Fridman (1:17:01.860)
I can see that long term vision,
Jeff Hawkins (1:17:02.980)
I'm sure it's gonna happen.
Lex Fridman (1:17:03.820)
How do I get there without killing myself?
Lex Fridman (1:17:05.780)
Without going out of business, right?
Lex Fridman (1:17:07.380)
That's the challenge.
Jeff Hawkins (1:17:08.740)
That's the dilemma.
Lex Fridman (1:17:09.580)
That's the really difficult thing to do.
Lex Fridman (1:17:11.100)
So we're facing that right now.
Lex Fridman (1:17:13.100)
So ideally what you'd wanna do
Jeff Hawkins (1:17:14.660)
is find some steps along the way
Lex Fridman (1:17:16.100)
that you can get there incrementally.
Jeff Hawkins (1:17:17.420)
You don't have to like throw it all out
Lex Fridman (1:17:19.180)
and start over again.
Jeff Hawkins (1:17:20.460)
The first thing that we've done
Lex Fridman (1:17:22.340)
is we focus on the sparse representations.
Lex Fridman (1:17:25.380)
So just in case you don't know what that means
Lex Fridman (1:17:28.420)
or some of the listeners don't know what that means,
Jeff Hawkins (1:17:31.220)
in the brain, if I have like 10,000 neurons,
Lex Fridman (1:17:34.100)
what you would see is maybe 2% of them active at a time.
Jeff Hawkins (1:17:36.980)
You don't see 50%, you don't see 30%,
Lex Fridman (1:17:39.540)
you might see 2%.
Lex Fridman (1:17:41.220)
And it's always like that.
Lex Fridman (1:17:42.660)
For any set of sensory inputs?
Jeff Hawkins (1:17:44.380)
It doesn't matter if anything,
Lex Fridman (1:17:45.340)
doesn't matter any part of the brain.
Lex Fridman (1:17:47.380)
But which neurons differs?
Lex Fridman (1:17:51.100)
Which neurons are active?
Jeff Hawkins (1:17:52.620)
Yeah, so let's say I take 10,000 neurons
Lex Fridman (1:17:55.380)
that are representing something.
Jeff Hawkins (1:17:56.300)
They're sitting there in a little block together.
Lex Fridman (1:17:57.940)
It's a teeny little block of neurons, 10,000 neurons.
Lex Fridman (1:18:00.060)
And they're representing a location,
Lex Fridman (1:18:01.620)
they're representing a cup,
Jeff Hawkins (1:18:02.500)
they're representing the input from my sensors.
Lex Fridman (1:18:04.060)
I don't know, it doesn't matter.
Jeff Hawkins (1:18:05.380)
It's representing something.
Lex Fridman (1:18:07.020)
The way the representations occur,
Jeff Hawkins (1:18:09.140)
it's always a sparse representation.
Lex Fridman (1:18:10.620)
Meaning it's a population code.
Lex Fridman (1:18:11.860)
So which 200 cells are active tells me what's going on.
Lex Fridman (1:18:14.980)
It's not, individual cells aren't that important at all.
Jeff Hawkins (1:18:18.060)
It's the population code that matters.
Lex Fridman (1:18:20.260)
And when you have sparse population codes,
Jeff Hawkins (1:18:23.140)
then all kinds of beautiful properties come out of them.
Lex Fridman (1:18:26.300)
So the brain uses sparse population codes.
Jeff Hawkins (1:18:28.100)
We've written and described these benefits
Lex Fridman (1:18:30.780)
in some of our papers.
Lex Fridman (1:18:32.420)
So they give this tremendous robustness to the systems.
Lex Fridman (1:18:37.660)
Brains are incredibly robust.
Jeff Hawkins (1:18:39.180)
Neurons are dying all the time and spasming
Lex Fridman (1:18:41.140)
and synapses are falling apart all the time.
Lex Fridman (1:18:43.940)
And it keeps working.
Lex Fridman (1:18:45.340)
So what Sibutai and Louise, one of our other engineers here
Jeff Hawkins (1:18:51.220)
have done, have shown they're introducing sparseness
Lex Fridman (1:18:55.740)
into convolutional neural networks.
Jeff Hawkins (1:18:56.860)
Now other people are thinking along these lines,
Lex Fridman (1:18:58.140)
but we're going about it in a more principled way, I think.
Lex Fridman (1:19:00.980)
And we're showing that if you enforce sparseness
Lex Fridman (1:19:04.100)
throughout these convolutional neural networks
Jeff Hawkins (1:19:07.340)
in both the act, which sort of,
Lex Fridman (1:19:09.660)
which neurons are active and the connections between them,
Jeff Hawkins (1:19:12.780)
that you get some very desirable properties.
Lex Fridman (1:19:15.660)
So one of the current hot topics in deep learning right now
Jeff Hawkins (1:19:18.860)
are these adversarial examples.
Lex Fridman (1:19:20.900)
So, you know, you give me any deep learning network
Lex Fridman (1:19:23.500)
and I can give you a picture that looks perfect
Lex Fridman (1:19:26.060)
and you're going to call it, you know,
Jeff Hawkins (1:19:27.100)
you're going to say the monkey is, you know, an airplane.
Lex Fridman (1:19:30.300)
So that's a problem.
Lex Fridman (1:19:32.540)
And DARPA just announced some big thing.
Lex Fridman (1:19:34.140)
They're trying to, you know, have some contest for this.
Lex Fridman (1:19:36.580)
But if you enforce sparse representations here,
Lex Fridman (1:19:40.180)
many of these problems go away.
Jeff Hawkins (1:19:41.500)
They're much more robust and they're not easy to fool.
Lex Fridman (1:19:44.940)
So we've already shown some of those results,
Jeff Hawkins (1:19:48.340)
just literally in January or February,
Lex Fridman (1:19:51.140)
just like last month we did that.
Lex Fridman (1:19:53.740)
And you can, I think it's on bioRxiv right now,
Lex Fridman (1:19:57.340)
or on iRxiv, you can read about it.
Lex Fridman (1:19:59.540)
But, so that's like a baby step, okay?
Lex Fridman (1:20:03.100)
That's taking something from the brain.
Jeff Hawkins (1:20:04.340)
We know about sparseness.
Lex Fridman (1:20:05.620)
We know why it's important.
Jeff Hawkins (1:20:06.500)
We know what it gives the brain.
Lex Fridman (1:20:08.060)
So let's try to enforce that onto this.
Lex Fridman (1:20:09.500)
What's your intuition why sparsity leads to robustness?
Lex Fridman (1:20:12.420)
Because it feels like it would be less robust.
Lex Fridman (1:20:15.060)
Why would you feel the rest robust to you?
Lex Fridman (1:20:17.260)
So it just feels like if the fewer neurons are involved,
Jeff Hawkins (1:20:24.380)
the more fragile the representation.
Lex Fridman (1:20:26.660)
But I didn't say there was lots of few neurons.
Jeff Hawkins (1:20:28.260)
I said, let's say 200.
Lex Fridman (1:20:29.860)
That's a lot.
Jeff Hawkins (1:20:31.020)
There's still a lot, it's just.
Lex Fridman (1:20:32.620)
So here's an intuition for it.
Jeff Hawkins (1:20:35.260)
This is a bit technical, so for engineers,
Lex Fridman (1:20:39.860)
machine learning people, this will be easy,
Lex Fridman (1:20:41.260)
but all the listeners, maybe not.
Lex Fridman (1:20:44.300)
If you're trying to classify something,
Jeff Hawkins (1:20:45.740)
you're trying to divide some very high dimensional space
Lex Fridman (1:20:48.380)
into different pieces, A and B.
Lex Fridman (1:20:50.380)
And you're trying to create some point where you say,
Lex Fridman (1:20:52.820)
all these points in this high dimensional space are A,
Lex Fridman (1:20:54.780)
and all these points in this high dimensional space are B.
Lex Fridman (1:20:57.580)
And if you have points that are close to that line,
Jeff Hawkins (1:21:01.980)
it's not very robust.
Lex Fridman (1:21:02.900)
It works for all the points you know about,
Lex Fridman (1:21:04.940)
but it's not very robust,
Lex Fridman (1:21:07.100)
because you can just move a little bit
Lex Fridman (1:21:08.260)
and you've crossed over the line.
Lex Fridman (1:21:10.300)
When you have sparse representations,
Jeff Hawkins (1:21:12.700)
imagine I pick, I'm gonna pick 200 cells active
Lex Fridman (1:21:16.060)
out of 10,000, okay?
Lex Fridman (1:21:19.260)
So I have 200 cells active.
Lex Fridman (1:21:20.340)
Now let's say I pick randomly another,
Jeff Hawkins (1:21:22.220)
a different representation, 200.
Lex Fridman (1:21:24.420)
The overlap between those is gonna be very small,
Jeff Hawkins (1:21:26.740)
just a few.
Lex Fridman (1:21:28.060)
I can pick millions of samples randomly of 200 neurons,
Lex Fridman (1:21:32.740)
and not one of them will overlap more than just a few.
Lex Fridman (1:21:36.980)
So one way to think about it is,
Jeff Hawkins (1:21:39.140)
if I wanna fool one of these representations
Lex Fridman (1:21:41.460)
to look like one of those other representations,
Jeff Hawkins (1:21:43.460)
I can't move just one cell, or two cells,
Lex Fridman (1:21:45.660)
or three cells, or four cells.
Jeff Hawkins (1:21:46.780)
I have to move 100 cells.
Lex Fridman (1:21:49.140)
And that makes them robust.
Jeff Hawkins (1:21:52.700)
In terms of further, so you mentioned sparsity.
Lex Fridman (1:21:56.180)
What would be the next thing?
Jeff Hawkins (1:21:57.260)
Yeah.
Lex Fridman (1:21:58.100)
Okay, so we have, we picked one.
Jeff Hawkins (1:22:00.460)
We don't know if it's gonna work well yet.
Lex Fridman (1:22:02.380)
So again, we're trying to come up with incremental ways
Jeff Hawkins (1:22:04.540)
to moving from brain theory to add pieces
Lex Fridman (1:22:07.860)
to machine learning, current machine learning world,
Lex Fridman (1:22:10.140)
and one step at a time.
Lex Fridman (1:22:12.260)
So the next thing we're gonna try to do
Jeff Hawkins (1:22:13.740)
is sort of incorporate some of the ideas
Lex Fridman (1:22:15.820)
of the thousand brains theory,
Jeff Hawkins (1:22:19.100)
that you have many, many models that are voting.
Lex Fridman (1:22:22.580)
Now that idea is not new.
Jeff Hawkins (1:22:23.700)
There's a mixture of models that's been around
Lex Fridman (1:22:25.300)
for a long time.
Lex Fridman (1:22:27.160)
But the way the brain does it is a little different.
Lex Fridman (1:22:29.740)
And the way it votes is different.
Lex Fridman (1:22:33.620)
And the kind of way it represents uncertainty
Lex Fridman (1:22:36.220)
is different.
Lex Fridman (1:22:37.180)
So we're just starting this work,
Lex Fridman (1:22:39.980)
but we're gonna try to see if we can sort of incorporate
Jeff Hawkins (1:22:42.280)
some of the principles of voting,
Lex Fridman (1:22:43.760)
or principles of the thousand brain theory.
Jeff Hawkins (1:22:45.940)
Like lots of simple models that talk to each other
Lex Fridman (1:22:49.420)
in a certain way.
Lex Fridman (1:22:53.940)
And can we build more machines, systems that learn faster
Lex Fridman (1:22:57.700)
and also, well mostly are multimodal
Lex Fridman (1:23:03.220)
and robust to multimodal type of issues.
Lex Fridman (1:23:07.500)
So one of the challenges there
Jeff Hawkins (1:23:09.580)
is the machine learning computer vision community
Lex Fridman (1:23:13.100)
has certain sets of benchmarks,
Jeff Hawkins (1:23:15.600)
sets of tests based on which they compete.
Lex Fridman (1:23:18.180)
And I would argue, especially from your perspective,
Jeff Hawkins (1:23:22.060)
that those benchmarks aren't that useful
Lex Fridman (1:23:24.660)
for testing the aspects that the brain is good at,
Jeff Hawkins (1:23:28.860)
or intelligence.
Lex Fridman (1:23:29.940)
They're not really testing intelligence.
Jeff Hawkins (1:23:31.300)
They're very fine.
Lex Fridman (1:23:32.980)
And it's been extremely useful
Jeff Hawkins (1:23:34.780)
for developing specific mathematical models,
Lex Fridman (1:23:37.420)
but it's not useful in the long term
Jeff Hawkins (1:23:40.420)
for creating intelligence.
Lex Fridman (1:23:41.680)
So you think you also have a role in proposing
Lex Fridman (1:23:44.660)
better tests?
Lex Fridman (1:23:47.020)
Yeah, this is a very,
Jeff Hawkins (1:23:48.460)
you've identified a very serious problem.
Lex Fridman (1:23:51.440)
First of all, the tests that they have
Jeff Hawkins (1:23:53.340)
are the tests that they want.
Lex Fridman (1:23:54.580)
Not the tests of the other things
Lex Fridman (1:23:55.860)
that we're trying to do, right?
Lex Fridman (1:23:58.740)
You know, what are the, so on.
Jeff Hawkins (1:24:01.700)
The second thing is sometimes these,
Lex Fridman (1:24:04.220)
to be competitive in these tests,
Jeff Hawkins (1:24:06.620)
you have to have huge data sets and huge computing power.
Lex Fridman (1:24:10.820)
And so, you know, and we don't have that here.
Jeff Hawkins (1:24:13.420)
We don't have it as well as other big teams
Lex Fridman (1:24:15.500)
that big companies do.
Lex Fridman (1:24:18.700)
So there's numerous issues there.
Lex Fridman (1:24:20.900)
You know, we come out, you know,
Jeff Hawkins (1:24:22.420)
where our approach to this is all based on,
Lex Fridman (1:24:24.260)
in some sense, you might argue, elegance.
Jeff Hawkins (1:24:26.100)
We're coming at it from like a theoretical base
Lex Fridman (1:24:27.780)
that we think, oh my God, this is so clearly elegant.
Jeff Hawkins (1:24:29.980)
This is how brains work.
Lex Fridman (1:24:30.820)
This is what intelligence is.
Lex Fridman (1:24:31.860)
But the machine learning world has gotten in this phase
Lex Fridman (1:24:33.940)
where they think it doesn't matter.
Jeff Hawkins (1:24:35.500)
Doesn't matter what you think,
Lex Fridman (1:24:36.600)
as long as you do, you know, 0.1% better on this benchmark,
Jeff Hawkins (1:24:39.440)
that's what, that's all that matters.
Lex Fridman (1:24:40.780)
And that's a problem.
Jeff Hawkins (1:24:43.860)
You know, we have to figure out how to get around that.
Lex Fridman (1:24:46.060)
That's a challenge for us.
Jeff Hawkins (1:24:47.300)
That's one of the challenges that we have to deal with.
Lex Fridman (1:24:50.500)
So I agree, you've identified a big issue.
Jeff Hawkins (1:24:52.820)
It's difficult for those reasons.
Lex Fridman (1:24:55.900)
But you know, part of the reasons I'm talking to you here
Jeff Hawkins (1:24:59.580)
today is I hope I'm gonna get some machine learning people
Lex Fridman (1:25:01.620)
to say, I'm gonna read those papers.
Jeff Hawkins (1:25:03.260)
Those might be some interesting ideas.
Lex Fridman (1:25:04.500)
I'm tired of doing this 0.1% improvement stuff, you know?
Jeff Hawkins (1:25:08.460)
Well, that's why I'm here as well,
Lex Fridman (1:25:10.340)
because I think machine learning now as a community
Jeff Hawkins (1:25:13.020)
is at a place where the next step needs to be orthogonal
Lex Fridman (1:25:18.500)
to what has received success in the past.
Jeff Hawkins (1:25:21.300)
Well, you see other leaders saying this,
Lex Fridman (1:25:23.100)
machine learning leaders, you know,
Jeff Hawkins (1:25:25.500)
Jeff Hinton with his capsules idea.
Lex Fridman (1:25:27.940)
Many people have gotten up to say, you know,
Jeff Hawkins (1:25:29.300)
we're gonna hit road map, maybe we should look at the brain,
Lex Fridman (1:25:32.100)
you know, things like that.
Lex Fridman (1:25:33.460)
So hopefully that thinking will occur organically.
Lex Fridman (1:25:38.100)
And then we're in a nice position for people to come
Lex Fridman (1:25:40.740)
and look at our work and say,
Lex Fridman (1:25:41.740)
well, what can we learn from these guys?
Jeff Hawkins (1:25:43.180)
Yeah, MIT is launching a billion dollar computing college
Lex Fridman (1:25:47.500)
that's centered around this idea, so.
Lex Fridman (1:25:49.220)
Is it on this idea of what?
Lex Fridman (1:25:50.980)
Well, the idea that, you know,
Jeff Hawkins (1:25:52.700)
the humanities, psychology, and neuroscience
Lex Fridman (1:25:54.980)
have to work all together to get to build the S.
Jeff Hawkins (1:25:58.860)
Yeah, I mean, Stanford just did
Lex Fridman (1:26:00.340)
this Human Centered AI Center.
Jeff Hawkins (1:26:02.500)
I'm a little disappointed in these initiatives
Lex Fridman (1:26:04.420)
because, you know, they're focusing
Jeff Hawkins (1:26:08.340)
on sort of the human side of it,
Lex Fridman (1:26:09.940)
and it could very easily slip into
Lex Fridman (1:26:12.140)
how humans interact with intelligent machines,
Lex Fridman (1:26:16.060)
which is nothing wrong with that,
Lex Fridman (1:26:17.620)
but that's not, that is orthogonal
Lex Fridman (1:26:19.420)
to what we're trying to do.
Jeff Hawkins (1:26:20.380)
We're trying to say, like,
Lex Fridman (1:26:21.340)
what is the essence of intelligence?
Jeff Hawkins (1:26:22.860)
I don't care.
Lex Fridman (1:26:23.700)
In fact, I wanna build intelligent machines
Jeff Hawkins (1:26:25.500)
that aren't emotional, that don't smile at you,
Lex Fridman (1:26:28.620)
that, you know, that aren't trying to tuck you in at night.
Jeff Hawkins (1:26:31.820)
Yeah, there is that pattern that you,
Lex Fridman (1:26:34.020)
when you talk about understanding humans
Jeff Hawkins (1:26:36.500)
is important for understanding intelligence,
Lex Fridman (1:26:38.380)
that you start slipping into topics of ethics
Jeff Hawkins (1:26:41.140)
or, yeah, like you said,
Lex Fridman (1:26:43.700)
the interactive elements as opposed to,
Jeff Hawkins (1:26:45.700)
no, no, no, we have to zoom in on the brain,
Lex Fridman (1:26:47.380)
study what the human brain, the baby, the...
Jeff Hawkins (1:26:51.460)
Let's study what a brain does.
Lex Fridman (1:26:52.900)
Does.
Lex Fridman (1:26:53.740)
And then we can decide which parts of that
Lex Fridman (1:26:54.780)
we wanna recreate in some system,
Lex Fridman (1:26:57.740)
but until you have that theory about what the brain does,
Lex Fridman (1:26:59.900)
what's the point, you know, it's just,
Jeff Hawkins (1:27:01.300)
you're gonna be wasting time, I think.
Lex Fridman (1:27:02.740)
Right, just to break it down
Jeff Hawkins (1:27:04.060)
on the artificial neural network side,
Lex Fridman (1:27:05.620)
maybe you could speak to this
Jeff Hawkins (1:27:06.740)
on the biological neural network side,
Lex Fridman (1:27:09.180)
the process of learning versus the process of inference.
Jeff Hawkins (1:27:13.300)
Maybe you can explain to me,
Lex Fridman (1:27:15.620)
is there a difference between,
Jeff Hawkins (1:27:18.460)
you know, in artificial neural networks,
Lex Fridman (1:27:19.860)
there's a difference between the learning stage
Lex Fridman (1:27:21.500)
and the inference stage.
Lex Fridman (1:27:22.940)
Do you see the brain as something different?
Jeff Hawkins (1:27:24.980)
One of the big distinctions that people often say,
Lex Fridman (1:27:29.020)
I don't know how correct it is,
Jeff Hawkins (1:27:30.660)
is artificial neural networks need a lot of data.
Lex Fridman (1:27:32.940)
They're very inefficient learning.
Lex Fridman (1:27:34.820)
Do you see that as a correct distinction
Lex Fridman (1:27:37.340)
from the biology of the human brain,
Jeff Hawkins (1:27:40.300)
that the human brain is very efficient,
Lex Fridman (1:27:41.980)
or is that just something we deceive ourselves?
Jeff Hawkins (1:27:44.220)
No, it is efficient, obviously.
Lex Fridman (1:27:45.420)
We can learn new things almost instantly.
Lex Fridman (1:27:47.580)
And so what elements do you think are useful?
Lex Fridman (1:27:50.020)
Yeah, I can talk about that.
Jeff Hawkins (1:27:50.860)
You brought up two issues there.
Lex Fridman (1:27:52.300)
So remember I talked early about the constraints
Jeff Hawkins (1:27:54.820)
we always feel, well, one of those constraints
Lex Fridman (1:27:57.260)
is the fact that brains are continually learning.
Jeff Hawkins (1:28:00.940)
That's not something we said, oh, we can add that later.
Lex Fridman (1:28:03.780)
That's something that was upfront,
Jeff Hawkins (1:28:05.780)
had to be there from the start,
Lex Fridman (1:28:08.900)
made our problems harder.
Lex Fridman (1:28:11.260)
But we showed, going back to the 2016 paper
Lex Fridman (1:28:14.420)
on sequence memory, we showed how that happens,
Lex Fridman (1:28:16.780)
how the brains infer and learn at the same time.
Lex Fridman (1:28:19.940)
And our models do that.
Lex Fridman (1:28:21.740)
And they're not two separate phases,
Lex Fridman (1:28:24.060)
or two separate sets of time.
Jeff Hawkins (1:28:26.340)
I think that's a big, big problem in AI,
Lex Fridman (1:28:29.780)
at least for many applications, not for all.
Lex Fridman (1:28:33.420)
So I can talk about that.
Lex Fridman (1:28:34.380)
There are some, it gets detailed,
Jeff Hawkins (1:28:37.180)
there are some parts of the neocortex in the brain
Lex Fridman (1:28:39.660)
where actually what's going on,
Jeff Hawkins (1:28:41.740)
there's these cycles of activity in the brain.
Lex Fridman (1:28:46.860)
And there's very strong evidence
Jeff Hawkins (1:28:49.260)
that you're doing more of inference
Lex Fridman (1:28:51.260)
on one part of the phase,
Lex Fridman (1:28:52.300)
and more of learning on the other part of the phase.
Lex Fridman (1:28:54.100)
So the brain can actually sort of separate
Jeff Hawkins (1:28:55.500)
different populations of cells
Lex Fridman (1:28:56.660)
or going back and forth like this.
Lex Fridman (1:28:58.340)
But in general, I would say that's an important problem.
Lex Fridman (1:29:01.540)
We have all of our networks that we've come up with do both.
Lex Fridman (1:29:05.620)
And they're continuous learning networks.
Lex Fridman (1:29:08.220)
And you mentioned benchmarks earlier.
Jeff Hawkins (1:29:10.980)
Well, there are no benchmarks about that.
Lex Fridman (1:29:12.500)
So we have to, we get in our little soapbox,
Lex Fridman (1:29:17.180)
and hey, by the way, this is important,
Lex Fridman (1:29:19.220)
and here's a mechanism for doing that.
Lex Fridman (1:29:20.580)
But until you can prove it to someone
Lex Fridman (1:29:23.900)
in some commercial system or something, it's a little harder.
Lex Fridman (1:29:26.700)
So yeah, one of the things I had to linger on that
Lex Fridman (1:29:28.980)
is in some ways to learn the concept of a coffee cup,
Jeff Hawkins (1:29:33.780)
you only need this one coffee cup
Lex Fridman (1:29:35.900)
and maybe some time alone in a room with it.
Jeff Hawkins (1:29:37.980)
Well, the first thing is,
Lex Fridman (1:29:39.940)
imagine I reach my hand into a black box
Lex Fridman (1:29:41.820)
and I'm reaching, I'm trying to touch something.
Lex Fridman (1:29:43.700)
I don't know upfront if it's something I already know
Jeff Hawkins (1:29:46.220)
or if it's a new thing.
Lex Fridman (1:29:47.860)
And I have to, I'm doing both at the same time.
Jeff Hawkins (1:29:50.460)
I don't say, oh, let's see if it's a new thing.
Lex Fridman (1:29:53.260)
Oh, let's see if it's an old thing.
Jeff Hawkins (1:29:54.740)
I don't do that.
Lex Fridman (1:29:55.580)
As I go, my brain says, oh, it's new or it's not new.
Lex Fridman (1:29:59.420)
And if it's new, I start learning what it is.
Lex Fridman (1:30:02.300)
And by the way, it starts learning from the get go,
Jeff Hawkins (1:30:04.820)
even if it's gonna recognize it.
Lex Fridman (1:30:06.020)
So they're not separate problems.
Lex Fridman (1:30:08.900)
And so that's the thing there.
Lex Fridman (1:30:10.060)
The other thing you mentioned was the fast learning.
Lex Fridman (1:30:13.540)
So I was just talking about continuous learning,
Lex Fridman (1:30:15.580)
but there's also fast learning.
Jeff Hawkins (1:30:16.660)
Literally, I can show you this coffee cup
Lex Fridman (1:30:18.780)
and I say, here's a new coffee cup.
Jeff Hawkins (1:30:20.060)
It's got the logo on it.
Lex Fridman (1:30:21.340)
Take a look at it, done, you're done.
Jeff Hawkins (1:30:23.860)
You can predict what it's gonna look like,
Lex Fridman (1:30:25.380)
you know, in different positions.
Lex Fridman (1:30:27.460)
So I can talk about that too.
Lex Fridman (1:30:29.540)
In the brain, the way learning occurs,
Jeff Hawkins (1:30:34.220)
I mentioned this earlier, but I'll mention it again.
Lex Fridman (1:30:35.700)
The way learning occurs,
Jeff Hawkins (1:30:36.820)
imagine I am a section of a dendrite of a neuron,
Lex Fridman (1:30:40.140)
and I'm gonna learn something new.
Jeff Hawkins (1:30:43.740)
Doesn't matter what it is.
Lex Fridman (1:30:44.580)
I'm just gonna learn something new.
Jeff Hawkins (1:30:46.180)
I need to recognize a new pattern.
Lex Fridman (1:30:48.900)
So what I'm gonna do is I'm gonna form new synapses.
Jeff Hawkins (1:30:52.540)
New synapses, we're gonna rewire the brain
Lex Fridman (1:30:55.140)
onto that section of the dendrite.
Jeff Hawkins (1:30:57.900)
Once I've done that, everything else that neuron has learned
Lex Fridman (1:31:01.020)
is not affected by it.
Jeff Hawkins (1:31:02.580)
That's because it's isolated
Lex Fridman (1:31:04.340)
to that small section of the dendrite.
Jeff Hawkins (1:31:06.380)
They're not all being added together, like a point neuron.
Lex Fridman (1:31:09.580)
So if I learn something new on this segment here,
Jeff Hawkins (1:31:11.740)
it doesn't change any of the learning
Lex Fridman (1:31:13.180)
that occur anywhere else in that neuron.
Lex Fridman (1:31:14.860)
So I can add something without affecting previous learning.
Lex Fridman (1:31:18.420)
And I can do it quickly.
Jeff Hawkins (1:31:20.940)
Now let's talk, we can talk about the quickness,
Lex Fridman (1:31:22.300)
how it's done in real neurons.
Lex Fridman (1:31:24.020)
You might say, well, doesn't it take time to form synapses?
Lex Fridman (1:31:26.740)
Yes, it can take maybe an hour to form a new synapse.
Jeff Hawkins (1:31:30.900)
We can form memories quicker than that,
Lex Fridman (1:31:32.500)
and I can explain that how it happens too, if you want.
Lex Fridman (1:31:35.860)
But it's getting a bit neurosciencey.
Lex Fridman (1:31:39.460)
That's great, but is there an understanding
Lex Fridman (1:31:41.380)
of these mechanisms at every level?
Lex Fridman (1:31:43.100)
Yeah.
Lex Fridman (1:31:43.940)
So from the short term memories and the forming.
Lex Fridman (1:31:48.620)
So this idea of synaptogenesis, the growth of new synapses,
Jeff Hawkins (1:31:51.580)
that's well described, it's well understood.
Lex Fridman (1:31:54.100)
And that's an essential part of learning.
Jeff Hawkins (1:31:55.820)
That is learning.
Lex Fridman (1:31:56.780)
That is learning.
Jeff Hawkins (1:31:58.180)
Okay.
Lex Fridman (1:32:01.980)
Going back many, many years,
Jeff Hawkins (1:32:03.860)
people, you know, it was, what's his name,
Lex Fridman (1:32:06.340)
the psychologist who proposed, Hebb, Donald Hebb.
Jeff Hawkins (1:32:09.580)
He proposed that learning was the modification
Lex Fridman (1:32:12.020)
of the strength of a connection between two neurons.
Jeff Hawkins (1:32:15.460)
People interpreted that as the modification
Lex Fridman (1:32:18.180)
of the strength of a synapse.
Jeff Hawkins (1:32:19.660)
He didn't say that.
Lex Fridman (1:32:20.980)
He just said there's a modification
Jeff Hawkins (1:32:22.340)
between the effect of one neuron and another.
Lex Fridman (1:32:24.540)
So synaptogenesis is totally consistent
Jeff Hawkins (1:32:26.500)
with what Donald Hebb said.
Lex Fridman (1:32:28.180)
But anyway, there's these mechanisms,
Jeff Hawkins (1:32:29.860)
the growth of new synapses.
Lex Fridman (1:32:30.860)
You can go online, you can watch a video
Jeff Hawkins (1:32:32.260)
of a synapse growing in real time.
Lex Fridman (1:32:33.900)
It's literally, you can see this little thing going boop.
Jeff Hawkins (1:32:37.140)
It's pretty impressive.
Lex Fridman (1:32:38.420)
So those mechanisms are known.
Jeff Hawkins (1:32:39.740)
Now there's another thing that we've speculated
Lex Fridman (1:32:42.340)
and we've written about,
Jeff Hawkins (1:32:43.540)
which is consistent with known neuroscience,
Lex Fridman (1:32:45.780)
but it's less proven.
Lex Fridman (1:32:48.340)
And this is the idea, how do I form a memory
Lex Fridman (1:32:50.580)
really, really quickly?
Jeff Hawkins (1:32:51.620)
Like instantaneous.
Lex Fridman (1:32:52.820)
If it takes an hour to grow a synapse,
Jeff Hawkins (1:32:54.580)
like that's not instantaneous.
Lex Fridman (1:32:56.820)
So there are types of synapses called silent synapses.
Jeff Hawkins (1:33:01.700)
They look like a synapse, but they don't do anything.
Lex Fridman (1:33:04.060)
They're just sitting there.
Jeff Hawkins (1:33:04.900)
It's like if an action potential comes in,
Lex Fridman (1:33:07.900)
it doesn't release any neurotransmitter.
Jeff Hawkins (1:33:10.140)
Some parts of the brain have more of these than others.
Lex Fridman (1:33:12.500)
For example, the hippocampus has a lot of them,
Jeff Hawkins (1:33:14.020)
which is where we associate most short term memory with.
Lex Fridman (1:33:18.540)
So what we speculated, again, in that 2016 paper,
Jeff Hawkins (1:33:22.100)
we proposed that the way we form very quick memories,
Lex Fridman (1:33:26.420)
very short term memories, or quick memories,
Jeff Hawkins (1:33:28.940)
is that we convert silent synapses into active synapses.
Lex Fridman (1:33:33.860)
It's like saying a synapse has a zero weight
Lex Fridman (1:33:36.060)
and a one weight,
Lex Fridman (1:33:37.860)
but the longterm memory has to be formed by synaptogenesis.
Lex Fridman (1:33:41.460)
So you can remember something really quickly
Lex Fridman (1:33:43.300)
by just flipping a bunch of these guys from silent to active.
Jeff Hawkins (1:33:46.220)
It's not from 0.1 to 0.15.
Lex Fridman (1:33:49.140)
It's like, it doesn't do anything
Jeff Hawkins (1:33:50.700)
till it releases transmitter.
Lex Fridman (1:33:52.260)
And if I do that over a bunch of these,
Jeff Hawkins (1:33:53.500)
I've got a very quick short term memory.
Lex Fridman (1:33:56.860)
So I guess the lesson behind this
Jeff Hawkins (1:33:58.500)
is that most neural networks today are fully connected.
Lex Fridman (1:34:01.860)
Every neuron connects every other neuron
Jeff Hawkins (1:34:03.380)
from layer to layer.
Lex Fridman (1:34:04.580)
That's not correct in the brain.
Jeff Hawkins (1:34:06.060)
We don't want that.
Lex Fridman (1:34:06.980)
We actually don't want that.
Jeff Hawkins (1:34:08.340)
It's bad.
Lex Fridman (1:34:09.260)
You want a very sparse connectivity
Lex Fridman (1:34:10.700)
so that any neuron connects to some subset of the neurons
Lex Fridman (1:34:14.500)
in the other layer.
Lex Fridman (1:34:15.340)
And it does so on a dendrite by dendrite segment basis.
Lex Fridman (1:34:18.980)
So it's a very some parcelated out type of thing.
Lex Fridman (1:34:21.580)
And that then learning is not adjusting all these weights,
Lex Fridman (1:34:25.380)
but learning is just saying,
Jeff Hawkins (1:34:26.340)
okay, connect to these 10 cells here right now.
Lex Fridman (1:34:30.180)
In that process, you know, with artificial neural networks,
Jeff Hawkins (1:34:32.980)
it's a very simple process of backpropagation
Lex Fridman (1:34:36.060)
that adjusts the weights.
Jeff Hawkins (1:34:37.180)
The process of synaptogenesis.
Lex Fridman (1:34:40.100)
Synaptogenesis.
Jeff Hawkins (1:34:40.940)
Synaptogenesis.
Lex Fridman (1:34:42.300)
It's even easier.
Jeff Hawkins (1:34:43.140)
It's even easier.
Lex Fridman (1:34:43.980)
It's even easier.
Jeff Hawkins (1:34:44.820)
Backpropagation requires something
Lex Fridman (1:34:47.260)
that really can't happen in brains.
Jeff Hawkins (1:34:48.700)
This backpropagation of this error signal,
Lex Fridman (1:34:51.220)
that really can't happen.
Jeff Hawkins (1:34:52.060)
People are trying to make it happen in brains,
Lex Fridman (1:34:53.500)
but it doesn't happen in brains.
Jeff Hawkins (1:34:54.740)
This is pure Hebbian learning.
Lex Fridman (1:34:56.780)
Well, synaptogenesis is pure Hebbian learning.
Jeff Hawkins (1:34:58.660)
It's basically saying,
Lex Fridman (1:35:00.140)
there's a population of cells over here
Jeff Hawkins (1:35:01.540)
that are active right now.
Lex Fridman (1:35:03.020)
And there's a population of cells over here
Jeff Hawkins (1:35:04.340)
active right now.
Lex Fridman (1:35:05.380)
How do I form connections between those active cells?
Lex Fridman (1:35:07.980)
And it's literally saying this guy became active.
Lex Fridman (1:35:11.100)
These 100 neurons here became active
Jeff Hawkins (1:35:13.260)
before this neuron became active.
Lex Fridman (1:35:15.080)
So form connections to those ones.
Jeff Hawkins (1:35:17.140)
That's it.
Lex Fridman (1:35:17.960)
There's no propagation of error, nothing.
Jeff Hawkins (1:35:19.940)
All the networks we do,
Lex Fridman (1:35:20.980)
all the models we have work on almost completely on
Jeff Hawkins (1:35:25.700)
Hebbian learning,
Lex Fridman (1:35:26.540)
but on dendritic segments
Lex Fridman (1:35:30.260)
and multiple synapses at the same time.
Lex Fridman (1:35:33.060)
So now let's sort of turn the question
Jeff Hawkins (1:35:34.540)
that you already answered,
Lex Fridman (1:35:35.820)
and maybe you can answer it again.
Jeff Hawkins (1:35:38.820)
If you look at the history of artificial intelligence,
Lex Fridman (1:35:41.260)
where do you think we stand?
Lex Fridman (1:35:43.540)
How far are we from solving intelligence?
Lex Fridman (1:35:45.780)
You said you were very optimistic.
Lex Fridman (1:35:47.700)
Can you elaborate on that?
Lex Fridman (1:35:48.900)
Yeah, it's always the crazy question to ask
Jeff Hawkins (1:35:53.500)
because no one can predict the future.
Lex Fridman (1:35:55.100)
Absolutely.
Lex Fridman (1:35:55.940)
So I'll tell you a story.
Lex Fridman (1:35:58.180)
I used to run a different neuroscience institute
Jeff Hawkins (1:36:01.400)
called the Redwood Neuroscience Institute,
Lex Fridman (1:36:02.620)
and we would hold these symposiums
Lex Fridman (1:36:04.740)
and we'd get like 35 scientists
Lex Fridman (1:36:06.340)
from around the world to come together.
Lex Fridman (1:36:08.060)
And I used to ask them all the same question.
Lex Fridman (1:36:10.380)
I would say, well, how long do you think it'll be
Lex Fridman (1:36:11.740)
before we understand how the neocortex works?
Lex Fridman (1:36:14.540)
And everyone went around the room
Lex Fridman (1:36:15.540)
and they had introduced the name
Lex Fridman (1:36:16.560)
and they have to answer that question.
Lex Fridman (1:36:18.240)
So I got, the typical answer was 50 to 100 years.
Lex Fridman (1:36:22.940)
Some people would say 500 years.
Jeff Hawkins (1:36:24.780)
Some people said never.
Lex Fridman (1:36:25.860)
I said, why are you a neuroscientist?
Jeff Hawkins (1:36:27.820)
It's never gonna, it's a good pay.
Lex Fridman (1:36:32.780)
It's interesting.
Jeff Hawkins (1:36:34.380)
So, you know, but it doesn't work like that.
Lex Fridman (1:36:36.300)
As I mentioned earlier, these are not,
Jeff Hawkins (1:36:38.740)
these are step functions.
Lex Fridman (1:36:39.620)
Things happen and then bingo, they happen.
Jeff Hawkins (1:36:41.780)
You can't predict that.
Lex Fridman (1:36:43.620)
I feel I've already passed a step function.
Lex Fridman (1:36:45.620)
So if I can do my job correctly over the next five years,
Lex Fridman (1:36:50.740)
then, meaning I can proselytize these ideas.
Jeff Hawkins (1:36:53.540)
I can convince other people they're right.
Lex Fridman (1:36:56.140)
We can show that other people,
Jeff Hawkins (1:36:58.740)
machine learning people should pay attention
Lex Fridman (1:37:00.260)
to these ideas.
Jeff Hawkins (1:37:01.420)
Then we're definitely in an under 20 year timeframe.
Lex Fridman (1:37:04.580)
If I can do those things, if I'm not successful in that,
Lex Fridman (1:37:07.780)
and this is the last time anyone talks to me
Lex Fridman (1:37:09.780)
and no one reads our papers and you know,
Lex Fridman (1:37:12.180)
and I'm wrong or something like that,
Lex Fridman (1:37:13.980)
then I don't know.
Lex Fridman (1:37:15.940)
But it's not 50 years.
Lex Fridman (1:37:21.820)
Think about electric cars.
Lex Fridman (1:37:22.940)
How quickly are they gonna populate the world?
Lex Fridman (1:37:24.940)
It probably takes about a 20 year span.
Jeff Hawkins (1:37:27.900)
It'll be something like that.
Lex Fridman (1:37:28.820)
But I think if I can do what I said, we're starting it.
Lex Fridman (1:37:31.740)
And of course there could be other,
Lex Fridman (1:37:34.220)
you said step functions.
Jeff Hawkins (1:37:35.400)
It could be everybody gives up on your ideas for 20 years
Lex Fridman (1:37:40.100)
and then all of a sudden somebody picks it up again.
Jeff Hawkins (1:37:42.180)
Wait, that guy was onto something.
Lex Fridman (1:37:43.620)
Yeah, so that would be a failure on my part, right?
Jeff Hawkins (1:37:47.540)
Think about Charles Babbage.
Lex Fridman (1:37:49.820)
Charles Babbage, he's the guy who invented the computer
Jeff Hawkins (1:37:52.220)
back in the 18 something, 1800s.
Lex Fridman (1:37:55.820)
And everyone forgot about it until 100 years later.
Lex Fridman (1:37:59.460)
And say, hey, this guy figured this stuff out
Lex Fridman (1:38:00.900)
a long time ago.
Lex Fridman (1:38:02.380)
But he was ahead of his time.
Lex Fridman (1:38:03.940)
I don't think, as I said,
Jeff Hawkins (1:38:06.460)
I recognize this is part of any entrepreneur's challenge.
Lex Fridman (1:38:09.780)
I use entrepreneur broadly in this case.
Jeff Hawkins (1:38:11.500)
I'm not meaning like I'm building a business
Lex Fridman (1:38:12.980)
or trying to sell something.
Jeff Hawkins (1:38:13.820)
I mean, I'm trying to sell ideas.
Lex Fridman (1:38:15.900)
And this is the challenge as to how you get people
Jeff Hawkins (1:38:19.380)
to pay attention to you, how do you get them
Lex Fridman (1:38:21.540)
to give you positive or negative feedback,
Lex Fridman (1:38:24.700)
how do you get the people to act differently
Lex Fridman (1:38:25.960)
based on your ideas.
Lex Fridman (1:38:27.220)
So we'll see how well we do on that.
Lex Fridman (1:38:30.180)
So you know that there's a lot of hype
Jeff Hawkins (1:38:32.300)
behind artificial intelligence currently.
Lex Fridman (1:38:34.640)
Do you, as you look to spread the ideas
Jeff Hawkins (1:38:39.540)
that are of neocortical theory, the things you're working on,
Lex Fridman (1:38:43.300)
do you think there's some possibility
Lex Fridman (1:38:45.100)
we'll hit an AI winter once again?
Lex Fridman (1:38:47.300)
Yeah, it's certainly a possibility.
Jeff Hawkins (1:38:48.940)
No question about it.
Lex Fridman (1:38:49.780)
Is that something you worry about?
Lex Fridman (1:38:50.600)
Yeah, well, I guess, do I worry about it?
Lex Fridman (1:38:54.340)
I haven't decided yet if that's good or bad for my mission.
Jeff Hawkins (1:38:57.540)
That's true, that's very true.
Lex Fridman (1:38:59.660)
Because it's almost like you need the winter
Jeff Hawkins (1:39:02.940)
to refresh the palette.
Lex Fridman (1:39:04.300)
Yeah, it's like, I want, here's what you wanna have it is.
Jeff Hawkins (1:39:07.860)
You want, like to the extent that everyone is so thrilled
Lex Fridman (1:39:12.180)
about the current state of machine learning and AI
Lex Fridman (1:39:15.460)
and they don't imagine they need anything else,
Lex Fridman (1:39:18.100)
it makes my job harder.
Jeff Hawkins (1:39:19.740)
If everything crashed completely
Lex Fridman (1:39:22.580)
and every student left the field
Lex Fridman (1:39:24.260)
and there was no money for anybody to do anything
Lex Fridman (1:39:26.200)
and it became an embarrassment
Jeff Hawkins (1:39:27.460)
to talk about machine intelligence and AI,
Lex Fridman (1:39:29.020)
that wouldn't be good for us either.
Lex Fridman (1:39:30.740)
You want sort of the soft landing approach, right?
Lex Fridman (1:39:33.400)
You want enough people, the senior people in AI
Lex Fridman (1:39:36.620)
and machine learning to say, you know,
Lex Fridman (1:39:37.860)
we need other approaches.
Jeff Hawkins (1:39:38.900)
We really need other approaches.
Lex Fridman (1:39:40.460)
Damn, we need other approaches.
Jeff Hawkins (1:39:42.020)
Maybe we should look to the brain.
Lex Fridman (1:39:43.100)
Okay, let's look to the brain.
Lex Fridman (1:39:44.220)
Who's got some brain ideas?
Lex Fridman (1:39:45.380)
Okay, let's start a little project on the side here
Jeff Hawkins (1:39:47.900)
trying to do brain idea related stuff.
Lex Fridman (1:39:49.700)
That's the ideal outcome we would want.
Lex Fridman (1:39:51.820)
So I don't want a total winter
Lex Fridman (1:39:53.980)
and yet I don't want it to be sunny all the time either.
Lex Fridman (1:39:57.680)
So what do you think it takes to build a system
Lex Fridman (1:40:00.300)
with human level intelligence
Lex Fridman (1:40:03.020)
where once demonstrated you would be very impressed?
Lex Fridman (1:40:06.820)
So does it have to have a body?
Jeff Hawkins (1:40:08.700)
Does it have to have the C word we used before,
Lex Fridman (1:40:12.780)
consciousness as an entirety in a holistic sense?
Jeff Hawkins (1:40:19.140)
First of all, I don't think the goal
Lex Fridman (1:40:20.500)
is to create a machine that is human level intelligence.
Jeff Hawkins (1:40:23.740)
I think it's a false goal.
Lex Fridman (1:40:24.980)
Back to Turing, I think it was a false statement.
Jeff Hawkins (1:40:27.380)
We want to understand what intelligence is
Lex Fridman (1:40:29.060)
and then we can build intelligent machines
Jeff Hawkins (1:40:30.780)
of all different scales, all different capabilities.
Lex Fridman (1:40:34.260)
A dog is intelligent.
Jeff Hawkins (1:40:35.300)
I don't need, that'd be pretty good to have a dog.
Lex Fridman (1:40:38.460)
But what about something that doesn't look
Lex Fridman (1:40:39.580)
like an animal at all, in different spaces?
Lex Fridman (1:40:41.660)
So my thinking about this is that
Jeff Hawkins (1:40:44.300)
we want to define what intelligence is,
Lex Fridman (1:40:46.060)
agree upon what makes an intelligent system.
Jeff Hawkins (1:40:48.840)
We can then say, okay, we're now gonna build systems
Lex Fridman (1:40:51.100)
that work on those principles or some subset of them
Lex Fridman (1:40:54.340)
and we can apply them to all different types of problems.
Lex Fridman (1:40:57.380)
And the kind, the idea, it's not computing.
Jeff Hawkins (1:41:00.860)
We don't ask, if I take a little one chip computer,
Lex Fridman (1:41:05.340)
I don't say, well, that's not a computer
Jeff Hawkins (1:41:06.660)
because it's not as powerful as this big server over here.
Lex Fridman (1:41:09.660)
No, no, because we know that what the principles
Jeff Hawkins (1:41:11.260)
of computing are and I can apply those principles
Lex Fridman (1:41:12.940)
to a small problem or into a big problem.
Lex Fridman (1:41:14.860)
And same, intelligence needs to get there.
Lex Fridman (1:41:16.520)
We have to say, these are the principles.
Jeff Hawkins (1:41:17.620)
I can make a small one, a big one.
Lex Fridman (1:41:19.020)
I can make them distributed.
Jeff Hawkins (1:41:19.940)
I can put them on different sensors.
Lex Fridman (1:41:21.620)
They don't have to be human like at all.
Jeff Hawkins (1:41:23.220)
Now, you did bring up a very interesting question
Lex Fridman (1:41:24.740)
about embodiment.
Lex Fridman (1:41:25.620)
Does it have to have a body?
Lex Fridman (1:41:27.500)
It has to have some concept of movement.
Jeff Hawkins (1:41:30.660)
It has to be able to move through these reference frames
Lex Fridman (1:41:33.260)
I talked about earlier.
Jeff Hawkins (1:41:34.460)
Whether it's physically moving,
Lex Fridman (1:41:35.820)
like I need, if I'm gonna have an AI
Jeff Hawkins (1:41:37.420)
that understands coffee cups,
Lex Fridman (1:41:38.780)
it's gonna have to pick up the coffee cup
Lex Fridman (1:41:40.500)
and touch it and look at it with its eyes and hands
Lex Fridman (1:41:43.180)
or something equivalent to that.
Jeff Hawkins (1:41:45.380)
If I have a mathematical AI,
Lex Fridman (1:41:48.100)
maybe it needs to move through mathematical spaces.
Jeff Hawkins (1:41:51.340)
I could have a virtual AI that lives in the internet
Lex Fridman (1:41:55.240)
and its movements are traversing links
Lex Fridman (1:41:58.980)
and digging into files,
Lex Fridman (1:42:00.260)
but it's got a location that it's traveling
Jeff Hawkins (1:42:03.100)
through some space.
Lex Fridman (1:42:04.940)
You can't have an AI that just take some flash thing input.
Jeff Hawkins (1:42:09.060)
We call it flash inference.
Lex Fridman (1:42:10.620)
Here's a pattern, done.
Jeff Hawkins (1:42:12.860)
No, it's movement pattern, movement pattern,
Lex Fridman (1:42:15.740)
movement pattern, attention, digging, building structure,
Jeff Hawkins (1:42:19.020)
figuring out the model of the world.
Lex Fridman (1:42:20.420)
So some sort of embodiment,
Jeff Hawkins (1:42:22.740)
whether it's physical or not, has to be part of it.
Lex Fridman (1:42:25.780)
So self awareness and the way to be able to answer
Lex Fridman (1:42:28.020)
where am I?
Lex Fridman (1:42:28.860)
Well, you're bringing up self,
Jeff Hawkins (1:42:29.680)
that's a different topic, self awareness.
Lex Fridman (1:42:31.460)
No, the very narrow definition of self,
Jeff Hawkins (1:42:33.700)
meaning knowing a sense of self enough to know
Lex Fridman (1:42:37.740)
where am I in the space where it's actually.
Jeff Hawkins (1:42:39.980)
Yeah, basically the system needs to know its location
Lex Fridman (1:42:43.500)
or each component of the system needs to know
Jeff Hawkins (1:42:46.020)
where it is in the world at that point in time.
Lex Fridman (1:42:48.620)
So self awareness and consciousness.
Lex Fridman (1:42:51.660)
Do you think one, from the perspective of neuroscience
Lex Fridman (1:42:55.620)
and neurocortex, these are interesting topics,
Jeff Hawkins (1:42:58.180)
solvable topics.
Lex Fridman (1:42:59.780)
Do you have any ideas of why the heck it is
Lex Fridman (1:43:02.180)
that we have a subjective experience at all?
Lex Fridman (1:43:04.420)
Yeah, I have a lot of thoughts on that.
Lex Fridman (1:43:05.260)
And is it useful or is it just a side effect of us?
Lex Fridman (1:43:08.460)
It's interesting to think about.
Jeff Hawkins (1:43:10.140)
I don't think it's useful as a means to figure out
Lex Fridman (1:43:13.460)
how to build intelligent machines.
Jeff Hawkins (1:43:16.360)
It's something that systems do
Lex Fridman (1:43:20.180)
and we can talk about what it is that are like,
Jeff Hawkins (1:43:22.780)
well, if I build a system like this,
Lex Fridman (1:43:23.980)
then it would be self aware.
Jeff Hawkins (1:43:25.300)
Or if I build it like this, it wouldn't be self aware.
Lex Fridman (1:43:28.340)
So that's a choice I can have.
Jeff Hawkins (1:43:30.040)
It's not like, oh my God, it's self aware.
Lex Fridman (1:43:32.300)
I can't turn, I heard an interview recently
Jeff Hawkins (1:43:35.800)
with this philosopher from Yale,
Lex Fridman (1:43:37.120)
I can't remember his name, I apologize for that.
Lex Fridman (1:43:39.020)
But he was talking about,
Lex Fridman (1:43:39.860)
well, if these computers are self aware,
Jeff Hawkins (1:43:41.420)
then it would be a crime to unplug them.
Lex Fridman (1:43:42.900)
And I'm like, oh, come on, that's not,
Jeff Hawkins (1:43:45.060)
I unplug myself every night, I go to sleep.
Lex Fridman (1:43:47.260)
Is that a crime?
Jeff Hawkins (1:43:48.260)
I plug myself in again in the morning and there I am.
Lex Fridman (1:43:51.340)
So people get kind of bent out of shape about this.
Jeff Hawkins (1:43:56.020)
I have very definite, very detailed understanding
Lex Fridman (1:43:59.500)
or opinions about what it means to be conscious
Lex Fridman (1:44:02.260)
and what it means to be self aware.
Lex Fridman (1:44:04.580)
I don't think it's that interesting a problem.
Jeff Hawkins (1:44:06.780)
You've talked to Christoph Koch.
Lex Fridman (1:44:08.740)
He thinks that's the only problem.
Jeff Hawkins (1:44:10.900)
I didn't actually listen to your interview with him,
Lex Fridman (1:44:12.380)
but I know him and I know that's the thing he cares about.
Jeff Hawkins (1:44:15.820)
He also thinks intelligence and consciousness are disjoint.
Lex Fridman (1:44:18.260)
So I mean, it's not, you don't have to have one or the other.
Lex Fridman (1:44:21.020)
So he is.
Lex Fridman (1:44:21.860)
I disagree with that.
Jeff Hawkins (1:44:22.740)
I just totally disagree with that.
Lex Fridman (1:44:24.600)
So where's your thoughts and consciousness,
Lex Fridman (1:44:26.300)
where does it emerge from?
Lex Fridman (1:44:27.660)
Because it is.
Lex Fridman (1:44:28.500)
So then we have to break it down to the two parts, okay?
Lex Fridman (1:44:30.860)
Because consciousness isn't one thing.
Jeff Hawkins (1:44:32.140)
That's part of the problem with that term
Lex Fridman (1:44:33.660)
is it means different things to different people
Lex Fridman (1:44:35.460)
and there's different components of it.
Lex Fridman (1:44:37.600)
There is a concept of self awareness, okay?
Jeff Hawkins (1:44:40.820)
That can be very easily explained.
Lex Fridman (1:44:43.100)
You have a model of your own body.
Jeff Hawkins (1:44:46.060)
The neocortex models things in the world
Lex Fridman (1:44:48.140)
and it also models your own body.
Lex Fridman (1:44:50.500)
And then it has a memory.
Lex Fridman (1:44:53.340)
It can remember what you've done, okay?
Lex Fridman (1:44:55.860)
So it can remember what you did this morning,
Lex Fridman (1:44:57.540)
can remember what you had for breakfast and so on.
Lex Fridman (1:44:59.640)
And so I can say to you, okay, Lex,
Lex Fridman (1:45:03.080)
were you conscious this morning when you had your bagel?
Lex Fridman (1:45:06.900)
And you'd say, yes, I was conscious.
Lex Fridman (1:45:08.820)
Now what if I could take your brain
Lex Fridman (1:45:10.300)
and revert all the synapses back
Lex Fridman (1:45:12.020)
to the state they were this morning?
Lex Fridman (1:45:14.180)
And then I said to you, Lex,
Lex Fridman (1:45:15.900)
were you conscious when you ate the bagel?
Lex Fridman (1:45:17.220)
And you said, no, I wasn't conscious.
Lex Fridman (1:45:18.540)
I said, here's a video of eating the bagel.
Lex Fridman (1:45:19.740)
And you said, I wasn't there.
Lex Fridman (1:45:22.420)
That's not possible
Jeff Hawkins (1:45:23.380)
because I must've been unconscious at that time.
Lex Fridman (1:45:25.660)
So we can just make this one to one correlation
Jeff Hawkins (1:45:27.460)
between memory of your body's trajectory through the world
Lex Fridman (1:45:31.000)
over some period of time,
Jeff Hawkins (1:45:32.100)
a memory and the ability to recall that memory
Lex Fridman (1:45:34.260)
is what you would call conscious.
Jeff Hawkins (1:45:35.900)
I was conscious of that, it's a self awareness.
Lex Fridman (1:45:38.940)
And any system that can recall,
Jeff Hawkins (1:45:41.340)
memorize what it's done recently
Lex Fridman (1:45:43.540)
and bring that back and invoke it again
Jeff Hawkins (1:45:46.340)
would say, yeah, I'm aware.
Lex Fridman (1:45:48.220)
I remember what I did.
Jeff Hawkins (1:45:49.380)
All right, I got it.
Lex Fridman (1:45:51.340)
That's an easy one.
Jeff Hawkins (1:45:52.420)
Although some people think that's a hard one.
Lex Fridman (1:45:54.780)
The more challenging part of consciousness
Jeff Hawkins (1:45:57.380)
is this one that's sometimes used
Lex Fridman (1:45:59.060)
going by the word of qualia,
Lex Fridman (1:46:01.300)
which is, why does an object seem red?
Lex Fridman (1:46:04.860)
Or what is pain?
Lex Fridman (1:46:06.860)
And why does pain feel like something?
Lex Fridman (1:46:08.740)
Why do I feel redness?
Lex Fridman (1:46:10.380)
Or why do I feel painness?
Lex Fridman (1:46:12.660)
And then I could say, well,
Lex Fridman (1:46:13.500)
why does sight seems different than hearing?
Lex Fridman (1:46:15.620)
It's the same problem.
Jeff Hawkins (1:46:16.460)
It's really, these are all just neurons.
Lex Fridman (1:46:18.580)
And so how is it that,
Lex Fridman (1:46:20.300)
why does looking at you feel different than hearing you?
Lex Fridman (1:46:24.140)
It feels different, but there's just neurons in my head.
Jeff Hawkins (1:46:26.080)
They're all doing the same thing.
Lex Fridman (1:46:27.820)
So that's an interesting question.
Jeff Hawkins (1:46:29.820)
The best treatise I've read about this
Lex Fridman (1:46:31.540)
is by a guy named Oregon.
Jeff Hawkins (1:46:33.580)
He wrote a book called,
Lex Fridman (1:46:35.740)
Why Red Doesn't Sound Like a Bell.
Jeff Hawkins (1:46:37.480)
It's a little, it's not a trade book, easy to read,
Lex Fridman (1:46:42.040)
but it, and it's an interesting question.
Jeff Hawkins (1:46:46.040)
Take something like color.
Lex Fridman (1:46:47.880)
Color really doesn't exist in the world.
Jeff Hawkins (1:46:49.360)
It's not a property of the world.
Lex Fridman (1:46:51.160)
Property of the world that exists is light frequency.
Lex Fridman (1:46:54.240)
And that gets turned into,
Lex Fridman (1:46:55.640)
we have certain cells in the retina
Jeff Hawkins (1:46:57.960)
that respond to different frequencies
Lex Fridman (1:46:59.320)
different than others.
Lex Fridman (1:47:00.240)
And so when they enter the brain,
Lex Fridman (1:47:01.440)
you just have a bunch of axons
Jeff Hawkins (1:47:02.440)
that are firing at different rates.
Lex Fridman (1:47:04.500)
And from that, we perceive color.
Lex Fridman (1:47:06.680)
But there is no color in the brain.
Lex Fridman (1:47:07.960)
I mean, there's no color coming in on those synapses.
Jeff Hawkins (1:47:10.840)
It's just a correlation between some axons
Lex Fridman (1:47:14.380)
and some property of frequency.
Lex Fridman (1:47:17.360)
And that isn't even color itself.
Lex Fridman (1:47:18.880)
Frequency doesn't have a color.
Jeff Hawkins (1:47:20.140)
It's just what it is.
Lex Fridman (1:47:22.940)
So then the question is,
Lex Fridman (1:47:24.120)
well, why does it even appear to have a color at all?
Lex Fridman (1:47:27.880)
Just as you're describing it,
Jeff Hawkins (1:47:29.080)
there seems to be a connection to those ideas
Lex Fridman (1:47:31.000)
of reference frames.
Jeff Hawkins (1:47:32.560)
I mean, it just feels like consciousness
Lex Fridman (1:47:37.040)
having the subject,
Jeff Hawkins (1:47:38.400)
assigning the feeling of red to the actual color
Lex Fridman (1:47:42.600)
or to the wavelength is useful for intelligence.
Jeff Hawkins (1:47:47.920)
Yeah, I think that's a good way of putting it.
Lex Fridman (1:47:49.600)
It's useful as a predictive mechanism
Jeff Hawkins (1:47:51.600)
or useful as a generalization idea.
Lex Fridman (1:47:53.840)
It's a way of grouping things together to say,
Jeff Hawkins (1:47:55.660)
it's useful to have a model like this.
Lex Fridman (1:47:57.560)
So think about the well known syndrome
Jeff Hawkins (1:48:02.640)
that people who've lost a limb experience
Lex Fridman (1:48:04.800)
called phantom limbs.
Lex Fridman (1:48:06.960)
And what they claim is they can have their arm is removed,
Lex Fridman (1:48:12.120)
but they feel their arm.
Jeff Hawkins (1:48:13.280)
That not only feel it, they know it's there.
Lex Fridman (1:48:15.960)
It's there, I know it's there.
Jeff Hawkins (1:48:17.740)
They'll swear to you that it's there.
Lex Fridman (1:48:19.000)
And then they can feel pain in their arm
Lex Fridman (1:48:20.360)
and they'll feel pain in their finger.
Lex Fridman (1:48:21.840)
And if they move their non existent arm behind their back,
Jeff Hawkins (1:48:25.280)
then they feel the pain behind their back.
Lex Fridman (1:48:27.320)
So this whole idea that your arm exists
Jeff Hawkins (1:48:30.120)
is a model of your brain.
Lex Fridman (1:48:31.360)
It may or may not really exist.
Lex Fridman (1:48:34.360)
And just like, but it's useful to have a model of something
Lex Fridman (1:48:38.520)
that sort of correlates to things in the world.
Lex Fridman (1:48:40.360)
So you can make predictions about what would happen
Lex Fridman (1:48:41.960)
when those things occur.
Jeff Hawkins (1:48:43.520)
It's a little bit of a fuzzy,
Lex Fridman (1:48:44.640)
but I think you're getting quite towards the answer there.
Jeff Hawkins (1:48:46.480)
It's useful for the model to express things certain ways
Lex Fridman (1:48:51.280)
that we can then map them into these reference frames
Lex Fridman (1:48:53.640)
and make predictions about them.
Lex Fridman (1:48:55.780)
I need to spend more time on this topic.
Jeff Hawkins (1:48:57.680)
It doesn't bother me.
Lex Fridman (1:48:58.880)
Do you really need to spend more time?
Jeff Hawkins (1:49:00.360)
Yeah, I know.
Lex Fridman (1:49:01.840)
It does feel special that we have subjective experience,
Lex Fridman (1:49:04.720)
but I'm yet to know why.
Lex Fridman (1:49:07.320)
I'm just personally curious.
Jeff Hawkins (1:49:09.040)
It's not necessary for the work we're doing here.
Lex Fridman (1:49:11.400)
I don't think I need to solve that problem
Jeff Hawkins (1:49:13.080)
to build intelligent machines at all, not at all.
Lex Fridman (1:49:15.560)
But there is sort of the silly notion
Jeff Hawkins (1:49:17.800)
that you described briefly
Lex Fridman (1:49:20.440)
that doesn't seem so silly to us humans is,
Jeff Hawkins (1:49:23.280)
if you're successful building intelligent machines,
Lex Fridman (1:49:27.080)
it feels wrong to then turn them off.
Jeff Hawkins (1:49:30.240)
Because if you're able to build a lot of them,
Lex Fridman (1:49:33.240)
it feels wrong to then be able to turn off the...
Lex Fridman (1:49:38.760)
Well, why?
Lex Fridman (1:49:39.600)
Let's break that down a bit.
Lex Fridman (1:49:41.840)
As humans, why do we fear death?
Lex Fridman (1:49:43.920)
There's two reasons we fear death.
Jeff Hawkins (1:49:47.060)
Well, first of all, I'll say,
Lex Fridman (1:49:47.900)
when you're dead, it doesn't matter at all.
Lex Fridman (1:49:48.960)
Who cares?
Lex Fridman (1:49:49.800)
You're dead.
Lex Fridman (1:49:50.640)
So why do we fear death?
Lex Fridman (1:49:51.840)
We fear death for two reasons.
Jeff Hawkins (1:49:53.480)
One is because we are programmed genetically to fear death.
Lex Fridman (1:49:57.760)
That's a survival and pop beginning of the genes thing.
Lex Fridman (1:50:02.940)
And we also are programmed to feel sad
Lex Fridman (1:50:05.120)
when people we know die.
Jeff Hawkins (1:50:06.880)
We don't feel sad for someone we don't know dies.
Lex Fridman (1:50:08.560)
There's people dying right now,
Jeff Hawkins (1:50:09.600)
they're only just gonna say,
Lex Fridman (1:50:10.420)
I don't feel bad about them,
Jeff Hawkins (1:50:11.260)
because I don't know them.
Lex Fridman (1:50:12.080)
But if I knew them, I'd feel really bad.
Lex Fridman (1:50:13.420)
So again, these are old brain,
Lex Fridman (1:50:16.840)
genetically embedded things that we fear death.
Jeff Hawkins (1:50:19.880)
It's outside of those uncomfortable feelings.
Lex Fridman (1:50:24.280)
There's nothing else to worry about.
Jeff Hawkins (1:50:25.840)
Well, wait, hold on a second.
Lex Fridman (1:50:27.360)
Do you know the denial of death by Becker?
Jeff Hawkins (1:50:30.440)
No.
Lex Fridman (1:50:31.360)
There's a thought that death is,
Jeff Hawkins (1:50:36.760)
our whole conception of our world model
Lex Fridman (1:50:41.280)
kind of assumes immortality.
Lex Fridman (1:50:43.800)
And then death is this terror that underlies it all.
Lex Fridman (1:50:47.040)
So like...
Jeff Hawkins (1:50:47.880)
Some people's world model, not mine.
Lex Fridman (1:50:50.400)
But, okay, so what Becker would say
Jeff Hawkins (1:50:52.760)
is that you're just living in an illusion.
Lex Fridman (1:50:54.520)
You've constructed an illusion for yourself
Jeff Hawkins (1:50:56.200)
because it's such a terrible terror,
Lex Fridman (1:50:59.000)
the fact that this...
Lex Fridman (1:51:00.160)
What's the illusion?
Lex Fridman (1:51:01.160)
The illusion that death doesn't matter.
Jeff Hawkins (1:51:02.640)
You're still not coming to grips with...
Lex Fridman (1:51:04.800)
The illusion of what?
Jeff Hawkins (1:51:05.620)
That death is...
Lex Fridman (1:51:07.120)
Going to happen.
Lex Fridman (1:51:08.700)
Oh, like it's not gonna happen?
Lex Fridman (1:51:10.440)
You're actually operating.
Jeff Hawkins (1:51:11.880)
You haven't, even though you said you've accepted it,
Lex Fridman (1:51:14.280)
you haven't really accepted the notion that you're gonna die
Jeff Hawkins (1:51:16.120)
is what you say.
Lex Fridman (1:51:16.960)
So it sounds like you disagree with that notion.
Jeff Hawkins (1:51:21.440)
Yeah, yeah, totally.
Lex Fridman (1:51:22.400)
I literally, every night I go to bed, it's like dying.
Jeff Hawkins (1:51:28.040)
Like little deaths.
Lex Fridman (1:51:28.880)
It's little deaths.
Lex Fridman (1:51:29.720)
And if I didn't wake up, it wouldn't matter to me.
Lex Fridman (1:51:32.960)
Only if I knew that was gonna happen would it be bothersome.
Lex Fridman (1:51:35.160)
If I didn't know it was gonna happen, how would I know?
Lex Fridman (1:51:37.600)
Then I would worry about my wife.
Lex Fridman (1:51:39.520)
So imagine I was a loner and I lived in Alaska
Lex Fridman (1:51:43.040)
and I lived out there and there was no animals.
Jeff Hawkins (1:51:45.420)
Nobody knew I existed.
Lex Fridman (1:51:46.480)
I was just eating these roots all the time.
Lex Fridman (1:51:48.720)
And nobody knew I was there.
Lex Fridman (1:51:51.120)
And one day I didn't wake up.
Lex Fridman (1:51:54.680)
What pain in the world would there exist?
Lex Fridman (1:51:57.040)
Well, so most people that think about this problem
Jeff Hawkins (1:51:59.800)
would say that you're just deeply enlightened
Lex Fridman (1:52:01.960)
or are completely delusional.
Jeff Hawkins (1:52:04.120)
One of the two.
Lex Fridman (1:52:05.920)
But I would say that's a very enlightened way
Jeff Hawkins (1:52:10.720)
to see the world.
Lex Fridman (1:52:13.120)
That's the rational one as well.
Jeff Hawkins (1:52:14.760)
It's rational, that's right.
Lex Fridman (1:52:15.760)
But the fact is we don't,
Jeff Hawkins (1:52:19.040)
I mean, we really don't have an understanding
Lex Fridman (1:52:22.360)
of why the heck it is we're born and why we die
Lex Fridman (1:52:24.920)
and what happens after we die.
Lex Fridman (1:52:25.960)
Well, maybe there isn't a reason, maybe there is.
Lex Fridman (1:52:27.880)
So I'm interested in those big problems too, right?
Lex Fridman (1:52:30.120)
You interviewed Max Tegmark,
Lex Fridman (1:52:32.560)
and there's people like that, right?
Lex Fridman (1:52:33.600)
I'm interested in those big problems as well.
Lex Fridman (1:52:35.240)
And in fact, when I was young,
Lex Fridman (1:52:38.240)
I made a list of the biggest problems I could think of.
Lex Fridman (1:52:41.200)
First, why does anything exist?
Lex Fridman (1:52:43.360)
Second, why do we have the laws of physics that we have?
Lex Fridman (1:52:46.600)
Third, is life inevitable?
Lex Fridman (1:52:49.200)
And why is it here?
Lex Fridman (1:52:50.120)
Fourth, is intelligence inevitable?
Lex Fridman (1:52:52.240)
And why is it here?
Jeff Hawkins (1:52:53.080)
I stopped there because I figured
Lex Fridman (1:52:55.000)
if you can make a truly intelligent system,
Jeff Hawkins (1:52:57.840)
that will be the quickest way
Lex Fridman (1:52:59.240)
to answer the first three questions.
Jeff Hawkins (1:53:01.040)
I'm serious.
Lex Fridman (1:53:04.440)
And so I said, my mission, you asked me earlier,
Jeff Hawkins (1:53:07.960)
my first mission is to understand the brain,
Lex Fridman (1:53:09.440)
but I felt that is the shortest way
Jeff Hawkins (1:53:10.760)
to get to true machine intelligence.
Lex Fridman (1:53:12.160)
And I wanna get to true machine intelligence
Jeff Hawkins (1:53:13.680)
because even if it doesn't occur in my lifetime,
Lex Fridman (1:53:15.920)
other people will benefit from it
Jeff Hawkins (1:53:17.480)
because I think it'll occur in my lifetime,
Lex Fridman (1:53:19.200)
but 20 years, you never know.
Lex Fridman (1:53:23.640)
But that will be the quickest way for us to,
Lex Fridman (1:53:26.080)
we can make super mathematicians,
Jeff Hawkins (1:53:27.800)
we can make super space explorers,
Lex Fridman (1:53:29.520)
we can make super physicist brains that do these things
Lex Fridman (1:53:34.240)
and that can run experiments that we can't run.
Lex Fridman (1:53:37.440)
We don't have the abilities to manipulate things and so on,
Lex Fridman (1:53:40.360)
but we can build intelligent machines that do all those things
Lex Fridman (1:53:42.800)
with the ultimate goal of finding out the answers
Jeff Hawkins (1:53:46.560)
to the other questions.
Lex Fridman (1:53:48.800)
Let me ask you another depressing and difficult question,
Jeff Hawkins (1:53:51.480)
which is once we achieve that goal of creating,
Lex Fridman (1:53:57.880)
no, of understanding intelligence,
Lex Fridman (1:54:01.200)
do you think we would be happier,
Lex Fridman (1:54:02.960)
more fulfilled as a species?
Jeff Hawkins (1:54:04.760)
The understanding intelligence
Lex Fridman (1:54:05.720)
or understanding the answers to the big questions?
Jeff Hawkins (1:54:07.920)
Understanding intelligence.
Lex Fridman (1:54:08.960)
Oh, totally, totally.
Jeff Hawkins (1:54:11.800)
It would be far more fun place to live.
Lex Fridman (1:54:13.960)
You think so?
Lex Fridman (1:54:14.800)
Oh yeah, why not?
Lex Fridman (1:54:15.680)
I mean, just put aside this terminator nonsense
Lex Fridman (1:54:19.720)
and just think about, you can think about,
Lex Fridman (1:54:24.320)
we can talk about the risks of AI if you want.
Jeff Hawkins (1:54:26.840)
I'd love to, so let's talk about.
Lex Fridman (1:54:28.240)
But I think the world would be far better knowing things.
Jeff Hawkins (1:54:30.640)
We're always better than know things.
Lex Fridman (1:54:32.080)
Do you think it's better, is it a better place to live in
Jeff Hawkins (1:54:35.080)
that I know that our planet is one of many
Lex Fridman (1:54:37.440)
in the solar system and the solar system's one of many
Lex Fridman (1:54:39.240)
in the galaxy?
Lex Fridman (1:54:40.080)
I think it's a more, I dread, I sometimes think like,
Lex Fridman (1:54:43.400)
God, what would it be like to live 300 years ago?
Lex Fridman (1:54:45.360)
I'd be looking up at the sky, I can't understand anything.
Jeff Hawkins (1:54:47.440)
Oh my God, I'd be like going to bed every night going,
Lex Fridman (1:54:49.240)
what's going on here?
Jeff Hawkins (1:54:50.160)
Well, I mean, in some sense I agree with you,
Lex Fridman (1:54:52.480)
but I'm not exactly sure.
Lex Fridman (1:54:54.800)
So I'm also a scientist, so I share your views,
Lex Fridman (1:54:57.240)
but I'm not, we're like rolling down the hill together.
Lex Fridman (1:55:02.640)
What's down the hill?
Lex Fridman (1:55:03.480)
I feel like we're climbing a hill.
Jeff Hawkins (1:55:05.280)
Whatever.
Lex Fridman (1:55:06.120)
We're getting closer to enlightenment
Lex Fridman (1:55:07.640)
and you're going down the hill.
Lex Fridman (1:55:10.200)
We're climbing, we're getting pulled up a hill
Jeff Hawkins (1:55:12.240)
by our curiosity.
Lex Fridman (1:55:13.840)
Our curiosity is, we're pulling ourselves up the hill
Jeff Hawkins (1:55:16.120)
by our curiosity.
Lex Fridman (1:55:16.960)
Yeah, Sisyphus was doing the same thing with the rock.
Jeff Hawkins (1:55:19.160)
Yeah, yeah, yeah, yeah.
Lex Fridman (1:55:20.880)
But okay, our happiness aside, do you have concerns
Jeff Hawkins (1:55:24.280)
about, you talk about Sam Harris, Elon Musk,
Lex Fridman (1:55:29.040)
of existential threats of intelligent systems?
Jeff Hawkins (1:55:31.880)
No, I'm not worried about existential threats at all.
Lex Fridman (1:55:33.800)
There are some things we really do need to worry about.
Jeff Hawkins (1:55:36.400)
Even today's AI, we have things we have to worry about.
Lex Fridman (1:55:38.440)
We have to worry about privacy
Lex Fridman (1:55:39.560)
and about how it impacts false beliefs in the world.
Lex Fridman (1:55:42.800)
And we have real problems and things to worry about
Jeff Hawkins (1:55:47.000)
with today's AI.
Lex Fridman (1:55:48.280)
And that will continue as we create more intelligent systems.
Jeff Hawkins (1:55:51.480)
There's no question, the whole issue
Lex Fridman (1:55:53.080)
about making intelligent armaments and weapons
Jeff Hawkins (1:55:57.080)
is something that really we have to think about carefully.
Lex Fridman (1:55:59.920)
I don't think of those as existential threats.
Jeff Hawkins (1:56:01.880)
I think those are the kind of threats we always face
Lex Fridman (1:56:04.320)
and we'll have to face them here
Lex Fridman (1:56:05.840)
and we'll have to deal with them.
Lex Fridman (1:56:10.400)
We could talk about what people think
Jeff Hawkins (1:56:12.040)
are the existential threats,
Lex Fridman (1:56:13.880)
but when I hear people talking about them,
Jeff Hawkins (1:56:16.200)
they all sound hollow to me.
Lex Fridman (1:56:17.760)
They're based on ideas, they're based on people
Jeff Hawkins (1:56:20.000)
who really have no idea what intelligence is.
Lex Fridman (1:56:22.160)
And if they knew what intelligence was,
Jeff Hawkins (1:56:24.920)
they wouldn't say those things.
Lex Fridman (1:56:26.640)
So those are not experts in the field.
Lex Fridman (1:56:28.600)
Yeah, so there's two, right?
Lex Fridman (1:56:32.040)
So one is like super intelligence.
Lex Fridman (1:56:33.720)
So a system that becomes far, far superior
Lex Fridman (1:56:37.720)
in reasoning ability than us humans.
Lex Fridman (1:56:43.160)
How is that an existential threat?
Lex Fridman (1:56:46.200)
Then, so there's a lot of ways in which it could be.
Jeff Hawkins (1:56:49.120)
One way is us humans are actually irrational, inefficient
Lex Fridman (1:56:54.040)
and get in the way of, not happiness,
Lex Fridman (1:57:00.520)
but whatever the objective function is
Lex Fridman (1:57:02.120)
of maximizing that objective function.
Jeff Hawkins (1:57:04.320)
Super intelligent.
Lex Fridman (1:57:05.160)
The paperclip problem and things like that.
Lex Fridman (1:57:06.720)
So the paperclip problem but with the super intelligent.
Lex Fridman (1:57:09.440)
Yeah, yeah, yeah, yeah.
Lex Fridman (1:57:10.480)
So we already face this threat in some sense.
Lex Fridman (1:57:15.680)
They're called bacteria.
Jeff Hawkins (1:57:17.320)
These are organisms in the world
Lex Fridman (1:57:18.960)
that would like to turn everything into bacteria.
Lex Fridman (1:57:21.400)
And they're constantly morphing,
Lex Fridman (1:57:23.040)
they're constantly changing to evade our protections.
Lex Fridman (1:57:26.360)
And in the past, they have killed huge swaths
Lex Fridman (1:57:30.680)
of populations of humans on this planet.
Lex Fridman (1:57:33.400)
So if you wanna worry about something
Lex Fridman (1:57:34.560)
that's gonna multiply endlessly, we have it.
Lex Fridman (1:57:38.360)
And I'm far more worried in that regard.
Lex Fridman (1:57:40.600)
I'm far more worried that some scientists in the laboratory
Jeff Hawkins (1:57:43.280)
will create a super virus or a super bacteria
Lex Fridman (1:57:45.440)
that we cannot control.
Jeff Hawkins (1:57:47.120)
That is a more of an existential threat.
Lex Fridman (1:57:49.640)
Putting an intelligence thing on top of it
Jeff Hawkins (1:57:52.160)
actually seems to make it less existential to me.
Lex Fridman (1:57:54.240)
It's like, it limits its power.
Jeff Hawkins (1:57:56.640)
It limits where it can go.
Lex Fridman (1:57:57.720)
It limits the number of things it can do in many ways.
Jeff Hawkins (1:57:59.820)
A bacteria is something you can't even see.
Lex Fridman (1:58:03.080)
So that's only one of those problems.
Jeff Hawkins (1:58:04.240)
Yes, exactly.
Lex Fridman (1:58:05.080)
So the other one, just in your intuition about intelligence,
Jeff Hawkins (1:58:09.600)
when you think about intelligence of us humans,
Lex Fridman (1:58:12.480)
do you think of that as something,
Jeff Hawkins (1:58:14.880)
if you look at intelligence on a spectrum
Lex Fridman (1:58:16.960)
from zero to us humans,
Lex Fridman (1:58:18.900)
do you think you can scale that to something far,
Lex Fridman (1:58:22.080)
far superior to all the mechanisms we've been talking about?
Jeff Hawkins (1:58:24.760)
I wanna make another point here, Lex, before I get there.
Lex Fridman (1:58:28.360)
Intelligence is the neocortex.
Jeff Hawkins (1:58:30.920)
It is not the entire brain.
Lex Fridman (1:58:34.080)
The goal is not to make a human.
Jeff Hawkins (1:58:36.200)
The goal is not to make an emotional system.
Lex Fridman (1:58:38.400)
The goal is not to make a system
Jeff Hawkins (1:58:39.560)
that wants to have sex and reproduce.
Lex Fridman (1:58:41.440)
Why would I build that?
Jeff Hawkins (1:58:42.880)
If I wanna have a system that wants to reproduce
Lex Fridman (1:58:44.560)
and have sex, make bacteria, make computer viruses.
Jeff Hawkins (1:58:47.260)
Those are bad things, don't do that.
Lex Fridman (1:58:49.800)
Those are really bad, don't do those things.
Jeff Hawkins (1:58:52.360)
Regulate those.
Lex Fridman (1:58:53.560)
But if I just say I want an intelligent system,
Lex Fridman (1:58:56.120)
why does it have to have any of the human like emotions?
Lex Fridman (1:58:58.560)
Why does it even care if it lives?
Lex Fridman (1:59:00.400)
Why does it even care if it has food?
Lex Fridman (1:59:02.560)
It doesn't care about those things.
Jeff Hawkins (1:59:03.840)
It's just, you know, it's just in a trance
Lex Fridman (1:59:06.320)
thinking about mathematics or it's out there
Jeff Hawkins (1:59:08.280)
just trying to build the space for it on Mars.
Lex Fridman (1:59:14.000)
That's a choice we make.
Jeff Hawkins (1:59:15.320)
Don't make human like things,
Lex Fridman (1:59:17.160)
don't make replicating things,
Jeff Hawkins (1:59:18.480)
don't make things that have emotions,
Lex Fridman (1:59:19.840)
just stick to the neocortex.
Lex Fridman (1:59:21.000)
So that's a view actually that I share
Lex Fridman (1:59:23.120)
but not everybody shares in the sense that
Jeff Hawkins (1:59:25.400)
you have faith and optimism about us as engineers of systems,
Lex Fridman (1:59:29.840)
humans as builders of systems to not put in stupid, not.
Lex Fridman (1:59:34.840)
So this is why I mentioned the bacteria one.
Lex Fridman (1:59:37.640)
Because you might say, well, some person's gonna do that.
Jeff Hawkins (1:59:40.760)
Well, some person today could create a bacteria
Lex Fridman (1:59:42.920)
that's resistant to all the known antibacterial agents.
Lex Fridman (1:59:46.920)
So we already have that threat.
Lex Fridman (1:59:49.200)
We already know this is going on.
Jeff Hawkins (1:59:51.320)
It's not a new threat.
Lex Fridman (1:59:52.760)
So just accept that and then we have to deal with it, right?
Jeff Hawkins (1:59:56.280)
Yeah, so my point is nothing to do with intelligence.
Lex Fridman (1:59:59.600)
Intelligence is a separate component
Lex Fridman (20:01.220)
and there's a sense of time related to them.
Lex Fridman (20:03.740)
Some don't, but most things do actually.
Lex Fridman (20:08.260)
So it's sort of infused throughout the models of the world.
Lex Fridman (20:12.100)
You build a model of the world,
Jeff Hawkins (20:13.700)
you're learning the structure of the objects in the world,
Lex Fridman (20:16.420)
and you're also learning how those things change
Jeff Hawkins (20:18.980)
through time.
Lex Fridman (20:20.780)
Okay, so it really is just a fourth dimension
Jeff Hawkins (20:23.900)
that's infused deeply, and you have to make sure
Lex Fridman (20:26.760)
that your models of intelligence incorporate it.
Jeff Hawkins (20:30.940)
So, like you mentioned, the state of neuroscience
Lex Fridman (20:34.840)
is deeply empirical, a lot of data collection.
Jeff Hawkins (20:37.800)
It's, you know, that's where it is.
Lex Fridman (20:41.420)
You mentioned Thomas Kuhn, right?
Jeff Hawkins (20:43.100)
Yeah.
Lex Fridman (20:44.580)
And then you're proposing a theory of intelligence,
Lex Fridman (20:48.020)
and which is really the next step,
Lex Fridman (20:50.460)
the really important step to take,
Lex Fridman (20:52.900)
but why is HTM, or what we'll talk about soon,
Lex Fridman (21:00.840)
the right theory?
Lex Fridman (21:03.700)
So is it more in the, is it backed by intuition?
Lex Fridman (21:07.700)
Is it backed by evidence?
Lex Fridman (21:09.920)
Is it backed by a mixture of both?
Lex Fridman (21:11.980)
Is it kind of closer to where string theory is in physics,
Jeff Hawkins (21:15.580)
where there's mathematical components
Lex Fridman (21:18.460)
which show that, you know what,
Jeff Hawkins (21:21.060)
it seems that this, it fits together too well
Lex Fridman (21:24.740)
for it not to be true, which is where string theory is.
Lex Fridman (21:28.100)
Is that where you're kind of seeing?
Lex Fridman (21:29.500)
It's a mixture of all those things,
Jeff Hawkins (21:30.740)
although definitely where we are right now
Lex Fridman (21:32.780)
is definitely much more on the empirical side
Jeff Hawkins (21:34.620)
than, let's say, string theory.
Lex Fridman (21:37.060)
The way this goes about, we're theorists, right?
Lex Fridman (21:39.280)
So we look at all this data, and we're trying to come up
Lex Fridman (21:41.580)
with some sort of model that explains it, basically,
Lex Fridman (21:44.340)
and there's, unlike string theory,
Lex Fridman (21:46.860)
there's vast more amounts of empirical data here
Jeff Hawkins (21:50.220)
that I think than most physicists deal with.
Lex Fridman (21:54.660)
And so our challenge is to sort through that
Lex Fridman (21:57.540)
and figure out what kind of constructs would explain this.
Lex Fridman (22:02.020)
And when we have an idea,
Jeff Hawkins (22:04.940)
you come up with a theory of some sort,
Lex Fridman (22:06.400)
you have lots of ways of testing it.
Jeff Hawkins (22:08.740)
First of all, there are 100 years of assimilated,
Lex Fridman (22:13.740)
assimilated, unassimilated empirical data from neuroscience.
Lex Fridman (22:16.620)
So we go back and read papers,
Lex Fridman (22:18.140)
and we say, oh, did someone find this already?
Jeff Hawkins (22:20.680)
We can predict X, Y, and Z,
Lex Fridman (22:23.280)
and maybe no one's even talked about it
Jeff Hawkins (22:25.220)
since 1972 or something, but we go back and find that,
Lex Fridman (22:28.180)
and we say, oh, either it can support the theory
Jeff Hawkins (22:31.140)
or it can invalidate the theory.
Lex Fridman (22:33.420)
And we say, okay, we have to start over again.
Jeff Hawkins (22:34.880)
Oh, no, it's supportive, let's keep going with that one.
Lex Fridman (22:38.140)
So the way I kind of view it, when we do our work,
Jeff Hawkins (22:42.260)
we look at all this empirical data,
Lex Fridman (22:45.460)
and what I call it is a set of constraints.
Jeff Hawkins (22:47.700)
We're not interested in something
Lex Fridman (22:48.700)
that's biologically inspired.
Jeff Hawkins (22:49.900)
We're trying to figure out how the actual brain works.
Lex Fridman (22:52.140)
So every piece of empirical data
Jeff Hawkins (22:53.660)
is a constraint on a theory.
Lex Fridman (22:55.500)
In theory, if you have the correct theory,
Lex Fridman (22:57.020)
it needs to explain every pin, right?
Lex Fridman (22:59.420)
So we have this huge number of constraints on the problem,
Jeff Hawkins (23:03.140)
which initially makes it very, very difficult.
Lex Fridman (23:05.960)
If you don't have many constraints,
Jeff Hawkins (23:07.220)
you can make up stuff all the day.
Lex Fridman (23:08.500)
You can say, oh, here's an answer on how you can do this,
Jeff Hawkins (23:10.200)
you can do that, you can do this.
Lex Fridman (23:11.360)
But if you consider all biology as a set of constraints,
Jeff Hawkins (23:13.760)
all neuroscience as a set of constraints,
Lex Fridman (23:15.580)
and even if you're working in one little part
Jeff Hawkins (23:17.240)
of the neocortex, for example,
Lex Fridman (23:18.380)
there are hundreds and hundreds of constraints.
Jeff Hawkins (23:20.620)
These are empirical constraints
Lex Fridman (23:22.460)
that it's very, very difficult initially
Jeff Hawkins (23:24.840)
to come up with a theoretical framework for that.
Lex Fridman (23:27.260)
But when you do, and it solves all those constraints
Jeff Hawkins (23:30.100)
at once, you have a high confidence
Lex Fridman (23:32.980)
that you got something close to correct.
Jeff Hawkins (23:35.660)
It's just mathematically almost impossible not to be.
Lex Fridman (23:39.160)
So that's the curse and the advantage of what we have.
Jeff Hawkins (23:43.900)
The curse is we have to solve,
Lex Fridman (23:45.260)
we have to meet all these constraints, which is really hard.
Lex Fridman (23:48.960)
But when you do meet them,
Lex Fridman (23:50.900)
then you have a great confidence
Jeff Hawkins (23:53.220)
that you've discovered something.
Lex Fridman (23:54.940)
In addition, then we work with scientific labs.
Lex Fridman (23:58.040)
So we'll say, oh, there's something we can't find,
Lex Fridman (24:00.000)
we can predict something,
Lex Fridman (24:01.260)
but we can't find it anywhere in the literature.
Lex Fridman (24:04.180)
So we will then, we have people we've collaborated with,
Lex Fridman (24:06.900)
we'll say, sometimes they'll say, you know what?
Lex Fridman (24:09.220)
I have some collected data, which I didn't publish,
Lex Fridman (24:11.740)
but we can go back and look at it
Lex Fridman (24:13.020)
and see if we can find that,
Jeff Hawkins (24:14.780)
which is much easier than designing a new experiment.
Lex Fridman (24:17.020)
You know, neuroscience experiments take a long time, years.
Jeff Hawkins (24:20.340)
So, although some people are doing that now too.
Lex Fridman (24:23.300)
So, but between all of these things,
Jeff Hawkins (24:27.740)
I think it's a reasonable,
Lex Fridman (24:30.020)
actually a very, very good approach.
Jeff Hawkins (24:31.620)
We are blessed with the fact that we can test our theories
Lex Fridman (24:35.020)
out the yin yang here because there's so much
Jeff Hawkins (24:37.100)
unassimilar data and we can also falsify our theories
Lex Fridman (24:39.640)
very easily, which we do often.
Lex Fridman (24:41.460)
So it's kind of reminiscent to whenever that was
Lex Fridman (24:44.380)
with Copernicus, you know, when you figure out
Jeff Hawkins (24:47.300)
that the sun's at the center of the solar system
Lex Fridman (24:51.140)
as opposed to earth, the pieces just fall into place.
Jeff Hawkins (24:54.900)
Yeah, I think that's the general nature of aha moments
Lex Fridman (24:59.580)
is, and it's Copernicus, it could be,
Jeff Hawkins (25:02.020)
you could say the same thing about Darwin,
Lex Fridman (25:05.220)
you could say the same thing about, you know,
Jeff Hawkins (25:07.580)
about the double helix,
Lex Fridman (25:09.660)
that people have been working on a problem for so long
Lex Fridman (25:12.780)
and have all this data and they can't make sense of it,
Lex Fridman (25:14.580)
they can't make sense of it.
Lex Fridman (25:15.820)
But when the answer comes to you
Lex Fridman (25:17.420)
and everything falls into place,
Jeff Hawkins (25:19.380)
it's like, oh my gosh, that's it.
Lex Fridman (25:21.660)
That's got to be right.
Jeff Hawkins (25:23.080)
I asked both Jim Watson and Francis Crick about this.
Lex Fridman (25:29.140)
I asked them, you know, when you were working on
Jeff Hawkins (25:31.700)
trying to discover the structure of the double helix,
Lex Fridman (25:35.760)
and when you came up with the sort of the structure
Jeff Hawkins (25:39.620)
that ended up being correct, but it was sort of a guess,
Lex Fridman (25:44.020)
you know, it wasn't really verified yet.
Lex Fridman (25:45.700)
I said, did you know that it was right?
Lex Fridman (25:48.460)
And they both said, absolutely.
Lex Fridman (25:50.220)
So we absolutely knew it was right.
Lex Fridman (25:51.860)
And it doesn't matter if other people didn't believe it
Jeff Hawkins (25:54.740)
or not, we knew it was right.
Lex Fridman (25:55.660)
They'd get around to thinking it
Lex Fridman (25:56.700)
and agree with it eventually anyway.
Lex Fridman (25:59.060)
And that's the kind of thing you hear a lot with scientists
Jeff Hawkins (26:01.300)
who really are studying a difficult problem.
Lex Fridman (26:04.220)
And I feel that way too about our work.
Jeff Hawkins (26:07.140)
Have you talked to Crick or Watson about the problem
Lex Fridman (26:10.700)
you're trying to solve, the, of finding the DNA of the brain?
Jeff Hawkins (26:15.940)
Yeah, in fact, Francis Crick was very interested in this
Lex Fridman (26:19.960)
in the latter part of his life.
Lex Fridman (26:21.540)
And in fact, I got interested in brains
Lex Fridman (26:23.780)
by reading an essay he wrote in 1979
Jeff Hawkins (26:26.900)
called Thinking About the Brain.
Lex Fridman (26:28.800)
And that was when I decided I'm gonna leave my profession
Jeff Hawkins (26:32.620)
of computers and engineering and become a neuroscientist.
Lex Fridman (26:35.580)
Just reading that one essay from Francis Crick.
Jeff Hawkins (26:37.660)
I got to meet him later in life.
Lex Fridman (26:41.640)
I spoke at the Salk Institute and he was in the audience.
Lex Fridman (26:44.660)
And then I had a tea with him afterwards.
Lex Fridman (26:48.820)
He was interested in a different problem.
Jeff Hawkins (26:50.620)
He was focused on consciousness.
Lex Fridman (26:53.380)
The easy problem, right?
Jeff Hawkins (26:54.260)
Well, I think it's the red herring.
Lex Fridman (26:58.640)
And so we weren't really overlapping a lot there.
Jeff Hawkins (27:02.260)
Jim Watson, who's still alive,
Lex Fridman (27:05.380)
is also interested in this problem.
Lex Fridman (27:07.420)
And he was, when he was director
Lex Fridman (27:09.020)
of the Cold Spring Harbor Laboratories,
Jeff Hawkins (27:12.420)
he was really sort of behind moving in the direction
Lex Fridman (27:15.140)
of neuroscience there.
Lex Fridman (27:16.580)
And so he had a personal interest in this field.
Lex Fridman (27:20.220)
And I have met with him numerous times.
Lex Fridman (27:23.620)
And in fact, the last time was a little bit over a year ago,
Lex Fridman (27:27.680)
I gave a talk at Cold Spring Harbor Labs
Jeff Hawkins (27:30.340)
about the progress we were making in our work.
Lex Fridman (27:34.620)
And it was a lot of fun because he said,
Jeff Hawkins (27:39.860)
well, you wouldn't be coming here
Lex Fridman (27:41.100)
unless you had something important to say.
Lex Fridman (27:42.380)
So I'm gonna go attend your talk.
Lex Fridman (27:44.740)
So he sat in the very front row.
Jeff Hawkins (27:46.620)
Next to him was the director of the lab, Bruce Stillman.
Lex Fridman (27:50.140)
So these guys are in the front row of this auditorium.
Jeff Hawkins (27:52.540)
Nobody else in the auditorium wants to sit in the front row
Lex Fridman (27:54.620)
because there's Jim Watson and there's the director.
Lex Fridman (27:56.980)
And I gave a talk and then I had dinner with him afterwards.
Lex Fridman (28:03.700)
But there's a great picture of my colleague Subitai Amantak
Jeff Hawkins (28:07.060)
where I'm up there sort of like screaming the basics
Lex Fridman (28:09.860)
of this new framework we have.
Lex Fridman (28:11.700)
And Jim Watson's on the edge of his chair.
Lex Fridman (28:13.780)
He's literally on the edge of his chair,
Jeff Hawkins (28:15.180)
like intently staring up at the screen.
Lex Fridman (28:17.820)
And when he discovered the structure of DNA,
Jeff Hawkins (28:21.740)
the first public talk he gave
Lex Fridman (28:23.800)
was at Cold Spring Harbor Labs.
Lex Fridman (28:25.940)
And there's a picture, there's a famous picture
Lex Fridman (28:27.460)
of Jim Watson standing at the whiteboard
Jeff Hawkins (28:29.340)
with an overrated thing pointing at something,
Lex Fridman (28:31.540)
pointing at the double helix with his pointer.
Lex Fridman (28:33.180)
And it actually looks a lot like the picture of me.
Lex Fridman (28:34.980)
So there was a sort of funny,
Jeff Hawkins (28:36.100)
there's Arian talking about the brain
Lex Fridman (28:37.460)
and there's Jim Watson staring intently at it.
Lex Fridman (28:39.300)
And of course there with, whatever, 60 years earlier,
Lex Fridman (28:41.620)
he was standing pointing at the double helix.
Jeff Hawkins (28:44.260)
That's one of the great discoveries in all of,
Lex Fridman (28:47.260)
whatever, biology, science, all science and DNA.
Lex Fridman (28:49.740)
So it's funny that there's echoes of that in your presentation.
Lex Fridman (28:54.540)
Do you think, in terms of evolutionary timeline and history,
Lex Fridman (28:58.360)
the development of the neocortex was a big leap?
Lex Fridman (29:01.960)
Or is it just a small step?
Lex Fridman (29:07.020)
So like, if we ran the whole thing over again,
Lex Fridman (29:09.780)
from the birth of life on Earth,
Lex Fridman (29:12.660)
how likely would we develop the mechanism of the neocortex?
Lex Fridman (29:15.260)
Okay, well those are two separate questions.
Lex Fridman (29:17.220)
One is, was it a big leap?
Lex Fridman (29:18.660)
And one was how likely it is, okay?
Jeff Hawkins (29:21.380)
They're not necessarily related.
Lex Fridman (29:22.880)
Maybe correlated.
Jeff Hawkins (29:23.720)
Maybe correlated, maybe not.
Lex Fridman (29:25.100)
And we don't really have enough data
Jeff Hawkins (29:26.100)
to make a judgment about that.
Lex Fridman (29:28.100)
I would say definitely it was a big leap.
Lex Fridman (29:29.980)
And I can tell you why.
Lex Fridman (29:30.980)
I don't think it was just another incremental step.
Jeff Hawkins (29:34.060)
I don't get that at the moment.
Lex Fridman (29:35.900)
I don't really have any idea how likely it is.
Jeff Hawkins (29:38.420)
If we look at evolution,
Lex Fridman (29:39.860)
we have one data point, which is Earth, right?
Jeff Hawkins (29:42.540)
Life formed on Earth billions of years ago,
Lex Fridman (29:45.220)
whether it was introduced here or it created it here,
Jeff Hawkins (29:48.100)
or someone introduced it, we don't really know,
Lex Fridman (29:49.560)
but it was here early.
Jeff Hawkins (29:51.220)
It took a long, long time to get to multicellular life.
Lex Fridman (29:55.140)
And then for multicellular life,
Jeff Hawkins (29:58.940)
it took a long, long time to get the neocortex.
Lex Fridman (2:00:01.920)
that you might apply to a system
Jeff Hawkins (2:00:03.560)
that wants to reproduce and do stupid things.
Lex Fridman (2:00:06.040)
Let's not do that.
Jeff Hawkins (2:00:07.240)
Yeah, in fact, it is a mystery
Lex Fridman (2:00:08.400)
why people haven't done that yet.
Jeff Hawkins (2:00:10.520)
My dad is a physicist, believes that the reason,
Lex Fridman (2:00:14.320)
he says, for example, nuclear weapons haven't proliferated
Jeff Hawkins (2:00:18.080)
amongst evil people.
Lex Fridman (2:00:19.040)
So one belief that I share is that
Jeff Hawkins (2:00:21.680)
there's not that many evil people in the world
Lex Fridman (2:00:25.160)
that would use, whether it's bacteria or nuclear weapons
Jeff Hawkins (2:00:31.280)
or maybe the future AI systems to do bad.
Lex Fridman (2:00:35.000)
So the fraction is small.
Lex Fridman (2:00:36.200)
And the second is that it's actually really hard,
Lex Fridman (2:00:38.400)
technically, so the intersection between evil
Lex Fridman (2:00:42.480)
and competent is small in terms of, and that's the.
Lex Fridman (2:00:45.160)
And by the way, to really annihilate humanity,
Jeff Hawkins (2:00:47.000)
you'd have to have sort of the nuclear winter phenomenon,
Lex Fridman (2:00:50.800)
which is not one person shooting or even 10 bombs.
Jeff Hawkins (2:00:54.080)
You'd have to have some automated system
Lex Fridman (2:00:56.440)
that detonates a million bombs
Jeff Hawkins (2:00:58.520)
or whatever many thousands we have.
Lex Fridman (2:01:00.400)
So extreme evil combined with extreme competence.
Lex Fridman (2:01:03.080)
And to start with building some stupid system
Lex Fridman (2:01:05.080)
that would automatically, Dr. Strangelove type of thing,
Jeff Hawkins (2:01:08.000)
you know, I mean, look, we could have
Lex Fridman (2:01:11.920)
some nuclear bomb go off in some major city in the world.
Jeff Hawkins (2:01:14.600)
I think that's actually quite likely, even in my lifetime.
Lex Fridman (2:01:17.120)
I don't think that's an unlikely thing.
Lex Fridman (2:01:18.480)
And it'd be a tragedy.
Lex Fridman (2:01:20.600)
But it won't be an existential threat.
Lex Fridman (2:01:23.160)
And it's the same as, you know, the virus of 1917,
Lex Fridman (2:01:26.560)
whatever it was, you know, the influenza.
Jeff Hawkins (2:01:30.000)
These bad things can happen and the plague and so on.
Lex Fridman (2:01:33.880)
We can't always prevent them.
Jeff Hawkins (2:01:35.360)
We always try, but we can't.
Lex Fridman (2:01:37.040)
But they're not existential threats
Jeff Hawkins (2:01:38.240)
until we combine all those crazy things together.
Lex Fridman (2:01:41.200)
So on the spectrum of intelligence from zero to human,
Lex Fridman (2:01:45.440)
do you have a sense of whether it's possible
Lex Fridman (2:01:47.960)
to create several orders of magnitude
Lex Fridman (2:01:51.560)
or at least double that of human intelligence?
Lex Fridman (2:01:54.680)
Talking about neuro context.
Jeff Hawkins (2:01:55.920)
I think it's the wrong thing to say double the intelligence.
Lex Fridman (2:01:59.000)
Break it down into different components.
Jeff Hawkins (2:02:01.600)
Can I make something that's a million times fast
Lex Fridman (2:02:03.640)
than a human brain?
Jeff Hawkins (2:02:04.520)
Yes, I can do that.
Lex Fridman (2:02:06.280)
Could I make something that is,
Lex Fridman (2:02:09.160)
has a lot more storage than the human brain?
Lex Fridman (2:02:10.960)
Yes, I could do that.
Jeff Hawkins (2:02:11.880)
More common, more copies of common.
Lex Fridman (2:02:13.600)
Can I make something that attaches
Lex Fridman (2:02:14.720)
to different sensors than human brain?
Lex Fridman (2:02:16.160)
Yes, I can do that.
Lex Fridman (2:02:17.280)
Could I make something that's distributed?
Lex Fridman (2:02:19.280)
So these people, yeah, we talked early
Jeff Hawkins (2:02:21.160)
about the departure of the neocortex voting.
Lex Fridman (2:02:23.120)
They don't have to be co located.
Jeff Hawkins (2:02:24.240)
Like, you know, they can be all around the place.
Lex Fridman (2:02:25.680)
I could do that too.
Lex Fridman (2:02:29.080)
Those are the levers I have, but is it more intelligent?
Lex Fridman (2:02:32.440)
Well, it depends what I train it on.
Lex Fridman (2:02:33.760)
What is it doing?
Lex Fridman (2:02:34.800)
If it's.
Jeff Hawkins (2:02:35.640)
Well, so here's the thing.
Lex Fridman (2:02:36.720)
So let's say larger neocortex
Lex Fridman (2:02:39.400)
and or whatever size that allows for higher
Lex Fridman (2:02:44.720)
and higher hierarchies to form,
Jeff Hawkins (2:02:47.920)
we're talking about reference frames and concepts.
Lex Fridman (2:02:50.160)
Could I have something that's a super physicist
Lex Fridman (2:02:51.920)
or a super mathematician?
Lex Fridman (2:02:52.920)
Yes.
Lex Fridman (2:02:53.760)
And the question is, once you have a super physicist,
Lex Fridman (2:02:56.680)
will they be able to understand something?
Lex Fridman (2:03:00.440)
Do you have a sense that it will be orders of math,
Lex Fridman (2:03:02.200)
like us compared to ants?
Lex Fridman (2:03:03.040)
Could we ever understand it?
Lex Fridman (2:03:04.560)
Yeah.
Jeff Hawkins (2:03:06.080)
Most people cannot understand general relativity.
Lex Fridman (2:03:11.920)
It's a really hard thing to get.
Jeff Hawkins (2:03:13.280)
I mean, yeah, you can paint it in a fuzzy picture,
Lex Fridman (2:03:15.800)
stretchy space, you know?
Lex Fridman (2:03:17.560)
But the field equations to do that
Lex Fridman (2:03:19.920)
and the deep intuitions are really, really hard.
Lex Fridman (2:03:23.080)
And I've tried, I'm unable to do it.
Lex Fridman (2:03:25.960)
Like it's easy to get special relativity,
Lex Fridman (2:03:28.800)
but general relativity, man, that's too much.
Lex Fridman (2:03:32.360)
And so we already live with this to some extent.
Jeff Hawkins (2:03:34.960)
The vast majority of people can't understand actually
Lex Fridman (2:03:37.320)
what the vast majority of other people actually know.
Jeff Hawkins (2:03:40.280)
We're just, either we don't have the effort to,
Lex Fridman (2:03:41.960)
or we can't, or we don't have time,
Jeff Hawkins (2:03:43.280)
or just not smart enough, whatever.
Lex Fridman (2:03:46.920)
But we have ways of communicating.
Jeff Hawkins (2:03:48.560)
Einstein has spoken in a way that I can understand.
Lex Fridman (2:03:51.600)
He's given me analogies that are useful.
Jeff Hawkins (2:03:54.600)
I can use those analogies from my own work
Lex Fridman (2:03:56.880)
and think about concepts that are similar.
Jeff Hawkins (2:04:01.040)
It's not stupid.
Lex Fridman (2:04:02.200)
It's not like he's existing some other plane
Lex Fridman (2:04:04.040)
and there's no connection with my plane in the world here.
Lex Fridman (2:04:06.680)
So that will occur.
Jeff Hawkins (2:04:07.840)
It already has occurred.
Lex Fridman (2:04:09.280)
That's what my point of this story is.
Jeff Hawkins (2:04:10.760)
It already has occurred.
Lex Fridman (2:04:11.720)
We live it every day.
Jeff Hawkins (2:04:14.360)
One could argue that when we create machine intelligence
Lex Fridman (2:04:17.040)
that think a million times faster than us
Jeff Hawkins (2:04:18.720)
that it'll be so far we can't make the connections.
Lex Fridman (2:04:20.920)
But you know, at the moment,
Jeff Hawkins (2:04:23.480)
everything that seems really, really hard
Lex Fridman (2:04:25.640)
to figure out in the world,
Jeff Hawkins (2:04:26.680)
when you actually figure it out, it's not that hard.
Lex Fridman (2:04:29.000)
You know, almost everyone can understand the multiverses.
Jeff Hawkins (2:04:32.160)
Almost everyone can understand quantum physics.
Lex Fridman (2:04:34.040)
Almost everyone can understand these basic things,
Jeff Hawkins (2:04:36.120)
even though hardly any people could figure those things out.
Lex Fridman (2:04:39.000)
Yeah, but really understand.
Lex Fridman (2:04:41.320)
But you don't need to really.
Lex Fridman (2:04:42.360)
Only a few people really understand.
Jeff Hawkins (2:04:43.800)
You need to only understand the projections,
Lex Fridman (2:04:47.880)
the sprinkles of the useful insights from that.
Lex Fridman (2:04:50.120)
That was my example of Einstein, right?
Lex Fridman (2:04:51.760)
His general theory of relativity is one thing
Jeff Hawkins (2:04:53.800)
that very, very, very few people can get.
Lex Fridman (2:04:56.000)
And what if we just said those other few people
Lex Fridman (2:04:58.240)
are also artificial intelligences?
Lex Fridman (2:05:00.600)
How bad is that?
Lex Fridman (2:05:01.440)
In some sense they are, right?
Lex Fridman (2:05:02.720)
Yeah, they say already.
Jeff Hawkins (2:05:04.280)
I mean, Einstein wasn't a really normal person.
Lex Fridman (2:05:06.280)
He had a lot of weird quirks.
Lex Fridman (2:05:07.560)
And so did the other people who worked with him.
Lex Fridman (2:05:09.440)
So, you know, maybe they already were sort of
Jeff Hawkins (2:05:11.360)
this astral plane of intelligence that,
Lex Fridman (2:05:14.200)
we live with it already.
Jeff Hawkins (2:05:15.240)
It's not a problem.
Lex Fridman (2:05:17.000)
It's still useful and, you know.
Lex Fridman (2:05:20.160)
So do you think we are the only intelligent life
Lex Fridman (2:05:22.960)
out there in the universe?
Jeff Hawkins (2:05:24.880)
I would say that intelligent life
Lex Fridman (2:05:27.880)
has and will exist elsewhere in the universe.
Jeff Hawkins (2:05:29.760)
I'll say that.
Lex Fridman (2:05:31.480)
There was a question about
Jeff Hawkins (2:05:32.600)
contemporaneous intelligence life,
Lex Fridman (2:05:34.080)
which is hard to even answer
Jeff Hawkins (2:05:35.600)
when we think about relativity and the nature of space time.
Lex Fridman (2:05:39.480)
Can't say what exactly is this time
Jeff Hawkins (2:05:41.120)
someplace else in the world.
Lex Fridman (2:05:43.160)
But I think it's, you know,
Jeff Hawkins (2:05:44.600)
I do worry a lot about the filter idea,
Lex Fridman (2:05:48.440)
which is that perhaps intelligent species
Jeff Hawkins (2:05:52.240)
don't last very long.
Lex Fridman (2:05:54.040)
And so we haven't been around very long.
Lex Fridman (2:05:55.720)
And as a technological species,
Lex Fridman (2:05:57.200)
we've been around for almost nothing, you know.
Jeff Hawkins (2:05:59.800)
What, 200 years, something like that.
Lex Fridman (2:06:02.720)
And we don't have any data,
Jeff Hawkins (2:06:04.160)
a good data point on whether it's likely
Lex Fridman (2:06:06.040)
that we'll survive or not.
Lex Fridman (2:06:08.480)
So do I think that there have been intelligent life
Lex Fridman (2:06:10.960)
elsewhere in the universe?
Jeff Hawkins (2:06:11.800)
Almost certainly, of course.
Lex Fridman (2:06:13.400)
In the past, in the future, yes.
Lex Fridman (2:06:16.440)
Does it survive for a long time?
Lex Fridman (2:06:17.880)
I don't know.
Jeff Hawkins (2:06:18.840)
This is another reason I'm excited about our work,
Lex Fridman (2:06:21.120)
is our work meaning the general world of AI.
Jeff Hawkins (2:06:24.240)
I think we can build intelligent machines
Lex Fridman (2:06:28.640)
that outlast us.
Jeff Hawkins (2:06:32.040)
You know, they don't have to be tied to Earth.
Lex Fridman (2:06:34.080)
They don't have to, you know,
Jeff Hawkins (2:06:35.800)
I'm not saying they're recreating, you know,
Lex Fridman (2:06:38.200)
aliens, I'm just saying,
Jeff Hawkins (2:06:40.680)
if I asked myself,
Lex Fridman (2:06:41.920)
and this might be a good point to end on here.
Jeff Hawkins (2:06:44.280)
If I asked myself, you know,
Lex Fridman (2:06:45.120)
what's special about our species?
Jeff Hawkins (2:06:47.240)
We're not particularly interesting physically.
Lex Fridman (2:06:49.040)
We don't fly, we're not good swimmers,
Jeff Hawkins (2:06:51.480)
we're not very fast, we're not very strong, you know.
Lex Fridman (2:06:54.000)
It's our brain, that's the only thing.
Lex Fridman (2:06:55.480)
And we are the only species on this planet
Lex Fridman (2:06:57.440)
that's built the model of the world
Jeff Hawkins (2:06:58.760)
that extends beyond what we can actually sense.
Lex Fridman (2:07:01.160)
We're the only people who know about
Jeff Hawkins (2:07:03.000)
the far side of the moon, and the other universes,
Lex Fridman (2:07:05.160)
and I mean, other galaxies, and other stars,
Lex Fridman (2:07:07.280)
and about what happens in the atom.
Lex Fridman (2:07:09.520)
There's no, that knowledge doesn't exist anywhere else.
Jeff Hawkins (2:07:12.440)
It's only in our heads.
Lex Fridman (2:07:13.800)
Cats don't do it, dogs don't do it,
Jeff Hawkins (2:07:15.000)
monkeys don't do it, it's just on.
Lex Fridman (2:07:16.360)
And that is what we've created that's unique.
Jeff Hawkins (2:07:18.320)
Not our genes, it's knowledge.
Lex Fridman (2:07:20.360)
And if I asked me, what is the legacy of humanity?
Lex Fridman (2:07:23.160)
What should our legacy be?
Lex Fridman (2:07:25.120)
It should be knowledge.
Jeff Hawkins (2:07:25.960)
We should preserve our knowledge
Lex Fridman (2:07:27.560)
in a way that it can exist beyond us.
Lex Fridman (2:07:30.080)
And I think the best way of doing that,
Lex Fridman (2:07:32.040)
in fact you have to do it,
Jeff Hawkins (2:07:33.080)
is it has to go along with intelligent machines
Lex Fridman (2:07:34.880)
that understand that knowledge.
Jeff Hawkins (2:07:37.800)
It's a very broad idea, but we should be thinking,
Lex Fridman (2:07:41.920)
I call it a state planning for humanity.
Jeff Hawkins (2:07:43.800)
We should be thinking about what we wanna leave behind
Lex Fridman (2:07:46.560)
when as a species we're no longer here.
Lex Fridman (2:07:49.320)
And that'll happen sometime.
Lex Fridman (2:07:51.080)
Sooner or later it's gonna happen.
Lex Fridman (2:07:52.480)
And understanding intelligence and creating intelligence
Lex Fridman (2:07:56.080)
gives us a better chance to prolong.
Jeff Hawkins (2:07:58.400)
It does give us a better chance to prolong life, yes.
Lex Fridman (2:08:01.120)
It gives us a chance to live on other planets.
Lex Fridman (2:08:03.200)
But even beyond that, I mean our solar system
Lex Fridman (2:08:06.080)
will disappear one day, just given enough time.
Lex Fridman (2:08:08.680)
So I don't know, I doubt we'll ever be able to travel
Lex Fridman (2:08:12.880)
to other things, but we could tell the stars,
Lex Fridman (2:08:15.480)
but we could send intelligent machines to do that.
Lex Fridman (2:08:17.800)
So you have an optimistic, a hopeful view of our knowledge
Jeff Hawkins (2:08:23.160)
of the echoes of human civilization
Lex Fridman (2:08:26.040)
living through the intelligent systems we create?
Jeff Hawkins (2:08:29.240)
Oh, totally.
Lex Fridman (2:08:30.080)
Well, I think the intelligent systems we create
Jeff Hawkins (2:08:31.400)
are in some sense the vessel for bringing them beyond Earth
Lex Fridman (2:08:36.200)
or making them last beyond humans themselves.
Lex Fridman (2:08:39.840)
How do you feel about that?
Lex Fridman (2:08:41.280)
That they won't be human, quote unquote?
Lex Fridman (2:08:43.640)
Who cares?
Lex Fridman (2:08:45.080)
Human, what is human?
Jeff Hawkins (2:08:46.120)
Our species are changing all the time.
Lex Fridman (2:08:48.640)
Human today is not the same as human just 50 years ago.
Lex Fridman (2:08:52.600)
What is human?
Lex Fridman (2:08:53.440)
Do we care about our genetics?
Lex Fridman (2:08:54.520)
Why is that important?
Lex Fridman (2:08:56.160)
As I point out, our genetics are no more interesting
Jeff Hawkins (2:08:58.320)
than a bacterium's genetics.
Lex Fridman (2:08:59.440)
It's no more interesting than a monkey's genetics.
Lex Fridman (2:09:01.720)
What we have, what's unique and what's valuable
Lex Fridman (2:09:04.560)
is our knowledge, what we've learned about the world.
Lex Fridman (2:09:07.400)
And that is the rare thing.
Lex Fridman (2:09:09.640)
That's the thing we wanna preserve.
Lex Fridman (2:09:11.480)
It's, who cares about our genes?
Lex Fridman (2:09:13.640)
That's not.
Jeff Hawkins (2:09:14.480)
It's the knowledge.
Lex Fridman (2:09:16.280)
It's the knowledge.
Jeff Hawkins (2:09:17.120)
That's a really good place to end.
Lex Fridman (2:09:19.080)
Thank you so much for talking to me.
Jeff Hawkins (2:09:20.120)
No, it was fun.
Lex Fridman (30:02.300)
And we've only had the neocortex for a few 100,000 years.
Lex Fridman (30:05.460)
So that's like nothing, okay?
Lex Fridman (30:08.000)
So is it likely?
Jeff Hawkins (30:09.600)
Well, it certainly isn't something
Lex Fridman (30:10.740)
that happened right away on Earth.
Lex Fridman (30:13.560)
And there were multiple steps to get there.
Lex Fridman (30:15.200)
So I would say it's probably not gonna be something
Jeff Hawkins (30:17.220)
that would happen instantaneously
Lex Fridman (30:18.260)
on other planets that might have life.
Jeff Hawkins (30:20.620)
It might take several billion years on average.
Lex Fridman (30:23.160)
Is it likely?
Jeff Hawkins (30:24.380)
I don't know, but you'd have to survive
Lex Fridman (30:25.740)
for several billion years to find out.
Jeff Hawkins (30:27.900)
Probably.
Lex Fridman (30:29.340)
Is it a big leap?
Jeff Hawkins (30:30.260)
Yeah, I think it is a qualitative difference
Lex Fridman (30:35.500)
in all other evolutionary steps.
Jeff Hawkins (30:37.860)
I can try to describe that if you'd like.
Lex Fridman (30:39.820)
Sure, in which way?
Jeff Hawkins (30:41.980)
Yeah, I can tell you how.
Lex Fridman (30:43.940)
Pretty much, let's start with a little preface.
Jeff Hawkins (30:47.740)
Many of the things that humans are able to do
Lex Fridman (30:50.500)
do not have obvious survival advantages precedent.
Jeff Hawkins (30:58.620)
We could create music, is that,
Lex Fridman (31:00.260)
is there a really survival advantage to that?
Jeff Hawkins (31:02.700)
Maybe, maybe not.
Lex Fridman (31:03.900)
What about mathematics?
Lex Fridman (31:04.900)
Is there a real survival advantage to mathematics?
Lex Fridman (31:07.020)
Well, you could stretch it.
Lex Fridman (31:09.340)
You can try to figure these things out, right?
Lex Fridman (31:13.140)
But most of evolutionary history,
Jeff Hawkins (31:14.800)
everything had immediate survival advantages to it.
Lex Fridman (31:18.700)
So, I'll tell you a story, which I like,
Jeff Hawkins (31:22.020)
may or may not be true, but the story goes as follows.
Lex Fridman (31:29.140)
Organisms have been evolving for,
Jeff Hawkins (31:30.860)
since the beginning of life here on Earth,
Lex Fridman (31:33.740)
and adding this sort of complexity onto that,
Lex Fridman (31:35.700)
and this sort of complexity onto that,
Lex Fridman (31:36.860)
and the brain itself is evolved this way.
Jeff Hawkins (31:39.700)
In fact, there's old parts, and older parts,
Lex Fridman (31:42.420)
and older, older parts of the brain
Jeff Hawkins (31:43.740)
that kind of just keeps calming on new things,
Lex Fridman (31:45.500)
and we keep adding capabilities.
Jeff Hawkins (31:47.260)
When we got to the neocortex,
Lex Fridman (31:48.700)
initially it had a very clear survival advantage
Jeff Hawkins (31:52.500)
in that it produced better vision,
Lex Fridman (31:54.380)
and better hearing, and better touch,
Lex Fridman (31:55.700)
and maybe, and so on.
Lex Fridman (31:57.780)
But what I think happens is that evolution discovered,
Jeff Hawkins (32:01.140)
it took a mechanism, and this is in our recent theories,
Lex Fridman (32:05.100)
but it took a mechanism evolved a long time ago
Jeff Hawkins (32:08.140)
for navigating in the world, for knowing where you are.
Lex Fridman (32:10.380)
These are the so called grid cells and place cells
Jeff Hawkins (32:13.360)
of an old part of the brain.
Lex Fridman (32:15.160)
And it took that mechanism for building maps of the world,
Lex Fridman (32:20.900)
and knowing where you are on those maps,
Lex Fridman (32:22.580)
and how to navigate those maps,
Lex Fridman (32:24.140)
and turns it into a sort of a slimmed down,
Lex Fridman (32:27.060)
idealized version of it.
Lex Fridman (32:29.540)
And that idealized version could now apply
Lex Fridman (32:31.600)
to building maps of other things.
Jeff Hawkins (32:32.820)
Maps of coffee cups, and maps of phones,
Lex Fridman (32:35.100)
maps of mathematics.
Jeff Hawkins (32:36.460)
Concepts almost.
Lex Fridman (32:37.300)
Concepts, yes, and not just almost, exactly.
Lex Fridman (32:40.260)
And so, and it just started replicating this stuff, right?
Lex Fridman (32:44.140)
You just think more, and more, and more.
Lex Fridman (32:45.220)
So we went from being sort of dedicated purpose
Lex Fridman (32:48.780)
neural hardware to solve certain problems
Jeff Hawkins (32:51.460)
that are important to survival,
Lex Fridman (32:53.200)
to a general purpose neural hardware
Jeff Hawkins (32:55.820)
that could be applied to all problems.
Lex Fridman (32:58.100)
And now it's escaped the orbit of survival.
Jeff Hawkins (33:02.600)
We are now able to apply it to things
Lex Fridman (33:04.460)
which we find enjoyment,
Lex Fridman (33:08.700)
but aren't really clearly survival characteristics.
Lex Fridman (33:13.700)
And that it seems to only have happened in humans,
Jeff Hawkins (33:16.740)
to the large extent.
Lex Fridman (33:19.260)
And so that's what's going on,
Jeff Hawkins (33:20.980)
where we sort of have,
Lex Fridman (33:22.940)
we've sort of escaped the gravity of evolutionary pressure,
Jeff Hawkins (33:26.360)
in some sense, in the neocortex.
Lex Fridman (33:28.620)
And it now does things which are not,
Jeff Hawkins (33:31.540)
that are really interesting,
Lex Fridman (33:32.780)
discovering models of the universe,
Jeff Hawkins (33:34.340)
which may not really help us.
Lex Fridman (33:36.100)
Does it matter?
Lex Fridman (33:37.100)
How does it help us surviving,
Lex Fridman (33:38.600)
knowing that there might be multiverses,
Jeff Hawkins (33:40.240)
or that there might be the age of the universe,
Lex Fridman (33:42.940)
or how do various stellar things occur?
Jeff Hawkins (33:46.140)
It doesn't really help us survive at all.
Lex Fridman (33:47.820)
But we enjoy it, and that's what happened.
Jeff Hawkins (33:50.460)
Or at least not in the obvious way, perhaps.
Lex Fridman (33:53.300)
It is required,
Jeff Hawkins (33:56.200)
if you look at the entire universe in an evolutionary way,
Lex Fridman (33:58.540)
it's required for us to do interplanetary travel,
Lex Fridman (34:00.900)
and therefore survive past our own sun.
Lex Fridman (34:03.140)
But you know, let's not get too.
Jeff Hawkins (34:04.500)
Yeah, but evolution works at one time frame,
Lex Fridman (34:07.220)
it's survival, if you think of survival of the phenotype,
Jeff Hawkins (34:11.340)
survival of the individual.
Lex Fridman (34:13.180)
What you're talking about there is spans well beyond that.
Lex Fridman (34:16.360)
So there's no genetic,
Lex Fridman (34:18.740)
I'm not transferring any genetic traits to my children
Jeff Hawkins (34:23.420)
that are gonna help them survive better on Mars.
Lex Fridman (34:26.540)
Totally different mechanism, that's right.
Lex Fridman (34:28.260)
So let's get into the new, as you've mentioned,
Lex Fridman (34:31.340)
this idea of the, I don't know if you have a nice name,
Jeff Hawkins (34:34.860)
thousand.
Lex Fridman (34:35.700)
We call it the thousand brain theory of intelligence.
Jeff Hawkins (34:37.340)
I like it.
Lex Fridman (34:38.180)
Can you talk about this idea of a spatial view of concepts
Lex Fridman (34:43.620)
and so on?
Lex Fridman (34:44.460)
Yeah, so can I just describe sort of the,
Jeff Hawkins (34:46.500)
there's an underlying core discovery,
Lex Fridman (34:49.300)
which then everything comes from that.
Jeff Hawkins (34:51.140)
That's a very simple, this is really what happened.
Lex Fridman (34:55.660)
We were deep into problems about understanding
Lex Fridman (34:58.580)
how we build models of stuff in the world
Lex Fridman (35:00.540)
and how we make predictions about things.
Lex Fridman (35:03.020)
And I was holding a coffee cup just like this in my hand.
Lex Fridman (35:07.220)
And my finger was touching the side, my index finger.
Lex Fridman (35:10.540)
And then I moved it to the top
Lex Fridman (35:12.700)
and I was gonna feel the rim at the top of the cup.
Lex Fridman (35:15.460)
And I asked myself a very simple question.
Lex Fridman (35:18.280)
I said, well, first of all, I say,
Jeff Hawkins (35:20.100)
I know that my brain predicts what it's gonna feel
Lex Fridman (35:22.260)
before it touches it.
Jeff Hawkins (35:23.300)
You can just think about it and imagine it.
Lex Fridman (35:26.040)
And so we know that the brain's making predictions
Jeff Hawkins (35:27.660)
all the time.
Lex Fridman (35:28.500)
So the question is, what does it take to predict that?
Lex Fridman (35:31.540)
And there's a very interesting answer.
Lex Fridman (35:33.620)
First of all, it says the brain has to know
Jeff Hawkins (35:35.400)
it's touching a coffee cup.
Lex Fridman (35:36.500)
It has to have a model of a coffee cup.
Jeff Hawkins (35:38.020)
It needs to know where the finger currently is
Lex Fridman (35:41.020)
on the cup relative to the cup.
Jeff Hawkins (35:43.260)
Because when I make a movement,
Lex Fridman (35:44.420)
it needs to know where it's going to be on the cup
Jeff Hawkins (35:46.340)
after the movement is completed relative to the cup.
Lex Fridman (35:50.380)
And then it can make a prediction
Jeff Hawkins (35:51.900)
about what it's gonna sense.
Lex Fridman (35:53.340)
So this told me that the neocortex,
Jeff Hawkins (35:54.960)
which is making this prediction,
Lex Fridman (35:56.380)
needs to know that it's sensing it's touching a cup.
Lex Fridman (35:59.420)
And it needs to know the location of my finger
Lex Fridman (36:01.420)
relative to that cup in a reference frame of the cup.
Jeff Hawkins (36:04.380)
It doesn't matter where the cup is relative to my body.
Lex Fridman (36:06.300)
It doesn't matter its orientation.
Jeff Hawkins (36:08.260)
None of that matters.
Lex Fridman (36:09.160)
It's where my finger is relative to the cup,
Jeff Hawkins (36:10.940)
which tells me then that the neocortex
Lex Fridman (36:13.540)
has a reference frame that's anchored to the cup.
Jeff Hawkins (36:17.340)
Because otherwise I wouldn't be able to say the location
Lex Fridman (36:19.280)
and I wouldn't be able to predict my new location.
Lex Fridman (36:21.500)
And then we quickly, very instantly can say,
Lex Fridman (36:24.120)
well, every part of my skin could touch this cup.
Lex Fridman (36:26.240)
And therefore every part of my skin is making predictions
Lex Fridman (36:28.100)
and every part of my skin must have a reference frame
Jeff Hawkins (36:30.940)
that it's using to make predictions.
Lex Fridman (36:33.520)
So the big idea is that throughout the neocortex,
Jeff Hawkins (36:39.500)
there are, everything is being stored
Lex Fridman (36:44.940)
and referenced in reference frames.
Jeff Hawkins (36:46.740)
You can think of them like XYZ reference frames,
Lex Fridman (36:48.820)
but they're not like that.
Jeff Hawkins (36:50.380)
We know a lot about the neural mechanisms for this,
Lex Fridman (36:52.060)
but the brain thinks in reference frames.
Lex Fridman (36:54.860)
And as an engineer, if you're an engineer,
Lex Fridman (36:56.700)
this is not surprising.
Jeff Hawkins (36:57.740)
You'd say, if I wanted to build a CAD model
Lex Fridman (37:00.340)
of the coffee cup, well, I would bring it up
Lex Fridman (37:02.120)
and some CAD software, and I would assign
Lex Fridman (37:04.100)
some reference frame and say this features
Jeff Hawkins (37:05.460)
at this locations and so on.
Lex Fridman (37:06.980)
But the fact that this, the idea that this is occurring
Jeff Hawkins (37:09.700)
throughout the neocortex everywhere, it was a novel idea.
Lex Fridman (37:14.360)
And then a zillion things fell into place after that,
Jeff Hawkins (37:19.080)
a zillion.
Lex Fridman (37:19.940)
So now we think about the neocortex
Jeff Hawkins (37:21.860)
as processing information quite differently
Lex Fridman (37:23.420)
than we used to do it.
Jeff Hawkins (37:24.260)
We used to think about the neocortex
Lex Fridman (37:25.540)
as processing sensory data and extracting features
Jeff Hawkins (37:28.700)
from that sensory data and then extracting features
Lex Fridman (37:30.860)
from the features, very much like a deep learning network
Jeff Hawkins (37:33.580)
does today.
Lex Fridman (37:34.900)
But that's not how the brain works at all.
Jeff Hawkins (37:36.620)
The brain works by assigning everything,
Lex Fridman (37:39.300)
every input, everything to reference frames.
Lex Fridman (37:41.860)
And there are thousands, hundreds of thousands
Lex Fridman (37:44.380)
of them active at once in your neocortex.
Jeff Hawkins (37:47.660)
It's a surprising thing to think about,
Lex Fridman (37:49.580)
but once you sort of internalize this,
Jeff Hawkins (37:51.060)
you understand that it explains almost every,
Lex Fridman (37:54.380)
almost all the mysteries we've had about this structure.
Lex Fridman (37:57.780)
So one of the consequences of that
Lex Fridman (38:00.200)
is that every small part of the neocortex,
Jeff Hawkins (38:02.620)
say a millimeter square, and there's 150,000 of those.
Lex Fridman (38:06.340)
So it's about 150,000 square millimeters.
Jeff Hawkins (38:08.620)
If you take every little square millimeter of the cortex,
Lex Fridman (38:11.380)
it's got some input coming into it
Lex Fridman (38:13.260)
and it's gonna have reference frames
Lex Fridman (38:14.940)
where it's assigned that input to.
Lex Fridman (38:16.800)
And each square millimeter can learn
Lex Fridman (38:19.320)
complete models of objects.
Lex Fridman (38:20.980)
So what do I mean by that?
Lex Fridman (38:22.020)
If I'm touching the coffee cup,
Jeff Hawkins (38:23.300)
well, if I just touch it in one place,
Lex Fridman (38:25.580)
I can't learn what this coffee cup is
Jeff Hawkins (38:27.180)
because I'm just feeling one part.
Lex Fridman (38:28.980)
But if I move it around the cup
Lex Fridman (38:31.060)
and touch it at different areas,
Lex Fridman (38:32.540)
I can build up a complete model of the cup
Jeff Hawkins (38:34.060)
because I'm now filling in that three dimensional map,
Lex Fridman (38:36.700)
which is the coffee cup.
Jeff Hawkins (38:37.540)
I can say, oh, what am I feeling
Lex Fridman (38:38.660)
at all these different locations?
Jeff Hawkins (38:39.900)
That's the basic idea, it's more complicated than that.
Lex Fridman (38:43.020)
But so through time, and we talked about time earlier,
Jeff Hawkins (38:46.220)
through time, even a single column,
Lex Fridman (38:48.180)
which is only looking at, or a single part of the cortex,
Jeff Hawkins (38:50.300)
which is only looking at a small part of the world,
Lex Fridman (38:52.720)
can build up a complete model of an object.
Lex Fridman (38:55.060)
And so if you think about the part of the brain,
Lex Fridman (38:57.100)
which is getting input from all my fingers,
Lex Fridman (38:59.100)
so they're spread across the top of your head here.
Lex Fridman (39:01.700)
This is the somatosensory cortex.
Jeff Hawkins (39:04.040)
There's columns associated
Lex Fridman (39:05.180)
with all the different areas of my skin.
Lex Fridman (39:07.380)
And what we believe is happening
Lex Fridman (39:10.100)
is that all of them are building models of this cup,
Jeff Hawkins (39:12.900)
every one of them, or things.
Lex Fridman (39:15.340)
They're not all building,
Jeff Hawkins (39:16.620)
not every column or every part of the cortex
Lex Fridman (39:18.180)
builds models of everything,
Lex Fridman (39:19.500)
but they're all building models of something.
Lex Fridman (39:21.700)
And so you have, so when I touch this cup with my hand,
Jeff Hawkins (39:26.700)
there are multiple models of the cup being invoked.
Lex Fridman (39:28.980)
If I look at it with my eyes,
Jeff Hawkins (39:30.460)
there are, again, many models of the cup being invoked,
Lex Fridman (39:32.540)
because each part of the visual system,
Jeff Hawkins (39:34.300)
the brain doesn't process an image.
Lex Fridman (39:35.820)
That's a misleading idea.
Jeff Hawkins (39:38.740)
It's just like your fingers touching the cup,
Lex Fridman (39:40.460)
so different parts of my retina
Jeff Hawkins (39:41.300)
are looking at different parts of the cup.
Lex Fridman (39:42.980)
And thousands and thousands of models of the cup
Jeff Hawkins (39:45.540)
are being invoked at once.
Lex Fridman (39:47.380)
And they're all voting with each other,
Jeff Hawkins (39:48.900)
trying to figure out what's going on.
Lex Fridman (39:50.140)
So that's why we call it the thousand brains theory
Jeff Hawkins (39:51.740)
of intelligence, because there isn't one model of a cup.
Lex Fridman (39:54.700)
There are thousands of models of this cup.
Jeff Hawkins (39:56.300)
There are thousands of models of your cellphone
Lex Fridman (39:57.940)
and about cameras and microphones and so on.
Jeff Hawkins (40:00.860)
It's a distributed modeling system,
Lex Fridman (40:02.860)
which is very different
Jeff Hawkins (40:03.700)
than the way people have thought about it.
Lex Fridman (40:04.860)
And so that's a really compelling and interesting idea.
Jeff Hawkins (40:07.340)
I have two first questions.
Lex Fridman (40:08.700)
So one, on the ensemble part of everything coming together,
Jeff Hawkins (40:12.060)
you have these thousand brains.
Lex Fridman (40:14.860)
How do you know which one has done the best job
Jeff Hawkins (40:17.900)
of forming the...
Lex Fridman (40:18.740)
Great question.
Jeff Hawkins (40:19.580)
Let me try to explain it.
Lex Fridman (40:20.420)
There's a problem that's known in neuroscience
Jeff Hawkins (40:23.500)
called the sensor fusion problem.
Lex Fridman (40:25.220)
Yes.
Lex Fridman (40:26.060)
And so the idea is there's something like,
Lex Fridman (40:27.740)
oh, the image comes from the eye.
Jeff Hawkins (40:29.140)
There's a picture on the retina
Lex Fridman (40:30.620)
and then it gets projected to the neocortex.
Jeff Hawkins (40:32.380)
Oh, by now it's all spread out all over the place
Lex Fridman (40:35.100)
and it's kind of squirrely and distorted
Lex Fridman (40:37.100)
and pieces are all over the...
Lex Fridman (40:39.020)
It doesn't look like a picture anymore.
Lex Fridman (40:40.900)
When does it all come back together again?
Lex Fridman (40:43.660)
Or you might say, well, yes,
Lex Fridman (40:45.380)
but I also have sounds or touches associated with the cup.
Lex Fridman (40:48.620)
So I'm seeing the cup and touching the cup.
Lex Fridman (40:50.660)
How do they get combined together again?
Lex Fridman (40:52.620)
So it's called the sensor fusion problem.
Jeff Hawkins (40:54.260)
As if all these disparate parts
Lex Fridman (40:55.860)
have to be brought together into one model someplace.
Jeff Hawkins (40:59.020)
That's the wrong idea.
Lex Fridman (41:01.140)
The right idea is that you've got all these guys voting.
Jeff Hawkins (41:03.500)
There's auditory models of the cup.
Lex Fridman (41:05.420)
There's visual models of the cup.
Jeff Hawkins (41:06.620)
There's tactile models of the cup.
Lex Fridman (41:09.860)
In the vision system,
Jeff Hawkins (41:10.700)
there might be ones that are more focused on black and white
Lex Fridman (41:12.580)
and ones focusing on color.
Jeff Hawkins (41:13.620)
It doesn't really matter.
Lex Fridman (41:14.460)
There's just thousands and thousands of models of this cup.
Lex Fridman (41:17.020)
And they vote.
Lex Fridman (41:17.900)
They don't actually come together in one spot.
Jeff Hawkins (41:20.620)
Just literally think of it this way.
Lex Fridman (41:21.900)
Imagine you have these columns
Jeff Hawkins (41:24.100)
that are like about the size of a little piece of spaghetti.
Lex Fridman (41:26.660)
Like a two and a half millimeters tall
Lex Fridman (41:28.500)
and about a millimeter in wide.
Lex Fridman (41:30.020)
They're not physical, but you could think of them that way.
Lex Fridman (41:33.300)
And each one's trying to guess what this thing is
Lex Fridman (41:35.300)
or touching.
Jeff Hawkins (41:36.140)
Now, they can do a pretty good job
Lex Fridman (41:38.060)
if they're allowed to move over time.
Lex Fridman (41:40.060)
So I can reach my hand into a black box
Lex Fridman (41:41.620)
and move my finger around an object.
Lex Fridman (41:43.540)
And if I touch enough spaces, I go, okay,
Lex Fridman (41:45.540)
now I know what it is.
Lex Fridman (41:46.980)
But often we don't do that.
Lex Fridman (41:48.300)
Often I can just reach and grab something with my hand
Jeff Hawkins (41:49.940)
all at once and I get it.
Lex Fridman (41:51.020)
Or if I had to look through the world through a straw,
Lex Fridman (41:53.740)
so I'm only invoking one little column,
Lex Fridman (41:55.860)
I can only see part of something
Jeff Hawkins (41:56.700)
because I have to move the straw around.
Lex Fridman (41:58.140)
But if I open my eyes, I see the whole thing at once.
Lex Fridman (42:00.460)
So what we think is going on
Lex Fridman (42:01.460)
is all these little pieces of spaghetti,
Jeff Hawkins (42:03.180)
if you will, all these little columns in the cortex,
Lex Fridman (42:05.300)
are all trying to guess what it is that they're sensing.
Jeff Hawkins (42:08.580)
They'll do a better guess if they have time
Lex Fridman (42:10.740)
and can move over time.
Lex Fridman (42:11.700)
So if I move my eyes, I move my fingers.
Lex Fridman (42:13.620)
But if they don't, they have a poor guess.
Jeff Hawkins (42:16.580)
It's a probabilistic guess of what they might be touching.
Lex Fridman (42:20.060)
Now, imagine they can post their probability
Jeff Hawkins (42:22.940)
at the top of a little piece of spaghetti.
Lex Fridman (42:24.580)
Each one of them says,
Jeff Hawkins (42:25.420)
I think, and it's not really a probability distribution.
Lex Fridman (42:27.420)
It's more like a set of possibilities.
Jeff Hawkins (42:29.460)
In the brain, it doesn't work as a probability distribution.
Lex Fridman (42:31.980)
It works as more like what we call a union.
Lex Fridman (42:34.020)
So you could say, and one column says,
Lex Fridman (42:35.860)
I think it could be a coffee cup,
Jeff Hawkins (42:37.540)
a soda can, or a water bottle.
Lex Fridman (42:39.940)
And another column says,
Jeff Hawkins (42:40.900)
I think it could be a coffee cup
Lex Fridman (42:42.300)
or a telephone or a camera or whatever, right?
Lex Fridman (42:46.460)
And all these guys are saying what they think it might be.
Lex Fridman (42:49.940)
And there's these long range connections
Jeff Hawkins (42:51.620)
in certain layers in the cortex.
Lex Fridman (42:53.460)
So there's in some layers in some cells types
Jeff Hawkins (42:56.660)
in each column, send the projections across the brain.
Lex Fridman (43:00.060)
And that's the voting occurs.
Lex Fridman (43:01.740)
And so there's a simple associative memory mechanism.
Lex Fridman (43:04.100)
We've described this in a recent paper
Lex Fridman (43:06.140)
and we've modeled this that says,
Lex Fridman (43:09.500)
they can all quickly settle on the only
Jeff Hawkins (43:11.940)
or the one best answer for all of them.
Lex Fridman (43:14.900)
If there is a single best answer,
Jeff Hawkins (43:16.420)
they all vote and say, yep, it's gotta be the coffee cup.
Lex Fridman (43:18.940)
And at that point, they all know it's a coffee cup.
Lex Fridman (43:21.060)
And at that point, everyone acts as if it's a coffee cup.
Lex Fridman (43:23.380)
They're like, yep, we know it's a coffee,
Jeff Hawkins (43:24.220)
even though I've only seen one little piece of this world,
Lex Fridman (43:26.380)
I know it's a coffee cup I'm touching
Jeff Hawkins (43:27.700)
or I'm seeing or whatever.
Lex Fridman (43:28.980)
And so you can think of all these columns
Jeff Hawkins (43:30.900)
are looking at different parts in different places,
Lex Fridman (43:33.020)
different sensory input, different locations,
Jeff Hawkins (43:35.220)
they're all different.
Lex Fridman (43:36.180)
But this layer that's doing the voting, it solidifies.
Jeff Hawkins (43:40.460)
It's just like it crystallizes and says,
Lex Fridman (43:42.260)
oh, we all know what we're doing.
Lex Fridman (43:44.140)
And so you don't bring these models together in one model,
Lex Fridman (43:46.460)
you just vote and there's a crystallization of the vote.
Jeff Hawkins (43:49.140)
Great, that's at least a compelling way
Lex Fridman (43:51.780)
to think about the way you form a model of the world.
Jeff Hawkins (43:58.180)
Now, you talk about a coffee cup.
Lex Fridman (44:00.420)
Do you see this, as far as I understand,
Jeff Hawkins (44:03.220)
you are proposing this as well,
Lex Fridman (44:04.660)
that this extends to much more than coffee cups?
Jeff Hawkins (44:06.900)
Yeah.
Lex Fridman (44:07.740)
It does.
Jeff Hawkins (44:09.540)
Or at least the physical world,
Lex Fridman (44:10.780)
it expands to the world of concepts.
Jeff Hawkins (44:14.100)
Yeah, it does.
Lex Fridman (44:15.020)
And well, first, the primary thing is evidence for that
Jeff Hawkins (44:18.220)
is that the regions of the neocortex
Lex Fridman (44:20.700)
that are associated with language
Jeff Hawkins (44:22.340)
or high level thought or mathematics
Lex Fridman (44:23.860)
or things like that,
Jeff Hawkins (44:24.700)
they look like the regions of the neocortex
Lex Fridman (44:26.180)
that process vision, hearing, and touch.
Jeff Hawkins (44:28.300)
They don't look any different.
Lex Fridman (44:29.700)
Or they look only marginally different.
Lex Fridman (44:32.820)
And so one would say, well, if Vernon Mountcastle,
Lex Fridman (44:36.420)
who proposed that all the parts of the neocortex
Jeff Hawkins (44:38.860)
do the same thing, if he's right,
Lex Fridman (44:41.060)
then the parts that are doing language
Jeff Hawkins (44:42.820)
or mathematics or physics
Lex Fridman (44:44.540)
are working on the same principle.
Jeff Hawkins (44:45.700)
They must be working on the principle of reference frames.
Lex Fridman (44:48.500)
So that's a little odd thought.
Lex Fridman (44:51.820)
But of course, we had no prior idea
Lex Fridman (44:53.940)
how these things happen.
Lex Fridman (44:55.020)
So let's go with that.
Lex Fridman (44:57.340)
And we, in our recent paper,
Jeff Hawkins (44:59.900)
we talked a little bit about that.
Lex Fridman (45:01.620)
I've been working on it more since.
Jeff Hawkins (45:02.820)
I have better ideas about it now.
Lex Fridman (45:05.380)
I'm sitting here very confident
Jeff Hawkins (45:06.980)
that that's what's happening.
Lex Fridman (45:08.020)
And I can give you some examples
Jeff Hawkins (45:09.260)
that help you think about that.
Lex Fridman (45:11.220)
It's not we understand it completely,
Lex Fridman (45:12.500)
but I understand it better than I've described it
Lex Fridman (45:14.300)
in any paper so far.
Jeff Hawkins (45:15.660)
So, but we did put that idea out there.
Lex Fridman (45:17.700)
It says, okay, this is,
Lex Fridman (45:18.940)
it's a good place to start, you know?
Lex Fridman (45:22.620)
And the evidence would suggest it's how it's happening.
Lex Fridman (45:24.900)
And then we can start tackling that problem
Lex Fridman (45:26.660)
one piece at a time.
Lex Fridman (45:27.500)
Like, what does it mean to do high level thought?
Lex Fridman (45:29.060)
What does it mean to do language?
Lex Fridman (45:30.020)
How would that fit into a reference frame framework?
Lex Fridman (45:34.220)
Yeah, so there's a,
Jeff Hawkins (45:35.980)
I don't know if you could tell me if there's a connection,
Lex Fridman (45:37.580)
but there's an app called Anki
Jeff Hawkins (45:40.180)
that helps you remember different concepts.
Lex Fridman (45:42.420)
And they talk about like a memory palace
Jeff Hawkins (45:45.100)
that helps you remember completely random concepts
Lex Fridman (45:47.780)
by trying to put them in a physical space in your mind
Lex Fridman (45:51.380)
and putting them next to each other.
Lex Fridman (45:52.220)
It's called the method of loci.
Jeff Hawkins (45:53.580)
Loci, yeah.
Lex Fridman (45:54.700)
For some reason, that seems to work really well.
Jeff Hawkins (45:57.580)
Now, that's a very narrow kind of application
Lex Fridman (45:59.420)
of just remembering some facts.
Lex Fridman (46:00.580)
But that's a very, very telling one.
Lex Fridman (46:03.260)
Yes, exactly.
Lex Fridman (46:04.100)
So this seems like you're describing a mechanism
Lex Fridman (46:06.740)
why this seems to work.
Lex Fridman (46:09.620)
So basically the way what we think is going on
Lex Fridman (46:11.820)
is all things you know, all concepts, all ideas,
Jeff Hawkins (46:15.060)
words, everything you know are stored in reference frames.
Lex Fridman (46:20.460)
And so if you want to remember something,
Jeff Hawkins (46:24.300)
you have to basically navigate through a reference frame
Lex Fridman (46:26.860)
the same way a rat navigates through a maze
Lex Fridman (46:28.620)
and the same way my finger navigates to this coffee cup.
Lex Fridman (46:31.420)
You are moving through some space.
Lex Fridman (46:33.500)
And so if you have a random list of things
Lex Fridman (46:35.900)
you were asked to remember,
Jeff Hawkins (46:37.460)
by assigning them to a reference frame,
Lex Fridman (46:39.300)
you've already know very well to see your house, right?
Lex Fridman (46:42.100)
And the idea of the method of loci is you can say,
Lex Fridman (46:43.580)
okay, in my lobby, I'm going to put this thing.
Lex Fridman (46:45.820)
And then the bedroom, I put this one.
Lex Fridman (46:47.660)
I go down the hall, I put this thing.
Lex Fridman (46:48.940)
And then you want to recall those facts
Lex Fridman (46:50.820)
or recall those things.
Jeff Hawkins (46:51.660)
You just walk mentally, you walk through your house.
Lex Fridman (46:54.100)
You're mentally moving through a reference frame
Jeff Hawkins (46:56.540)
that you already had.
Lex Fridman (46:57.660)
And that tells you,
Jeff Hawkins (46:59.260)
there's two things that are really important about that.
Lex Fridman (47:00.580)
It tells us the brain prefers to store things
Jeff Hawkins (47:02.740)
in reference frames.
Lex Fridman (47:03.940)
And that the method of recalling things
Jeff Hawkins (47:06.820)
or thinking, if you will,
Lex Fridman (47:08.220)
is to move mentally through those reference frames.
Jeff Hawkins (47:11.500)
You could move physically through some reference frames,
Lex Fridman (47:13.540)
like I could physically move through the reference frame
Jeff Hawkins (47:15.220)
of this coffee cup.
Lex Fridman (47:16.300)
I can also mentally move through the reference frame
Jeff Hawkins (47:17.900)
of the coffee cup, imagining me touching it.
Lex Fridman (47:19.980)
But I can also mentally move my house.
Lex Fridman (47:22.420)
And so now we can ask yourself,
Lex Fridman (47:24.660)
or are all concepts stored this way?
Jeff Hawkins (47:26.740)
There was some recent research using human subjects
Lex Fridman (47:31.380)
in fMRI, and I'm going to apologize for not knowing
Jeff Hawkins (47:33.540)
the name of the scientists who did this.
Lex Fridman (47:36.660)
But what they did is they put humans in this fMRI machine,
Jeff Hawkins (47:41.060)
which is one of these imaging machines.
Lex Fridman (47:42.780)
And they gave the humans tasks to think about birds.
Lex Fridman (47:46.460)
So they had different types of birds,
Lex Fridman (47:47.780)
and birds that look big and small,
Lex Fridman (47:49.660)
and long necks and long legs, things like that.
Lex Fridman (47:52.220)
And what they could tell from the fMRI
Jeff Hawkins (47:55.260)
was a very clever experiment.
Lex Fridman (47:57.580)
You get to tell when humans were thinking about the birds,
Jeff Hawkins (48:00.780)
that the birds, the knowledge of birds
Lex Fridman (48:03.580)
was arranged in a reference frame,
Jeff Hawkins (48:05.500)
similar to the ones that are used
Lex Fridman (48:07.100)
when you navigate in a room.
Jeff Hawkins (48:08.980)
That these are called grid cells,
Lex Fridman (48:10.340)
and there are grid cell like patterns of activity
Jeff Hawkins (48:12.820)
in the neocortex when they do this.
Lex Fridman (48:15.380)
So it's a very clever experiment.
Lex Fridman (48:18.980)
And what it basically says,
Lex Fridman (48:20.180)
that even when you're thinking about something abstract,
Lex Fridman (48:22.140)
and you're not really thinking about it as a reference frame,
Lex Fridman (48:24.700)
it tells us the brain is actually using a reference frame.
Lex Fridman (48:26.980)
And it's using the same neural mechanisms.
Lex Fridman (48:28.780)
These grid cells are the basic same neural mechanism
Jeff Hawkins (48:30.780)
that we propose that grid cells,
Lex Fridman (48:32.860)
which exist in the old part of the brain,
Jeff Hawkins (48:34.980)
the entorhinal cortex, that that mechanism
Lex Fridman (48:37.340)
is now similar mechanism is used throughout the neocortex.
Jeff Hawkins (48:40.060)
It's the same nature to preserve this interesting way
Lex Fridman (48:43.180)
of creating reference frames.
Lex Fridman (48:44.580)
And so now they have empirical evidence
Lex Fridman (48:46.940)
that when you think about concepts like birds,
Jeff Hawkins (48:49.500)
that you're using reference frames
Lex Fridman (48:51.220)
that are built on grid cells.
Lex Fridman (48:53.180)
So that's similar to the method of loci,
Lex Fridman (48:55.180)
but in this case, the birds are related.
Lex Fridman (48:56.820)
So they create their own reference frame,
Lex Fridman (48:58.620)
which is consistent with bird space.
Lex Fridman (49:01.100)
And when you think about something, you go through that.
Lex Fridman (49:03.540)
You can make the same example,
Jeff Hawkins (49:04.820)
let's take mathematics.
Lex Fridman (49:06.620)
Let's say you wanna prove a conjecture.
Lex Fridman (49:09.260)
What is a conjecture?
Lex Fridman (49:10.100)
A conjecture is a statement you believe to be true,
Lex Fridman (49:13.300)
but you haven't proven it.
Lex Fridman (49:15.140)
And so it might be an equation.
Jeff Hawkins (49:16.540)
I wanna show that this is equal to that.
Lex Fridman (49:19.140)
And you have some places you start with.
Jeff Hawkins (49:21.180)
You say, well, I know this is true,
Lex Fridman (49:22.340)
and I know this is true.
Lex Fridman (49:23.420)
And I think that maybe to get to the final proof,
Lex Fridman (49:25.900)
I need to go through some intermediate results.
Lex Fridman (49:28.700)
What I believe is happening is literally these equations
Lex Fridman (49:33.140)
or these points are assigned to a reference frame,
Jeff Hawkins (49:36.380)
a mathematical reference frame.
Lex Fridman (49:37.980)
And when you do mathematical operations,
Jeff Hawkins (49:39.820)
a simple one might be multiply or divide,
Lex Fridman (49:41.660)
but you might be a little plus transform or something else.
Jeff Hawkins (49:44.060)
That is like a movement in the reference frame of the math.
Lex Fridman (49:47.500)
And so you're literally trying to discover a path
Jeff Hawkins (49:50.260)
from one location to another location
Lex Fridman (49:52.660)
in a space of mathematics.
Lex Fridman (49:56.140)
And if you can get to these intermediate results,
Lex Fridman (49:58.220)
then you know your map is pretty good,
Lex Fridman (50:00.420)
and you know you're using the right operations.
Lex Fridman (50:02.940)
Much of what we think about is solving hard problems
Jeff Hawkins (50:05.940)
is designing the correct reference frame for that problem,
Lex Fridman (50:08.820)
figuring out how to organize the information
Lex Fridman (50:11.100)
and what behaviors I wanna use in that space
Lex Fridman (50:14.300)
to get me there.
Jeff Hawkins (50:16.220)
Yeah, so if you dig in an idea of this reference frame,
Lex Fridman (50:19.260)
whether it's the math, you start a set of axioms
Jeff Hawkins (50:21.700)
to try to get to proving the conjecture.
Lex Fridman (50:25.140)
Can you try to describe, maybe take a step back,
Lex Fridman (50:28.140)
how you think of the reference frame in that context?
Lex Fridman (50:30.660)
Is it the reference frame that the axioms are happy in?
Lex Fridman (50:36.140)
Is it the reference frame that might contain everything?
Lex Fridman (50:38.780)
Is it a changing thing as you?
Jeff Hawkins (50:41.780)
You have many, many reference frames.
Lex Fridman (50:43.140)
I mean, in fact, the way the theory,
Jeff Hawkins (50:44.580)
the thousand brain theory of intelligence says
Lex Fridman (50:46.140)
that every single thing in the world
Jeff Hawkins (50:47.380)
has its own reference frame.
Lex Fridman (50:48.300)
So every word has its own reference frames.
Lex Fridman (50:50.860)
And we can talk about this.
Lex Fridman (50:52.940)
The mathematics work out,
Jeff Hawkins (50:54.460)
this is no problem for neurons to do this.
Lex Fridman (50:55.940)
But how many reference frames does a coffee cup have?
Jeff Hawkins (50:58.740)
Well, it's on a table.
Lex Fridman (51:00.140)
Let's say you ask how many reference frames
Jeff Hawkins (51:03.700)
could a column in my finger
Lex Fridman (51:06.020)
that's touching the coffee cup have?
Jeff Hawkins (51:07.420)
Because there are many, many copy,
Lex Fridman (51:09.060)
there are many, many models of the coffee cup.
Lex Fridman (51:10.500)
So the coffee, there is no one model of a coffee cup.
Lex Fridman (51:13.020)
There are many models of a coffee cup.
Lex Fridman (51:14.220)
And you could say, well,
Lex Fridman (51:15.220)
how many different things can my finger learn?
Lex Fridman (51:17.260)
Is this the question you want to ask?
Lex Fridman (51:19.540)
Imagine I say every concept, every idea,
Jeff Hawkins (51:21.780)
everything you've ever know about that you can say,
Lex Fridman (51:23.860)
I know that thing has a reference frame
Jeff Hawkins (51:27.260)
associated with it.
Lex Fridman (51:28.180)
And what we do when we build composite objects,
Jeff Hawkins (51:30.180)
we assign reference frames
Lex Fridman (51:32.460)
to point another reference frame.
Lex Fridman (51:33.940)
So my coffee cup has multiple components to it.
Lex Fridman (51:37.060)
It's got a limb, it's got a cylinder, it's got a handle.
Lex Fridman (51:40.660)
And those things have their own reference frames
Lex Fridman (51:42.820)
and they're assigned to a master reference frame,
Jeff Hawkins (51:45.060)
which is called this cup.
Lex Fridman (51:46.380)
And now I have this Numenta logo on it.
Jeff Hawkins (51:48.180)
Well, that's something that exists elsewhere in the world.
Lex Fridman (51:50.420)
It's its own thing.
Lex Fridman (51:51.260)
So it has its own reference frame.
Lex Fridman (51:52.300)
So we now have to say,
Lex Fridman (51:53.140)
how can I assign the Numenta logo reference frame
Lex Fridman (51:56.740)
onto the cylinder or onto the coffee cup?
Lex Fridman (51:59.180)
So it's all, we talked about this in the paper
Lex Fridman (52:01.500)
that came out in December of this last year.
Jeff Hawkins (52:06.860)
The idea of how you can assign reference frames
Lex Fridman (52:08.780)
to reference frames, how neurons could do this.
Jeff Hawkins (52:10.540)
So, well, my question is,
Lex Fridman (52:12.620)
even though you mentioned reference frames a lot,
Jeff Hawkins (52:14.740)
I almost feel it's really useful to dig into
Lex Fridman (52:16.940)
how you think of what a reference frame is.
Jeff Hawkins (52:20.140)
I mean, it was already helpful for me to understand
Lex Fridman (52:22.020)
that you think of reference frames
Jeff Hawkins (52:23.700)
as something there is a lot of.
Lex Fridman (52:26.340)
Okay, so let's just say that we're gonna have
Jeff Hawkins (52:28.780)
some neurons in the brain, not many, actually,
Lex Fridman (52:31.060)
10,000, 20,000 are gonna create
Jeff Hawkins (52:32.740)
a whole bunch of reference frames.
Lex Fridman (52:34.300)
What does it mean?
Lex Fridman (52:35.540)
What is a reference frame?
Lex Fridman (52:37.300)
First of all, these reference frames are different
Jeff Hawkins (52:40.060)
than the ones you might be used to.
Lex Fridman (52:42.220)
We know lots of reference frames.
Jeff Hawkins (52:43.420)
For example, we know the Cartesian coordinates, X, Y, Z,
Lex Fridman (52:46.060)
that's a type of reference frame.
Jeff Hawkins (52:47.580)
We know longitude and latitude,
Lex Fridman (52:50.260)
that's a different type of reference frame.
Jeff Hawkins (52:52.780)
If I look at a printed map,
Lex Fridman (52:54.540)
you might have columns A through M,
Lex Fridman (52:58.460)
and rows one through 20,
Lex Fridman (53:00.060)
that's a different type of reference frame.
Jeff Hawkins (53:01.420)
It's kind of a Cartesian coordinate reference frame.
Lex Fridman (53:04.660)
The interesting thing about the reference frames
Jeff Hawkins (53:06.580)
in the brain, and we know this because these
Lex Fridman (53:08.580)
have been established through neuroscience
Jeff Hawkins (53:10.820)
studying the entorhinal cortex.
Lex Fridman (53:12.260)
So I'm not speculating here, okay?
Jeff Hawkins (53:13.580)
This is known neuroscience in an old part of the brain.
Lex Fridman (53:16.780)
The way these cells create reference frames,
Jeff Hawkins (53:18.860)
they have no origin.
Lex Fridman (53:20.700)
So what it's more like, you have a point,
Jeff Hawkins (53:24.340)
a point in some space, and you,
Lex Fridman (53:27.620)
given a particular movement,
Jeff Hawkins (53:29.060)
you can then tell what the next point should be.
Lex Fridman (53:32.340)
And you can then tell what the next point would be,
Lex Fridman (53:34.100)
and so on.
Lex Fridman (53:35.460)
You can use this to calculate
Lex Fridman (53:38.700)
how to get from one point to another.
Lex Fridman (53:40.340)
So how do I get from my house to my home,
Jeff Hawkins (53:43.180)
or how do I get my finger from the side of my cup
Lex Fridman (53:44.940)
to the top of the cup?
Lex Fridman (53:46.740)
How do I get from the axioms to the conjecture?
Lex Fridman (53:50.540)
So it's a different type of reference frame,
Lex Fridman (53:54.420)
and I can, if you want, I can describe in more detail,
Lex Fridman (53:57.380)
I can paint a picture of how you might want
Jeff Hawkins (53:59.060)
to think about that.
Lex Fridman (53:59.900)
It's really helpful to think it's something
Jeff Hawkins (54:00.980)
you can move through, but is there,
Lex Fridman (54:03.740)
is it helpful to think of it as spatial in some sense,
Lex Fridman (54:08.700)
or is there something that's more?
Lex Fridman (54:09.540)
No, it's definitely spatial.
Jeff Hawkins (54:11.140)
It's spatial in a mathematical sense.
Lex Fridman (54:13.820)
How many dimensions?
Lex Fridman (54:14.820)
Can it be a crazy number of dimensions?
Lex Fridman (54:16.260)
Well, that's an interesting question.
Jeff Hawkins (54:17.460)
In the old part of the brain, the entorhinal cortex,
Lex Fridman (54:20.260)
they studied rats, and initially it looks like,
Jeff Hawkins (54:22.940)
oh, this is just two dimensional.
Lex Fridman (54:24.220)
It's like the rat is in some box in the maze or whatever,
Lex Fridman (54:27.260)
and they know where the rat is using
Lex Fridman (54:28.820)
these two dimensional reference frames
Jeff Hawkins (54:30.300)
to know where it is in the maze.
Lex Fridman (54:32.380)
We said, well, okay, but what about bats?
Jeff Hawkins (54:35.540)
That's a mammal, and they fly in three dimensional space.
Lex Fridman (54:38.740)
How do they do that?
Lex Fridman (54:39.580)
They seem to know where they are, right?
Lex Fridman (54:41.700)
So this is a current area of active research,
Lex Fridman (54:44.300)
and it seems like somehow the neurons
Lex Fridman (54:46.380)
in the entorhinal cortex can learn three dimensional space.
Jeff Hawkins (54:50.300)
We just, two members of our team,
Lex Fridman (54:52.700)
along with Elif Fett from MIT,
Jeff Hawkins (54:55.940)
just released a paper this literally last week.
Lex Fridman (54:59.580)
It's on bioRxiv, where they show that you can,
Jeff Hawkins (55:03.620)
if you, the way these things work,
Lex Fridman (55:05.460)
and I won't get, unless you want to,
Jeff Hawkins (55:06.700)
I won't get into the detail,
Lex Fridman (55:08.100)
but grid cells can represent any n dimensional space.
Jeff Hawkins (55:12.540)
It's not inherently limited.
Lex Fridman (55:15.340)
You can think of it this way.
Jeff Hawkins (55:16.620)
If you had two dimensional, the way it works
Lex Fridman (55:18.620)
is you had a bunch of two dimensional slices.
Jeff Hawkins (55:20.780)
That's the way these things work.
Lex Fridman (55:21.940)
There's a whole bunch of two dimensional models,
Lex Fridman (55:24.260)
and you can just, you can slice up
Lex Fridman (55:26.140)
any n dimensional space with two dimensional projections.
Jeff Hawkins (55:29.300)
So, and you could have one dimensional models.
Lex Fridman (55:31.660)
So there's nothing inherent about the mathematics
Jeff Hawkins (55:34.420)
about the way the neurons do this,
Lex Fridman (55:35.780)
which constrain the dimensionality of the space,
Jeff Hawkins (55:39.460)
which I think was important.
Lex Fridman (55:41.460)
So obviously I have a three dimensional map of this cup.
Jeff Hawkins (55:44.060)
Maybe it's even more than that, I don't know.
Lex Fridman (55:46.340)
But it's clearly a three dimensional map of the cup.
Jeff Hawkins (55:48.340)
I don't just have a projection of the cup.
Lex Fridman (55:50.900)
But when I think about birds,
Jeff Hawkins (55:52.020)
or when I think about mathematics,
Lex Fridman (55:53.180)
perhaps it's more than three dimensions.
Lex Fridman (55:55.260)
Who knows?
Lex Fridman (55:56.260)
So in terms of each individual column
Jeff Hawkins (56:00.100)
building up more and more information over time,
Lex Fridman (56:04.020)
do you think that mechanism is well understood?
Jeff Hawkins (56:06.380)
In your mind, you've proposed a lot of architectures there.
Lex Fridman (56:09.860)
Is that a key piece, or is it,
Jeff Hawkins (56:11.820)
is the big piece, the thousand brain theory of intelligence,
Lex Fridman (56:16.220)
the ensemble of it all?
Jeff Hawkins (56:17.500)
Well, I think they're both big.
Lex Fridman (56:18.460)
I mean, clearly the concept, as a theorist,
Lex Fridman (56:20.940)
the concept is most exciting, right?
Lex Fridman (56:23.060)
The high level concept.
Jeff Hawkins (56:23.900)
The high level concept.
Lex Fridman (56:24.740)
This is a totally new way of thinking
Jeff Hawkins (56:26.140)
about how the neocortex works.
Lex Fridman (56:27.220)
So that is appealing.
Jeff Hawkins (56:28.660)
It has all these ramifications.
Lex Fridman (56:30.700)
And with that, as a framework for how the brain works,
Jeff Hawkins (56:33.780)
you can make all kinds of predictions
Lex Fridman (56:34.980)
and solve all kinds of problems.
Jeff Hawkins (56:36.220)
Now we're trying to work through
Lex Fridman (56:37.260)
many of these details right now.
Lex Fridman (56:38.460)
Okay, how do the neurons actually do this?
Lex Fridman (56:40.540)
Well, it turns out, if you think about grid cells
Lex Fridman (56:42.500)
and place cells in the old parts of the brain,
Lex Fridman (56:44.740)
there's a lot that's known about them,
Lex Fridman (56:45.980)
but there's still some mysteries.
Lex Fridman (56:47.020)
There's a lot of debate about exactly the details,
Lex Fridman (56:49.060)
how these work and what are the signs.
Lex Fridman (56:50.740)
And we have that still, that same level of detail,
Jeff Hawkins (56:52.860)
that same level of concern.
Lex Fridman (56:54.140)
What we spend here most of our time doing
Jeff Hawkins (56:56.820)
is trying to make a very good list
Lex Fridman (57:00.060)
of the things we don't understand yet.
Jeff Hawkins (57:02.660)
That's the key part here.
Lex Fridman (57:04.020)
What are the constraints?
Jeff Hawkins (57:05.260)
It's not like, oh, this thing seems to work, we're done.
Lex Fridman (57:07.020)
No, it's like, okay, it kind of works,
Lex Fridman (57:08.820)
but these are other things we know it has to do
Lex Fridman (57:10.700)
and it's not doing those yet.
Jeff Hawkins (57:12.860)
I would say we're well on the way here.
Lex Fridman (57:15.060)
We're not done yet.
Jeff Hawkins (57:17.100)
There's a lot of trickiness to this system,
Lex Fridman (57:20.020)
but the basic principles about how different layers
Jeff Hawkins (57:23.180)
in the neocortex are doing much of this, we understand.
Lex Fridman (57:27.340)
But there's some fundamental parts
Jeff Hawkins (57:28.620)
that we don't understand as well.
Lex Fridman (57:30.020)
So what would you say is one of the harder open problems
Jeff Hawkins (57:34.100)
or one of the ones that have been bothering you,
Lex Fridman (57:37.220)
keeping you up at night the most?
Jeff Hawkins (57:38.460)
Oh, well, right now, this is a detailed thing
Lex Fridman (57:40.620)
that wouldn't apply to most people, okay?
Jeff Hawkins (57:42.980)
Sure.
Lex Fridman (57:43.820)
But you want me to answer that question?
Jeff Hawkins (57:44.660)
Yeah, please.
Lex Fridman (57:46.180)
We've talked about as if, oh,
Jeff Hawkins (57:48.380)
to predict what you're going to sense on this coffee cup,
Lex Fridman (57:50.660)
I need to know where my finger is gonna be
Jeff Hawkins (57:52.300)
on the coffee cup.
Lex Fridman (57:53.580)
That is true, but it's insufficient.
Jeff Hawkins (57:56.340)
Think about my finger touches the edge of the coffee cup.
Lex Fridman (57:58.460)
My finger can touch it at different orientations.
Jeff Hawkins (58:01.660)
I can rotate my finger around here and that doesn't change.
Lex Fridman (58:06.340)
I can make that prediction and somehow,
Lex Fridman (58:08.780)
so it's not just the location.
Lex Fridman (58:10.100)
There's an orientation component of this as well.
Jeff Hawkins (58:13.300)
This is known in the old parts of the brain too.
Lex Fridman (58:15.140)
There's things called head direction cells,
Jeff Hawkins (58:16.620)
which way the rat is facing.
Lex Fridman (58:18.020)
It's the same kind of basic idea.
Lex Fridman (58:20.460)
So if my finger were a rat, you know, in three dimensions,
Lex Fridman (58:23.620)
I have a three dimensional orientation
Lex Fridman (58:25.740)
and I have a three dimensional location.
Lex Fridman (58:27.220)
If I was a rat, I would have a,
Jeff Hawkins (58:28.620)
you might think of it as a two dimensional location,
Lex Fridman (58:30.620)
a two dimensional orientation,
Jeff Hawkins (58:31.460)
a one dimensional orientation,
Lex Fridman (58:32.540)
like just which way is it facing?
Lex Fridman (58:35.100)
So how the two components work together,
Lex Fridman (58:38.260)
how it is that I combine orientation,
Jeff Hawkins (58:41.500)
the orientation of my sensor,
Lex Fridman (58:43.940)
as well as the location is a tricky problem.
Lex Fridman (58:49.660)
And I think I've made progress on it.
Lex Fridman (58:52.740)
So at a bigger version of that,
Lex Fridman (58:55.140)
so perspective is super interesting, but super specific.
Lex Fridman (58:58.460)
Yeah, I warned you.
Jeff Hawkins (59:00.060)
No, no, no, that's really good,
Lex Fridman (59:01.260)
but there's a more general version of that.
Lex Fridman (59:03.740)
Do you think context matters,
Lex Fridman (59:06.940)
the fact that we're in a building in North America,
Lex Fridman (59:10.700)
that we, in the day and age where we have mugs?
Lex Fridman (59:15.940)
I mean, there's all this extra information
Jeff Hawkins (59:19.180)
that you bring to the table about everything else
Lex Fridman (59:22.060)
in the room that's outside of just the coffee cup.
Lex Fridman (59:24.700)
How does it get connected, do you think?
Lex Fridman (59:27.340)
Yeah, and that is another really interesting question.
Jeff Hawkins (59:30.300)
I'm gonna throw that under the rubric
Lex Fridman (59:32.140)
or the name of attentional problems.
Jeff Hawkins (59:35.100)
First of all, we have this model,
Lex Fridman (59:36.180)
I have many, many models.
Lex Fridman (59:38.020)
And also the question, does it matter?
Lex Fridman (59:40.140)
Well, it matters for certain things, of course it does.
Jeff Hawkins (59:42.620)
Maybe what we think of that as a coffee cup
Lex Fridman (59:44.980)
in another part of the world
Jeff Hawkins (59:45.900)
is viewed as something completely different.
Lex Fridman (59:47.660)
Or maybe our logo, which is very benign
Jeff Hawkins (59:50.420)
in this part of the world,
Lex Fridman (59:51.340)
it means something very different
Jeff Hawkins (59:52.540)
in another part of the world.
Lex Fridman (59:53.780)
So those things do matter.
Jeff Hawkins (59:57.380)
I think the way to think about it is the following,
🔗 相关节目