Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI
AI 与机器学习心理与人性音乐与艺术生物与进化技术与编程
📋 章节目录
暂无章节信息
🔑 关键词
donlearningdeepmachineknowledgeintelligencebookhumanstuffsystemssaidhardbettergeneralforthdoesnhumansmachinesphysicallanguage
💬 精彩语录
暂无语录
🎙️ 完整对话（2128 条）
Lex Fridman (00:00.000)
The following is a conversation with Gary Marcus.
以下是与加里·马库斯的对话。
Lex Fridman (00:02.740)
He's a professor emeritus at NYU,
他是纽约大学的名誉教授
Lex Fridman (00:04.980)
founder of Robust AI and Geometric Intelligence.
鲁棒人工智能和几何智能的创始人。
Lex Fridman (00:08.200)
The latter is a machine learning company
后者是一家机器学习公司
Lex Fridman (00:10.300)
that was acquired by Uber in 2016.
2016 年被 Uber 收购。
Gary Marcus (00:13.500)
He's the author of several books,
他是多本书的作者
Lex Fridman (00:15.740)
Unnatural and Artificial Intelligence,
非自然和人工智能，
Gary Marcus (00:18.180)
including his new book, Rebooting AI,
包括他的新书《重启人工智能》，
Lex Fridman (00:20.840)
Building Machines We Can Trust.
打造值得信赖的机器。
Gary Marcus (00:23.340)
Gary has been a critical voice,
加里一直发出批评的声音，
Lex Fridman (00:25.480)
highlighting the limits of deep learning and AI in general
强调深度学习和人工智能的总体局限性
Lex Fridman (00:28.780)
and discussing the challenges before our AI community
并讨论我们的人工智能社区面临的挑战
Lex Fridman (00:33.700)
that must be solved in order to achieve
为了实现这一目标必须解决这个问题
Gary Marcus (00:35.740)
artificial general intelligence.
通用人工智能。
Lex Fridman (00:38.300)
As I'm having these conversations,
当我进行这些谈话时，
Gary Marcus (00:40.100)
I try to find paths toward insight, towards new ideas.
我试图寻找通向洞察力、通向新想法的途径。
Lex Fridman (00:43.600)
I try to have no ego in the process.
在这个过程中我尽量不去自负。
Gary Marcus (00:45.940)
It gets in the way.
它妨碍了。
Lex Fridman (00:47.640)
I'll often continuously try on several hats, several roles.
我经常会不断尝试不同的帽子、不同的角色。
Gary Marcus (00:52.300)
One, for example, is the role of a three year old
例如，一个是三岁孩子的角色
Lex Fridman (00:54.740)
who understands very little about anything
Lex Fridman (00:57.140)
and asks big what and why questions.
Lex Fridman (01:00.340)
The other might be a role of a devil's advocate
Gary Marcus (01:02.940)
who presents counter ideas with the goal of arriving
Lex Fridman (01:05.600)
at greater understanding through debate.
Gary Marcus (01:08.240)
Hopefully, both are useful, interesting,
Lex Fridman (01:11.240)
and even entertaining at times.
Gary Marcus (01:13.400)
I ask for your patience as I learn
Lex Fridman (01:15.400)
to have better conversations.
Gary Marcus (01:17.760)
This is the Artificial Intelligence Podcast.
Lex Fridman (01:20.800)
If you enjoy it, subscribe on YouTube,
Gary Marcus (01:23.140)
give it five stars on iTunes, support it on Patreon,
Lex Fridman (01:26.340)
or simply connect with me on Twitter
Gary Marcus (01:28.560)
at Lex Friedman, spelled F R I D M A N.
Lex Fridman (01:32.540)
And now, here's my conversation with Gary Marcus.
Lex Fridman (01:37.220)
Do you think human civilization will one day have
Lex Fridman (01:40.400)
to face an AI driven technological singularity
Gary Marcus (01:42.960)
that will, in a societal way,
Lex Fridman (01:45.620)
modify our place in the food chain
Lex Fridman (01:47.260)
of intelligent living beings on this planet?
Lex Fridman (01:50.140)
I think our place in the food chain has already changed.
Lex Fridman (01:54.860)
So there are lots of things people used to do by hand
Lex Fridman (01:57.340)
that they do with machine.
Gary Marcus (01:59.180)
If you think of a singularity as like one single moment,
Lex Fridman (02:01.800)
which is, I guess, what it suggests,
Gary Marcus (02:03.220)
I don't know if it'll be like that,
Lex Fridman (02:04.580)
but I think that there's a lot of gradual change
Lex Fridman (02:07.340)
and AI is getting better and better.
Lex Fridman (02:09.220)
I mean, I'm here to tell you why I think it's not nearly
Gary Marcus (02:11.420)
as good as people think, but the overall trend is clear.
Lex Fridman (02:14.380)
Maybe Rick Hertzweil thinks it's an exponential
Lex Fridman (02:17.380)
and I think it's linear.
Lex Fridman (02:18.440)
In some cases, it's close to zero right now,
Lex Fridman (02:20.800)
but it's all gonna happen.
Lex Fridman (02:21.820)
I mean, we are gonna get to human level intelligence
Gary Marcus (02:24.780)
or whatever you want, artificial general intelligence
Lex Fridman (02:28.660)
at some point, and that's certainly gonna change
Gary Marcus (02:31.380)
our place in the food chain,
Lex Fridman (02:32.500)
because a lot of the tedious things that we do now,
Gary Marcus (02:35.200)
we're gonna have machines do,
Lex Fridman (02:36.040)
and a lot of the dangerous things that we do now,
Gary Marcus (02:38.540)
we're gonna have machines do.
Lex Fridman (02:39.900)
I think our whole lives are gonna change
Gary Marcus (02:41.660)
from people finding their meaning through their work
Lex Fridman (02:45.020)
through people finding their meaning
Gary Marcus (02:46.700)
through creative expression.
Lex Fridman (02:48.660)
So the singularity will be a very gradual,
Gary Marcus (02:53.660)
in fact, removing the meaning of the word singularity.
Lex Fridman (02:56.620)
It'll be a very gradual transformation in your view.
Gary Marcus (03:00.540)
I think that it'll be somewhere in between,
Lex Fridman (03:03.460)
and I guess it depends what you mean by gradual and sudden.
Gary Marcus (03:05.700)
I don't think it's gonna be one day.
Lex Fridman (03:07.340)
I think it's important to realize
Gary Marcus (03:08.860)
that intelligence is a multidimensional variable.
Lex Fridman (03:11.820)
So people sort of write this stuff
Gary Marcus (03:14.420)
as if IQ was one number, and the day that you hit 262
Lex Fridman (03:20.620)
or whatever, you displace the human beings.
Lex Fridman (03:22.700)
And really, there's lots of facets to intelligence.
Lex Fridman (03:25.300)
So there's verbal intelligence,
Lex Fridman (03:26.740)
and there's motor intelligence,
Lex Fridman (03:28.580)
and there's mathematical intelligence and so forth.
Gary Marcus (03:32.060)
Machines, in their mathematical intelligence,
Lex Fridman (03:34.620)
far exceed most people already.
Gary Marcus (03:36.900)
In their ability to play games,
Lex Fridman (03:38.140)
they far exceed most people already.
Gary Marcus (03:40.080)
In their ability to understand language,
Lex Fridman (03:41.760)
they lag behind my five year old,
Gary Marcus (03:43.140)
far behind my five year old.
Lex Fridman (03:44.740)
So there are some facets of intelligence
Gary Marcus (03:46.860)
that machines have grasped, and some that they haven't,
Lex Fridman (03:49.460)
and we have a lot of work left to do
Gary Marcus (03:51.780)
to get them to, say, understand natural language,
Lex Fridman (03:54.300)
or to understand how to flexibly approach
Gary Marcus (03:57.780)
some kind of novel MacGyver problem solving
Lex Fridman (04:01.340)
kind of situation.
Lex Fridman (04:03.020)
And I don't know that all of these things will come at once.
Lex Fridman (04:05.620)
I think there are certain vital prerequisites
Gary Marcus (04:07.940)
that we're missing now.
Lex Fridman (04:09.320)
So for example, machines don't really have common sense now.
Lex Fridman (04:12.500)
So they don't understand that bottles contain water,
Lex Fridman (04:15.540)
and that people drink water to quench their thirst,
Lex Fridman (04:18.160)
and that they don't wanna dehydrate.
Lex Fridman (04:19.500)
They don't know these basic facts about human beings,
Lex Fridman (04:22.100)
and I think that that's a rate limiting step
Lex Fridman (04:24.440)
for many things.
Gary Marcus (04:25.300)
It's a great limiting step for reading, for example,
Lex Fridman (04:27.680)
because stories depend on things like,
Gary Marcus (04:29.740)
oh my God, that person's running out of water.
Lex Fridman (04:31.540)
That's why they did this thing.
Gary Marcus (04:33.040)
Or if they only had water, they could put out the fire.
Lex Fridman (04:37.100)
So you watch a movie, and your knowledge
Gary Marcus (04:39.380)
about how things work matter.
Lex Fridman (04:41.220)
And so a computer can't understand that movie
Gary Marcus (04:44.320)
if it doesn't have that background knowledge.
Lex Fridman (04:45.780)
Same thing if you read a book.
Lex Fridman (04:47.900)
And so there are lots of places where,
Lex Fridman (04:49.660)
if we had a good machine interpretable set of common sense,
Gary Marcus (04:53.740)
many things would accelerate relatively quickly,
Lex Fridman (04:56.580)
but I don't think even that is a single point.
Gary Marcus (04:59.940)
There's many different aspects of knowledge.
Lex Fridman (05:02.540)
And we might, for example, find that we make a lot
Gary Marcus (05:05.260)
of progress on physical reasoning,
Lex Fridman (05:06.660)
getting machines to understand, for example,
Lex Fridman (05:09.140)
how keys fit into locks, or that kind of stuff,
Lex Fridman (05:11.980)
or how this gadget here works, and so forth and so on.
Lex Fridman (05:16.980)
And so machines might do that long before they do
Lex Fridman (05:19.500)
really good psychological reasoning,
Gary Marcus (05:21.780)
because it's easier to get kind of labeled data
Lex Fridman (05:24.380)
or to do direct experimentation on a microphone stand
Gary Marcus (05:28.680)
than it is to do direct experimentation on human beings
Lex Fridman (05:31.780)
to understand the levers that guide them.
Gary Marcus (05:34.860)
That's a really interesting point, actually,
Lex Fridman (05:36.860)
whether it's easier to gain common sense knowledge
Gary Marcus (05:39.740)
or psychological knowledge.
Lex Fridman (05:41.740)
I would say the common sense knowledge
Gary Marcus (05:43.300)
includes both physical knowledge and psychological knowledge.
Lex Fridman (05:46.860)
And the argument I was making.
Gary Marcus (05:47.700)
Well, you said physical versus psychological.
Lex Fridman (05:49.660)
Yeah, physical versus psychological.
Lex Fridman (05:51.100)
And the argument I was making is physical knowledge
Lex Fridman (05:53.260)
might be more accessible, because you could have a robot,
Gary Marcus (05:55.300)
for example, lift a bottle, try putting a bottle cap on it,
Lex Fridman (05:58.420)
see that it falls off if it does this,
Lex Fridman (06:00.420)
and see that it could turn it upside down,
Lex Fridman (06:02.020)
and so the robot could do some experimentation.
Gary Marcus (06:04.700)
We do some of our psychological reasoning
Lex Fridman (06:07.220)
by looking at our own minds.
Lex Fridman (06:09.240)
So I can sort of guess how you might react to something
Lex Fridman (06:11.940)
based on how I think I would react to it.
Lex Fridman (06:13.660)
And robots don't have that intuition,
Lex Fridman (06:15.980)
and they also can't do experiments on people
Gary Marcus (06:18.460)
in the same way or we'll probably shut them down.
Lex Fridman (06:20.500)
So if we wanted to have robots figure out
Lex Fridman (06:24.260)
how I respond to pain by pinching me in different ways,
Lex Fridman (06:27.800)
like that's probably, it's not gonna make it
Gary Marcus (06:29.660)
past the human subjects board
Lex Fridman (06:31.020)
and companies are gonna get sued or whatever.
Lex Fridman (06:32.900)
So there's certain kinds of practical experience
Lex Fridman (06:35.860)
that are limited or off limits to robots.
Gary Marcus (06:39.660)
That's a really interesting point.
Lex Fridman (06:41.060)
What is more difficult to gain a grounding in?
Gary Marcus (06:47.540)
Because to play devil's advocate,
Lex Fridman (06:49.940)
I would say that human behavior is easier expressed
Gary Marcus (06:54.980)
in data and digital form.
Lex Fridman (06:56.940)
And so when you look at Facebook algorithms,
Gary Marcus (06:59.100)
they get to observe human behavior.
Lex Fridman (07:01.100)
So you get to study and manipulate even a human behavior
Gary Marcus (07:04.620)
in a way that you perhaps cannot study
Lex Fridman (07:07.740)
or manipulate the physical world.
Lex Fridman (07:09.540)
So it's true why you said pain is like physical pain,
Lex Fridman (07:14.400)
but that's again, the physical world.
Gary Marcus (07:16.020)
Emotional pain might be much easier to experiment with,
Lex Fridman (07:20.080)
perhaps unethical, but nevertheless,
Gary Marcus (07:22.740)
some would argue it's already going on.
Lex Fridman (07:25.380)
I think that you're right, for example,
Gary Marcus (07:27.340)
that Facebook does a lot of experimentation
Lex Fridman (07:30.980)
in psychological reasoning.
Gary Marcus (07:32.900)
In fact, Zuckerberg talked about AI
Lex Fridman (07:36.040)
at a talk that he gave in NIPS.
Gary Marcus (07:38.400)
I wasn't there, but the conference
Lex Fridman (07:40.300)
has been renamed NeurIPS,
Lex Fridman (07:41.300)
but he used to be called NIPS when he gave the talk.
Lex Fridman (07:43.740)
And he talked about Facebook basically
Gary Marcus (07:45.300)
having a gigantic theory of mind.
Lex Fridman (07:47.100)
So I think it is certainly possible.
Gary Marcus (07:49.540)
I mean, Facebook does some of that.
Lex Fridman (07:51.220)
I think they have a really good idea
Gary Marcus (07:52.620)
of how to addict people to things.
Lex Fridman (07:53.900)
They understand what draws people back to things.
Gary Marcus (07:56.420)
I think they exploit it in ways
Lex Fridman (07:57.580)
that I'm not very comfortable with.
Lex Fridman (07:59.220)
But even so, I think that there are only some slices
Lex Fridman (08:03.300)
of human experience that they can access
Gary Marcus (08:05.620)
through the kind of interface they have.
Lex Fridman (08:07.220)
And of course, they're doing all kinds of VR stuff,
Lex Fridman (08:08.980)
and maybe that'll change and they'll expand their data.
Lex Fridman (08:11.940)
And I'm sure that that's part of their goal.
Lex Fridman (08:14.940)
So it is an interesting question.
Lex Fridman (08:16.860)
I think love, fear, insecurity,
Gary Marcus (08:21.700)
all of the things that,
Lex Fridman (08:24.300)
I would say some of the deepest things
Gary Marcus (08:26.620)
about human nature and the human mind
Lex Fridman (08:28.620)
could be explored through digital form.
Gary Marcus (08:30.500)
It's that you're actually the first person
Lex Fridman (08:32.220)
just now that brought up,
Gary Marcus (08:33.680)
I wonder what is more difficult.
Lex Fridman (08:35.860)
Because I think folks who are the slow,
Lex Fridman (08:40.220)
and we'll talk a lot about deep learning,
Lex Fridman (08:41.820)
but the people who are thinking beyond deep learning
Gary Marcus (08:44.860)
are thinking about the physical world.
Lex Fridman (08:46.420)
You're starting to think about robotics
Gary Marcus (08:48.060)
in the home robotics.
Lex Fridman (08:49.180)
How do we make robots manipulate objects,
Gary Marcus (08:52.300)
which requires an understanding of the physical world
Lex Fridman (08:55.020)
and then requires common sense reasoning.
Lex Fridman (08:57.300)
And that has felt to be like the next step
Lex Fridman (08:59.440)
for common sense reasoning,
Lex Fridman (09:00.420)
but you've now brought up the idea
Lex Fridman (09:02.100)
that there's also the emotional part.
Lex Fridman (09:03.620)
And it's interesting whether that's hard or easy.
Lex Fridman (09:06.840)
I think some parts of it are and some aren't.
Lex Fridman (09:08.540)
So my company that I recently founded with Rod Brooks,
Lex Fridman (09:12.660)
from MIT for many years and so forth,
Gary Marcus (09:15.940)
we're interested in both.
Lex Fridman (09:17.240)
We're interested in physical reasoning
Lex Fridman (09:18.580)
and psychological reasoning, among many other things.
Lex Fridman (09:21.500)
And there are pieces of each of these that are accessible.
Lex Fridman (09:26.140)
So if you want a robot to figure out
Lex Fridman (09:28.020)
whether it can fit under a table,
Gary Marcus (09:29.720)
that's a relatively accessible piece of physical reasoning.
Lex Fridman (09:33.660)
If you know the height of the table
Lex Fridman (09:34.760)
and you know the height of the robot, it's not that hard.
Lex Fridman (09:36.980)
If you wanted to do physical reasoning about Jenga,
Gary Marcus (09:39.900)
it gets a little bit more complicated
Lex Fridman (09:41.500)
and you have to have higher resolution data
Gary Marcus (09:43.820)
in order to do it.
Lex Fridman (09:45.260)
With psychological reasoning,
Gary Marcus (09:46.900)
it's not that hard to know, for example,
Lex Fridman (09:49.320)
that people have goals and they like to act on those goals,
Lex Fridman (09:51.700)
but it's really hard to know exactly what those goals are.
Lex Fridman (09:54.900)
But ideas of frustration.
Gary Marcus (09:56.780)
I mean, you could argue it's extremely difficult
Lex Fridman (09:58.780)
to understand the sources of human frustration
Gary Marcus (10:01.460)
as they're playing Jenga with you, or not.
Lex Fridman (10:05.740)
You could argue that it's very accessible.
Gary Marcus (10:08.020)
There's some things that are gonna be obvious
Lex Fridman (10:09.740)
and some not.
Lex Fridman (10:10.580)
So I don't think anybody really can do this well yet,
Lex Fridman (10:14.220)
but I think it's not inconceivable
Gary Marcus (10:16.620)
to imagine machines in the not so distant future
Lex Fridman (10:20.120)
being able to understand that if people lose in a game,
Gary Marcus (10:24.220)
that they don't like that.
Lex Fridman (10:26.260)
That's not such a hard thing to program
Lex Fridman (10:27.940)
and it's pretty consistent across people.
Lex Fridman (10:29.980)
Most people don't enjoy losing
Lex Fridman (10:31.540)
and so that makes it relatively easy to code.
Lex Fridman (10:34.620)
On the other hand, if you wanted to capture everything
Gary Marcus (10:36.860)
about frustration, well, people can get frustrated
Lex Fridman (10:39.180)
for a lot of different reasons.
Gary Marcus (10:40.320)
They might get sexually frustrated,
Lex Fridman (10:42.340)
they might get frustrated,
Gary Marcus (10:43.180)
they can get their promotion at work,
Lex Fridman (10:45.140)
all kinds of different things.
Lex Fridman (10:46.900)
And the more you expand the scope,
Lex Fridman (10:48.580)
the harder it is for anything like the existing techniques
Gary Marcus (10:51.540)
to really do that.
Lex Fridman (10:53.000)
So I'm talking to Garret Kasparov next week
Lex Fridman (10:55.660)
and he seemed pretty frustrated
Lex Fridman (10:57.220)
with his game against Deep Blue, so.
Gary Marcus (10:58.940)
Yeah, well, I'm frustrated with my game
Lex Fridman (11:00.300)
against him last year,
Gary Marcus (11:01.340)
because I played him, I had two excuses,
Lex Fridman (11:03.620)
I'll give you my excuses up front,
Lex Fridman (11:04.900)
but it won't mitigate the outcome.
Lex Fridman (11:07.060)
I was jet lagged and I hadn't played in 25 or 30 years,
Lex Fridman (11:11.100)
but the outcome is he completely destroyed me
Lex Fridman (11:13.020)
and it wasn't even close.
Lex Fridman (11:14.420)
Have you ever been beaten in any board game by a machine?
Lex Fridman (11:19.740)
I have, I actually played the predecessor to Deep Blue.
Gary Marcus (11:24.740)
Deep Thought, I believe it was called,
Lex Fridman (11:27.940)
and that too crushed me.
Lex Fridman (11:30.000)
And that was, and after that you realize it's over for us.
Lex Fridman (11:35.340)
Well, there's no point in my playing Deep Blue.
Gary Marcus (11:36.820)
I mean, it's a waste of Deep Blue's computation.
Lex Fridman (11:40.260)
I mean, I played Kasparov
Gary Marcus (11:41.540)
because we both gave lectures this same event
Lex Fridman (11:44.820)
and he was playing 30 people.
Gary Marcus (11:46.020)
I forgot to mention that.
Lex Fridman (11:46.900)
Not only did he crush me,
Lex Fridman (11:47.980)
but he crushed 29 other people at the same time.
Lex Fridman (11:50.660)
I mean, but the actual philosophical and emotional experience
Gary Marcus (11:55.460)
of being beaten by a machine, I imagine is a,
Lex Fridman (11:59.100)
I mean, to you who thinks about these things
Gary Marcus (12:01.380)
may be a profound experience.
Lex Fridman (12:03.580)
Or no, it was a simple mathematical experience.
Gary Marcus (12:07.780)
Yeah, I think a game like chess particularly
Lex Fridman (12:10.300)
where you have perfect information,
Gary Marcus (12:12.740)
it's two player closed end
Lex Fridman (12:14.780)
and there's more computation for the computer,
Gary Marcus (12:16.940)
it's no surprise the machine wins.
Lex Fridman (12:18.860)
I mean, I'm not sad when a computer,
Gary Marcus (12:22.020)
I'm not sad when a computer calculates
Lex Fridman (12:23.940)
a cube root faster than me.
Gary Marcus (12:25.220)
Like, I know I can't win that game.
Lex Fridman (12:27.860)
I'm not gonna try.
Gary Marcus (12:28.900)
Well, with a system like AlphaGo or AlphaZero,
Lex Fridman (12:32.080)
do you see a little bit more magic in a system like that
Lex Fridman (12:35.060)
even though it's simply playing a board game?
Lex Fridman (12:37.260)
But because there's a strong learning component?
Gary Marcus (12:39.940)
You know, I find you should mention that
Lex Fridman (12:41.300)
in the context of this conversation
Gary Marcus (12:42.580)
because Kasparov and I are working on an article
Lex Fridman (12:45.300)
that's gonna be called AI is not magic.
Gary Marcus (12:47.300)
And, you know, neither one of us thinks that it's magic.
Lex Fridman (12:50.500)
And part of the point of this article
Gary Marcus (12:51.980)
is that AI is actually a grab bag of different techniques
Lex Fridman (12:55.140)
and some of them have,
Gary Marcus (12:56.060)
or they each have their own unique strengths and weaknesses.
Lex Fridman (13:00.060)
So, you know, you read media accounts
Lex Fridman (13:02.820)
and it's like, ooh, AI, it must be magical
Lex Fridman (13:05.200)
or it can solve any problem.
Gary Marcus (13:06.580)
Well, no, some problems are really accessible
Lex Fridman (13:09.500)
like chess and go and other problems like reading
Gary Marcus (13:11.980)
are completely outside the current technology.
Lex Fridman (13:14.940)
And it's not like you can take the technology,
Gary Marcus (13:17.100)
that drives AlphaGo and apply it to reading
Lex Fridman (13:20.100)
and get anywhere.
Gary Marcus (13:21.340)
You know, DeepMind has tried that a bit.
Lex Fridman (13:23.180)
They have all kinds of resources.
Gary Marcus (13:24.500)
You know, they built AlphaGo and they have,
Lex Fridman (13:26.180)
you know, I wrote a piece recently that they lost
Lex Fridman (13:29.460)
and you can argue about the word lost,
Lex Fridman (13:30.540)
but they spent $530 million more than they made last year.
Gary Marcus (13:34.900)
So, you know, they're making huge investments.
Lex Fridman (13:36.620)
They have a large budget
Lex Fridman (13:37.860)
and they have applied the same kinds of techniques
Lex Fridman (13:40.900)
to reading or to language.
Gary Marcus (13:43.220)
It's just much less productive there
Lex Fridman (13:45.540)
because it's a fundamentally different kind of problem.
Gary Marcus (13:47.900)
Chess and go and so forth are closed end problems.
Lex Fridman (13:50.660)
The rules haven't changed in 2,500 years.
Gary Marcus (13:52.980)
There's only so many moves you can make.
Lex Fridman (13:54.700)
You can talk about the exponential
Gary Marcus (13:56.460)
as you look at the combinations of moves,
Lex Fridman (13:58.180)
but fundamentally, you know, the go board has 361 squares.
Gary Marcus (14:01.240)
That's it.
Lex Fridman (14:02.080)
That's the only, you know, those intersections
Gary Marcus (14:04.100)
are the only places that you can place your stone.
Lex Fridman (14:07.300)
Whereas when you're reading,
Gary Marcus (14:09.140)
the next sentence could be anything.
Lex Fridman (14:11.460)
You know, it's completely up to the writer
Lex Fridman (14:13.300)
what they're gonna do next.
Lex Fridman (14:14.460)
That's fascinating that you think this way.
Gary Marcus (14:16.260)
You're clearly a brilliant mind
Lex Fridman (14:17.980)
who points out the emperor has no clothes,
Lex Fridman (14:19.700)
but so I'll play the role of a person who says.
Lex Fridman (14:22.300)
You're gonna put clothes on the emperor?
Gary Marcus (14:23.300)
Good luck with it.
Lex Fridman (14:24.140)
It romanticizes the notion of the emperor, period,
Gary Marcus (14:27.980)
suggesting that clothes don't even matter.
Lex Fridman (14:30.140)
Okay, so that's really interesting
Gary Marcus (14:33.580)
that you're talking about language.
Lex Fridman (14:36.260)
So there's the physical world
Gary Marcus (14:37.780)
of being able to move about the world,
Lex Fridman (14:39.680)
making an omelet and coffee and so on.
Gary Marcus (14:41.940)
There's language where you first understand
Lex Fridman (14:46.020)
what's being written and then maybe even more complicated
Gary Marcus (14:48.860)
than that, having a natural dialogue.
Lex Fridman (14:51.260)
And then there's the game of go and chess.
Gary Marcus (14:53.620)
I would argue that language is much closer to go
Lex Fridman (14:57.540)
than it is to the physical world.
Gary Marcus (14:59.700)
Like it is still very constrained.
Lex Fridman (15:01.460)
When you say the possibility of the number of sentences
Gary Marcus (15:04.740)
that could come, it is huge,
Lex Fridman (15:06.500)
but it nevertheless is much more constrained.
Gary Marcus (15:09.260)
It feels maybe I'm wrong than the possibilities
Lex Fridman (15:12.740)
that the physical world brings us.
Gary Marcus (15:14.540)
There's something to what you say
Lex Fridman (15:15.860)
in some ways in which I disagree.
Lex Fridman (15:17.700)
So one interesting thing about language
Lex Fridman (15:20.620)
is that it abstracts away.
Gary Marcus (15:23.340)
This bottle, I don't know if it would be in the field of view
Lex Fridman (15:26.140)
is on this table and I use the word on here
Lex Fridman (15:28.900)
and I can use the word on here, maybe not here,
Lex Fridman (15:32.980)
but that one word encompasses in analog space
Gary Marcus (15:36.980)
sort of infinite number of possibilities.
Lex Fridman (15:39.340)
So there is a way in which language filters down
Gary Marcus (15:43.060)
the variation of the world and there's other ways.
Lex Fridman (15:46.660)
So we have a grammar and more or less
Gary Marcus (15:49.900)
you have to follow the rules of that grammar.
Lex Fridman (15:51.700)
You can break them a little bit,
Lex Fridman (15:52.700)
but by and large we follow the rules of grammar
Lex Fridman (15:55.420)
and so that's a constraint on language.
Lex Fridman (15:57.020)
So there are ways in which language is a constrained system.
Lex Fridman (15:59.460)
On the other hand, there are many arguments
Gary Marcus (16:02.300)
that say there's an infinite number of possible sentences
Lex Fridman (16:04.740)
and you can establish that by just stacking them up.
Lex Fridman (16:07.660)
So I think there's water on the table,
Lex Fridman (16:09.500)
you think that I think there's water on the table,
Gary Marcus (16:11.740)
your mother thinks that you think that I think
Lex Fridman (16:13.340)
that water's on the table, your brother thinks
Gary Marcus (16:15.620)
that maybe your mom is wrong to think
Lex Fridman (16:17.300)
that you think that I think, right?
Lex Fridman (16:18.660)
So we can make sentences of infinite length
Lex Fridman (16:21.980)
or we can stack up adjectives.
Gary Marcus (16:23.580)
This is a very silly example, a very, very silly example,
Lex Fridman (16:26.420)
a very, very, very, very, very, very silly example
Lex Fridman (16:28.780)
and so forth.
Lex Fridman (16:29.620)
So there are good arguments
Gary Marcus (16:30.980)
that there's an infinite range of sentences.
Lex Fridman (16:32.420)
In any case, it's vast by any reasonable measure
Lex Fridman (16:35.780)
and for example, almost anything in the physical world
Lex Fridman (16:37.980)
we can talk about in the language world
Lex Fridman (16:40.460)
and interestingly, many of the sentences that we understand,
Lex Fridman (16:43.820)
we can only understand if we have a very rich model
Gary Marcus (16:46.820)
of the physical world.
Lex Fridman (16:47.820)
So I don't ultimately want to adjudicate the debate
Gary Marcus (16:50.620)
that I think you just set up, but I find it interesting.
Lex Fridman (16:54.420)
Maybe the physical world is even more complicated
Gary Marcus (16:57.180)
than language, I think that's fair, but.
Lex Fridman (16:59.580)
Language is really, really complicated.
Gary Marcus (17:03.100)
It's really, really hard.
Lex Fridman (17:04.100)
Well, it's really, really hard for machines,
Gary Marcus (17:06.100)
for linguists, people trying to understand it.
Lex Fridman (17:08.500)
It's not that hard for children
Lex Fridman (17:09.660)
and that's part of what's driven my whole career.
Lex Fridman (17:12.100)
I was a student of Steven Pinker's
Lex Fridman (17:14.340)
and we were trying to figure out
Lex Fridman (17:15.340)
why kids could learn language when machines couldn't.
Gary Marcus (17:18.700)
I think we're gonna get into language,
Lex Fridman (17:20.540)
we're gonna get into communication intelligence
Lex Fridman (17:22.460)
and neural networks and so on,
Lex Fridman (17:24.220)
but let me return to the high level,
Gary Marcus (17:28.860)
the futuristic for a brief moment.
Lex Fridman (17:32.540)
So you've written in your book, in your new book,
Gary Marcus (17:37.300)
it would be arrogant to suppose that we could forecast
Lex Fridman (17:39.940)
where AI will be or the impact it will have
Gary Marcus (17:42.500)
in a thousand years or even 500 years.
Lex Fridman (17:45.180)
So let me ask you to be arrogant.
Lex Fridman (17:48.340)
What do AI systems with or without physical bodies
Lex Fridman (17:51.500)
look like 100 years from now?
Gary Marcus (17:53.500)
If you would just, you can't predict,
Lex Fridman (17:56.820)
but if you were to philosophize and imagine, do.
Gary Marcus (18:00.540)
Can I first justify the arrogance
Lex Fridman (18:02.020)
before you try to push me beyond it?
Gary Marcus (18:04.100)
Sure.
Lex Fridman (18:05.940)
I mean, there are examples like,
Gary Marcus (18:07.700)
people figured out how electricity worked,
Lex Fridman (18:09.720)
they had no idea that that was gonna lead to cell phones.
Gary Marcus (18:13.060)
I mean, things can move awfully fast
Lex Fridman (18:15.600)
once new technologies are perfected.
Gary Marcus (18:17.940)
Even when they made transistors,
Lex Fridman (18:19.460)
they weren't really thinking that cell phones
Gary Marcus (18:21.100)
would lead to social networking.
Lex Fridman (18:23.340)
There are nevertheless predictions of the future,
Gary Marcus (18:25.740)
which are statistically unlikely to come to be,
Lex Fridman (18:28.820)
but nevertheless is the best.
Gary Marcus (18:29.660)
You're asking me to be wrong.
Lex Fridman (18:31.380)
Asking you to be statistically.
Lex Fridman (18:32.220)
In which way would I like to be wrong?
Lex Fridman (18:34.020)
Pick the least unlikely to be wrong thing,
Gary Marcus (18:37.500)
even though it's most very likely to be wrong.
Lex Fridman (18:39.760)
I mean, here's some things
Gary Marcus (18:40.600)
that we can safely predict, I suppose.
Lex Fridman (18:42.740)
We can predict that AI will be faster than it is now.
Gary Marcus (18:47.300)
It will be cheaper than it is now.
Lex Fridman (18:49.520)
It will be better in the sense of being more general
Lex Fridman (18:52.880)
and applicable in more places.
Lex Fridman (18:56.980)
It will be pervasive.
Gary Marcus (18:59.300)
I mean, these are easy predictions.
Lex Fridman (19:01.620)
I'm sort of modeling them in my head
Gary Marcus (19:03.320)
on Jeff Bezos's famous predictions.
Lex Fridman (19:05.820)
He says, I can't predict the future,
Gary Marcus (19:07.340)
not in every way, I'm paraphrasing.
Lex Fridman (19:09.820)
But I can predict that people
Gary Marcus (19:11.060)
will never wanna pay more money for their stuff.
Lex Fridman (19:13.220)
They're never gonna want it to take longer to get there.
Lex Fridman (19:15.580)
So you can't predict everything,
Lex Fridman (19:17.800)
but you can predict something.
Gary Marcus (19:18.880)
Sure, of course it's gonna be faster and better.
Lex Fridman (19:21.220)
But what we can't really predict
Gary Marcus (19:24.500)
is the full scope of where AI will be in a certain period.
Lex Fridman (19:28.700)
I mean, I think it's safe to say that,
Gary Marcus (19:31.900)
although I'm very skeptical about current AI,
Lex Fridman (19:35.660)
that it's possible to do much better.
Gary Marcus (19:37.700)
You know, there's no in principled argument
Lex Fridman (19:39.700)
that says AI is an insolvable problem,
Gary Marcus (19:42.100)
that there's magic inside our brains
Lex Fridman (19:43.620)
that will never be captured.
Gary Marcus (19:44.980)
I mean, I've heard people make those kind of arguments.
Lex Fridman (19:46.780)
I don't think they're very good.
Lex Fridman (19:48.980)
So AI's gonna come, and probably 500 years
Lex Fridman (19:54.100)
is plenty to get there.
Lex Fridman (19:55.540)
And then once it's here, it really will change everything.
Lex Fridman (19:59.260)
So when you say AI's gonna come,
Lex Fridman (1:00:01.540)
But I would say that it's a pretty interesting
Lex Fridman (1:00:06.940)
set of things that we are equipped with
Gary Marcus (1:00:08.660)
that allows us to do a lot of interesting things.
Lex Fridman (1:00:10.500)
So I would argue or guess, based on my reading
Gary Marcus (1:00:13.740)
of the developmental psychology literature,
Lex Fridman (1:00:15.220)
which I've also participated in,
Gary Marcus (1:00:17.980)
that children are born with a notion of space,
Lex Fridman (1:00:21.740)
time, other agents, places,
Lex Fridman (1:00:25.740)
and also this kind of mental algebra
Lex Fridman (1:00:27.620)
that I was describing before.
Gary Marcus (1:00:30.220)
No certain causation if I didn't just say that.
Lex Fridman (1:00:33.060)
So at least those kinds of things.
Gary Marcus (1:00:35.220)
They're like frameworks for learning the other things.
Lex Fridman (1:00:38.940)
Are they disjoint in your view
Lex Fridman (1:00:40.340)
or is it just somehow all connected?
Lex Fridman (1:00:42.860)
You've talked a lot about language.
Gary Marcus (1:00:44.340)
Is it all kind of connected in some mesh
Lex Fridman (1:00:47.940)
that's language like?
Lex Fridman (1:00:50.260)
If understanding concepts all together or?
Lex Fridman (1:00:52.740)
I don't think we know for people how they're represented
Lex Fridman (1:00:55.740)
and machines just don't really do this yet.
Lex Fridman (1:00:58.180)
So I think it's an interesting open question
Gary Marcus (1:01:00.540)
both for science and for engineering.
Lex Fridman (1:01:03.540)
Some of it has to be at least interrelated
Gary Marcus (1:01:06.340)
in the way that the interfaces of a software package
Lex Fridman (1:01:10.180)
have to be able to talk to one another.
Lex Fridman (1:01:12.140)
So the systems that represent space and time
Lex Fridman (1:01:16.620)
can't be totally disjoint because a lot of the things
Gary Marcus (1:01:19.820)
that we reason about are the relations
Lex Fridman (1:01:21.500)
between space and time and cause.
Lex Fridman (1:01:22.980)
So I put this on and I have expectations
Lex Fridman (1:01:26.460)
about what's gonna happen with the bottle cap
Gary Marcus (1:01:28.180)
on top of the bottle and those span space and time.
Lex Fridman (1:01:32.540)
If the cap is over here, I get a different outcome.
Gary Marcus (1:01:35.740)
If the timing is different, if I put this here,
Lex Fridman (1:01:38.540)
after I move that, then I get a different outcome.
Gary Marcus (1:01:41.900)
That relates to causality.
Lex Fridman (1:01:43.060)
So obviously these mechanisms, whatever they are,
Gary Marcus (1:01:47.840)
can certainly communicate with each other.
Lex Fridman (1:01:50.100)
So I think evolution had a significant role
Lex Fridman (1:01:53.180)
to play in the development of this whole kluge, right?
Lex Fridman (1:01:57.100)
How efficient do you think is evolution?
Gary Marcus (1:01:59.220)
Oh, it's terribly inefficient except that.
Lex Fridman (1:02:01.620)
Okay, well, can we do better?
Gary Marcus (1:02:03.980)
Well, I'll come to that in a sec.
Lex Fridman (1:02:05.740)
It's inefficient except that.
Gary Marcus (1:02:08.100)
Once it gets a good idea, it runs with it.
Lex Fridman (1:02:10.900)
So it took, I guess, a billion years,
Gary Marcus (1:02:15.660)
if I went roughly a billion years, to evolve
Lex Fridman (1:02:20.420)
to a vertebrate brain plan.
Lex Fridman (1:02:24.040)
And once that vertebrate brain plan evolved,
Lex Fridman (1:02:26.920)
it spread everywhere.
Lex Fridman (1:02:28.480)
So fish have it and dogs have it and we have it.
Lex Fridman (1:02:31.700)
We have adaptations of it and specializations of it,
Gary Marcus (1:02:34.140)
but, and the same thing with a primate brain plan.
Lex Fridman (1:02:37.160)
So monkeys have it and apes have it and we have it.
Lex Fridman (1:02:41.100)
So there are additional innovations like color vision
Lex Fridman (1:02:43.780)
and those spread really rapidly.
Lex Fridman (1:02:45.860)
So it takes evolution a long time to get a good idea,
Lex Fridman (1:02:48.820)
but, and I'm being anthropomorphic and not literal here,
Lex Fridman (1:02:53.300)
but once it has that idea, so to speak,
Lex Fridman (1:02:55.580)
which cashes out into one set of genes or in the genome,
Gary Marcus (1:02:58.540)
those genes spread very rapidly
Lex Fridman (1:03:00.420)
and they're like subroutines or libraries,
Gary Marcus (1:03:02.660)
I guess the word people might use nowadays
Lex Fridman (1:03:04.540)
or be more familiar with.
Gary Marcus (1:03:05.620)
They're libraries that get used over and over again.
Lex Fridman (1:03:08.780)
So once you have the library for building something
Gary Marcus (1:03:11.740)
with multiple digits, you can use it for a hand,
Lex Fridman (1:03:13.840)
but you can also use it for a foot.
Gary Marcus (1:03:15.540)
You just kind of reuse the library
Lex Fridman (1:03:17.420)
with slightly different parameters.
Gary Marcus (1:03:19.080)
Evolution does a lot of that,
Lex Fridman (1:03:20.660)
which means that the speed over time picks up.
Lex Fridman (1:03:23.500)
So evolution can happen faster
Lex Fridman (1:03:25.560)
because you have bigger and bigger libraries.
Lex Fridman (1:03:28.380)
And what I think has happened in attempts
Lex Fridman (1:03:32.220)
at evolutionary computation is that people start
Gary Marcus (1:03:35.740)
with libraries that are very, very minimal,
Lex Fridman (1:03:40.340)
like almost nothing, and then progress is slow
Lex Fridman (1:03:44.260)
and it's hard for someone to get a good PhD thesis
Lex Fridman (1:03:46.620)
out of it and they give up.
Gary Marcus (1:03:48.260)
If we had richer libraries to begin with,
Lex Fridman (1:03:50.260)
if you were evolving from systems
Gary Marcus (1:03:52.580)
that had an rich innate structure to begin with,
Lex Fridman (1:03:55.320)
then things might speed up.
Gary Marcus (1:03:56.780)
Or more PhD students, if the evolutionary process
Lex Fridman (1:03:59.900)
is indeed in a meta way runs away with good ideas,
Gary Marcus (1:04:04.260)
you need to have a lot of ideas,
Lex Fridman (1:04:06.740)
pool of ideas in order for it to discover one
Gary Marcus (1:04:08.820)
that you can run away with.
Lex Fridman (1:04:10.260)
And PhD students representing individual ideas as well.
Gary Marcus (1:04:13.220)
Yeah, I mean, you could throw
Lex Fridman (1:04:14.340)
a billion PhD students at it.
Gary Marcus (1:04:16.220)
Yeah, the monkeys are typewriters with Shakespeare, yep.
Lex Fridman (1:04:20.180)
Well, I mean, those aren't cumulative, right?
Gary Marcus (1:04:22.060)
That's just random.
Lex Fridman (1:04:23.420)
And part of the point that I'm making
Gary Marcus (1:04:24.940)
is that evolution is cumulative.
Lex Fridman (1:04:26.780)
So if you have a billion monkeys independently,
Gary Marcus (1:04:31.140)
you don't really get anywhere.
Lex Fridman (1:04:32.420)
But if you have a billion monkeys,
Lex Fridman (1:04:33.820)
and I think Dawkins made this point originally,
Lex Fridman (1:04:35.700)
or probably other people, Dawkins made it very nice
Lex Fridman (1:04:37.580)
and either a selfish gene or blind watchmaker.
Lex Fridman (1:04:40.420)
If there is some sort of fitness function
Gary Marcus (1:04:44.060)
that can drive you towards something,
Lex Fridman (1:04:45.860)
I guess that's Dawkins point.
Lex Fridman (1:04:47.060)
And my point, which is a variation on that,
Lex Fridman (1:04:49.420)
is that if the evolution is cumulative,
Gary Marcus (1:04:51.940)
I mean, the related points,
Lex Fridman (1:04:53.820)
then you can start going faster.
Lex Fridman (1:04:55.600)
Do you think something like the process of evolution
Lex Fridman (1:04:57.760)
is required to build intelligent systems?
Lex Fridman (1:05:00.180)
So if we... Not logically.
Lex Fridman (1:05:01.560)
So all the stuff that evolution did,
Gary Marcus (1:05:04.040)
a good engineer might be able to do.
Lex Fridman (1:05:07.040)
So for example, evolution made quadrupeds,
Gary Marcus (1:05:10.540)
which distribute the load across a horizontal surface.
Lex Fridman (1:05:14.180)
A good engineer could come up with that idea.
Gary Marcus (1:05:16.980)
I mean, sometimes good engineers come up with ideas
Lex Fridman (1:05:18.740)
by looking at biology.
Gary Marcus (1:05:19.760)
There's lots of ways to get your ideas.
Lex Fridman (1:05:22.500)
Part of what I'm suggesting
Gary Marcus (1:05:23.660)
is we should look at biology a lot more.
Lex Fridman (1:05:25.980)
We should look at the biology of thought and understanding
Lex Fridman (1:05:30.180)
and the biology by which creatures intuitively reason
Lex Fridman (1:05:33.480)
about physics or other agents,
Lex Fridman (1:05:35.960)
or like how do dogs reason about people?
Lex Fridman (1:05:37.900)
Like they're actually pretty good at it.
Gary Marcus (1:05:39.620)
If we could understand, at my college we joked dognition,
Lex Fridman (1:05:44.000)
if we could understand dognition well,
Lex Fridman (1:05:46.280)
and how it was implemented, that might help us with our AI.
Lex Fridman (1:05:49.780)
So do you think it's possible
Gary Marcus (1:05:53.780)
that the kind of timescale that evolution took
Lex Fridman (1:05:57.180)
is the kind of timescale that will be needed
Lex Fridman (1:05:58.940)
to build intelligent systems?
Lex Fridman (1:06:00.500)
Or can we significantly accelerate that process
Lex Fridman (1:06:02.980)
inside a computer?
Lex Fridman (1:06:04.020)
I mean, I think the way that we accelerate that process
Gary Marcus (1:06:07.580)
is we borrow from biology, not slavishly,
Lex Fridman (1:06:12.100)
but I think we look at how biology has solved problems
Lex Fridman (1:06:15.260)
and we say, does that inspire
Lex Fridman (1:06:16.780)
any engineering solutions here?
Gary Marcus (1:06:18.940)
Try to mimic biological systems
Lex Fridman (1:06:20.700)
and then therefore have a shortcut.
Gary Marcus (1:06:22.380)
Yeah, I mean, there's a field called biomimicry
Lex Fridman (1:06:25.020)
and people do that for like material science all the time.
Gary Marcus (1:06:28.980)
We should be doing the analog of that for AI
Lex Fridman (1:06:32.940)
and the analog for that for AI
Gary Marcus (1:06:34.460)
is to look at cognitive science or the cognitive sciences,
Lex Fridman (1:06:37.020)
which is psychology, maybe neuroscience, linguistics,
Lex Fridman (1:06:40.380)
and so forth, look to those for insight.
Lex Fridman (1:06:43.460)
What do you think is a good test of intelligence
Lex Fridman (1:06:45.340)
in your view?
Lex Fridman (1:06:46.180)
So I don't think there's one good test.
Gary Marcus (1:06:48.500)
In fact, I tried to organize a movement
Lex Fridman (1:06:51.780)
towards something called a Turing Olympics
Lex Fridman (1:06:53.380)
and my hope is that Francois is actually gonna take,
Lex Fridman (1:06:56.140)
Francois Chollet is gonna take over this.
Gary Marcus (1:06:58.260)
I think he's interested and I don't,
Lex Fridman (1:06:59.940)
I just don't have place in my busy life at this moment,
Lex Fridman (1:07:03.500)
but the notion is that there'd be many tests
Lex Fridman (1:07:06.460)
and not just one because intelligence is multifaceted.
Gary Marcus (1:07:09.500)
There can't really be a single measure of it
Lex Fridman (1:07:12.900)
because it isn't a single thing.
Gary Marcus (1:07:15.620)
Like just the crudest level,
Lex Fridman (1:07:17.340)
the SAT has a verbal component and a math component
Gary Marcus (1:07:19.860)
because they're not identical.
Lex Fridman (1:07:21.340)
And Howard Gardner has talked about multiple intelligences
Gary Marcus (1:07:23.660)
like kinesthetic intelligence
Lex Fridman (1:07:25.420)
and verbal intelligence and so forth.
Gary Marcus (1:07:27.740)
There are a lot of things that go into intelligence
Lex Fridman (1:07:29.940)
and people can get good at one or the other.
Gary Marcus (1:07:32.580)
I mean, in some sense, like every expert has developed
Lex Fridman (1:07:35.260)
a very specific kind of intelligence
Lex Fridman (1:07:37.260)
and then there are people that are generalists
Lex Fridman (1:07:39.300)
and I think of myself as a generalist
Gary Marcus (1:07:41.740)
with respect to cognitive science,
Lex Fridman (1:07:43.380)
which doesn't mean I know anything about quantum mechanics,
Lex Fridman (1:07:45.620)
but I know a lot about the different facets of the mind.
Lex Fridman (1:07:49.260)
And there's a kind of intelligence
Gary Marcus (1:07:51.380)
to thinking about intelligence.
Lex Fridman (1:07:52.660)
I like to think that I have some of that,
Lex Fridman (1:07:54.740)
but social intelligence, I'm just okay.
Lex Fridman (1:07:57.500)
There are people that are much better at that than I am.
Lex Fridman (1:08:00.140)
Sure, but what would be really impressive to you?
Lex Fridman (1:08:04.140)
I think the idea of a touring Olympics is really interesting
Gary Marcus (1:08:07.060)
especially if somebody like Francois is running it,
Lex Fridman (1:08:09.660)
but to you in general, not as a benchmark,
Lex Fridman (1:08:14.380)
but if you saw an AI system being able to accomplish
Lex Fridman (1:08:17.300)
something that would impress the heck out of you,
Lex Fridman (1:08:21.740)
what would that thing be?
Lex Fridman (1:08:22.740)
Would it be natural language conversation?
Gary Marcus (1:08:24.700)
For me personally, I would like to see
Lex Fridman (1:08:28.580)
a kind of comprehension that relates to what you just said.
Lex Fridman (1:08:30.660)
So I wrote a piece in the New Yorker in I think 2015
Lex Fridman (1:08:34.980)
right after Eugene Guestman, which was a software package,
Gary Marcus (1:08:39.940)
won a version of the Turing test.
Lex Fridman (1:08:42.940)
And the way that it did this is it be,
Gary Marcus (1:08:45.060)
well, the way you win the Turing test,
Lex Fridman (1:08:46.900)
so called win it, is the Turing test is you fool a person
Gary Marcus (1:08:50.700)
into thinking that a machine is a person,
Lex Fridman (1:08:54.420)
is you're evasive, you pretend to have limitations
Lex Fridman (1:08:57.940)
so you don't have to answer certain questions and so forth.
Lex Fridman (1:09:00.540)
So this particular system pretended to be a 13 year old boy
Gary Marcus (1:09:04.300)
from Odessa who didn't understand English
Lex Fridman (1:09:06.980)
and was kind of sarcastic
Lex Fridman (1:09:08.060)
and wouldn't answer your questions and so forth.
Lex Fridman (1:09:09.660)
And so judges got fooled into thinking briefly
Gary Marcus (1:09:12.460)
with a very little exposure, it was a 13 year old boy,
Lex Fridman (1:09:14.660)
and it docked all the questions
Gary Marcus (1:09:16.340)
Turing was actually interested in,
Lex Fridman (1:09:17.540)
which is like how do you make the machine
Lex Fridman (1:09:18.780)
actually intelligent?
Lex Fridman (1:09:20.420)
So that test itself is not that good.
Lex Fridman (1:09:22.100)
And so in New Yorker, I proposed an alternative, I guess,
Lex Fridman (1:09:26.100)
and the one that I proposed there
Gary Marcus (1:09:27.260)
was a comprehension test.
Lex Fridman (1:09:30.020)
And I must like Breaking Bad
Gary Marcus (1:09:31.060)
because I've already given you one Breaking Bad example
Lex Fridman (1:09:32.900)
and in that article, I have one as well,
Gary Marcus (1:09:35.660)
which was something like if Walter,
Lex Fridman (1:09:37.660)
you should be able to watch an episode of Breaking Bad
Gary Marcus (1:09:40.340)
or maybe you have to watch the whole series
Lex Fridman (1:09:41.700)
to be able to answer the question and say,
Gary Marcus (1:09:43.500)
if Walter White took a hit out on Jesse,
Lex Fridman (1:09:45.580)
why did he do that?
Lex Fridman (1:09:47.180)
So if you could answer kind of arbitrary questions
Lex Fridman (1:09:49.380)
about characters motivations, I would be really impressed
Gary Marcus (1:09:52.700)
with that and he built software to do that.
Lex Fridman (1:09:55.380)
They could watch a film or there are different versions.
Lex Fridman (1:09:58.500)
And so ultimately, I wrote this up with Praveen Paritosh
Lex Fridman (1:10:01.940)
in a special issue of AI Magazine
Gary Marcus (1:10:04.060)
that basically was about the Turing Olympics.
Lex Fridman (1:10:05.780)
There were like 14 tests proposed.
Gary Marcus (1:10:07.700)
The one that I was pushing was a comprehension challenge
Lex Fridman (1:10:10.100)
and Praveen who's at Google was trying to figure out
Gary Marcus (1:10:12.380)
like how we would actually run it
Lex Fridman (1:10:13.460)
and so we wrote a paper together.
Lex Fridman (1:10:15.340)
And you could have a text version too
Lex Fridman (1:10:17.300)
or you could have an auditory podcast version,
Gary Marcus (1:10:19.780)
you could have a written version.
Lex Fridman (1:10:20.620)
But the point is that you win at this test
Gary Marcus (1:10:23.820)
if you can do, let's say human level or better than humans
Lex Fridman (1:10:27.060)
at answering kind of arbitrary questions.
Lex Fridman (1:10:29.780)
Why did this person pick up the stone?
Lex Fridman (1:10:31.660)
What were they thinking when they picked up the stone?
Lex Fridman (1:10:34.180)
Were they trying to knock down glass?
Lex Fridman (1:10:36.260)
And I mean, ideally these wouldn't be multiple choice either
Gary Marcus (1:10:38.700)
because multiple choice is pretty easily gamed.
Lex Fridman (1:10:41.140)
So if you could have relatively open ended questions
Lex Fridman (1:10:44.180)
and you can answer why people are doing this stuff,
Lex Fridman (1:10:47.380)
I would be very impressed.
Lex Fridman (1:10:48.220)
And of course, humans can do this, right?
Lex Fridman (1:10:50.060)
If you watch a well constructed movie
Lex Fridman (1:10:52.820)
and somebody picks up a rock,
Lex Fridman (1:10:55.540)
everybody watching the movie
Lex Fridman (1:10:56.940)
knows why they picked up the rock, right?
Lex Fridman (1:10:59.420)
They all know, oh my gosh,
Gary Marcus (1:11:01.140)
he's gonna hit this character or whatever.
Lex Fridman (1:11:03.620)
We have an example in the book about
Gary Marcus (1:11:06.220)
when a whole bunch of people say, I am Spartacus,
Lex Fridman (1:11:08.700)
you know, this famous scene.
Gary Marcus (1:11:11.780)
The viewers understand,
Lex Fridman (1:11:13.540)
first of all, that everybody or everybody minus one
Gary Marcus (1:11:18.220)
has to be lying.
Lex Fridman (1:11:19.060)
They can't all be Spartacus.
Gary Marcus (1:11:20.340)
We have enough common sense knowledge
Lex Fridman (1:11:21.780)
to know they couldn't all have the same name.
Gary Marcus (1:11:24.100)
We know that they're lying
Lex Fridman (1:11:25.340)
and we can infer why they're lying, right?
Gary Marcus (1:11:27.100)
They're lying to protect someone
Lex Fridman (1:11:28.460)
and to protect things they believe in.
Gary Marcus (1:11:30.340)
You get a machine that can do that.
Lex Fridman (1:11:32.340)
They can say, this is why these guys all got up
Lex Fridman (1:11:35.100)
and said, I am Spartacus.
Lex Fridman (1:11:36.940)
I will sit down and say, AI has really achieved a lot.
Gary Marcus (1:11:40.540)
Thank you.
Lex Fridman (1:11:41.380)
Without cheating any part of the system.
Gary Marcus (1:11:43.860)
Yeah, I mean, if you do it,
Lex Fridman (1:11:45.620)
there are lots of ways you could cheat.
Gary Marcus (1:11:46.700)
You could build a Spartacus machine
Lex Fridman (1:11:48.820)
that works on that film.
Gary Marcus (1:11:50.260)
That's not what I'm talking about.
Lex Fridman (1:11:51.100)
I'm talking about, you can do this
Gary Marcus (1:11:52.860)
with essentially arbitrary films
Lex Fridman (1:11:54.860)
or from a large set. Even beyond films
Gary Marcus (1:11:56.580)
because it's possible such a system would discover
Lex Fridman (1:11:58.980)
that the number of narrative arcs in film
Gary Marcus (1:12:02.580)
is limited to 1930. Well, there's a famous thing
Lex Fridman (1:12:04.740)
about the classic seven plots or whatever.
Gary Marcus (1:12:07.060)
I don't care.
Lex Fridman (1:12:07.900)
If you wanna build in the system,
Gary Marcus (1:12:09.140)
boy meets girl, boy loses girl, boy finds girl.
Lex Fridman (1:12:11.660)
That's fine.
Gary Marcus (1:12:12.500)
I don't mind having some head stories on it.
Lex Fridman (1:12:13.980)
And they acknowledge.
Gary Marcus (1:12:14.820)
Okay, good.
Lex Fridman (1:12:16.340)
I mean, you could build it in innately
Gary Marcus (1:12:17.980)
or you could have your system watch a lot of films again.
Lex Fridman (1:12:20.460)
If you can do this at all,
Lex Fridman (1:12:22.380)
but with a wide range of films,
Lex Fridman (1:12:23.740)
not just one film in one genre.
Lex Fridman (1:12:27.340)
But even if you could do it for all Westerns,
Lex Fridman (1:12:28.860)
I'd be reasonably impressed.
Gary Marcus (1:12:30.300)
Yeah.
Lex Fridman (1:12:31.940)
So in terms of being impressed,
Gary Marcus (1:12:34.100)
just for the fun of it,
Lex Fridman (1:12:35.820)
because you've put so many interesting ideas out there
Gary Marcus (1:12:38.420)
in your book,
Lex Fridman (1:12:40.420)
challenging the community for further steps.
Gary Marcus (1:12:43.700)
Is it possible on the deep learning front
Lex Fridman (1:12:46.740)
that you're wrong about its limitations?
Gary Marcus (1:12:50.260)
That deep learning will unlock,
Lex Fridman (1:12:52.260)
Yann LeCun next year will publish a paper
Gary Marcus (1:12:54.500)
that achieves this comprehension.
Lex Fridman (1:12:56.940)
So do you think that way often as a scientist?
Lex Fridman (1:13:00.300)
Do you consider that your intuition
Lex Fridman (1:13:03.060)
that deep learning could actually run away with it?
Gary Marcus (1:13:06.740)
I'm more worried about rebranding
Lex Fridman (1:13:09.780)
as a kind of political thing.
Gary Marcus (1:13:11.380)
So, I mean, what's gonna happen, I think,
Lex Fridman (1:13:14.100)
is the deep learning is gonna start
Gary Marcus (1:13:15.660)
to encompass symbol manipulation.
Lex Fridman (1:13:17.380)
So I think Hinton's just wrong.
Gary Marcus (1:13:19.260)
Hinton says we don't want hybrids.
Lex Fridman (1:13:20.860)
I think people will work towards hybrids
Lex Fridman (1:13:22.380)
and they will relabel their hybrids as deep learning.
Lex Fridman (1:13:24.740)
We've already seen some of that.
Lex Fridman (1:13:25.860)
So AlphaGo is often described as a deep learning system,
Lex Fridman (1:13:29.620)
but it's more correctly described as a system
Gary Marcus (1:13:31.740)
that has deep learning, but also Monte Carlo tree search,
Lex Fridman (1:13:33.940)
which is a classical AI technique.
Lex Fridman (1:13:35.580)
And people will start to blur the lines
Lex Fridman (1:13:37.540)
in the way that IBM blurred Watson.
Gary Marcus (1:13:39.820)
First, Watson meant this particular system,
Lex Fridman (1:13:41.580)
and then it was just anything that IBM built
Gary Marcus (1:13:43.140)
in their cognitive division.
Lex Fridman (1:13:44.140)
But purely, let me ask, for sure,
Gary Marcus (1:13:45.740)
that's a branding question and that's like a giant mess.
Lex Fridman (1:13:49.500)
I mean, purely, a single neural network
Gary Marcus (1:13:51.940)
being able to accomplish reasonable comprehension.
Lex Fridman (1:13:54.060)
I don't stay up at night worrying
Gary Marcus (1:13:55.780)
that that's gonna happen.
Lex Fridman (1:13:57.780)
And I'll just give you two examples.
Gary Marcus (1:13:59.220)
One is a guy at DeepMind thought he had finally outfoxed me.
Lex Fridman (1:14:03.540)
At Zergilord, I think is his Twitter handle.
Lex Fridman (1:14:06.980)
And he said, he specifically made an example.
Lex Fridman (1:14:10.580)
Marcus said that such and such.
Gary Marcus (1:14:12.620)
He fed it into GP2, which is the AI system
Lex Fridman (1:14:16.420)
that is so smart that OpenAI couldn't release it
Lex Fridman (1:14:19.060)
because it would destroy the world, right?
Lex Fridman (1:14:21.180)
You remember that a few months ago.
Lex Fridman (1:14:22.940)
So he feeds it into GPT2, and my example
Lex Fridman (1:14:27.220)
was something like a rose is a rose,
Gary Marcus (1:14:28.740)
a tulip is a tulip, a lily is a blank.
Lex Fridman (1:14:31.340)
And he got it to actually do that,
Gary Marcus (1:14:32.860)
which was a little bit impressive.
Lex Fridman (1:14:34.020)
And I wrote back and I said, that's impressive,
Lex Fridman (1:14:35.340)
but can I ask you a few questions?
Lex Fridman (1:14:37.740)
I said, was that just one example?
Lex Fridman (1:14:40.060)
Can it do it generally?
Lex Fridman (1:14:41.620)
And can it do it with novel words,
Gary Marcus (1:14:43.220)
which was part of what I was talking about in 1998
Lex Fridman (1:14:45.300)
when I first raised the example.
Lex Fridman (1:14:46.740)
So a dax is a dax, right?
Lex Fridman (1:14:50.340)
And he sheepishly wrote back about 20 minutes later.
Lex Fridman (1:14:53.020)
And the answer was, well, it had some problems with those.
Lex Fridman (1:14:55.340)
So I made some predictions 21 years ago that still hold.
Lex Fridman (1:15:00.500)
In the world of computer science, that's amazing, right?
Lex Fridman (1:15:02.660)
Because there's a thousand or a million times more memory
Lex Fridman (1:15:06.500)
and computations a million times,
Lex Fridman (1:15:10.020)
do million times more operations per second
Gary Marcus (1:15:13.140)
spread across a cluster.
Lex Fridman (1:15:15.340)
And there's been advances in replacing sigmoids
Gary Marcus (1:15:20.780)
with other functions and so forth.
Lex Fridman (1:15:23.380)
There's all kinds of advances,
Lex Fridman (1:15:25.380)
but the fundamental architecture hasn't changed
Lex Fridman (1:15:27.100)
and the fundamental limit hasn't changed.
Lex Fridman (1:15:28.580)
And what I said then is kind of still true.
Lex Fridman (1:15:30.860)
Then here's a second example.
Gary Marcus (1:15:32.220)
I recently had a piece in Wired
Lex Fridman (1:15:34.020)
that's adapted from the book.
Lex Fridman (1:15:35.260)
And the book went to press before GP2 came out,
Lex Fridman (1:15:40.140)
but we described this children's story
Lex Fridman (1:15:42.300)
and all the inferences that you make in this story
Lex Fridman (1:15:45.580)
about a boy finding a lost wallet.
Lex Fridman (1:15:48.260)
And for fun, in the Wired piece, we ran it through GP2.
Lex Fridman (1:15:52.860)
GPT2, something called talktotransformer.com,
Lex Fridman (1:15:55.460)
and your viewers can try this experiment themselves.
Lex Fridman (1:15:58.180)
Go to the Wired piece that has the link
Lex Fridman (1:15:59.700)
and it has the story.
Lex Fridman (1:16:01.100)
And the system made perfectly fluent text
Gary Marcus (1:16:04.300)
that was totally inconsistent
Lex Fridman (1:16:06.420)
with the conceptual underpinnings of the story, right?
Gary Marcus (1:16:10.260)
This is what, again, I predicted in 1998.
Lex Fridman (1:16:13.220)
And for that matter, Chomsky and Miller
Gary Marcus (1:16:14.700)
made the same prediction in 1963.
Lex Fridman (1:16:16.660)
I was just updating their claim for a slightly new text.
Lex Fridman (1:16:19.420)
So those particular architectures
Lex Fridman (1:16:22.580)
that don't have any built in knowledge,
Gary Marcus (1:16:24.820)
they're basically just a bunch of layers
Lex Fridman (1:16:27.020)
doing correlational stuff.
Gary Marcus (1:16:28.940)
They're not gonna solve these problems.
Lex Fridman (1:16:31.220)
So 20 years ago, you said the emperor has no clothes.
Gary Marcus (1:16:34.500)
Today, the emperor still has no clothes.
Lex Fridman (1:16:36.860)
The lighting's better though.
Gary Marcus (1:16:38.020)
The lighting is better.
Lex Fridman (1:16:39.020)
And I think you yourself are also, I mean.
Lex Fridman (1:16:42.260)
And we found out some things to do with naked emperors.
Lex Fridman (1:16:44.340)
I mean, it's not like stuff is worthless.
Gary Marcus (1:16:46.420)
I mean, they're not really naked.
Lex Fridman (1:16:48.260)
It's more like they're in their briefs
Gary Marcus (1:16:49.580)
than everybody thinks they are.
Lex Fridman (1:16:50.820)
And so like, I mean, they are great at speech recognition,
Lex Fridman (1:16:54.340)
but the problems that I said were hard.
Lex Fridman (1:16:56.460)
I didn't literally say the emperor has no clothes.
Gary Marcus (1:16:58.220)
I said, this is a set of problems
Lex Fridman (1:17:00.140)
that humans are really good at.
Lex Fridman (1:17:01.780)
And it wasn't couched as AI.
Lex Fridman (1:17:03.140)
It was couched as cognitive science.
Lex Fridman (1:17:04.300)
But I said, if you wanna build a neural model
Lex Fridman (1:17:07.700)
of how humans do certain class of things,
Gary Marcus (1:17:10.340)
you're gonna have to change the architecture.
Lex Fridman (1:17:11.940)
And I stand by those claims.
Gary Marcus (1:17:13.620)
So, and I think people should understand
Lex Fridman (1:17:16.740)
you're quite entertaining in your cynicism,
Lex Fridman (1:17:19.020)
but you're also very optimistic and a dreamer
Lex Fridman (1:17:22.220)
about the future of AI too.
Lex Fridman (1:17:23.900)
So you're both, it's just.
Lex Fridman (1:17:25.340)
There's a famous saying about being,
Gary Marcus (1:17:27.820)
people overselling technology in the short run
Lex Fridman (1:17:30.700)
and underselling it in the long run.
Lex Fridman (1:17:34.100)
And so I actually end the book,
Lex Fridman (1:17:37.180)
Ernie Davis and I end our book with an optimistic chapter,
Gary Marcus (1:17:40.500)
which kind of killed Ernie
Lex Fridman (1:17:41.700)
because he's even more pessimistic than I am.
Gary Marcus (1:17:44.380)
He describes me as a contrarian and him as a pessimist.
Lex Fridman (1:17:47.580)
But I persuaded him that we should end the book
Gary Marcus (1:17:49.820)
with a look at what would happen
Lex Fridman (1:17:52.620)
if AI really did incorporate, for example,
Gary Marcus (1:17:55.340)
the common sense reasoning and the nativism
Lex Fridman (1:17:57.300)
and so forth, the things that we counseled for.
Lex Fridman (1:17:59.660)
And we wrote it and it's an optimistic chapter
Lex Fridman (1:18:02.140)
that AI suitably reconstructed so that we could trust it,
Gary Marcus (1:18:05.900)
which we can't now, could really be world changing.
Lex Fridman (1:18:09.500)
So on that point, if you look at the future trajectories
Gary Marcus (1:18:13.100)
of AI, people have worries about negative effects of AI,
Lex Fridman (1:18:17.140)
whether it's at the large existential scale
Gary Marcus (1:18:21.020)
or smaller short term scale of negative impact on society.
Lex Fridman (1:18:25.220)
So you write about trustworthy AI,
Lex Fridman (1:18:27.140)
how can we build AI systems that align with our values,
Lex Fridman (1:18:31.500)
that make for a better world,
Lex Fridman (1:18:32.780)
that we can interact with, that we can trust?
Lex Fridman (1:18:34.980)
The first thing we have to do
Gary Marcus (1:18:35.820)
is to replace deep learning with deep understanding.
Lex Fridman (1:18:38.260)
So you can't have alignment with a system
Gary Marcus (1:18:42.460)
that traffics only in correlations
Lex Fridman (1:18:44.620)
and doesn't understand concepts like bottles or harm.
Lex Fridman (1:18:47.900)
So Asimov talked about these famous laws
Lex Fridman (1:18:51.340)
and the first one was first do no harm.
Lex Fridman (1:18:54.060)
And you can quibble about the details of Asimov's laws,
Lex Fridman (1:18:56.860)
but we have to, if we're gonna build real robots
Gary Marcus (1:18:58.780)
in the real world, have something like that.
Lex Fridman (1:19:00.540)
That means we have to program in a notion
Gary Marcus (1:19:02.500)
that's at least something like harm.
Lex Fridman (1:19:04.240)
That means we have to have these more abstract ideas
Gary Marcus (1:19:06.620)
that deep learning is not particularly good at.
Lex Fridman (1:19:08.460)
They have to be in the mix somewhere.
Lex Fridman (1:19:10.620)
And you could do statistical analysis
Lex Fridman (1:19:12.380)
about probabilities of given harms or whatever,
Lex Fridman (1:19:14.380)
but you have to know what a harm is
Lex Fridman (1:19:15.820)
in the same way that you have to understand
Gary Marcus (1:19:17.420)
that a bottle isn't just a collection of pixels.
Lex Fridman (1:19:20.660)
And also be able to, you're implying
Gary Marcus (1:19:24.020)
that you need to also be able to communicate
Lex Fridman (1:19:25.940)
that to humans so the AI systems would be able
Gary Marcus (1:19:29.700)
to prove to humans that they understand
Lex Fridman (1:19:33.780)
that they know what harm means.
Gary Marcus (1:19:35.460)
I might run it in the reverse direction,
Lex Fridman (1:19:37.380)
but roughly speaking, I agree with you.
Lex Fridman (1:19:38.620)
So we probably need to have committees
Lex Fridman (1:19:42.500)
of wise people, ethicists and so forth.
Gary Marcus (1:19:45.660)
Think about what these rules ought to be
Lex Fridman (1:19:47.500)
and we shouldn't just leave it to software engineers.
Gary Marcus (1:19:49.780)
It shouldn't just be software engineers
Lex Fridman (1:19:51.620)
and it shouldn't just be people
Gary Marcus (1:19:53.900)
who own large mega corporations
Lex Fridman (1:19:56.580)
that are good at technology, ethicists
Lex Fridman (1:19:58.860)
and so forth should be involved.
Lex Fridman (1:20:00.260)
But there should be some assembly of wise people
Gary Marcus (1:20:04.660)
as I was putting it that tries to figure out
Lex Fridman (1:20:07.220)
what the rules ought to be.
Lex Fridman (1:20:08.700)
And those have to get translated into code.
Lex Fridman (1:20:12.460)
You can argue or code or neural networks or something.
Gary Marcus (1:20:15.460)
They have to be translated into something
Lex Fridman (1:20:18.660)
that machines can work with.
Lex Fridman (1:20:19.980)
And that means there has to be a way
Lex Fridman (1:20:21.940)
of working the translation.
Lex Fridman (1:20:23.380)
And right now we don't.
Lex Fridman (1:20:24.460)
We don't have a way.
Lex Fridman (1:20:25.340)
So let's say you and I were the committee
Lex Fridman (1:20:27.060)
and we decide that Asimov's first law is actually right.
Lex Fridman (1:20:29.820)
And let's say it's not just two white guys,
Lex Fridman (1:20:31.580)
which would be kind of unfortunate that we have abroad.
Lex Fridman (1:20:34.020)
And so we've representative sample of the world
Lex Fridman (1:20:36.260)
or however we wanna do this.
Lex Fridman (1:20:37.500)
And the committee decides eventually,
Lex Fridman (1:20:40.460)
okay, Asimov's first law is actually pretty good.
Gary Marcus (1:20:42.820)
There are these exceptions to it.
Lex Fridman (1:20:44.060)
We wanna program in these exceptions.
Lex Fridman (1:20:46.060)
But let's start with just the first one
Lex Fridman (1:20:47.460)
and then we'll get to the exceptions.
Gary Marcus (1:20:48.860)
First one is first do no harm.
Lex Fridman (1:20:50.620)
Well, somebody has to now actually turn that into
Gary Marcus (1:20:53.620)
a computer program or a neural network or something.
Lex Fridman (1:20:56.220)
And one way of taking the whole book,
Gary Marcus (1:20:58.740)
the whole argument that I'm making
Lex Fridman (1:21:00.260)
is that we just don't have to do that yet.
Lex Fridman (1:21:02.500)
And we're fooling ourselves
Lex Fridman (1:21:03.540)
if we think that we can build trustworthy AI
Gary Marcus (1:21:05.860)
if we can't even specify in any kind of,
Lex Fridman (1:21:09.500)
we can't do it in Python and we can't do it in TensorFlow.
Gary Marcus (1:21:13.140)
We're fooling ourselves in thinking
Lex Fridman (1:21:14.380)
that we can make trustworthy AI
Gary Marcus (1:21:15.820)
if we can't translate harm into something
Lex Fridman (1:21:18.780)
that we can execute.
Lex Fridman (1:21:19.940)
And if we can't, then we should be thinking really hard
Lex Fridman (1:21:22.820)
how could we ever do such a thing?
Gary Marcus (1:21:24.620)
Because if we're gonna use AI
Lex Fridman (1:21:26.500)
in the ways that we wanna use it,
Gary Marcus (1:21:27.940)
to make job interviews or to do surveillance,
Lex Fridman (1:21:31.060)
not that I personally wanna do that or whatever.
Gary Marcus (1:21:32.460)
I mean, if we're gonna use AI
Lex Fridman (1:21:33.780)
in ways that have practical impact on people's lives
Gary Marcus (1:21:36.180)
or medicine, it's gotta be able
Lex Fridman (1:21:38.980)
to understand stuff like that.
Lex Fridman (1:21:41.180)
So one of the things your book highlights
Lex Fridman (1:21:42.820)
is that a lot of people in the deep learning community,
Lex Fridman (1:21:47.380)
but also the general public, politicians,
Lex Fridman (1:21:50.220)
just people in all general groups and walks of life
Gary Marcus (1:21:53.220)
have different levels of misunderstanding of AI.
Lex Fridman (1:21:57.340)
So when you talk about committees,
Lex Fridman (1:22:00.940)
what's your advice to our society?
Lex Fridman (1:22:05.620)
How do we grow, how do we learn about AI
Gary Marcus (1:22:08.140)
such that such committees could emerge
Lex Fridman (1:22:10.820)
where large groups of people could have
Gary Marcus (1:22:13.500)
a productive discourse about
Lex Fridman (1:22:15.180)
how to build successful AI systems?
Gary Marcus (1:22:17.820)
Part of the reason we wrote the book
Lex Fridman (1:22:19.660)
was to try to inform those committees.
Lex Fridman (1:22:22.060)
So part of the reason we wrote the book
Lex Fridman (1:22:23.540)
was to inspire a future generation of students
Gary Marcus (1:22:25.660)
to solve what we think are the important problems.
Lex Fridman (1:22:27.860)
So a lot of the book is trying to pinpoint
Lex Fridman (1:22:29.860)
what we think are the hard problems
Lex Fridman (1:22:31.220)
where we think effort would most be rewarded.
Lex Fridman (1:22:34.020)
And part of it is to try to train people
Lex Fridman (1:22:37.780)
who talk about AI, but aren't experts in the field
Gary Marcus (1:22:41.020)
to understand what's realistic and what's not.
Lex Fridman (1:22:43.500)
One of my favorite parts in the book
Gary Marcus (1:22:44.660)
is the six questions you should ask
Lex Fridman (1:22:46.940)
anytime you read a media account.
Lex Fridman (1:22:48.380)
So like number one is if somebody talks about something,
Lex Fridman (1:22:51.060)
look for the demo.
Gary Marcus (1:22:51.900)
If there's no demo, don't believe it.
Lex Fridman (1:22:54.100)
Like the demo that you can try.
Gary Marcus (1:22:55.300)
If you can't try it at home,
Lex Fridman (1:22:56.460)
maybe it doesn't really work that well yet.
Lex Fridman (1:22:58.380)
So if, we don't have this example in the book,
Lex Fridman (1:23:00.620)
but if Sundar Pinchai says we have this thing
Gary Marcus (1:23:04.140)
that allows it to sound like human beings in conversation,
Lex Fridman (1:23:08.380)
you should ask, can I try it?
Lex Fridman (1:23:10.380)
And you should ask how general it is.
Lex Fridman (1:23:11.860)
And it turns out at that time,
Gary Marcus (1:23:13.060)
I'm alluding to Google Duplex when it was announced,
Lex Fridman (1:23:15.460)
it only worked on calling hairdressers,
Gary Marcus (1:23:18.220)
restaurants and finding opening hours.
Lex Fridman (1:23:20.020)
That's not very general, that's narrow AI.
Lex Fridman (1:23:22.260)
And I'm not gonna ask your thoughts about Sophia,
Lex Fridman (1:23:24.580)
but yeah, I understand that's a really good question
Gary Marcus (1:23:27.740)
to ask of any kind of hype top idea.
Lex Fridman (1:23:30.220)
Sophia has very good material written for her,
Lex Fridman (1:23:32.580)
but she doesn't understand the things that she's saying.
Lex Fridman (1:23:35.380)
So a while ago you've written a book
Gary Marcus (1:23:38.220)
on the science of learning, which I think is fascinating,
Lex Fridman (1:23:40.540)
but the learning case studies of playing guitar.
Gary Marcus (1:23:43.500)
That's called Guitar Zero.
Lex Fridman (1:23:45.100)
I love guitar myself, I've been playing my whole life.
Lex Fridman (1:23:47.340)
So let me ask a very important question.
Lex Fridman (1:23:50.260)
What is your favorite song, rock song,
Lex Fridman (1:23:53.500)
to listen to or try to play?
Lex Fridman (1:23:56.300)
Well, those would be different,
Lex Fridman (1:23:57.140)
but I'll say that my favorite rock song to listen to
Lex Fridman (1:23:59.660)
is probably All Along the Watchtower,
Gary Marcus (1:24:01.060)
the Jimi Hendrix version.
Lex Fridman (1:24:01.980)
The Jimi Hendrix version.
Gary Marcus (1:24:02.980)
It feels magic to me.
Lex Fridman (1:24:04.860)
I've actually recently learned it, I love that song.
Gary Marcus (1:24:07.040)
I've been trying to put it on YouTube, myself singing.
Lex Fridman (1:24:09.380)
Singing is the scary part.
Gary Marcus (1:24:11.300)
If you could party with a rock star for a weekend,
Lex Fridman (1:24:13.380)
living or dead, who would you choose?
Lex Fridman (1:24:17.780)
And pick their mind, it's not necessarily about the partying.
Lex Fridman (1:24:21.140)
Thanks for the clarification.
Gary Marcus (1:24:24.700)
I guess John Lennon's such an intriguing person,
Lex Fridman (1:24:26.980)
and I think a troubled person, but an intriguing one.
Gary Marcus (1:24:31.660)
Beautiful.
Lex Fridman (1:24:32.500)
Well, Imagine is one of my favorite songs.
Gary Marcus (1:24:35.460)
Also one of my favorite songs.
Lex Fridman (1:24:37.100)
That's a beautiful way to end it.
Gary Marcus (1:24:38.300)
Gary, thank you so much for talking to me.
Lex Fridman (1:24:39.780)
Thanks so much for having me.
Lex Fridman (20:01.060)
are you talking about human level intelligence?
Lex Fridman (20:03.660)
So maybe I...
Gary Marcus (20:04.980)
I like the term general intelligence.
Lex Fridman (20:06.660)
So I don't think that the ultimate AI,
Gary Marcus (20:09.500)
if there is such a thing, is gonna look just like humans.
Lex Fridman (20:11.980)
I think it's gonna do some things
Gary Marcus (20:13.600)
that humans do better than current machines,
Lex Fridman (20:16.580)
like reason flexibly.
Lex Fridman (20:18.580)
And understand language and so forth.
Lex Fridman (20:21.180)
But it doesn't mean they have to be identical to humans.
Lex Fridman (20:23.460)
So for example, humans have terrible memory,
Lex Fridman (20:25.980)
and they suffer from what some people
Gary Marcus (20:28.780)
call motivated reasoning.
Lex Fridman (20:29.920)
So they like arguments that seem to support them,
Lex Fridman (20:32.460)
and they dismiss arguments that they don't like.
Lex Fridman (20:35.460)
There's no reason that a machine should ever do that.
Lex Fridman (20:38.660)
So you see that those limitations of memory
Lex Fridman (20:42.280)
as a bug, not a feature.
Gary Marcus (20:43.940)
Absolutely.
Lex Fridman (20:44.820)
I'll say two things about that.
Gary Marcus (20:46.620)
One is I was on a panel with Danny Kahneman,
Lex Fridman (20:48.660)
the Nobel Prize winner, last night,
Lex Fridman (20:50.300)
and we were talking about this stuff.
Lex Fridman (20:51.760)
And I think what we converged on
Gary Marcus (20:53.480)
is that humans are a low bar to exceed.
Lex Fridman (20:56.120)
They may be outside of our skill right now,
Lex Fridman (20:58.940)
but as AI programmers, but eventually AI will exceed it.
Lex Fridman (21:04.300)
So we're not talking about human level AI.
Gary Marcus (21:06.060)
We're talking about general intelligence
Lex Fridman (21:07.900)
that can do all kinds of different things
Lex Fridman (21:09.420)
and do it without some of the flaws that human beings have.
Lex Fridman (21:12.220)
The other thing I'll say is I wrote a whole book,
Gary Marcus (21:13.700)
actually, about the flaws of humans.
Lex Fridman (21:15.280)
It's actually a nice bookend to the,
Gary Marcus (21:17.980)
or counterpoint to the current book.
Lex Fridman (21:19.180)
So I wrote a book called Cluj,
Gary Marcus (21:21.380)
which was about the limits of the human mind.
Lex Fridman (21:24.020)
The current book is kind of about those few things
Gary Marcus (21:26.380)
that humans do a lot better than machines.
Lex Fridman (21:28.760)
Do you think it's possible that the flaws
Gary Marcus (21:30.820)
of the human mind, the limits of memory,
Lex Fridman (21:33.260)
our mortality, our bias,
Gary Marcus (21:38.460)
is a strength, not a weakness,
Lex Fridman (21:40.300)
that that is the thing that enables,
Lex Fridman (21:43.500)
from which motivation springs and meaning springs or not?
Lex Fridman (21:47.940)
I've heard a lot of arguments like this.
Gary Marcus (21:49.460)
I've never found them that convincing.
Lex Fridman (21:50.860)
I think that there's a lot of making lemonade out of lemons.
Lex Fridman (21:55.120)
So we, for example, do a lot of free association
Lex Fridman (21:58.260)
where one idea just leads to the next
Lex Fridman (22:00.780)
and they're not really that well connected.
Lex Fridman (22:02.540)
And we enjoy that and we make poetry out of it
Lex Fridman (22:04.500)
and we make kind of movies with free associations
Lex Fridman (22:07.100)
and it's fun and whatever.
Gary Marcus (22:08.140)
I don't think that's really a virtue of the system.
Lex Fridman (22:12.300)
I think that the limitations in human reasoning
Gary Marcus (22:15.340)
actually get us in a lot of trouble.
Lex Fridman (22:16.580)
Like, for example, politically we can't see eye to eye
Gary Marcus (22:19.300)
because we have the motivational reasoning I was talking
Lex Fridman (22:21.780)
about and something related called confirmation bias.
Lex Fridman (22:25.080)
So we have all of these problems that actually make
Lex Fridman (22:27.460)
for a rougher society because we can't get along
Gary Marcus (22:29.920)
because we can't interpret the data in shared ways.
Lex Fridman (22:34.320)
And then we do some nice stuff with that.
Lex Fridman (22:36.460)
So my free associations are different from yours
Lex Fridman (22:38.900)
and you're kind of amused by them and that's great.
Lex Fridman (22:41.600)
And hence poetry.
Lex Fridman (22:42.620)
So there are lots of ways in which we take
Gary Marcus (22:45.060)
a lousy situation and make it good.
Lex Fridman (22:47.540)
Another example would be our memories are terrible.
Lex Fridman (22:50.580)
So we play games like Concentration where you flip over
Lex Fridman (22:53.300)
two cards, try to find a pair.
Lex Fridman (22:54.980)
Can you imagine a computer playing that?
Lex Fridman (22:56.480)
Computer's like, this is the dullest game in the world.
Gary Marcus (22:58.300)
I know where all the cards are, I see it once,
Lex Fridman (22:59.940)
I know where it is, what are you even talking about?
Lex Fridman (23:02.580)
So we make a fun game out of having this terrible memory.
Lex Fridman (23:07.040)
So we are imperfect in discovering and optimizing
Gary Marcus (23:12.220)
some kind of utility function.
Lex Fridman (23:13.540)
But you think in general, there is a utility function.
Gary Marcus (23:16.300)
There's an objective function that's better than others.
Lex Fridman (23:18.860)
I didn't say that.
Lex Fridman (23:20.340)
But see, the presumption, when you say...
Lex Fridman (23:24.420)
I think you could design a better memory system.
Gary Marcus (23:27.220)
You could argue about utility functions
Lex Fridman (23:29.900)
and how you wanna think about that.
Lex Fridman (23:32.100)
But objectively, it would be really nice
Lex Fridman (23:34.180)
to do some of the following things.
Gary Marcus (23:36.500)
To get rid of memories that are no longer useful.
Lex Fridman (23:41.140)
Objectively, that would just be good.
Lex Fridman (23:42.700)
And we're not that good at it.
Lex Fridman (23:43.580)
So when you park in the same lot every day,
Gary Marcus (23:46.540)
you confuse where you parked today
Lex Fridman (23:47.900)
with where you parked yesterday
Gary Marcus (23:48.860)
with where you parked the day before and so forth.
Lex Fridman (23:50.700)
So you blur together a series of memories.
Gary Marcus (23:52.620)
There's just no way that that's optimal.
Lex Fridman (23:55.380)
I mean, I've heard all kinds of wacky arguments
Gary Marcus (23:56.940)
of people trying to defend that.
Lex Fridman (23:58.140)
But in the end of the day,
Gary Marcus (23:58.980)
I don't think any of them hold water.
Lex Fridman (24:00.420)
It's just above.
Gary Marcus (24:01.260)
Or memories of traumatic events would be possibly
Lex Fridman (24:04.420)
a very nice feature to have to get rid of those.
Gary Marcus (24:06.780)
It'd be great if you could just be like,
Lex Fridman (24:08.300)
I'm gonna wipe this sector.
Gary Marcus (24:10.580)
I'm done with that.
Lex Fridman (24:12.020)
I didn't have fun last night.
Gary Marcus (24:13.260)
I don't wanna think about it anymore.
Lex Fridman (24:14.780)
Whoop, bye bye.
Gary Marcus (24:15.820)
I'm gone.
Lex Fridman (24:16.660)
But we can't.
Lex Fridman (24:17.740)
Do you think it's possible to build a system...
Lex Fridman (24:20.380)
So you said human level intelligence is a weird concept, but...
Gary Marcus (24:23.780)
Well, I'm saying I prefer general intelligence.
Lex Fridman (24:25.420)
General intelligence.
Gary Marcus (24:26.260)
I mean, human level intelligence is a real thing.
Lex Fridman (24:28.140)
And you could try to make a machine
Gary Marcus (24:29.820)
that matches people or something like that.
Lex Fridman (24:31.940)
I'm saying that per se shouldn't be the objective,
Lex Fridman (24:34.220)
but rather that we should learn from humans
Lex Fridman (24:37.220)
the things they do well and incorporate that into our AI,
Gary Marcus (24:39.660)
just as we incorporate the things that machines do well
Lex Fridman (24:42.100)
that people do terribly.
Gary Marcus (24:43.260)
So, I mean, it's great that AI systems
Lex Fridman (24:45.780)
can do all this brute force computation that people can't.
Lex Fridman (24:48.340)
And one of the reasons I work on this stuff
Lex Fridman (24:50.820)
is because I would like to see machines solve problems
Gary Marcus (24:53.300)
that people can't, that combine the strength,
Lex Fridman (24:56.020)
or that in order to be solved would combine
Gary Marcus (24:59.460)
the strengths of machines to do all this computation
Lex Fridman (25:02.220)
with the ability, let's say, of people to read.
Lex Fridman (25:04.220)
So I'd like machines that can read
Lex Fridman (25:06.180)
the entire medical literature in a day.
Gary Marcus (25:08.660)
7,000 new papers or whatever the numbers,
Lex Fridman (25:10.780)
comes out every day.
Gary Marcus (25:11.740)
There's no way for any doctor or whatever to read them all.
Lex Fridman (25:15.740)
A machine that could read would be a brilliant thing.
Lex Fridman (25:17.980)
And that would be strengths of brute force computation
Lex Fridman (25:21.060)
combined with kind of subtlety and understanding medicine
Gary Marcus (25:24.300)
that a good doctor or scientist has.
Lex Fridman (25:26.900)
So if we can linger a little bit
Gary Marcus (25:28.020)
on the idea of general intelligence.
Lex Fridman (25:29.660)
So Yann LeCun believes that human intelligence
Gary Marcus (25:32.860)
isn't general at all, it's very narrow.
Lex Fridman (25:35.580)
How do you think?
Gary Marcus (25:36.700)
I don't think that makes sense.
Lex Fridman (25:38.140)
We have lots of narrow intelligences for specific problems.
Lex Fridman (25:42.140)
But the fact is, like, anybody can walk into,
Lex Fridman (25:45.940)
let's say, a Hollywood movie,
Lex Fridman (25:47.620)
and reason about the content
Lex Fridman (25:49.140)
of almost anything that goes on there.
Lex Fridman (25:51.700)
So you can reason about what happens in a bank robbery,
Lex Fridman (25:55.180)
or what happens when someone is infertile
Lex Fridman (25:58.620)
and wants to go to IVF to try to have a child,
Lex Fridman (26:02.780)
or you can, the list is essentially endless.
Lex Fridman (26:05.940)
And not everybody understands every scene in the movie,
Lex Fridman (26:09.580)
but there's a huge range of things
Gary Marcus (26:11.740)
that pretty much any ordinary adult can understand.
Lex Fridman (26:15.060)
His argument is, is that actually,
Gary Marcus (26:18.220)
the set of things seems large for us humans
Lex Fridman (26:20.700)
because we're very limited in considering
Gary Marcus (26:24.380)
the kind of possibilities of experiences that are possible.
Lex Fridman (26:27.340)
But in fact, the amount of experience that are possible
Gary Marcus (26:30.180)
is infinitely larger.
Lex Fridman (26:32.500)
Well, I mean, if you wanna make an argument
Gary Marcus (26:35.140)
that humans are constrained in what they can understand,
Lex Fridman (26:38.780)
I have no issue with that.
Gary Marcus (26:40.940)
I think that's right.
Lex Fridman (26:41.780)
But it's still not the same thing at all
Gary Marcus (26:44.460)
as saying, here's a system that can play Go.
Lex Fridman (26:47.460)
It's been trained on five million games.
Lex Fridman (26:49.700)
And then I say, can it play on a rectangular board
Lex Fridman (26:52.580)
rather than a square board?
Lex Fridman (26:53.700)
And you say, well, if I retrain it from scratch
Lex Fridman (26:56.580)
on another five million games, it can.
Gary Marcus (26:58.340)
That's really, really narrow, and that's where we are.
Lex Fridman (27:01.140)
We don't have even a system that could play Go
Lex Fridman (27:05.140)
and then without further retraining,
Lex Fridman (27:07.100)
play on a rectangular board,
Gary Marcus (27:08.700)
which any human could do with very little problem.
Lex Fridman (27:12.600)
So that's what I mean by narrow.
Lex Fridman (27:14.860)
And so it's just wordplay to say.
Lex Fridman (27:16.900)
That is semantics, yeah.
Gary Marcus (27:18.060)
Then it's just words.
Lex Fridman (27:19.300)
Then yeah, you mean general in a sense
Gary Marcus (27:21.180)
that you can do all kinds of Go board shapes flexibly.
Lex Fridman (27:25.780)
Well, that would be like a first step
Gary Marcus (27:28.100)
in the right direction,
Lex Fridman (27:29.020)
but obviously that's not what it really meaning.
Gary Marcus (27:30.540)
You're kidding.
Lex Fridman (27:32.380)
What I mean by general is that you could transfer
Gary Marcus (27:36.140)
the knowledge you learn in one domain to another.
Lex Fridman (27:38.940)
So if you learn about bank robberies in movies
Lex Fridman (27:43.320)
and there's chase scenes,
Lex Fridman (27:44.780)
then you can understand that amazing scene in Breaking Bad
Gary Marcus (27:47.740)
when Walter White has a car chase scene
Lex Fridman (27:50.580)
with only one person.
Gary Marcus (27:51.500)
He's the only one in it.
Lex Fridman (27:52.620)
And you can reflect on how that car chase scene
Gary Marcus (27:55.540)
is like all the other car chase scenes you've ever seen
Lex Fridman (27:58.060)
and totally different and why that's cool.
Lex Fridman (28:01.140)
And the fact that the number of domains
Lex Fridman (28:03.100)
you can do that with is finite
Gary Marcus (28:04.540)
doesn't make it less general.
Lex Fridman (28:05.760)
So the idea of general is you could just do it
Gary Marcus (28:07.340)
on a lot of, don't transfer it across a lot of domains.
Lex Fridman (28:09.380)
Yeah, I mean, I'm not saying humans are infinitely general
Gary Marcus (28:11.740)
or that humans are perfect.
Lex Fridman (28:12.960)
I just said a minute ago, it's a low bar,
Lex Fridman (28:15.340)
but it's just, it's a low bar.
Lex Fridman (28:17.420)
But right now, like the bar is here and we're there
Lex Fridman (28:20.460)
and eventually we'll get way past it.
Lex Fridman (28:22.660)
So speaking of low bars,
Gary Marcus (28:25.600)
you've highlighted in your new book as well,
Lex Fridman (28:27.420)
but a couple of years ago wrote a paper
Gary Marcus (28:29.340)
titled Deep Learning, A Critical Appraisal
Lex Fridman (28:31.300)
that lists 10 challenges faced
Gary Marcus (28:33.340)
by current deep learning systems.
Lex Fridman (28:36.020)
So let me summarize them as data efficiency,
Gary Marcus (28:40.140)
transfer learning, hierarchical knowledge,
Lex Fridman (28:42.900)
open ended inference, explainability,
Gary Marcus (28:46.300)
integrating prior knowledge, cause of reasoning,
Lex Fridman (28:49.660)
modeling on a stable world, robustness, adversarial examples
Lex Fridman (28:53.220)
and so on.
Lex Fridman (28:54.140)
And then my favorite probably is reliability
Gary Marcus (28:56.900)
in the engineering of real world systems.
Lex Fridman (28:59.140)
So whatever people can read the paper,
Gary Marcus (29:01.600)
they should definitely read the paper,
Lex Fridman (29:02.940)
should definitely read your book.
Lex Fridman (29:04.320)
But which of these challenges is solved in your view
Lex Fridman (29:08.140)
has the biggest impact on the AI community?
Gary Marcus (29:11.060)
It's a very good question.
Lex Fridman (29:13.940)
And I'm gonna be evasive because I think that
Gary Marcus (29:16.300)
they go together a lot.
Lex Fridman (29:17.980)
So some of them might be solved independently of others,
Lex Fridman (29:21.420)
but I think a good solution to AI
Lex Fridman (29:23.700)
starts by having real,
Lex Fridman (29:25.460)
what I would call cognitive models of what's going on.
Lex Fridman (29:28.420)
So right now we have a approach that's dominant
Gary Marcus (29:31.340)
where you take statistical approximations of things,
Lex Fridman (29:33.920)
but you don't really understand them.
Lex Fridman (29:35.740)
So you know that bottles are correlated in your data
Lex Fridman (29:39.100)
with bottle caps,
Lex Fridman (29:40.300)
but you don't understand that there's a thread
Lex Fridman (29:42.220)
on the bottle cap that fits with the thread on the bottle
Lex Fridman (29:45.300)
and then that's what tightens it.
Lex Fridman (29:46.620)
If I tighten enough that there's a seal
Lex Fridman (29:48.540)
and the water won't come out.
Lex Fridman (29:49.660)
Like there's no machine that understands that.
Lex Fridman (29:51.980)
And having a good cognitive model
Lex Fridman (29:53.820)
of that kind of everyday phenomena
Gary Marcus (29:55.480)
is what we call common sense.
Lex Fridman (29:56.620)
And if you had that,
Gary Marcus (29:57.820)
then a lot of these other things start to fall
Lex Fridman (30:00.700)
into at least a little bit better place.
Gary Marcus (30:02.860)
Right now you're like learning correlations between pixels
Lex Fridman (30:05.640)
when you play a video game or something like that.
Lex Fridman (30:07.660)
And it doesn't work very well.
Lex Fridman (30:08.940)
It works when the video game is just the way
Gary Marcus (30:10.720)
that you studied it and then you alter the video game
Lex Fridman (30:12.940)
in small ways,
Gary Marcus (30:13.760)
like you move the paddle and break out a few pixels
Lex Fridman (30:15.780)
and the system falls apart.
Gary Marcus (30:17.460)
Because it doesn't understand,
Lex Fridman (30:19.020)
it doesn't have a representation of a paddle,
Gary Marcus (30:20.900)
a ball, a wall, a set of bricks and so forth.
Lex Fridman (30:23.340)
And so it's reasoning at the wrong level.
Lex Fridman (30:26.440)
So the idea of common sense,
Lex Fridman (30:29.220)
it's full of mystery,
Gary Marcus (30:30.220)
you've worked on it,
Lex Fridman (30:31.060)
but it's nevertheless full of mystery,
Gary Marcus (30:33.560)
full of promise.
Lex Fridman (30:34.720)
What does common sense mean?
Lex Fridman (30:36.540)
What does knowledge mean?
Lex Fridman (30:38.020)
So the way you've been discussing it now
Gary Marcus (30:40.020)
is very intuitive.
Lex Fridman (30:40.940)
It makes a lot of sense that that is something
Gary Marcus (30:42.580)
we should have and that's something
Lex Fridman (30:43.700)
deep learning systems don't have.
Lex Fridman (30:45.600)
But the argument could be that we're oversimplifying it
Lex Fridman (30:49.740)
because we're oversimplifying the notion of common sense
Gary Marcus (30:53.180)
because that's how it feels like we as humans
Lex Fridman (30:57.140)
at the cognitive level approach problems.
Lex Fridman (30:59.320)
So maybe.
Lex Fridman (31:00.160)
A lot of people aren't actually gonna read my book.
Lex Fridman (31:03.320)
But if they did read the book,
Lex Fridman (31:05.220)
one of the things that might come as a surprise to them
Gary Marcus (31:07.140)
is that we actually say common sense is really hard
Lex Fridman (31:10.660)
and really complicated.
Lex Fridman (31:11.640)
So they would probably,
Lex Fridman (31:13.020)
my critics know that I like common sense,
Lex Fridman (31:15.140)
but that chapter actually starts by us beating up
Lex Fridman (31:18.600)
not on deep learning,
Lex Fridman (31:19.900)
but kind of on our own home team as it will.
Lex Fridman (31:21.960)
So Ernie and I are first and foremost
Gary Marcus (31:25.180)
people that believe in at least some
Lex Fridman (31:26.780)
of what good old fashioned AI tried to do.
Lex Fridman (31:28.700)
So we believe in symbols and logic and programming.
Lex Fridman (31:32.500)
Things like that are important.
Lex Fridman (31:33.740)
And we go through why even those tools
Lex Fridman (31:37.020)
that we hold fairly dear aren't really enough.
Lex Fridman (31:39.560)
So we talk about why common sense is actually many things.
Lex Fridman (31:42.660)
And some of them fit really well with those
Gary Marcus (31:45.300)
classical sets of tools.
Lex Fridman (31:46.540)
So things like taxonomy.
Lex Fridman (31:48.240)
So I know that a bottle is an object
Lex Fridman (31:51.460)
or it's a vessel, let's say.
Lex Fridman (31:52.860)
And I know a vessel is an object
Lex Fridman (31:54.480)
and objects are material things in the physical world.
Lex Fridman (31:57.580)
So I can make some inferences.
Lex Fridman (32:00.500)
If I know that vessels need to not have holes in them,
Gary Marcus (32:07.020)
then I can infer that in order to carry their contents,
Lex Fridman (32:09.540)
then I can infer that a bottle
Gary Marcus (32:10.920)
shouldn't have a hole in it in order to carry its contents.
Lex Fridman (32:12.860)
So you can do hierarchical inference and so forth.
Lex Fridman (32:15.620)
And we say that's great,
Lex Fridman (32:17.260)
but it's only a tiny piece of what you need for common sense.
Gary Marcus (32:21.100)
We give lots of examples that don't fit into that.
Lex Fridman (32:23.460)
So another one that we talk about is a cheese grater.
Gary Marcus (32:26.500)
You've got holes in a cheese grater.
Lex Fridman (32:28.040)
You've got a handle on top.
Gary Marcus (32:29.500)
You can build a model in the game engine sense of a model
Lex Fridman (32:33.380)
so that you could have a little cartoon character
Gary Marcus (32:35.820)
flying around through the holes of the grater.
Lex Fridman (32:37.980)
But we don't have a system yet.
Gary Marcus (32:39.980)
Taxonomy doesn't help us that much
Lex Fridman (32:41.620)
that really understands why the handle is on top
Lex Fridman (32:43.780)
and what you do with the handle,
Lex Fridman (32:45.240)
or why all of those circles are sharp,
Gary Marcus (32:47.620)
or how you'd hold the cheese with respect to the grater
Lex Fridman (32:50.500)
in order to make it actually work.
Lex Fridman (32:52.120)
Do you think these ideas are just abstractions
Lex Fridman (32:55.020)
that could emerge on a system
Lex Fridman (32:57.140)
like a very large deep neural network?
Lex Fridman (32:59.920)
I'm a skeptic that that kind of emergence per se can work.
Lex Fridman (33:03.140)
So I think that deep learning might play a role
Lex Fridman (33:05.840)
in the systems that do what I want systems to do,
Lex Fridman (33:08.760)
but it won't do it by itself.
Lex Fridman (33:09.900)
I've never seen a deep learning system
Gary Marcus (33:13.140)
really extract an abstract concept.
Lex Fridman (33:15.900)
What they do, principled reasons for that
Gary Marcus (33:18.820)
stemming from how back propagation works,
Lex Fridman (33:20.540)
how the architectures are set up.
Gary Marcus (33:22.920)
One example is deep learning people
Lex Fridman (33:25.120)
actually all build in something called convolution,
Gary Marcus (33:29.620)
which Jan Lacune is famous for, which is an abstraction.
Lex Fridman (33:33.180)
They don't have their systems learn this.
Lex Fridman (33:34.960)
So the abstraction is an object that looks the same
Lex Fridman (33:37.740)
if it appears in different places.
Lex Fridman (33:39.220)
And what Lacune figured out and why,
Lex Fridman (33:41.940)
essentially why he was a co winner of the Turing Award
Gary Marcus (33:44.300)
was that if you programmed this in innately,
Lex Fridman (33:47.620)
then your system would be a whole lot more efficient.
Gary Marcus (33:50.680)
In principle, this should be learnable,
Lex Fridman (33:53.220)
but people don't have systems that kind of reify things
Lex Fridman (33:56.220)
and make them more abstract.
Lex Fridman (33:58.000)
And so what you'd really wind up with
Gary Marcus (34:00.420)
if you don't program that in advance is a system
Lex Fridman (34:02.700)
that kind of realizes that this is the same thing as this,
Lex Fridman (34:05.460)
but then I take your little clock there
Lex Fridman (34:06.980)
and I move it over and it doesn't realize
Gary Marcus (34:08.380)
that the same thing applies to the clock.
Lex Fridman (34:10.460)
So the really nice thing, you're right,
Gary Marcus (34:12.680)
that convolution is just one of the things
Lex Fridman (34:14.760)
that's like, it's an innate feature
Gary Marcus (34:17.160)
that's programmed by the human expert.
Lex Fridman (34:19.460)
We need more of those, not less.
Gary Marcus (34:21.260)
Yes, but the nice feature is it feels like
Lex Fridman (34:24.420)
that requires coming up with that brilliant idea,
Gary Marcus (34:28.200)
can get you a Turing Award,
Lex Fridman (34:29.780)
but it requires less effort than encoding
Lex Fridman (34:34.780)
and something we'll talk about, the expert system.
Lex Fridman (34:36.620)
So encoding a lot of knowledge by hand.
Lex Fridman (34:40.020)
So it feels like there's a huge amount of limitations
Lex Fridman (34:43.500)
which you clearly outline with deep learning,
Lex Fridman (34:46.500)
but the nice feature of deep learning,
Lex Fridman (34:47.820)
whatever it is able to accomplish,
Gary Marcus (34:49.600)
it does a lot of stuff automatically
Lex Fridman (34:53.500)
without human intervention.
Lex Fridman (34:54.900)
Well, and that's part of why people love it, right?
Lex Fridman (34:57.100)
But I always think of this quote from Bertrand Russell,
Gary Marcus (34:59.820)
which is it has all the advantages
Lex Fridman (35:02.740)
of theft over honest toil.
Gary Marcus (35:04.420)
It's really hard to program into a machine
Lex Fridman (35:08.140)
a notion of causality or even how a bottle works
Gary Marcus (35:11.300)
or what containers are.
Lex Fridman (35:12.640)
Ernie Davis and I wrote a, I don't know,
Gary Marcus (35:14.260)
45 page academic paper trying just to understand
Lex Fridman (35:17.980)
what a container is,
Gary Marcus (35:18.980)
which I don't think anybody ever read the paper,
Lex Fridman (35:21.100)
but it's a very detailed analysis of all the things,
Gary Marcus (35:25.260)
well, not even all of it,
Lex Fridman (35:26.100)
some of the things you need to do
Gary Marcus (35:27.140)
in order to understand a container.
Lex Fridman (35:28.580)
It would be a whole lot nice,
Lex Fridman (35:30.060)
and I'm a coauthor on the paper,
Lex Fridman (35:32.200)
I made it a little bit better,
Lex Fridman (35:33.180)
but Ernie did the hard work for that particular paper.
Lex Fridman (35:36.620)
And it took him like three months
Gary Marcus (35:38.060)
to get the logical statements correct.
Lex Fridman (35:40.660)
And maybe that's not the right way to do it,
Gary Marcus (35:42.860)
it's a way to do it.
Lex Fridman (35:44.100)
But on that way of doing it,
Gary Marcus (35:46.140)
it's really hard work to do something
Lex Fridman (35:48.440)
as simple as understanding containers.
Lex Fridman (35:50.220)
And nobody wants to do that hard work,
Lex Fridman (35:52.820)
even Ernie didn't want to do that hard work.
Gary Marcus (35:55.600)
Everybody would rather just like feed their system in
Lex Fridman (35:58.380)
with a bunch of videos with a bunch of containers
Lex Fridman (36:00.340)
and have the systems infer how containers work.
Lex Fridman (36:03.820)
It would be like so much less effort,
Gary Marcus (36:05.420)
let the machine do the work.
Lex Fridman (36:06.820)
And so I understand the impulse,
Gary Marcus (36:08.220)
I understand why people want to do that.
Lex Fridman (36:10.220)
I just don't think that it works.
Gary Marcus (36:11.860)
I've never seen anybody build a system
Lex Fridman (36:14.580)
that in a robust way can actually watch videos
Lex Fridman (36:18.700)
and predict exactly which containers would leak
Lex Fridman (36:21.300)
and which ones wouldn't or something like,
Lex Fridman (36:23.540)
and I know someone's gonna go out and do that
Lex Fridman (36:25.060)
since I said it, and I look forward to seeing it.
Lex Fridman (36:28.100)
But getting these things to work robustly
Lex Fridman (36:30.540)
is really, really hard.
Lex Fridman (36:32.900)
So Yann LeCun, who was my colleague at NYU for many years,
Lex Fridman (36:37.740)
thinks that the hard work should go into defining
Gary Marcus (36:40.760)
an unsupervised learning algorithm
Lex Fridman (36:43.180)
that will watch videos, use the next frame basically
Gary Marcus (36:46.680)
in order to tell it what's going on.
Lex Fridman (36:48.540)
And he thinks that's the Royal road
Lex Fridman (36:49.940)
and he's willing to put in the work
Lex Fridman (36:51.260)
in devising that algorithm.
Gary Marcus (36:53.300)
Then he wants the machine to do the rest.
Lex Fridman (36:55.580)
And again, I understand the impulse.
Gary Marcus (36:57.820)
My intuition, based on years of watching this stuff
Lex Fridman (37:01.700)
and making predictions 20 years ago that still hold
Gary Marcus (37:03.940)
even though there's a lot more computation and so forth,
Lex Fridman (37:06.500)
is that we actually have to do
Gary Marcus (37:07.460)
a different kind of hard work,
Lex Fridman (37:08.520)
which is more like building a design specification
Gary Marcus (37:11.320)
for what we want the system to do,
Lex Fridman (37:13.100)
doing hard engineering work to figure out
Lex Fridman (37:15.060)
how we do things like what Yann did for convolution
Lex Fridman (37:18.420)
in order to figure out how to encode complex knowledge
Gary Marcus (37:21.660)
into the systems.
Lex Fridman (37:22.620)
The current systems don't have that much knowledge
Gary Marcus (37:25.340)
other than convolution, which is again,
Lex Fridman (37:27.580)
this objects being in different places
Lex Fridman (37:30.500)
and having the same perception, I guess I'll say.
Lex Fridman (37:34.460)
Same appearance.
Gary Marcus (37:36.740)
People don't want to do that work.
Lex Fridman (37:38.260)
They don't see how to naturally fit one with the other.
Gary Marcus (37:41.580)
I think that's, yes, absolutely.
Lex Fridman (37:43.300)
But also on the expert system side,
Gary Marcus (37:45.540)
there's a temptation to go too far the other way.
Lex Fridman (37:47.620)
So we're just having an expert sort of sit down
Lex Fridman (37:49.860)
and encode the description,
Lex Fridman (37:51.940)
the framework for what a container is,
Lex Fridman (37:54.060)
and then having the system reason the rest.
Lex Fridman (37:56.540)
From my view, one really exciting possibility
Gary Marcus (37:59.260)
is of active learning where it's continuous interaction
Lex Fridman (38:02.180)
between a human and machine.
Gary Marcus (38:04.080)
As the machine, there's kind of deep learning type
Lex Fridman (38:07.060)
extraction of information from data patterns and so on,
Lex Fridman (38:10.120)
but humans also guiding the learning procedures,
Lex Fridman (38:14.660)
guiding both the process and the framework
Gary Marcus (38:19.940)
of how the machine learns, whatever the task is.
Lex Fridman (38:22.100)
I was with you with almost everything you said
Gary Marcus (38:24.100)
except the phrase deep learning.
Lex Fridman (38:26.460)
What I think you really want there
Gary Marcus (38:28.180)
is a new form of machine learning.
Lex Fridman (38:30.500)
So let's remember, deep learning is a particular way
Gary Marcus (38:32.980)
of doing machine learning.
Lex Fridman (38:33.980)
Most often it's done with supervised data
Gary Marcus (38:36.980)
for perceptual categories.
Lex Fridman (38:38.820)
There are other things you can do with deep learning,
Gary Marcus (38:41.780)
some of them quite technical,
Lex Fridman (38:42.740)
but the standard use of deep learning
Gary Marcus (38:44.600)
is I have a lot of examples and I have labels for them.
Lex Fridman (38:47.600)
So here are pictures.
Gary Marcus (38:48.820)
This one's the Eiffel Tower.
Lex Fridman (38:50.380)
This one's the Sears Tower.
Gary Marcus (38:51.660)
This one's the Empire State Building.
Lex Fridman (38:53.320)
This one's a cat.
Gary Marcus (38:54.160)
This one's a pig and so forth.
Lex Fridman (38:55.180)
You just get millions of examples, millions of labels,
Lex Fridman (38:58.900)
and deep learning is extremely good at that.
Lex Fridman (39:01.220)
It's better than any other solution that anybody has devised,
Lex Fridman (39:04.460)
but it is not good at representing abstract knowledge.
Lex Fridman (39:07.380)
It's not good at representing things
Gary Marcus (39:09.380)
like bottles contain liquid and have tops to them
Lex Fridman (39:13.980)
and so forth.
Gary Marcus (39:14.820)
It's not very good at learning
Lex Fridman (39:15.860)
or representing that kind of knowledge.
Gary Marcus (39:17.860)
It is an example of having a machine learn something,
Lex Fridman (39:21.300)
but it's a machine that learns a particular kind of thing,
Gary Marcus (39:23.900)
which is object classification.
Lex Fridman (39:25.540)
It's not a particularly good algorithm for learning
Gary Marcus (39:28.580)
about the abstractions that govern our world.
Lex Fridman (39:30.780)
There may be such a thing.
Gary Marcus (39:33.080)
Part of what we counsel in the book
Lex Fridman (39:34.300)
is maybe people should be working on devising such things.
Lex Fridman (39:36.980)
So one possibility, just I wonder what you think about it,
Lex Fridman (39:40.580)
is that deep neural networks do form abstractions,
Lex Fridman (39:45.180)
but they're not accessible to us humans
Lex Fridman (39:48.500)
in terms of we can't.
Gary Marcus (39:49.340)
There's some truth in that.
Lex Fridman (39:50.780)
So is it possible that either current or future
Gary Marcus (39:54.180)
neural networks form very high level abstractions,
Lex Fridman (39:56.520)
which are as powerful as our human abstractions
Gary Marcus (40:01.820)
of common sense.
Lex Fridman (40:02.660)
We just can't get a hold of them.
Lex Fridman (40:04.900)
And so the problem is essentially
Lex Fridman (40:06.620)
we need to make them explainable.
Gary Marcus (40:09.220)
This is an astute question,
Lex Fridman (40:10.640)
but I think the answer is at least partly no.
Gary Marcus (40:13.080)
One of the kinds of classical neural network architecture
Lex Fridman (40:16.060)
is what we call an auto associator.
Gary Marcus (40:17.620)
It just tries to take an input,
Lex Fridman (40:20.140)
goes through a set of hidden layers,
Lex Fridman (40:21.500)
and comes out with an output.
Lex Fridman (40:23.040)
And it's supposed to learn essentially
Gary Marcus (40:24.420)
the identity function,
Lex Fridman (40:25.460)
that your input is the same as your output.
Lex Fridman (40:27.260)
So you think of it as binary numbers.
Lex Fridman (40:28.460)
You've got the one, the two, the four, the eight,
Gary Marcus (40:30.660)
the 16, and so forth.
Lex Fridman (40:32.180)
And so if you want to input 24,
Gary Marcus (40:33.940)
you turn on the 16, you turn on the eight.
Lex Fridman (40:35.860)
It's like binary one, one, and a bunch of zeros.
Lex Fridman (40:38.940)
So I did some experiments in 1998
Lex Fridman (40:41.620)
with the precursors of contemporary deep learning.
Lex Fridman (40:46.620)
And what I showed was you could train these networks
Lex Fridman (40:50.460)
on all the even numbers,
Lex Fridman (40:52.060)
and they would never generalize to the odd number.
Lex Fridman (40:54.620)
A lot of people thought that I was, I don't know,
Gary Marcus (40:56.700)
an idiot or faking the experiment,
Lex Fridman (40:58.460)
or it wasn't true or whatever.
Lex Fridman (41:00.100)
But it is true that with this class of networks
Lex Fridman (41:03.260)
that we had in that day,
Gary Marcus (41:04.860)
that they would never ever make this generalization.
Lex Fridman (41:07.140)
And it's not that the networks were stupid,
Gary Marcus (41:09.660)
it's that they see the world in a different way than we do.
Lex Fridman (41:13.380)
They were basically concerned,
Lex Fridman (41:14.720)
what is the probability that the rightmost output node
Lex Fridman (41:18.580)
is going to be one?
Lex Fridman (41:19.980)
And as far as they were concerned,
Lex Fridman (41:21.220)
in everything they'd ever been trained on, it was a zero.
Gary Marcus (41:24.420)
That node had never been turned on,
Lex Fridman (41:27.020)
and so they figured, why turn it on now?
Gary Marcus (41:28.960)
Whereas a person would look at the same problem and say,
Lex Fridman (41:30.940)
well, it's obvious,
Gary Marcus (41:31.780)
we're just doing the thing that corresponds.
Lex Fridman (41:33.780)
The Latin for it is mutatis mutandis,
Gary Marcus (41:35.500)
we'll change what needs to be changed.
Lex Fridman (41:38.220)
And we do this, this is what algebra is.
Lex Fridman (41:40.500)
So I can do f of x equals y plus two,
Lex Fridman (41:43.840)
and I can do it for a couple of values,
Gary Marcus (41:45.380)
I can tell you if y is three,
Lex Fridman (41:46.500)
then x is five, and if y is four, x is six.
Lex Fridman (41:49.140)
And now I can do it with some totally different number,
Lex Fridman (41:50.980)
like a million, then you can say,
Gary Marcus (41:51.980)
well, obviously it's a million and two,
Lex Fridman (41:53.140)
because you have an algebraic operation
Gary Marcus (41:55.620)
that you're applying to a variable.
Lex Fridman (41:57.460)
And deep learning systems kind of emulate that,
Lex Fridman (42:00.620)
but they don't actually do it.
Lex Fridman (42:02.500)
The particular example,
Gary Marcus (42:04.140)
you could fudge a solution to that particular problem.
Lex Fridman (42:08.140)
The general form of that problem remains,
Gary Marcus (42:10.500)
that what they learn is really correlations
Lex Fridman (42:12.400)
between different input and output nodes.
Lex Fridman (42:14.180)
And they're complex correlations
Lex Fridman (42:16.140)
with multiple nodes involved and so forth.
Gary Marcus (42:18.780)
Ultimately, they're correlative,
Lex Fridman (42:20.260)
they're not structured over these operations over variables.
Gary Marcus (42:23.060)
Now, someday, people may do a new form of deep learning
Lex Fridman (42:25.960)
that incorporates that stuff,
Lex Fridman (42:27.300)
and I think it will help a lot.
Lex Fridman (42:28.460)
And there's some tentative work on things
Gary Marcus (42:30.260)
like differentiable programming right now
Lex Fridman (42:32.180)
that fall into that category.
Lex Fridman (42:34.240)
But the sort of classic stuff
Lex Fridman (42:35.500)
like people use for ImageNet doesn't have it.
Lex Fridman (42:38.780)
And you have people like Hinton going around saying,
Lex Fridman (42:41.060)
symbol manipulation, like what Marcus,
Lex Fridman (42:42.860)
what I advocate is like the gasoline engine.
Lex Fridman (42:45.680)
It's obsolete.
Gary Marcus (42:46.520)
We should just use this cool electric power
Lex Fridman (42:48.820)
that we've got with the deep learning.
Lex Fridman (42:50.320)
And that's really destructive,
Lex Fridman (42:51.980)
because we really do need to have the gasoline engine stuff
Gary Marcus (42:55.900)
that represents, I mean, I don't think it's a good analogy,
Lex Fridman (42:59.580)
but we really do need to have the stuff
Gary Marcus (43:02.180)
that represents symbols.
Lex Fridman (43:03.660)
Yeah, and Hinton as well would say
Gary Marcus (43:06.200)
that we do need to throw out everything and start over.
Lex Fridman (43:08.960)
Hinton said that to Axios,
Lex Fridman (43:12.820)
and I had a friend who interviewed him
Lex Fridman (43:15.540)
and tried to pin him down
Gary Marcus (43:16.460)
on what exactly we need to throw out,
Lex Fridman (43:17.820)
and he was very evasive.
Gary Marcus (43:19.900)
Well, of course, because we can't, if he knew.
Lex Fridman (43:22.700)
Then he'd throw it out himself.
Lex Fridman (43:23.940)
But I mean, you can't have it both ways.
Lex Fridman (43:25.400)
You can't be like, I don't know what to throw out,
Lex Fridman (43:27.520)
but I am gonna throw out the symbols.
Lex Fridman (43:29.980)
I mean, and not just the symbols,
Lex Fridman (43:32.140)
but the variables and the operations over variables.
Lex Fridman (43:34.100)
Don't forget, the operations over variables,
Gary Marcus (43:36.140)
the stuff that I'm endorsing
Lex Fridman (43:37.740)
and which John McCarthy did when he founded AI,
Gary Marcus (43:41.500)
that stuff is the stuff
Lex Fridman (43:42.660)
that we build most computers out of.
Gary Marcus (43:44.180)
There are people now who say,
Lex Fridman (43:45.460)
we don't need computer programmers anymore.
Gary Marcus (43:48.780)
Not quite looking at the statistics
Lex Fridman (43:50.240)
of how much computer programmers
Gary Marcus (43:51.180)
actually get paid right now.
Lex Fridman (43:52.980)
We need lots of computer programmers,
Lex Fridman (43:54.380)
and most of them, they do a little bit of machine learning,
Lex Fridman (43:57.780)
but they still do a lot of code, right?
Gary Marcus (43:59.900)
Code where it's like, if the value of X
Lex Fridman (44:02.220)
is greater than the value of Y,
Gary Marcus (44:03.580)
then do this kind of thing,
Lex Fridman (44:04.500)
like conditionals and comparing operations over variables.
Gary Marcus (44:08.100)
Like, there's this fantasy you can machine learn anything.
Lex Fridman (44:10.220)
There's some things you would never wanna machine learn.
Gary Marcus (44:12.580)
I would not use a phone operating system
Lex Fridman (44:14.980)
that was machine learned.
Gary Marcus (44:16.100)
Like, you made a bunch of phone calls
Lex Fridman (44:17.820)
and you recorded which packets were transmitted
Lex Fridman (44:19.740)
and you just machine learned it, it'd be insane.
Lex Fridman (44:22.500)
Or to build a web browser by taking logs of keystrokes
Lex Fridman (44:27.500)
and images, screenshots,
Lex Fridman (44:29.420)
and then trying to learn the relation between them.
Gary Marcus (44:31.500)
Nobody would ever,
Lex Fridman (44:32.860)
no rational person would ever try to build a browser
Gary Marcus (44:35.100)
that made, they would use symbol manipulation,
Lex Fridman (44:37.460)
the stuff that I think AI needs to avail itself of
Gary Marcus (44:40.140)
in addition to deep learning.
Lex Fridman (44:42.140)
Can you describe your view of symbol manipulation
Lex Fridman (44:46.540)
in its early days?
Lex Fridman (44:47.920)
Can you describe expert systems
Lex Fridman (44:49.540)
and where do you think they hit a wall
Lex Fridman (44:52.540)
or a set of challenges?
Gary Marcus (44:53.940)
Sure, so I mean, first I just wanna clarify,
Lex Fridman (44:56.580)
I'm not endorsing expert systems per se.
Gary Marcus (44:58.940)
You've been kind of contrasting them.
Lex Fridman (45:00.760)
There is a contrast,
Lex Fridman (45:01.600)
but that's not the thing that I'm endorsing.
Lex Fridman (45:04.220)
So expert systems tried to capture things
Gary Marcus (45:06.500)
like medical knowledge with a large set of rules.
Lex Fridman (45:09.460)
So if the patient has this symptom and this other symptom,
Gary Marcus (45:12.860)
then it is likely that they have this disease.
Lex Fridman (45:15.700)
So there are logical rules
Lex Fridman (45:16.860)
and they were symbol manipulating rules
Lex Fridman (45:18.340)
of just the sort that I'm talking about.
Lex Fridman (45:20.980)
And the problem.
Lex Fridman (45:21.820)
They encode a set of knowledge that the experts then put in.
Lex Fridman (45:24.980)
And very explicitly so.
Lex Fridman (45:26.260)
So you'd have somebody interview an expert
Lex Fridman (45:28.780)
and then try to turn that stuff into rules.
Lex Fridman (45:31.940)
And at some level I'm arguing for rules.
Lex Fridman (45:33.980)
But the difference is those guys did in the 80s
Lex Fridman (45:37.700)
was almost entirely rules,
Gary Marcus (45:40.040)
almost entirely handwritten with no machine learning.
Lex Fridman (45:42.980)
What a lot of people are doing now
Gary Marcus (45:44.340)
is almost entirely one species of machine learning
Lex Fridman (45:47.340)
with no rules.
Lex Fridman (45:48.260)
And what I'm counseling is actually a hybrid.
Lex Fridman (45:50.380)
I'm saying that both of these things have their advantage.
Lex Fridman (45:52.900)
So if you're talking about perceptual classification,
Lex Fridman (45:55.300)
how do I recognize a bottle?
Gary Marcus (45:57.140)
Deep learning is the best tool we've got right now.
Lex Fridman (45:59.540)
If you're talking about making inferences
Gary Marcus (46:00.940)
about what a bottle does,
Lex Fridman (46:02.420)
something closer to the expert systems
Gary Marcus (46:04.140)
is probably still the best available alternative.
Lex Fridman (46:07.340)
And probably we want something that is better able
Gary Marcus (46:09.860)
to handle quantitative and statistical information
Lex Fridman (46:12.620)
than those classical systems typically were.
Lex Fridman (46:14.940)
So we need new technologies
Lex Fridman (46:16.980)
that are gonna draw some of the strengths
Gary Marcus (46:18.620)
of both the expert systems and the deep learning,
Lex Fridman (46:21.060)
but are gonna find new ways to synthesize them.
Lex Fridman (46:23.260)
How hard do you think it is to add knowledge at the low level?
Lex Fridman (46:27.740)
So mine human intellects to add extra information
Lex Fridman (46:32.140)
to symbol manipulating systems?
Lex Fridman (46:36.540)
In some domains it's not that hard,
Lex Fridman (46:37.840)
but it's often really hard.
Lex Fridman (46:40.100)
Partly because a lot of the things that are important,
Gary Marcus (46:44.120)
people wouldn't bother to tell you.
Lex Fridman (46:46.060)
So if you pay someone on Amazon Mechanical Turk
Gary Marcus (46:49.680)
to tell you stuff about bottles,
Lex Fridman (46:52.060)
they probably won't even bother to tell you
Gary Marcus (46:55.060)
some of the basic level stuff
Lex Fridman (46:57.020)
that's just so obvious to a human being
Lex Fridman (46:59.180)
and yet so hard to capture in machines.
Lex Fridman (47:04.580)
They're gonna tell you more exotic things,
Lex Fridman (47:06.540)
and they're all well and good,
Lex Fridman (47:08.940)
but they're not getting to the root of the problem.
Lex Fridman (47:12.460)
So untutored humans aren't very good at knowing,
Lex Fridman (47:16.540)
and why should they be,
Lex Fridman (47:18.340)
what kind of knowledge the computer system developers
Lex Fridman (47:22.260)
actually need?
Gary Marcus (47:23.460)
I don't think that that's an irremediable problem.
Lex Fridman (47:26.620)
I think it's historically been a problem.
Gary Marcus (47:28.620)
People have had crowdsourcing efforts,
Lex Fridman (47:31.080)
and they don't work that well.
Gary Marcus (47:32.060)
There's one at MIT, we're recording this at MIT,
Lex Fridman (47:35.300)
called Virtual Home, where,
Lex Fridman (47:37.500)
and we talk about this in the book,
Lex Fridman (47:39.540)
find the exact example there,
Lex Fridman (47:40.740)
but people were asked to do things
Lex Fridman (47:42.800)
like describe an exercise routine.
Lex Fridman (47:44.880)
And the things that the people describe
Lex Fridman (47:47.460)
are at a very low level
Lex Fridman (47:48.580)
and don't really capture what's going on.
Lex Fridman (47:50.100)
So they're like, go to the room
Gary Marcus (47:52.340)
with the television and the weights,
Lex Fridman (47:54.700)
turn on the television,
Gary Marcus (47:56.100)
press the remote to turn on the television,
Lex Fridman (47:59.020)
lift weight, put weight down, whatever.
Gary Marcus (48:01.440)
It's like very micro level,
Lex Fridman (48:03.620)
and it's not telling you
Lex Fridman (48:04.900)
what an exercise routine is really about,
Lex Fridman (48:06.860)
which is like, I wanna fit a certain number of exercises
Gary Marcus (48:09.860)
in a certain time period,
Lex Fridman (48:10.940)
I wanna emphasize these muscles.
Gary Marcus (48:12.700)
You want some kind of abstract description.
Lex Fridman (48:15.060)
The fact that you happen to press the remote control
Gary Marcus (48:17.260)
in this room when you watch this television
Lex Fridman (48:20.020)
isn't really the essence of the exercise routine.
Lex Fridman (48:23.060)
But if you just ask people like, what did they do?
Lex Fridman (48:24.780)
Then they give you this fine grain.
Lex Fridman (48:26.980)
And so it takes a level of expertise
Lex Fridman (48:29.780)
about how the AI works
Gary Marcus (48:31.900)
in order to craft the right kind of knowledge.
Lex Fridman (48:34.540)
So there's this ocean of knowledge that we all operate on.
Gary Marcus (48:37.580)
Some of them may not even be conscious,
Lex Fridman (48:39.340)
or at least we're not able to communicate it effectively.
Gary Marcus (48:43.300)
Yeah, most of it we would recognize if somebody said it,
Lex Fridman (48:45.700)
if it was true or not,
Lex Fridman (48:47.420)
but we wouldn't think to say that it's true or not.
Lex Fridman (48:49.660)
That's a really interesting mathematical property.
Gary Marcus (48:53.060)
This ocean has the property
Lex Fridman (48:54.720)
that every piece of knowledge in it,
Gary Marcus (48:56.720)
we will recognize it as true if we're told,
Lex Fridman (48:59.940)
but we're unlikely to retrieve it in the reverse.
Lex Fridman (49:04.140)
So that interesting property,
Lex Fridman (49:07.180)
I would say there's a huge ocean of that knowledge.
Lex Fridman (49:10.580)
What's your intuition?
Lex Fridman (49:11.580)
Is it accessible to AI systems somehow?
Lex Fridman (49:14.700)
Can we?
Lex Fridman (49:15.940)
So you said this.
Gary Marcus (49:16.780)
I mean, most of it is not,
Lex Fridman (49:18.780)
well, I'll give you an asterisk on this in a second,
Lex Fridman (49:20.540)
but most of it has not ever been encoded
Lex Fridman (49:23.260)
in machine interpretable form.
Lex Fridman (49:25.660)
And so, I mean, if you say accessible,
Lex Fridman (49:27.300)
there's two meanings of that.
Lex Fridman (49:28.640)
One is like, could you build it into a machine?
Lex Fridman (49:31.540)
Yes.
Gary Marcus (49:32.380)
The other is like, is there some database
Lex Fridman (49:34.460)
that we could go download and stick into our machine?
Lex Fridman (49:38.380)
But the first thing, could we?
Lex Fridman (49:40.660)
What's your intuition? I think we could.
Gary Marcus (49:42.020)
I think it hasn't been done right.
Lex Fridman (49:45.020)
You know, the closest, and this is the asterisk,
Gary Marcus (49:47.300)
is the CYC psych system tried to do this.
Lex Fridman (49:51.140)
A lot of logicians worked for Doug Lennon
Gary Marcus (49:53.020)
for 30 years on this project.
Lex Fridman (49:55.460)
I think they stuck too closely to logic,
Gary Marcus (49:57.900)
didn't represent enough about probabilities,
Lex Fridman (50:00.220)
tried to hand code it.
Gary Marcus (50:01.180)
There are various issues,
Lex Fridman (50:02.180)
and it hasn't been that successful.
Gary Marcus (50:04.480)
That is the closest existing system
Lex Fridman (50:08.500)
to trying to encode this.
Lex Fridman (50:10.620)
Why do you think there's not more excitement
Lex Fridman (50:13.460)
slash money behind this idea currently?
Gary Marcus (50:16.420)
There was.
Lex Fridman (50:17.260)
People view that project as a failure.
Gary Marcus (50:19.180)
I think that they confuse the failure
Lex Fridman (50:22.060)
of a specific instance that was conceived 30 years ago
Gary Marcus (50:25.100)
for the failure of an approach,
Lex Fridman (50:26.180)
which they don't do for deep learning.
Lex Fridman (50:28.160)
So in 2010, people had the same attitude
Lex Fridman (50:31.940)
towards deep learning.
Gary Marcus (50:32.780)
They're like, this stuff doesn't really work.
Lex Fridman (50:35.500)
And all these other algorithms work better and so forth.
Lex Fridman (50:39.140)
And then certain key technical advances were made,
Lex Fridman (50:41.900)
but mostly it was the advent
Gary Marcus (50:43.780)
of graphics processing units that changed that.
Lex Fridman (50:46.400)
It wasn't even anything foundational in the techniques.
Lex Fridman (50:50.060)
And there was some new tricks,
Lex Fridman (50:51.220)
but mostly it was just more compute and more data,
Gary Marcus (50:55.300)
things like ImageNet that didn't exist before
Lex Fridman (50:57.900)
that allowed deep learning.
Lex Fridman (50:59.020)
And it could be, to work,
Lex Fridman (51:00.860)
it could be that CYC just needs a few more things
Gary Marcus (51:03.780)
or something like CYC,
Lex Fridman (51:05.500)
but the widespread view is that that just doesn't work.
Lex Fridman (51:08.820)
And people are reasoning from a single example.
Lex Fridman (51:11.820)
They don't do that with deep learning.
Gary Marcus (51:13.260)
They don't say nothing that existed in 2010,
Lex Fridman (51:16.580)
and there were many, many efforts in deep learning
Gary Marcus (51:18.860)
was really worth anything.
Lex Fridman (51:20.580)
I mean, really, there's no model from 2010
Gary Marcus (51:23.820)
in deep learning or the predecessors of deep learning
Lex Fridman (51:26.620)
that has any commercial value whatsoever at this point.
Gary Marcus (51:29.660)
They're all failures.
Lex Fridman (51:31.540)
But that doesn't mean that there wasn't anything there.
Gary Marcus (51:33.500)
I have a friend, I was getting to know him,
Lex Fridman (51:35.940)
and he said, I had a company too,
Gary Marcus (51:38.820)
I was talking about I had a new company.
Lex Fridman (51:40.580)
He said, I had a company too, and it failed.
Lex Fridman (51:42.900)
And I said, well, what did you do?
Lex Fridman (51:44.260)
And he said, deep learning.
Lex Fridman (51:45.660)
And the problem was he did it in 1986
Lex Fridman (51:47.940)
or something like that.
Lex Fridman (51:48.780)
And we didn't have the tools then, or 1990,
Lex Fridman (51:51.060)
we didn't have the tools then, not the algorithms.
Gary Marcus (51:53.980)
His algorithms weren't that different from model algorithms,
Lex Fridman (51:56.540)
but he didn't have the GPUs to run it fast enough.
Gary Marcus (51:58.420)
He didn't have the data.
Lex Fridman (51:59.620)
And so it failed.
Gary Marcus (52:01.340)
It could be that symbol manipulation per se
Lex Fridman (52:06.820)
with modern amounts of data and compute
Lex Fridman (52:09.580)
and maybe some advance in compute
Lex Fridman (52:11.940)
for that kind of compute might be great.
Gary Marcus (52:14.900)
My perspective on it is not that we want to resuscitate
Lex Fridman (52:19.340)
that stuff per se, but we want to borrow lessons from it,
Gary Marcus (52:21.540)
bring together with other things that we've learned.
Lex Fridman (52:23.380)
And it might have an ImageNet moment
Gary Marcus (52:25.900)
where it would spark the world's imagination
Lex Fridman (52:28.220)
and there'll be an explosion of symbol manipulation efforts.
Gary Marcus (52:31.460)
Yeah, I think that people at AI2,
Lex Fridman (52:33.660)
Paul Allen's AI Institute, are trying to build data sets.
Gary Marcus (52:39.060)
Well, they're not doing it
Lex Fridman (52:39.900)
for quite the reason that you say,
Lex Fridman (52:41.100)
but they're trying to build data sets
Lex Fridman (52:43.220)
that at least spark interest in common sense reasoning.
Gary Marcus (52:45.380)
To create benchmarks.
Lex Fridman (52:46.780)
Benchmarks for common sense.
Gary Marcus (52:48.220)
That's a large part of what the AI2.org
Lex Fridman (52:50.860)
is working on right now.
Lex Fridman (52:51.980)
So speaking of compute,
Lex Fridman (52:53.260)
Rich Sutton wrote a blog post titled Bitter Lesson.
Gary Marcus (52:56.380)
I don't know if you've read it,
Lex Fridman (52:57.220)
but he said that the biggest lesson that can be read
Gary Marcus (52:59.900)
from so many years of AI research
Lex Fridman (53:01.580)
is that general methods that leverage computation
Gary Marcus (53:04.180)
are ultimately the most effective.
Lex Fridman (53:06.300)
Do you think that?
Lex Fridman (53:07.140)
The most effective at what?
Lex Fridman (53:08.620)
Right, so they have been most effective
Gary Marcus (53:11.820)
for perceptual classification problems
Lex Fridman (53:14.500)
and for some reinforcement learning problems.
Lex Fridman (53:18.060)
And he works on reinforcement learning.
Lex Fridman (53:19.380)
Well, no, let me push back on that.
Gary Marcus (53:20.700)
You're actually absolutely right.
Lex Fridman (53:22.820)
But I would also say they have been most effective generally
Gary Marcus (53:28.060)
because everything we've done up to...
Lex Fridman (53:31.500)
Would you argue against that?
Gary Marcus (53:32.900)
Is, to me, deep learning is the first thing
Lex Fridman (53:36.220)
that has been successful at anything in AI.
Lex Fridman (53:41.860)
And you're pointing out that this success
Lex Fridman (53:45.300)
is very limited, folks,
Lex Fridman (53:47.100)
but has there been something truly successful
Lex Fridman (53:50.260)
before deep learning?
Gary Marcus (53:51.660)
Sure, I mean, I want to make a larger point,
Lex Fridman (53:54.860)
but on the narrower point, classical AI is used,
Gary Marcus (54:00.020)
for example, in doing navigation instructions.
Lex Fridman (54:03.660)
It's very successful.
Gary Marcus (54:06.020)
Everybody on the planet uses it now,
Lex Fridman (54:07.780)
like multiple times a day.
Lex Fridman (54:09.420)
That's a measure of success, right?
Lex Fridman (54:12.220)
So I don't think classical AI was wildly successful,
Lex Fridman (54:16.060)
but there are cases like that.
Lex Fridman (54:17.580)
They're just used all the time.
Gary Marcus (54:19.140)
Nobody even notices them because they're so pervasive.
Lex Fridman (54:23.740)
So there are some successes for classical AI.
Gary Marcus (54:26.580)
I think deep learning has been more successful,
Lex Fridman (54:28.700)
but my usual line about this, and I didn't invent it,
Lex Fridman (54:32.020)
but I like it a lot,
Lex Fridman (54:33.060)
is just because you can build a better ladder
Gary Marcus (54:34.780)
doesn't mean you can build a ladder to the moon.
Lex Fridman (54:37.140)
So the bitter lesson is if you have
Gary Marcus (54:39.660)
a perceptual classification problem,
Lex Fridman (54:42.220)
throwing a lot of data at it is better than anything else.
Lex Fridman (54:45.740)
But that has not given us any material progress
Lex Fridman (54:49.980)
in natural language understanding,
Gary Marcus (54:51.860)
common sense reasoning,
Lex Fridman (54:53.060)
like a robot would need to navigate a home.
Gary Marcus (54:56.220)
Problems like that, there's no actual progress there.
Lex Fridman (54:59.420)
So flip side of that, if we remove data from the picture,
Gary Marcus (55:02.220)
another bitter lesson is that you just have
Lex Fridman (55:05.780)
a very simple algorithm,
Lex Fridman (55:10.100)
and you wait for compute to scale.
Lex Fridman (55:12.500)
It doesn't have to be learning.
Gary Marcus (55:13.540)
It doesn't have to be deep learning.
Lex Fridman (55:14.580)
It doesn't have to be data driven,
Lex Fridman (55:16.420)
but just wait for the compute.
Lex Fridman (55:18.220)
So my question for you,
Lex Fridman (55:19.060)
do you think compute can unlock some of the things
Lex Fridman (55:21.660)
with either deep learning or symbol manipulation that?
Gary Marcus (55:25.460)
Sure, but I'll put a proviso on that.
Lex Fridman (55:29.780)
I think more compute's always better.
Gary Marcus (55:31.940)
Nobody's gonna argue with more compute.
Lex Fridman (55:33.660)
It's like having more money.
Gary Marcus (55:34.700)
I mean, there's the data.
Lex Fridman (55:36.020)
There's diminishing returns on more money.
Gary Marcus (55:37.460)
Exactly, there's diminishing returns on more money,
Lex Fridman (55:39.740)
but nobody's gonna argue
Lex Fridman (55:40.980)
if you wanna give them more money, right?
Lex Fridman (55:42.620)
Except maybe the people who signed the giving pledge,
Lex Fridman (55:44.620)
and some of them have a problem.
Lex Fridman (55:46.300)
They've promised to give away more money
Gary Marcus (55:47.980)
than they're able to.
Lex Fridman (55:49.660)
But the rest of us, if you wanna give me more money, fine.
Gary Marcus (55:52.500)
I'm saying more money, more problems, but okay.
Lex Fridman (55:54.580)
That's true too.
Lex Fridman (55:55.500)
What I would say to you is your brain uses like 20 watts,
Lex Fridman (56:00.100)
and it does a lot of things that deep learning doesn't do,
Gary Marcus (56:02.660)
or that symbol manipulation doesn't do,
Lex Fridman (56:05.140)
that AI just hasn't figured out how to do.
Lex Fridman (56:07.020)
So it's an existence proof
Lex Fridman (56:09.100)
that you don't need server resources
Gary Marcus (56:12.140)
that are Google scale in order to have an intelligence.
Lex Fridman (56:16.460)
I built, with a lot of help from my wife,
Gary Marcus (56:18.900)
two intelligences that are 20 watts each,
Lex Fridman (56:21.660)
and far exceed anything that anybody else
Gary Marcus (56:25.060)
has built at a silicon.
Lex Fridman (56:26.780)
Speaking of those two robots,
Lex Fridman (56:30.020)
what have you learned about AI from having?
Lex Fridman (56:33.260)
Well, they're not robots, but.
Gary Marcus (56:35.300)
Sorry, intelligent agents.
Lex Fridman (56:36.740)
Those two intelligent agents.
Gary Marcus (56:38.140)
I've learned a lot by watching my two intelligent agents.
Lex Fridman (56:42.780)
I think that what's fundamentally interesting,
Gary Marcus (56:45.820)
well, one of the many things
Lex Fridman (56:46.980)
that's fundamentally interesting about them
Gary Marcus (56:48.660)
is the way that they set their own problems to solve.
Lex Fridman (56:51.940)
So my two kids are a year and a half apart.
Gary Marcus (56:54.540)
They're both five and six and a half.
Lex Fridman (56:56.420)
They play together all the time,
Lex Fridman (56:58.180)
and they're constantly creating new challenges.
Lex Fridman (57:00.940)
That's what they do, is they make up games,
Lex Fridman (57:03.780)
and they're like, well, what if this, or what if that,
Lex Fridman (57:05.940)
or what if I had this superpower,
Lex Fridman (57:07.860)
or what if you could walk through this wall?
Lex Fridman (57:10.340)
So they're doing these what if scenarios all the time,
Lex Fridman (57:14.020)
and that's how they learn something about the world
Lex Fridman (57:17.540)
and grow their minds, and machines don't really do that.
Lex Fridman (57:22.580)
So that's interesting, and you've talked about this,
Lex Fridman (57:24.460)
you've written about it, you've thought about it,
Gary Marcus (57:26.100)
nature versus nurture.
Lex Fridman (57:29.260)
So what innate knowledge do you think we're born with,
Lex Fridman (57:33.580)
and what do we learn along the way
Lex Fridman (57:35.540)
in those early months and years?
Lex Fridman (57:38.260)
Can I just say how much I like that question?
Lex Fridman (57:41.540)
You phrased it just right, and almost nobody ever does,
Gary Marcus (57:45.780)
which is what is the innate knowledge
Lex Fridman (57:47.220)
and what's learned along the way?
Lex Fridman (57:49.180)
So many people dichotomize it,
Lex Fridman (57:51.180)
and they think it's nature versus nurture,
Gary Marcus (57:53.380)
when it is obviously has to be nature and nurture.
Lex Fridman (57:56.740)
They have to work together.
Gary Marcus (57:58.540)
You can't learn this stuff along the way
Lex Fridman (58:00.500)
unless you have some innate stuff,
Lex Fridman (58:02.340)
but just because you have the innate stuff
Lex Fridman (58:03.860)
doesn't mean you don't learn anything.
Lex Fridman (58:05.820)
And so many people get that wrong, including in the field.
Lex Fridman (58:09.340)
People think if I work in machine learning,
Gary Marcus (58:12.220)
the learning side, I must not be allowed to work
Lex Fridman (58:15.260)
on the innate side, or that will be cheating.
Gary Marcus (58:17.300)
Exactly, people have said that to me,
Lex Fridman (58:19.620)
and it's just absurd, so thank you.
Lex Fridman (58:23.380)
But you could break that apart more.
Lex Fridman (58:25.140)
I've talked to folks who studied
Gary Marcus (58:26.540)
the development of the brain,
Lex Fridman (58:28.260)
and the growth of the brain in the first few days
Gary Marcus (58:32.940)
in the first few months in the womb,
Lex Fridman (58:35.660)
all of that, is that innate?
Lex Fridman (58:39.500)
So that process of development from a stem cell
Lex Fridman (58:42.300)
to the growth of the central nervous system and so on,
Gary Marcus (58:46.020)
to the information that's encoded
Lex Fridman (58:49.300)
through the long arc of evolution.
Lex Fridman (58:52.300)
So all of that comes into play, and it's unclear.
Lex Fridman (58:55.300)
It's not just whether it's a dichotomy or not.
Gary Marcus (58:57.340)
It's where most, or where the knowledge is encoded.
Lex Fridman (59:02.060)
So what's your intuition about the innate knowledge,
Gary Marcus (59:07.780)
the power of it, what's contained in it,
Lex Fridman (59:09.700)
what can we learn from it?
Gary Marcus (59:11.340)
One of my earlier books was actually trying
Lex Fridman (59:12.740)
to understand the biology of this.
Gary Marcus (59:14.020)
The book was called The Birth of the Mind.
Lex Fridman (59:15.860)
Like how is it the genes even build innate knowledge?
Lex Fridman (59:18.900)
And from the perspective of the conversation
Lex Fridman (59:21.460)
we're having today, there's actually two questions.
Gary Marcus (59:23.580)
One is what innate knowledge or mechanisms,
Lex Fridman (59:26.460)
or what have you, people or other animals
Gary Marcus (59:29.660)
might be endowed with.
Lex Fridman (59:30.900)
I always like showing this video
Gary Marcus (59:32.260)
of a baby ibex climbing down a mountain.
Lex Fridman (59:34.620)
That baby ibex, a few hours after its birth,
Gary Marcus (59:37.380)
knows how to climb down a mountain.
Lex Fridman (59:38.420)
That means that it knows, not consciously,
Gary Marcus (59:40.940)
something about its own body and physics
Lex Fridman (59:43.020)
and 3D geometry and all of this kind of stuff.
Lex Fridman (59:47.500)
So there's one question about what does biology
Lex Fridman (59:49.660)
give its creatures and what has evolved in our brains?
Lex Fridman (59:53.220)
How is that represented in our brains?
Lex Fridman (59:54.940)
The question I thought about in the book
Gary Marcus (59:56.180)
The Birth of the Mind.
Lex Fridman (59:57.340)
And then there's a question of what AI should have.
Lex Fridman (59:59.300)
And they don't have to be the same.
🔗 相关节目