Richard Haier: IQ Tests, Human Intelligence, and Group Differences
心理与人性生物与进化音乐与艺术AI 与机器学习技术与编程
📋 章节目录
暂无章节信息
🔑 关键词
intelligencefactortestdontestsdatasaidresearchinterestingbrainhumandoesnbetterbookcalledgoingpersonsciencescorescores
💬 精彩语录
"It would have been obvious that school interventions, compensatory education, early childhood education,"
很明显,学校干预、补偿教育、幼儿教育、
— Richard Haier (2:04:45.900)
"And so one of the things you look at, at companies like Neuralink, you have brain computer interfaces,"
所以你看到的一件事是,在像 Neuralink 这样的公司,你有脑机接口,
— Richard Haier (2:32:01.140)
🎙️ 完整对话(2714 条)
Lex Fridman (00:00.000)
Let me ask you to this question,
让我来问你这个问题,
Lex Fridman (00:02.160)
whether it's bell curve or any research
无论是钟形曲线还是任何研究
Lex Fridman (00:04.240)
on race differences,
关于种族差异,
Lex Fridman (00:09.160)
can that be used to increase the amount of racism
这可以用来增加种族主义的数量吗
Lex Fridman (00:12.440)
in the world, can that be used to increase
在世界上,这可以用来增加
Lex Fridman (00:14.880)
the amount of hate in the world?
世界上的仇恨有多少?
Lex Fridman (00:16.920)
My sense is there is such enormous reservoirs
我的感觉是有如此巨大的水库
Richard Haier (00:24.120)
of hate and racism that have nothing to do
无关的仇恨和种族主义
Lex Fridman (00:27.940)
with scientific knowledge of the data
具有数据的科学知识
Richard Haier (00:31.480)
that speak against that,
反对这一点的人,
Lex Fridman (00:34.080)
that no, I don't want to give racist groups
不,我不想给种族主义团体
Richard Haier (00:39.200)
a veto power over what scientists study.
对科学家的研究拥有否决权。
Lex Fridman (00:44.200)
The following is a conversation with Richard Heyer
以下是与理查德·海尔的对话
Richard Haier (00:46.960)
on the science of human intelligence.
关于人类智力的科学。
Lex Fridman (00:49.320)
This is a highly controversial topic,
这是一个极具争议性的话题
Lex Fridman (00:51.720)
but a critically important one
但极其重要的一个
Lex Fridman (00:52.960)
for understanding the human mind.
为了了解人类的思想。
Richard Haier (00:54.980)
I hope you will join me in not shying away
我希望你能和我一起不回避
Lex Fridman (00:57.680)
from difficult topics like this,
从这样困难的话题中,
Lex Fridman (00:59.980)
and instead, let us try to navigate it
相反,让我们尝试导航它
Lex Fridman (01:03.700)
with empathy, rigor, and grace.
Richard Haier (01:06.460)
If you're watching this on video now,
Lex Fridman (01:08.800)
I should mention that I'm recording this introduction
Richard Haier (01:11.500)
in an undisclosed location somewhere in the world.
Lex Fridman (01:14.580)
I'm safe and happy and life is beautiful.
Richard Haier (01:18.640)
This is the Lex Friedman Podcast.
Lex Fridman (01:20.400)
To support it, please check out our sponsors
Richard Haier (01:22.360)
in the description, and now, dear friends,
Lex Fridman (01:25.200)
here's Richard Heyer.
Lex Fridman (01:27.920)
What are the measures of human intelligence,
Lex Fridman (01:29.880)
and how do we measure it?
Richard Haier (01:31.560)
Everybody has an idea of what they mean by intelligence.
Lex Fridman (01:35.840)
In the vernacular, what I mean by intelligence
Richard Haier (01:40.040)
is just being smart, how well you reason,
Lex Fridman (01:42.560)
how well you figure things out,
Lex Fridman (01:45.160)
what you do when you don't know what to do.
Lex Fridman (01:48.680)
Those are just kind of everyday common sense definitions
Richard Haier (01:53.680)
of how people use the word intelligence.
Lex Fridman (01:56.660)
If you wanna do research on intelligence,
Richard Haier (01:59.500)
measuring something that you can study scientifically
Lex Fridman (02:03.360)
is a little trickier, and what almost all researchers
Richard Haier (02:09.120)
who study intelligence use is the concept
Lex Fridman (02:12.820)
called the G factor, general intelligence,
Lex Fridman (02:16.980)
and that is what is common, that is a mental ability
Lex Fridman (02:21.640)
that is common to virtually all tests of mental abilities.
Richard Haier (02:26.500)
What's the origin of the term G factor,
Lex Fridman (02:28.500)
by the way, such a funny word
Lex Fridman (02:29.820)
for such a fundamental human thing?
Lex Fridman (02:32.060)
The general factor, I really started with Charles Spearman,
Lex Fridman (02:36.540)
and he noticed, this is like, boy,
Lex Fridman (02:39.060)
more than 100 years ago, he noticed that
Richard Haier (02:44.140)
when you tested people with different tests,
Lex Fridman (02:47.780)
all the tests were correlated positively,
Lex Fridman (02:53.100)
and so he was looking at student exams and things,
Lex Fridman (02:57.380)
and he invented the correlation coefficient, essentially,
Lex Fridman (03:01.620)
and when he used it to look at student performance
Lex Fridman (03:06.380)
on various topics, he found all the scores
Richard Haier (03:09.920)
were correlated with each other,
Lex Fridman (03:11.920)
and they were all positive correlations,
Lex Fridman (03:14.480)
so he inferred from this that there must be
Lex Fridman (03:17.540)
some common factor that was irrespective
Richard Haier (03:21.220)
of the content of the test.
Lex Fridman (03:23.360)
And positive correlation means if you do well
Richard Haier (03:27.420)
on the first test, you're likely to do well
Lex Fridman (03:29.500)
on the second test, and presumably,
Richard Haier (03:31.760)
that holds for tests across even disciplines,
Lex Fridman (03:35.820)
so not within subject, but across subjects,
Lex Fridman (03:39.340)
so that's where the general comes in,
Lex Fridman (03:43.020)
something about general intelligence.
Lex Fridman (03:45.020)
So when you were talking about measuring intelligence
Lex Fridman (03:46.980)
and trying to figure out something difficult
Richard Haier (03:50.140)
about this world and how to solve the puzzles
Lex Fridman (03:52.300)
of this world, that means, generally speaking,
Richard Haier (03:54.740)
not some specific test, but across all tests.
Lex Fridman (03:58.140)
Absolutely right, and people get hung up on this
Richard Haier (04:02.340)
because they say, well, what about the ability
Lex Fridman (04:04.860)
to do X, isn't that independent?
Lex Fridman (04:08.860)
And they said, I know somebody who's very good at this
Lex Fridman (04:12.000)
but not so good at this, this other thing.
Lex Fridman (04:15.360)
And so there are a lot of examples like that,
Lex Fridman (04:17.460)
but it's a general tendency, so exceptions
Richard Haier (04:21.660)
really don't disprove, your everyday experience
Lex Fridman (04:26.500)
is not the same as what the data actually show.
Lex Fridman (04:30.740)
And your everyday experience, when you say,
Lex Fridman (04:32.700)
oh, I know someone who's good at X, but not so good at Y,
Richard Haier (04:36.580)
that doesn't contradict the statement of about,
Lex Fridman (04:39.100)
he's not so good, but he's not the opposite.
Richard Haier (04:43.380)
He's not, it's not a negative correlation.
Lex Fridman (04:46.520)
Okay, so we're not, our anecdotal data,
Richard Haier (04:49.880)
I know a guy who's really good at solving
Lex Fridman (04:53.740)
some kind of visual thing, that's not sufficient
Richard Haier (04:58.140)
for us to understand actually the depths
Lex Fridman (05:00.020)
of that person's intelligence.
Lex Fridman (05:01.380)
So how, this idea of G factor,
Lex Fridman (05:07.480)
how much evidence is there, how strong,
Richard Haier (05:11.180)
you know, given across the decades that this idea
Lex Fridman (05:13.700)
has been around, how much has it been held up
Richard Haier (05:16.420)
that there is a universal sort of horsepower
Lex Fridman (05:21.580)
of intelligence that's underneath all of it,
Richard Haier (05:24.020)
all the different tests we do to try to get to this thing
Lex Fridman (05:28.300)
in the depths of the human mind that's a universal,
Richard Haier (05:32.000)
stable measure of a person's intelligence.
Lex Fridman (05:34.820)
You used a couple of words in there, stable and.
Lex Fridman (05:38.820)
We have to be precise with words?
Lex Fridman (05:40.580)
I was hoping we can get away with being poetic.
Richard Haier (05:42.700)
We can, there's a lot about research in general,
Lex Fridman (05:46.260)
not just intelligence research that is poetic.
Richard Haier (05:49.580)
Science has a punetic aspect to it.
Lex Fridman (05:52.900)
Good scientists are very intuitive.
Richard Haier (05:55.840)
They're not just, hey, these are the numbers.
Lex Fridman (05:59.220)
You have to kind of step back and see the big picture.
Richard Haier (06:02.100)
When it comes to intelligence research,
Lex Fridman (06:05.960)
you asked how well has this general concept held up?
Lex Fridman (06:09.660)
And I think I can say without fear
Lex Fridman (06:13.140)
of being empirically contradicted,
Richard Haier (06:16.260)
that it is the most replicated finding in all of psychology.
Lex Fridman (06:21.100)
Now, some cynics may say, well, big deal,
Richard Haier (06:22.980)
psychology, we all know there's a replication crisis
Lex Fridman (06:25.460)
in psychology and a lot of this stuff doesn't replicate.
Richard Haier (06:28.420)
That's all true.
Lex Fridman (06:29.700)
There is no replication crisis when it comes to studying
Richard Haier (06:33.540)
the existence of this general factor.
Lex Fridman (06:36.780)
Let me tell you some things about it.
Richard Haier (06:38.940)
It looks like it's universal
Lex Fridman (06:42.420)
that you find it in all cultures.
Richard Haier (06:44.660)
The way you find it, step back one step,
Lex Fridman (06:47.760)
the way you find it is to give a battery of mental tests.
Lex Fridman (06:51.760)
What battery?
Lex Fridman (06:52.780)
You choose.
Richard Haier (06:53.800)
Take a battery of any mental tests you want,
Lex Fridman (06:57.100)
give it to a large number of diverse people,
Lex Fridman (07:01.520)
and you will be able to extract statistically
Lex Fridman (07:05.740)
the commonality among all those tests.
Richard Haier (07:09.020)
It's done by a technique called factor analysis.
Lex Fridman (07:12.460)
People think that this may be a statistical artifact
Richard Haier (07:17.660)
of some kind, it is not a statistical artifact.
Lex Fridman (07:21.340)
What is factor analysis?
Richard Haier (07:22.700)
Factor analysis is a way of looking at a big set of data
Lex Fridman (07:26.180)
and look at the correlation among the different test scores
Lex Fridman (07:29.900)
and then find empirically the clusters of scores
Lex Fridman (07:33.840)
that go together.
Lex Fridman (07:35.620)
And there are different factors.
Lex Fridman (07:37.340)
So if you have a bunch of mental tests,
Richard Haier (07:39.380)
there may be a verbal factor,
Lex Fridman (07:41.120)
there may be a numerical factor,
Richard Haier (07:43.320)
there may be a visual spatial factor,
Lex Fridman (07:45.900)
but those factors have variance in common with each other.
Lex Fridman (07:50.220)
And that is the common,
Lex Fridman (07:53.060)
that's what's common among all the tests
Lex Fridman (07:55.360)
and that's what gets labeled the G factor.
Lex Fridman (07:58.100)
So if you give a diverse battery of mental tests
Lex Fridman (08:01.460)
and you extract a G factor from it,
Lex Fridman (08:04.740)
that factor usually accounts for around half of the variance.
Richard Haier (08:08.240)
It's the single biggest factor, but it's not the only factor,
Lex Fridman (08:12.860)
but it is the most reliable, it is the most stable,
Lex Fridman (08:17.220)
and it seems to be very much influenced by genetics.
Lex Fridman (08:23.820)
It's very hard to change the G factor with training
Richard Haier (08:28.220)
or drugs or anything else.
Lex Fridman (08:32.860)
You don't know how to increase the G factor.
Richard Haier (08:34.980)
Okay, you said a lot of really interesting things there.
Lex Fridman (08:36.920)
So first, I mean, just to get people used to it
Richard Haier (08:40.940)
in case they're not familiar with this idea,
Lex Fridman (08:43.260)
G factor is what we mean.
Lex Fridman (08:45.820)
So often there's this term used IQ,
Lex Fridman (08:50.340)
which is the way IQ is used,
Richard Haier (08:53.980)
they really mean G factor in regular conversation.
Lex Fridman (08:58.900)
Because what we mean by IQ, we mean intelligence
Lex Fridman (09:02.640)
and what we mean by intelligence,
Lex Fridman (09:04.500)
we mean general intelligence and general intelligence
Richard Haier (09:07.300)
in the human mind from a psychology,
Lex Fridman (09:09.860)
from a serious rigorous scientific perspective
Richard Haier (09:12.320)
actually means G factor.
Lex Fridman (09:13.900)
So G factor equals intelligence,
Richard Haier (09:15.780)
just in this conversation to define terms.
Lex Fridman (09:18.460)
Okay, so there's this stable thing called G factor.
Richard Haier (09:22.220)
You said, now factor, you said factor many times,
Lex Fridman (09:27.220)
means a measure that potentially could be reduced
Richard Haier (09:33.500)
to a single number across the different factors
Lex Fridman (09:35.820)
you mentioned.
Lex Fridman (09:37.180)
And what you said, it accounts for half, halfish.
Lex Fridman (09:45.060)
Accounts for halfish of what?
Richard Haier (09:46.640)
Of variance across the different set of tests.
Lex Fridman (09:51.640)
Set of tests, so if you do for some reason
Lex Fridman (09:56.500)
well on some set of tests, what does that mean?
Lex Fridman (10:00.940)
So that means there's some unique capabilities
Richard Haier (10:03.340)
outside of the G factor that might account for that.
Lex Fridman (10:05.900)
And what are those?
Lex Fridman (10:07.500)
What else is there besides the raw horsepower,
Lex Fridman (10:10.420)
the engine inside your mind that generates intelligence?
Richard Haier (10:13.380)
There are test taking skills.
Lex Fridman (10:16.380)
There are specific abilities.
Richard Haier (10:20.880)
Someone might be particularly good at mathematical things,
Lex Fridman (10:28.300)
mathematical concepts, even simple arithmetic.
Richard Haier (10:32.140)
Some people are much better than others.
Lex Fridman (10:34.340)
You might know people who can memorize,
Lex Fridman (10:36.420)
and short term memory is another component of this.
Lex Fridman (10:42.540)
Short term memory is one of the cognitive processes
Richard Haier (10:46.900)
that's most highly correlated with the G factor.
Lex Fridman (10:51.260)
So all those things like memory,
Richard Haier (10:56.260)
test taking skills account for variability
Lex Fridman (10:59.940)
across the test performances.
Lex Fridman (11:02.240)
But so you can run, but you can't hide
Lex Fridman (11:06.460)
from the thing that God gave you.
Richard Haier (11:08.540)
The genetics, so that G factor,
Lex Fridman (11:12.940)
science says that G factor's there.
Richard Haier (11:15.120)
Each one of us have.
Lex Fridman (11:16.900)
Each one of us has a G factor.
Richard Haier (11:19.380)
Oh boy.
Lex Fridman (11:20.220)
Some have more than others.
Richard Haier (11:21.380)
I'm getting uncomfortable already.
Lex Fridman (11:22.820)
Well, IQ is a score, and IQ, an IQ score
Richard Haier (11:28.500)
is a very good estimate of the G factor.
Lex Fridman (11:32.420)
You can't measure G directly, there's no direct measure.
Richard Haier (11:36.100)
You estimate it from these statistical techniques.
Lex Fridman (11:39.880)
But an IQ score is a good estimate, why?
Richard Haier (11:43.080)
Because a standard IQ test is a battery
Lex Fridman (11:46.420)
of different mental abilities.
Richard Haier (11:48.640)
You combined it into one score,
Lex Fridman (11:51.380)
and that score is highly correlated with the G factor,
Richard Haier (11:55.700)
even if you get better scores on some subtests than others.
Lex Fridman (12:00.140)
Because again, it's what's common
Richard Haier (12:02.300)
to all these mental abilities.
Lex Fridman (12:04.300)
So a good IQ test, and I'll ask you about that,
Lex Fridman (12:08.200)
but a good IQ test tries to compress down that battery
Lex Fridman (12:13.340)
of tests, like tries to get a nice battery,
Richard Haier (12:16.200)
the nice selection of variable tests into one test.
Lex Fridman (12:21.420)
And so in that way, it sneaks up to this G factor.
Lex Fridman (12:24.180)
And that's another interesting thing about G factor.
Lex Fridman (12:28.380)
Now you give, first of all, you have a great book
Richard Haier (12:32.380)
on the neuroscience of intelligence.
Lex Fridman (12:34.180)
You have a great course, which is when I first learned,
Richard Haier (12:38.300)
you're a great teacher, let me just say.
Lex Fridman (12:39.820)
Thank you.
Richard Haier (12:42.140)
Your course at the teaching company,
Lex Fridman (12:44.180)
I hope I'm saying that correctly.
Richard Haier (12:45.820)
The Intelligent Brain.
Lex Fridman (12:47.180)
The Intelligent Brain is when I first heard
Richard Haier (12:50.520)
about this G factor, this mysterious thing
Lex Fridman (12:53.820)
that lurks in the darkness that we cannot quite shine
Richard Haier (12:56.300)
a light on, we're trying to sneak up on.
Lex Fridman (12:59.000)
So the fact that there's this measure,
Richard Haier (13:00.580)
a stable measure of intelligence, we can't measure directly.
Lex Fridman (13:04.820)
But we can come up with a battery test
Richard Haier (13:07.820)
or one test that includes a battery
Lex Fridman (13:10.300)
of variable type of questions that can reliably
Richard Haier (13:17.460)
or attempt to estimate in a stable way that G factor.
Lex Fridman (13:21.700)
That's a fascinating idea.
Lex Fridman (13:23.340)
So for me as an AI person, it's fascinating.
Lex Fridman (13:25.860)
It's fascinating there's something stable like that
Richard Haier (13:27.900)
about the human mind, especially if it's grounded in genetics.
Lex Fridman (13:32.020)
It's both fascinating that as a researcher
Richard Haier (13:37.100)
of the human mind and all the human psychological,
Lex Fridman (13:43.220)
sociological, ethical questions that start arising,
Richard Haier (13:46.560)
it makes me uncomfortable.
Lex Fridman (13:48.260)
But truth can be uncomfortable.
Richard Haier (13:51.340)
I get that a lot about being uncomfortable
Lex Fridman (13:54.180)
talking about this.
Richard Haier (13:56.540)
Let me go back and just say one more empirical thing.
Lex Fridman (14:02.060)
It doesn't matter which battery of tests you use.
Lex Fridman (14:08.700)
So there are countless tests.
Lex Fridman (14:10.700)
You can take any 12 of them at random,
Richard Haier (14:13.500)
extract a G factor and another 12 at random
Lex Fridman (14:17.100)
and extract a G factor and those G factors
Richard Haier (14:19.940)
will be highly correlated like over 0.9 with each other.
Lex Fridman (14:23.160)
That's very, so it is a ubiquitous.
Richard Haier (14:26.100)
It doesn't depend on the content of the test
Lex Fridman (14:28.500)
is what I'm trying to say.
Richard Haier (14:30.260)
It is general among all those tests of mental ability.
Lex Fridman (14:34.020)
And tests of mental, mental abilities include things like,
Richard Haier (14:37.940)
geez, playing poker.
Lex Fridman (14:41.260)
Your skill at poker is not unrelated to G.
Richard Haier (14:46.300)
Your skill at anything that requires reasoning
Lex Fridman (14:49.460)
and thinking, anything, spelling, arithmetic,
Richard Haier (14:54.300)
more complex things, this concept is ubiquitous.
Lex Fridman (15:00.500)
And when you do batteries of tests in different cultures,
Richard Haier (15:03.880)
you get the same thing.
Lex Fridman (15:05.800)
So this says something interesting about the human mind
Richard Haier (15:08.880)
that as a computer is designed to be general.
Lex Fridman (15:12.780)
So that means you can, so it's not easily made specialized.
Richard Haier (15:17.780)
Meaning if you're going to be good at one thing,
Lex Fridman (15:21.940)
Miyamoto Musashi has this quote, he's an ancient warrior,
Richard Haier (15:26.940)
famous for the Book of Five Rings in the martial arts world.
Lex Fridman (15:30.940)
And the quote goes, if you know the way broadly,
Richard Haier (15:34.180)
you will see it in everything.
Lex Fridman (15:36.420)
Meaning if you do one thing is going to generalize
Richard Haier (15:42.660)
to everything.
Lex Fridman (15:44.500)
And that's an interesting quote.
Lex Fridman (15:46.620)
And that's an interesting thing about the human mind.
Lex Fridman (15:50.660)
So that's what the G factor reveals.
Richard Haier (15:54.380)
Okay, so what's the difference,
Lex Fridman (15:57.060)
if you can elaborate a little bit further
Lex Fridman (15:58.900)
between IQ and G factor?
Lex Fridman (16:00.900)
Just because it's a source of confusion for people.
Lex Fridman (16:03.560)
And IQ is a score.
Lex Fridman (16:05.660)
People use the word IQ to mean intelligence.
Lex Fridman (16:08.260)
But IQ has a more technical meaning
Lex Fridman (16:11.020)
for people who work in the field.
Lex Fridman (16:12.740)
And it's an IQ score, a score on a test
Lex Fridman (16:16.540)
that estimates the G factor.
Lex Fridman (16:20.060)
And the G factor is what's common
Lex Fridman (16:22.020)
among all these tests of mental ability.
Lex Fridman (16:24.200)
So if you think about, it's not a Venn diagram,
Lex Fridman (16:27.040)
but I guess you could make a Venn diagram out of it,
Lex Fridman (16:30.580)
but the G factor would be really at the core,
Lex Fridman (16:33.900)
what's common to everything.
Lex Fridman (16:37.540)
And what IQ scores do is they allow a rank order
Lex Fridman (16:42.540)
of people on the score.
Lex Fridman (16:44.500)
And this is what makes people uncomfortable.
Lex Fridman (16:46.900)
This is where there's a lot of controversy
Richard Haier (16:48.920)
about whether IQ tests are biased
Lex Fridman (16:51.540)
toward any one group or another.
Lex Fridman (16:54.380)
And a lot of the answers to these questions are very clear,
Lex Fridman (16:59.060)
but they also have a technical aspect of it
Richard Haier (17:02.060)
that's not so easy to explain.
Lex Fridman (17:04.180)
Well, we'll talk about the fascinating
Lex Fridman (17:06.180)
and the difficult things about all of this.
Lex Fridman (17:10.700)
So by the way, when you say rank order,
Richard Haier (17:12.560)
that means you get a number and that means one person,
Lex Fridman (17:15.420)
you can now compare.
Richard Haier (17:17.540)
Like you could say that this other person
Lex Fridman (17:20.900)
is more intelligent than me.
Richard Haier (17:23.020)
Well, what you can say is IQ scores
Lex Fridman (17:25.820)
are interpreted really as percentiles.
Lex Fridman (17:29.220)
So that if you have an IQ of 140
Lex Fridman (17:33.300)
and somebody else has 70,
Richard Haier (17:35.580)
the metric is such that you cannot say
Lex Fridman (17:38.140)
the person with an IQ of 140 is twice as smart
Richard Haier (17:42.940)
as a person with an IQ of 70.
Lex Fridman (17:45.980)
That would require a ratio scale with an absolute zero.
Richard Haier (17:49.900)
Now you may think you know people with zero intelligence,
Lex Fridman (17:53.140)
but in fact, there is no absolute zero on an IQ scale.
Richard Haier (17:57.980)
It's relative to other people.
Lex Fridman (18:01.140)
So relative to other people,
Richard Haier (18:03.220)
somebody with an IQ score of 140
Lex Fridman (18:06.300)
is in the upper less than 1%,
Richard Haier (18:09.260)
whereas somebody with an IQ of 70
Lex Fridman (18:12.580)
is two standard deviations below the mean.
Richard Haier (18:15.500)
That's a different percentile.
Lex Fridman (18:18.660)
So it's similar to like in chess,
Richard Haier (18:20.920)
you have an ELO rating that's designed to rank order people.
Lex Fridman (18:27.740)
So you can't say it's twice one person.
Richard Haier (18:30.500)
If your ELO rating is twice another person,
Lex Fridman (18:33.500)
I don't think you're twice as good at chess.
Richard Haier (18:35.740)
It's not stable in that way,
Lex Fridman (18:37.580)
but because it's very difficult
Richard Haier (18:39.060)
to do these kinds of comparisons.
Lex Fridman (18:41.440)
But so what can we say about the number itself?
Lex Fridman (18:47.020)
Is that stable across tests and so on, or no?
Lex Fridman (18:50.500)
There are a number of statistical properties of any test.
Richard Haier (18:54.060)
They're called psychometric properties.
Lex Fridman (18:56.460)
You have validity, you have reliability,
Richard Haier (18:59.340)
reliability, there are many different kinds of reliability.
Lex Fridman (19:02.780)
They all essentially measure stability.
Lex Fridman (19:05.980)
And IQ tests are stable within an individual.
Lex Fridman (19:09.780)
There are some longitudinal studies
Richard Haier (19:11.940)
where children were measured at age 11.
Lex Fridman (19:15.840)
And again, when they were 70 years old
Lex Fridman (19:18.140)
and the two IQ scores are highly correlated with each other.
Lex Fridman (19:22.140)
This comes from a fascinating study from Scotland.
Richard Haier (19:26.260)
In the 1930s, some researchers decided to get an IQ test
Lex Fridman (19:31.260)
on every single child age 11 in the whole country.
Lex Fridman (19:35.900)
And they did.
Lex Fridman (19:37.340)
And those records were discovered in an old storeroom
Richard Haier (19:42.980)
at the University of Edinburgh by a friend of mine,
Lex Fridman (19:47.020)
Ian Deary, who found the records, digitized them,
Lex Fridman (19:52.180)
and has done a lot of research
Lex Fridman (19:53.940)
on the people who are still alive today
Richard Haier (19:57.220)
from that original study,
Lex Fridman (19:58.540)
including brain imaging research, by the way.
Richard Haier (1:00:03.000)
The film was called Charlie
Lex Fridman (1:00:04.800)
for the younger people who are listening to this.
Richard Haier (1:00:08.680)
You might be able to stream it on Netflix or something,
Lex Fridman (1:00:11.780)
but it was a story about a person
Richard Haier (1:00:16.780)
with very low IQ who underwent a surgical procedure
Lex Fridman (1:00:20.620)
in the brain and he slowly became a genius.
Lex Fridman (1:00:24.860)
And the tragedy of the story is the effect was temporary.
Lex Fridman (1:00:31.180)
It's a fascinating story really.
Richard Haier (1:00:33.120)
That goes in contrast to the basic human experience
Lex Fridman (1:00:36.580)
that each of us individually have,
Lex Fridman (1:00:38.420)
but it raises the question of the full range of people
Lex Fridman (1:00:43.420)
you might be able to be given different levels
Richard Haier (1:00:47.700)
of intelligence.
Lex Fridman (1:00:48.780)
You've mentioned the normal distribution.
Lex Fridman (1:00:52.920)
So let's talk about it.
Lex Fridman (1:00:54.520)
There's a book called The Bell Curve written in 1994,
Richard Haier (1:00:58.360)
written by psychologist Richard Herrnstein
Lex Fridman (1:01:01.340)
and political scientist Charles Murray.
Lex Fridman (1:01:04.500)
Why was this book so controversial?
Lex Fridman (1:01:08.540)
This is a fascinating book.
Richard Haier (1:01:10.720)
I know Charles Murray.
Lex Fridman (1:01:12.620)
I've had many conversations with him.
Lex Fridman (1:01:15.340)
Yeah, what is the book about?
Lex Fridman (1:01:16.660)
The book is about the importance of intelligence
Richard Haier (1:01:22.720)
in everyday life.
Lex Fridman (1:01:25.420)
That's what the book is about.
Richard Haier (1:01:27.520)
It's an empirical book.
Lex Fridman (1:01:29.220)
It has statistical analyses of very large databases
Richard Haier (1:01:34.180)
that show that essentially IQ scores or their equivalent
Lex Fridman (1:01:39.180)
are correlated to all kinds of social problems
Lex Fridman (1:01:44.640)
and social benefits.
Lex Fridman (1:01:46.800)
And that in itself is not where the controversy
Richard Haier (1:01:51.740)
about that book came.
Lex Fridman (1:01:53.640)
The controversy was about one chapter in that book.
Lex Fridman (1:01:57.600)
And that is a chapter about the average difference
Lex Fridman (1:02:02.240)
in mean scores between black Americans and white Americans.
Lex Fridman (1:02:06.720)
And these are the terms that were used in the book
Lex Fridman (1:02:08.960)
at the time and are still used to some extent.
Lex Fridman (1:02:14.080)
And historically, or really for decades,
Lex Fridman (1:02:21.240)
it has been observed that disadvantaged groups
Richard Haier (1:02:28.020)
score on average lower than Caucasians
Lex Fridman (1:02:35.480)
on academic tests, tests of mental ability,
Lex Fridman (1:02:38.280)
and especially on IQ tests.
Lex Fridman (1:02:40.440)
And the difference is about a standard deviation,
Richard Haier (1:02:43.120)
which is about 15 points, which is a substantial difference.
Lex Fridman (1:02:48.920)
In the book, Herrnstein and Murray in this one chapter
Richard Haier (1:02:54.160)
assert clearly and unambiguously
Lex Fridman (1:02:59.200)
that whether this average difference
Richard Haier (1:03:01.960)
is due to genetics or not, they are agnostic.
Lex Fridman (1:03:06.960)
They don't know.
Richard Haier (1:03:08.280)
Moreover, they assert they don't care
Lex Fridman (1:03:11.160)
because you wouldn't treat anybody differently
Richard Haier (1:03:13.880)
knowing if there was a genetic component or not
Lex Fridman (1:03:17.640)
because that's a group average finding.
Richard Haier (1:03:20.560)
Every individual has to be treated as an individual.
Lex Fridman (1:03:23.960)
You can't make any assumption
Richard Haier (1:03:26.040)
about what that person's intellectual ability might be
Lex Fridman (1:03:30.640)
from the fact of a average group difference.
Richard Haier (1:03:33.160)
They're very clear about this.
Lex Fridman (1:03:34.880)
Nonetheless, people took away,
Richard Haier (1:03:41.680)
I'm gonna choose my words carefully
Lex Fridman (1:03:43.240)
because I have a feeling that many critics
Richard Haier (1:03:44.800)
didn't actually read the book.
Lex Fridman (1:03:49.160)
They took away that Herrnstein and Murray were saying
Richard Haier (1:03:51.880)
that blacks are genetically inferior.
Lex Fridman (1:03:54.960)
That was the take home message.
Lex Fridman (1:03:56.560)
And if they weren't saying it, they were implying it
Lex Fridman (1:03:59.680)
because they had a chapter that discussed
Richard Haier (1:04:02.720)
this empirical observation of a difference.
Lex Fridman (1:04:07.000)
And isn't this horrible?
Lex Fridman (1:04:10.560)
And so the reaction to that book was incendiary.
Lex Fridman (1:04:18.400)
What do we know about from that book
Lex Fridman (1:04:22.560)
and the research beyond about race differences
Lex Fridman (1:04:28.960)
and intelligence?
Richard Haier (1:04:30.620)
It's still the most incendiary topic in psychology.
Lex Fridman (1:04:33.800)
Nothing has changed that.
Richard Haier (1:04:35.880)
Anybody who even discusses it is easily called a racist
Lex Fridman (1:04:41.200)
just for discussing it.
Richard Haier (1:04:43.040)
It's become fashionable to find racism
Lex Fridman (1:04:45.560)
in any discussion like this.
Richard Haier (1:04:49.960)
It's unfortunate.
Lex Fridman (1:04:53.560)
The short answer to your question is
Richard Haier (1:04:57.080)
there's been very little actual research
Lex Fridman (1:05:00.160)
on this topic since 19...
Richard Haier (1:05:03.840)
Since the bell curve.
Lex Fridman (1:05:05.200)
Since the bell curve, even before.
Richard Haier (1:05:07.920)
This really became incendiary in 1969
Lex Fridman (1:05:12.460)
with an article published by an educational psychologist
Richard Haier (1:05:15.760)
named Arthur Jensen.
Lex Fridman (1:05:17.760)
Let's just take a minute and go back to that
Richard Haier (1:05:20.520)
to see the bell curve in a little bit more
Lex Fridman (1:05:22.920)
historical perspective.
Richard Haier (1:05:25.040)
Arthur Jensen was a educational psychologist
Lex Fridman (1:05:28.160)
at UC Berkeley.
Richard Haier (1:05:29.320)
I knew him as well.
Lex Fridman (1:05:31.440)
And in 1969 or 68, the Harvard Educational Review
Richard Haier (1:05:37.800)
asked him to do a review article
Lex Fridman (1:05:42.640)
on the early childhood education programs
Richard Haier (1:05:47.560)
that were designed to raise the IQs of minority students.
Lex Fridman (1:05:54.800)
This was before the federally funded Head Start program.
Richard Haier (1:05:58.560)
Head Start had not really gotten underway
Lex Fridman (1:06:01.280)
at the time Jensen undertook his review
Richard Haier (1:06:04.440)
of what were a number of demonstration programs.
Lex Fridman (1:06:08.640)
And these demonstration programs were for young children
Richard Haier (1:06:13.920)
who were around kindergarten age.
Lex Fridman (1:06:15.720)
And they were specially designed to be
Richard Haier (1:06:18.200)
cognitively stimulating, to provide lunches,
Lex Fridman (1:06:23.020)
do all the things that people thought would
Richard Haier (1:06:27.200)
minimize this average gap of intelligence tests.
Lex Fridman (1:06:31.800)
There was a strong belief among virtually all psychologists
Richard Haier (1:06:37.200)
that the cause of the gap was unequal opportunity
Lex Fridman (1:06:40.760)
due to racism, due to all negative things in the society.
Lex Fridman (1:06:45.600)
And if you could compensate for this, the gap would go away.
Lex Fridman (1:06:51.200)
So early childhood education back then was called
Richard Haier (1:06:54.740)
literally compensatory education.
Lex Fridman (1:06:58.640)
Jensen looked at these programs.
Richard Haier (1:07:00.920)
He was an empirical guy.
Lex Fridman (1:07:02.200)
He understood psychometrics.
Lex Fridman (1:07:04.600)
And he wrote a, it was over a hundred page article
Lex Fridman (1:07:08.800)
detailing these programs
Lex Fridman (1:07:12.120)
and the flaws in their research design.
Lex Fridman (1:07:15.360)
Some of the programs reported IQ gains
Richard Haier (1:07:17.640)
of on average five points,
Lex Fridman (1:07:20.400)
but a few reported 10, 20 and even 30 point gains.
Richard Haier (1:07:24.720)
One was called the miracle in Milwaukee.
Lex Fridman (1:07:28.000)
That investigator went to jail ultimately
Richard Haier (1:07:30.440)
for fabricating data.
Lex Fridman (1:07:33.280)
But the point is that Jensen wrote an article that said,
Richard Haier (1:07:36.000)
look, the opening sentence of his article is classic.
Lex Fridman (1:07:40.000)
The opening sentence is, I may not quote it exactly right,
Lex Fridman (1:07:43.520)
but it's, we have tried compensatory education
Lex Fridman (1:07:47.120)
and it has failed.
Lex Fridman (1:07:48.420)
And he showed that these gains were essentially nothing.
Lex Fridman (1:07:54.140)
You couldn't really document empirically any gains at all
Richard Haier (1:07:58.660)
from these really earnest efforts to increase IQ.
Lex Fridman (1:08:03.340)
But he went a step further, a fateful step further.
Richard Haier (1:08:08.060)
He said, not only have these efforts failed,
Lex Fridman (1:08:11.740)
but because they have had essentially no impact,
Richard Haier (1:08:15.220)
we have to reexamine our assumption
Lex Fridman (1:08:17.660)
that these differences are caused by environmental things
Richard Haier (1:08:22.100)
that we can address with education.
Lex Fridman (1:08:24.420)
We need to consider a genetic influence,
Richard Haier (1:08:28.500)
whether there's a genetic influence
Lex Fridman (1:08:30.580)
on this group difference.
Lex Fridman (1:08:32.420)
So you said that this is one of the more controversial works
Lex Fridman (1:08:36.060)
ever in science. I think it's the most infamous paper
Richard Haier (1:08:37.900)
in all of psychology, I would go on to say.
Lex Fridman (1:08:41.680)
Because in 1969, the genetic data was very skimpy
Richard Haier (1:08:46.680)
on this question, skimpy and controversial.
Lex Fridman (1:08:49.640)
It's always been controversial,
Lex Fridman (1:08:50.820)
but it was even skimpy and controversial.
Lex Fridman (1:08:53.820)
It's kind of a long story that I go into a little bit
Richard Haier (1:08:56.480)
in more detail in the book, Neuroscience of Intelligence.
Lex Fridman (1:09:02.200)
But to say he was vilified is an understatement.
Richard Haier (1:09:06.320)
I mean, he couldn't talk at the American
Lex Fridman (1:09:08.880)
Psychological Association without bomb threats
Richard Haier (1:09:13.140)
clearing the lecture hall.
Lex Fridman (1:09:15.240)
Campus security watched him all the time.
Richard Haier (1:09:18.120)
They opened his mail.
Lex Fridman (1:09:20.320)
He had to retreat to a different address.
Richard Haier (1:09:24.680)
This was one of the earliest kinds,
Lex Fridman (1:09:28.760)
this is before the internet
Lex Fridman (1:09:30.500)
and kind of internet social media mobs.
Lex Fridman (1:09:35.080)
But it was that intense.
Lex Fridman (1:09:38.120)
And I have written that overnight,
Lex Fridman (1:09:42.740)
after the publication of this article,
Richard Haier (1:09:45.960)
all intelligence research became radioactive.
Lex Fridman (1:09:49.760)
Nobody wanted to talk about it.
Richard Haier (1:09:56.200)
Nobody was doing more research.
Lex Fridman (1:09:58.920)
And then the bell curve came along.
Lex Fridman (1:10:02.080)
And the Jensen controversy was dying down.
Lex Fridman (1:10:05.720)
I have stories that Jensen told me about his interaction
Richard Haier (1:10:08.960)
with the Nixon White House on this issue.
Lex Fridman (1:10:10.920)
I mean, this was like a really big deal.
Richard Haier (1:10:14.360)
It was some unbelievable stories,
Lex Fridman (1:10:16.160)
but he told me this, so I kind of believe these stories.
Richard Haier (1:10:20.100)
Nonetheless.
Lex Fridman (1:10:21.640)
25 years later.
Richard Haier (1:10:22.920)
25 years later.
Lex Fridman (1:10:24.240)
All this silence, basically, saying,
Richard Haier (1:10:30.040)
nobody wants to do this kind of research.
Lex Fridman (1:10:32.280)
There's so much pressure, so much attack
Richard Haier (1:10:34.620)
against this kind of research.
Lex Fridman (1:10:36.360)
And here's sort of a bold, stupid, crazy people
Richard Haier (1:10:41.960)
that decide to dive right back in.
Lex Fridman (1:10:44.880)
I wonder how much discussion that was.
Lex Fridman (1:10:46.520)
Do we include this chapter or not?
Lex Fridman (1:10:48.720)
Murray has said they discussed it,
Lex Fridman (1:10:51.100)
and they felt they should include it.
Lex Fridman (1:10:55.640)
And they were very careful in the way they wrote it,
Richard Haier (1:10:59.580)
which did them no good.
Lex Fridman (1:11:01.040)
So, as a matter of fact, when the bell curve came out,
Richard Haier (1:11:06.560)
it was so controversial.
Lex Fridman (1:11:08.480)
I got a call from a television show called Nightline.
Richard Haier (1:11:13.320)
It was with a broadcaster called Ted Koppel.
Lex Fridman (1:11:16.840)
We had this evening show, I think it was on late at night.
Richard Haier (1:11:20.600)
Talked about news.
Lex Fridman (1:11:21.600)
It was a straight up news thing.
Lex Fridman (1:11:24.240)
And a producer called and asked if I would be on it
Lex Fridman (1:11:28.200)
to talk about the bell curve.
Lex Fridman (1:11:31.680)
And I said, she asked me what I thought
Lex Fridman (1:11:35.000)
about the bell curve as a book.
Lex Fridman (1:11:36.640)
And I said, look, it's a very good book.
Lex Fridman (1:11:38.560)
It talks about the role of intelligence in society.
Lex Fridman (1:11:43.080)
And she said, no, no, what do you think
Lex Fridman (1:11:44.980)
about the chapter on race?
Richard Haier (1:11:47.160)
That's what we want you to talk about.
Lex Fridman (1:11:49.940)
I remember this conversation.
Richard Haier (1:11:52.400)
I said, well, she said, what would you say
Lex Fridman (1:11:56.600)
if you were on TV?
Lex Fridman (1:11:58.800)
And I said, well, what I would say is that
Lex Fridman (1:12:02.240)
it's not at all clear if there's any genetic component
Richard Haier (1:12:07.760)
to intelligence, any differences.
Lex Fridman (1:12:13.600)
But if there were a strong genetic component,
Richard Haier (1:12:17.440)
that would be a good thing.
Lex Fridman (1:12:21.140)
And complete silence on the other end of the phone.
Lex Fridman (1:12:25.280)
And she said, well, what do you mean?
Lex Fridman (1:12:28.180)
And I said, well, if it's the more genetic
Richard Haier (1:12:31.160)
any difference is, the more it's biological.
Lex Fridman (1:12:35.080)
And if it's biological, we can figure out how to fix it.
Richard Haier (1:12:39.960)
I see, that's interesting.
Lex Fridman (1:12:41.480)
She said, would you say that on television?
Richard Haier (1:12:43.680)
Yes.
Lex Fridman (1:12:44.520)
And I said, no.
Lex Fridman (1:12:45.340)
And so that was the end of that.
Lex Fridman (1:12:47.400)
So that's for more like biology is within the reach
Richard Haier (1:12:56.400)
of science and the environment is a public policy,
Lex Fridman (1:13:02.000)
is social and all those kinds of things.
Richard Haier (1:13:05.500)
From your perspective, whichever one you think
Lex Fridman (1:13:09.120)
is more amenable to solutions in the short term
Richard Haier (1:13:11.760)
is the one that excites you.
Lex Fridman (1:13:13.560)
But you saying that is good, the truth of genetic differences,
Richard Haier (1:13:22.900)
no matter what, between groups is a painful, harmful,
Lex Fridman (1:13:32.680)
potentially dangerous thing.
Lex Fridman (1:13:35.320)
So let me ask you to this question,
Lex Fridman (1:13:38.080)
whether it's bell curve or any research
Richard Haier (1:13:40.160)
on race differences, can that be used to increase
Lex Fridman (1:13:47.080)
the amount of racism in the world?
Richard Haier (1:13:49.400)
Can that be used to increase the amount of hate
Lex Fridman (1:13:51.680)
in the world?
Lex Fridman (1:13:52.820)
Do you think about this kind of stuff?
Lex Fridman (1:13:54.400)
I've thought about this a lot, not as a scientist,
Lex Fridman (1:13:57.400)
but as a person.
Lex Fridman (1:14:00.680)
And my sense is there is such enormous reservoirs
Richard Haier (1:14:05.680)
of hate and racism that have nothing to do
Lex Fridman (1:14:13.040)
with scientific knowledge of the data,
Richard Haier (1:14:16.600)
that speak against that.
Lex Fridman (1:14:19.200)
That no, I don't wanna give racist groups a veto power
Richard Haier (1:14:25.360)
over what scientists study.
Lex Fridman (1:14:27.680)
If you think that the differences, and by the way,
Richard Haier (1:14:31.440)
virtually no one disagrees that there are differences
Lex Fridman (1:14:34.360)
in scores, it's all about what causes them
Lex Fridman (1:14:37.200)
and how to fix it.
Lex Fridman (1:14:39.400)
So if you think this is a cultural problem,
Richard Haier (1:14:42.920)
then you must ask the problem,
Lex Fridman (1:14:44.840)
what do you want to change anything about the culture?
Lex Fridman (1:14:49.360)
Or are you okay with the culture?
Lex Fridman (1:14:51.800)
Cause you don't feel it's appropriate
Richard Haier (1:14:53.380)
to change a person's culture.
Lex Fridman (1:14:55.600)
So are you okay with that?
Lex Fridman (1:14:57.160)
And the fact that that may lead to disadvantages
Lex Fridman (1:14:59.720)
in school achievement.
Richard Haier (1:15:01.600)
It's a question.
Lex Fridman (1:15:02.500)
If you think it's environmental,
Lex Fridman (1:15:05.940)
what are the environmental parameters that can be fixed?
Lex Fridman (1:15:10.380)
I'll tell you one, lead from gasoline in the atmosphere.
Richard Haier (1:15:15.860)
Lead in paint, lead in water.
Lex Fridman (1:15:18.780)
That's an environmental toxin that society
Richard Haier (1:15:22.340)
has the means to eliminate, and they should.
Lex Fridman (1:15:25.980)
Yeah, just to sort of try and define some insights
Lex Fridman (1:15:30.260)
and conclusion to this very difficult topic.
Lex Fridman (1:15:33.860)
Is there been research on environment versus genetics,
Richard Haier (1:15:38.740)
nature versus nurture, on this question
Lex Fridman (1:15:40.420)
of race differences?
Richard Haier (1:15:43.100)
There is not, no one wants to do this research.
Lex Fridman (1:15:46.380)
First of all, it's hard research to do.
Richard Haier (1:15:48.080)
Second of all, it's a minefield.
Lex Fridman (1:15:50.620)
No one wants to spend their career on it.
Richard Haier (1:15:52.940)
Tenured people don't want to do it, let alone students.
Lex Fridman (1:15:56.320)
The way I talk about it,
Richard Haier (1:16:00.600)
well, before I tell you the way I talk about it,
Lex Fridman (1:16:02.440)
I want to say one more thing about Jensen.
Richard Haier (1:16:05.640)
He was once asked by a journalist straight out,
Lex Fridman (1:16:08.120)
are you a racist?
Richard Haier (1:16:10.480)
His answer was very interesting.
Lex Fridman (1:16:12.520)
His answer was, I've thought about that a lot,
Lex Fridman (1:16:16.960)
and I've concluded it doesn't matter.
Lex Fridman (1:16:22.040)
Now, I know what he meant by this.
Richard Haier (1:16:23.640)
The guts to say that, wow.
Lex Fridman (1:16:25.840)
He was a very unusual person.
Richard Haier (1:16:27.560)
I think he had a touch of Asperger's syndrome,
Lex Fridman (1:16:29.920)
to tell you the truth,
Richard Haier (1:16:30.940)
because I saw him in many circumstances.
Lex Fridman (1:16:34.160)
He would be canceled on Twitter immediately
Richard Haier (1:16:36.140)
with that sentence.
Lex Fridman (1:16:37.520)
But what he meant was he had a hypothesis,
Lex Fridman (1:16:42.440)
and with respect to group differences,
Lex Fridman (1:16:44.920)
he called it the default hypothesis.
Richard Haier (1:16:47.400)
He said, whatever factors affect individual intelligence
Lex Fridman (1:16:51.240)
are likely the same factors that affect group differences.
Richard Haier (1:16:54.320)
It was the default.
Lex Fridman (1:16:55.600)
But it was a hypothesis.
Richard Haier (1:16:58.280)
It should be tested, and if it turned out
Lex Fridman (1:17:01.040)
empirical tests didn't support the hypothesis,
Richard Haier (1:17:03.680)
he was happy to move on to something else.
Lex Fridman (1:17:06.480)
He was absolutely committed to that scientific ideal,
Richard Haier (1:17:12.480)
that it's an empirical question,
Lex Fridman (1:17:16.040)
we should look at it, and let's see what happens.
Richard Haier (1:17:18.800)
The scientific method cannot be racist,
Lex Fridman (1:17:22.080)
from his perspective.
Richard Haier (1:17:23.480)
It doesn't matter what the scientists,
Lex Fridman (1:17:26.240)
if they follow the scientific method,
Richard Haier (1:17:30.440)
it doesn't matter what they believe.
Lex Fridman (1:17:32.080)
And if they are biased, and they consciously
Richard Haier (1:17:35.520)
or unconsciously bias the data,
Lex Fridman (1:17:39.520)
other people will come along to replicate it,
Richard Haier (1:17:42.680)
they will fail, and the process over time will work.
Lex Fridman (1:17:48.200)
So let me push back on this idea.
Richard Haier (1:17:50.980)
Because psychology to me is full of gray areas.
Lex Fridman (1:17:57.380)
And what I've observed about psychology,
Richard Haier (1:18:01.460)
even replication crisis aside,
Lex Fridman (1:18:04.060)
is that something about the media,
Richard Haier (1:18:06.220)
something about journalism,
Lex Fridman (1:18:08.020)
something about the virality of ideas in the public sphere,
Richard Haier (1:18:13.700)
they misinterpret, they take up things from studies,
Lex Fridman (1:18:18.360)
willfully or from ignorance, misinterpret findings,
Lex Fridman (1:18:23.640)
and tell narratives around that.
Lex Fridman (1:18:27.120)
I personally believe, for me,
Richard Haier (1:18:29.820)
I'm not saying that broadly about science,
Lex Fridman (1:18:31.520)
but for me, it's my responsibility to anticipate
Richard Haier (1:18:35.760)
the ways in which findings will be misinterpreted.
Lex Fridman (1:18:40.420)
So I thought about this a lot,
Richard Haier (1:18:42.840)
because I published papers on semi autonomous vehicles,
Lex Fridman (1:18:47.020)
and those cars, people die in cars.
Richard Haier (1:18:52.960)
There's people that have written me letters saying emails,
Lex Fridman (1:18:57.200)
nobody writes letters, I wish they did,
Richard Haier (1:19:00.000)
that have blood on my hands,
Lex Fridman (1:19:01.800)
because of things that I would say positive or negative,
Richard Haier (1:19:04.800)
there's consequences.
Lex Fridman (1:19:06.200)
In the same way, when you're a researcher of intelligence,
Richard Haier (1:19:09.260)
I'm sure you might get emails,
Lex Fridman (1:19:12.040)
or at least people might believe
Richard Haier (1:19:14.500)
that a finding of your study is going to be used
Lex Fridman (1:19:17.920)
by a large number of people
Richard Haier (1:19:19.080)
to increase the amount of hate in the world.
Lex Fridman (1:19:22.640)
I think there's some responsibility on scientists,
Lex Fridman (1:19:26.060)
but for me, I think there's a great responsibility
Lex Fridman (1:19:30.480)
to anticipate the ways things will be misinterpreted,
Lex Fridman (1:19:35.400)
and there, you have to, first of all,
Lex Fridman (1:19:37.880)
decide whether you want to say a thing at all,
Richard Haier (1:19:40.980)
do the study at all, publish the study at all,
Lex Fridman (1:19:43.440)
and two, the words with which you explain it.
Richard Haier (1:19:49.000)
I find this on Twitter a lot, actually,
Lex Fridman (1:19:50.640)
which is, when I write a tweet,
Lex Fridman (1:19:53.200)
and I'm usually just doing it so innocently,
Lex Fridman (1:19:58.560)
I'll write it, it takes me five seconds to write it,
Richard Haier (1:20:02.040)
or whatever, 30 seconds to write it,
Lex Fridman (1:20:04.320)
and then I'll think, all right, I close my eyes open,
Lex Fridman (1:20:08.240)
and try to see how will the world interpret this,
Lex Fridman (1:20:11.680)
what are the ways in which this will be misinterpreted,
Lex Fridman (1:20:14.400)
and I'll sometimes adjust that tweet to see,
Lex Fridman (1:20:18.480)
yeah, so in my mind, it's clear,
Lex Fridman (1:20:20.940)
but that's because it's my mind from which this tweet came,
Lex Fridman (1:20:24.040)
but you have to think, in a fresh mind that sees this,
Lex Fridman (1:20:28.160)
and it's spread across a large number of other minds,
Lex Fridman (1:20:32.840)
how will the interpretation morph?
Richard Haier (1:20:36.040)
I mean, for a tweet, it's a silly thing, it doesn't matter,
Lex Fridman (1:20:38.260)
but for a scientific paper and study and finding,
Richard Haier (1:20:45.640)
I think it matters.
Lex Fridman (1:20:47.240)
So I don't know what your thoughts are on that,
Richard Haier (1:20:49.720)
because maybe for Jensen, the data's there,
Lex Fridman (1:20:54.600)
what do you want me to do?
Richard Haier (1:20:55.600)
This is a scientific process that's been carried out,
Lex Fridman (1:20:59.160)
if you think the data was polluted by bias,
Richard Haier (1:21:02.200)
do other studies that reveal the bias,
Lex Fridman (1:21:05.800)
but the data's there.
Lex Fridman (1:21:07.440)
And I'm not a poet, I'm not a literary writer,
Lex Fridman (1:21:14.720)
what do you want me to do?
Richard Haier (1:21:15.640)
I'm just presenting you the data.
Lex Fridman (1:21:17.560)
What do you think on that spectrum?
Lex Fridman (1:21:19.360)
What's the role of a scientist?
Lex Fridman (1:21:21.120)
The reason I do podcasts,
Richard Haier (1:21:23.540)
the reason I write books for the public
Lex Fridman (1:21:27.280)
is to explain what I think the data mean
Lex Fridman (1:21:30.420)
and what I think the data don't mean.
Lex Fridman (1:21:32.680)
I don't do very much on Twitter other than to retweet
Richard Haier (1:21:38.240)
references to papers.
Lex Fridman (1:21:39.920)
I don't think it's my role to explain these,
Richard Haier (1:21:42.440)
because they're complicated, they're nuanced.
Lex Fridman (1:21:46.480)
But when you decide not to do a scientific study
Richard Haier (1:21:51.560)
because you're, or not to publish a result
Lex Fridman (1:21:54.360)
because you're afraid the result could be harmful
Richard Haier (1:21:59.360)
or insensitive, that's not an unreasonable thought.
Lex Fridman (1:22:05.080)
And people will make different conclusions
Lex Fridman (1:22:09.200)
and decisions about that.
Lex Fridman (1:22:11.560)
I wrote about this, I'm the editor
Richard Haier (1:22:14.680)
of a journal called Intelligence,
Lex Fridman (1:22:17.000)
which publishes scientific papers.
Richard Haier (1:22:20.560)
Sometimes we publish papers on group differences.
Lex Fridman (1:22:24.200)
Those papers sometimes are controversial.
Richard Haier (1:22:27.040)
These papers are written for a scientific audience.
Lex Fridman (1:22:29.600)
They're not written for the Twitter audience.
Richard Haier (1:22:32.160)
I don't promote them very much on Twitter.
Lex Fridman (1:22:37.480)
But in a scientific paper,
Richard Haier (1:22:41.240)
you have to now choose your words carefully also,
Lex Fridman (1:22:44.600)
because those papers are picked up by non scientists,
Richard Haier (1:22:49.480)
by writers of various kinds,
Lex Fridman (1:22:52.000)
and you have to be available to discuss what you're saying
Lex Fridman (1:22:56.120)
and what you're not saying.
Lex Fridman (1:22:58.720)
Sometimes you are successful at having a good conversation
Richard Haier (1:23:04.640)
like we are today, that doesn't start out pejorative.
Lex Fridman (1:23:09.200)
Other times I've been asked to participate in debates
Richard Haier (1:23:12.520)
where my role would be to justify race science.
Lex Fridman (1:23:16.500)
Well, you can see you start out.
Richard Haier (1:23:21.280)
That was a BBC request that I received.
Lex Fridman (1:23:25.680)
I have so much, it's a love hate relationship,
Richard Haier (1:23:28.280)
mostly hate with these shallow journalism organizations.
Lex Fridman (1:23:33.000)
So they would want to use you
Richard Haier (1:23:36.240)
as a kind of in a debate setting to communicate
Lex Fridman (1:23:39.180)
as to like there is raised differences between groups
Lex Fridman (1:23:42.300)
and make that into debate and put you in a role of...
Lex Fridman (1:23:47.880)
Justifying racism.
Richard Haier (1:23:49.280)
Justifying racism.
Lex Fridman (1:23:50.120)
That's what they're asking me to do.
Richard Haier (1:23:51.120)
Courses like educating about this field
Lex Fridman (1:23:54.360)
of the science of intelligence, yeah.
Richard Haier (1:23:56.040)
I wanna say one more thing
Lex Fridman (1:23:57.980)
before we get off the normal distribution.
Lex Fridman (1:24:01.320)
You also asked me what is the science after the bell curve?
Lex Fridman (1:24:06.160)
And the short answer is there's not much new work,
Lex Fridman (1:24:09.640)
but whatever work there is supports the idea
Lex Fridman (1:24:13.420)
that there still are group differences.
Richard Haier (1:24:16.280)
It's arguable whether those differences
Lex Fridman (1:24:18.560)
have diminished at all or not.
Lex Fridman (1:24:20.700)
And there is still a major problem
Lex Fridman (1:24:24.940)
in underperformance for school achievement
Richard Haier (1:24:29.100)
for many disadvantaged and minority students.
Lex Fridman (1:24:33.400)
And there so far is no way to fix it.
Lex Fridman (1:24:37.020)
What do we do with this information?
Lex Fridman (1:24:39.780)
Is this now a task?
Richard Haier (1:24:42.060)
Now we'll talk about the future
Lex Fridman (1:24:45.620)
on the neuroscience and the biology side,
Lex Fridman (1:24:47.700)
but in terms of this information as a society
Lex Fridman (1:24:51.100)
in the public policy, in the political space,
Lex Fridman (1:24:53.480)
in the social space, what do we do with this information?
Lex Fridman (1:24:56.260)
I've thought a lot about this.
Richard Haier (1:24:57.880)
The first step is to have people interested in policy
Lex Fridman (1:25:03.220)
understand what the data actually show
Richard Haier (1:25:06.940)
to pay attention to intelligence data.
Lex Fridman (1:25:09.800)
You can read policy papers about education
Lex Fridman (1:25:13.460)
and using your word processor,
Lex Fridman (1:25:15.840)
you can search for the word intelligence.
Richard Haier (1:25:17.860)
You can search a 20,000 word document in a second
Lex Fridman (1:25:22.100)
and find out the word intelligence does not appear anywhere.
Richard Haier (1:25:26.780)
In most discussions about what to do about achievement gaps,
Lex Fridman (1:25:32.500)
I'm not talking about test gaps,
Richard Haier (1:25:33.640)
I'm talking about actual achievement gaps in schools,
Lex Fridman (1:25:37.340)
which everyone agrees is a problem,
Richard Haier (1:25:40.140)
the word intelligence doesn't appear among educators.
Lex Fridman (1:25:43.940)
That's fascinating.
Richard Haier (1:25:44.780)
As a matter of fact, in California,
Lex Fridman (1:25:47.660)
there has been tremendous controversy
Richard Haier (1:25:50.020)
about recent attempts to revise the curriculum
Lex Fridman (1:25:53.100)
for math in high schools.
Lex Fridman (1:25:56.380)
And we had a Stanford professor of education
Lex Fridman (1:25:59.140)
who was running this review assert
Richard Haier (1:26:02.820)
there's no such thing as talent, mathematical talent.
Lex Fridman (1:26:07.860)
And she wanted to get rid of the advanced classes in math
Richard Haier (1:26:12.300)
because not everyone could do that.
Lex Fridman (1:26:15.340)
Now, of course, this has been very controversial,
Richard Haier (1:26:17.260)
they've retreated somewhat,
Lex Fridman (1:26:19.260)
but the idea that a university professor
Richard Haier (1:26:21.620)
was in charge of this who believes
Lex Fridman (1:26:26.140)
that there's no talent, that it doesn't exist,
Richard Haier (1:26:31.840)
this is rather shocking,
Lex Fridman (1:26:33.540)
let alone the complete absence of intelligence data.
Richard Haier (1:26:37.460)
By the way, let me tell you something
Lex Fridman (1:26:38.980)
about what the intelligence data show.
Richard Haier (1:26:41.980)
Let's take race out of it.
Lex Fridman (1:26:45.540)
Even though the origins of these studies
Richard Haier (1:26:48.260)
were a long time ago,
Lex Fridman (1:26:52.020)
I'm blocking on the name of the report,
Richard Haier (1:26:53.900)
the Coleman report was a famous report about education.
Lex Fridman (1:26:57.700)
And they measured all kinds of variables about schools,
Richard Haier (1:27:01.900)
about teachers,
Lex Fridman (1:27:03.460)
and they looked at academic achievement as an outcome.
Lex Fridman (1:27:08.100)
And they found the most predictive variables
Lex Fridman (1:27:12.660)
of education outcome were the variables
Richard Haier (1:27:15.980)
the student brought with him or her into the school,
Lex Fridman (1:27:20.340)
essentially their ability.
Lex Fridman (1:27:23.380)
And that when you combine the school
Lex Fridman (1:27:26.380)
and the teacher variables together,
Richard Haier (1:27:29.460)
the quality of the school, the funding of the school,
Lex Fridman (1:27:31.860)
the quality of the teachers, their education,
Richard Haier (1:27:34.620)
you put all the teacher and school variables together,
Lex Fridman (1:27:37.540)
it barely accounted for 10% of the variance.
Lex Fridman (1:27:41.660)
And this has been replicated now.
Lex Fridman (1:27:45.340)
So the best research we have shows that school variables
Lex Fridman (1:27:51.660)
and teacher variables together account
Lex Fridman (1:27:54.740)
for about 10% of student academic achievement.
Richard Haier (1:27:59.500)
Now, you wanna have some policy
Lex Fridman (1:28:02.060)
on improving academic achievement,
Lex Fridman (1:28:04.780)
how much money do you wanna put into teacher education?
Lex Fridman (1:28:08.340)
How much money do you wanna put into the quality
Lex Fridman (1:28:11.620)
of the school administration?
Lex Fridman (1:28:14.220)
You know who you can ask?
Richard Haier (1:28:15.340)
You can ask the Gates Foundation,
Lex Fridman (1:28:18.100)
because they spent a tremendous amount of money doing that.
Lex Fridman (1:28:21.500)
And at the end of it, because they're measurement people,
Lex Fridman (1:28:25.100)
they wanna know the data,
Richard Haier (1:28:27.540)
they found it had no impact at all.
Lex Fridman (1:28:29.660)
And they've kind of pulled out of that kind of program.
Richard Haier (1:28:33.460)
So, oh boy.
Lex Fridman (1:28:36.220)
Let me ask you, this is me talking, but there's...
Richard Haier (1:28:41.380)
Just the two of us.
Lex Fridman (1:28:42.660)
Just the two of us, but I'm gonna say
Richard Haier (1:28:44.900)
some funny and ridiculous things,
Lex Fridman (1:28:46.660)
so you're surely not approving of it.
Lex Fridman (1:28:51.260)
But there's a movie called Clerks.
Lex Fridman (1:28:53.940)
You probably...
Richard Haier (1:28:54.780)
I've seen it, I've seen it, yeah.
Lex Fridman (1:28:56.380)
There's a funny scene in there where a lovely couple
Richard Haier (1:28:59.860)
are talking about the number
Lex Fridman (1:29:01.460)
of previous sexual partners they had.
Lex Fridman (1:29:03.940)
And the woman says that,
Lex Fridman (1:29:07.820)
I believe she just had a handful,
Richard Haier (1:29:09.580)
like two or three or something like that sexual partners,
Lex Fridman (1:29:12.620)
but then she also mentioned that she...
Lex Fridman (1:29:17.860)
What's that called?
Lex Fridman (1:29:19.020)
Fallacia, what's the scientific?
Lex Fridman (1:29:20.380)
But she went, you know, gave a blow job
Lex Fridman (1:29:23.060)
to 37 guys, I believe it is.
Lex Fridman (1:29:26.180)
And so that has to do with the truth.
Lex Fridman (1:29:29.980)
So sometimes, knowing the truth
Richard Haier (1:29:35.940)
can get in the way of a successful relationship
Lex Fridman (1:29:39.540)
of love of some of the human flourishing.
Lex Fridman (1:29:43.020)
And that seems to me that's at the core here,
Lex Fridman (1:29:46.380)
that facing some kind of truth
Richard Haier (1:29:49.860)
that's not able to be changed
Lex Fridman (1:29:53.740)
makes it difficult to sort of...
Richard Haier (1:29:56.260)
Is limiting as opposed to empowering.
Lex Fridman (1:29:59.940)
That's the concern.
Richard Haier (1:30:01.300)
If you sort of test for intelligence
Lex Fridman (1:30:03.940)
and lay the data out,
Richard Haier (1:30:06.180)
it feels like you will give up on certain people.
Lex Fridman (1:30:09.140)
You will sort of start bidding people,
Richard Haier (1:30:12.580)
it's like, well, this person is like,
Lex Fridman (1:30:15.620)
let's focus on the average people
Richard Haier (1:30:18.060)
or let's focus on the very intelligent people.
Lex Fridman (1:30:20.060)
That's the concern.
Lex Fridman (1:30:21.500)
And there's a kind of intuition
Lex Fridman (1:30:26.700)
that if we just don't measure
Lex Fridman (1:30:29.580)
and we don't use that data,
Lex Fridman (1:30:31.660)
that we will treat everybody equal
Lex Fridman (1:30:33.620)
and give everybody equal opportunity.
Lex Fridman (1:30:37.860)
If we have the data in front of us,
Richard Haier (1:30:39.660)
we're likely to misdistribute
Lex Fridman (1:30:43.820)
the amount of sort of attention we allocate,
Richard Haier (1:30:46.340)
resources we allocate to people.
Lex Fridman (1:30:49.420)
That's probably the concern.
Richard Haier (1:30:52.180)
It's a realistic concern,
Lex Fridman (1:30:55.100)
but I think it's a misplaced concern
Richard Haier (1:30:57.660)
if you wanna fix the problem.
Lex Fridman (1:31:00.540)
If you wanna fix the problem,
Richard Haier (1:31:02.060)
you have to know what the problem is.
Lex Fridman (1:31:03.900)
Yep.
Richard Haier (1:31:05.140)
Now, let me tell you this,
Lex Fridman (1:31:06.820)
let's go back to the bell curve,
Richard Haier (1:31:08.860)
not the bell curve, but the normal distribution.
Lex Fridman (1:31:11.340)
Yes, 16% of the population on average has an IQ under 85,
Richard Haier (1:31:20.540)
which means they're very hard.
Lex Fridman (1:31:22.100)
If you have an IQ under 85,
Richard Haier (1:31:24.220)
it's very hard to find gainful employment
Lex Fridman (1:31:26.780)
at a salary that sustains you
Lex Fridman (1:31:31.260)
at least minimally in modern life, okay?
Lex Fridman (1:31:35.420)
Not impossible, but it's very difficult.
Richard Haier (1:31:37.540)
16% of the population of the United States
Lex Fridman (1:31:41.380)
is about 51 or 52 million people with IQs under 85.
Richard Haier (1:31:47.420)
This is not a small issue.
Lex Fridman (1:31:51.620)
14 million children have IQs under 85.
Lex Fridman (1:31:57.100)
Is this something we wanna ignore?
Lex Fridman (1:31:59.820)
Does this have any, what is the Venn diagram between,
Richard Haier (1:32:03.540)
you know, when you have people with IQs under 85,
Lex Fridman (1:32:07.140)
and you have achievement in school or achievement in life?
Richard Haier (1:32:13.540)
There's a lot of overlap there.
Lex Fridman (1:32:16.020)
This is why, to go back to the IQ pill,
Richard Haier (1:32:18.940)
if there were a way to shift that curve toward the higher end,
Lex Fridman (1:32:27.380)
that would have a big impact.
Richard Haier (1:32:30.980)
If I could maybe, before we talk about the impact on life
Lex Fridman (1:32:34.860)
and so on, some of the criticisms of the bell curve.
Lex Fridman (1:32:38.500)
So Steven Jay Gould wrote that the bell curve
Lex Fridman (1:32:41.940)
rests on four incorrect assumptions.
Richard Haier (1:32:44.860)
It would be just interesting to get your thoughts
Lex Fridman (1:32:46.900)
on the four assumptions, which are,
Richard Haier (1:32:48.940)
intelligence must be reducible to a single number,
Lex Fridman (1:32:51.980)
intelligence must be capable of rank ordering people
Richard Haier (1:32:54.740)
in a linear order,
Lex Fridman (1:32:56.100)
intelligence must be primarily genetically based,
Lex Fridman (1:32:59.860)
and intelligence must be essentially immutable.
Lex Fridman (1:33:03.860)
Maybe not as criticisms, but as thoughts about intelligence.
Richard Haier (1:33:09.740)
Oh yeah, we could spend a lot of time on him.
Lex Fridman (1:33:13.620)
On Steven Jay Gould?
Richard Haier (1:33:14.700)
Yes.
Lex Fridman (1:33:15.700)
He wrote that in what, about 1985, 1984?
Richard Haier (1:33:20.700)
His views were overtly political, not scientific.
Lex Fridman (1:33:25.980)
He was a scientist,
Lex Fridman (1:33:27.380)
but his views on this were overtly political,
Lex Fridman (1:33:30.700)
and I would encourage people listening to this,
Richard Haier (1:33:33.820)
if they really want to understand his criticisms,
Lex Fridman (1:33:38.940)
they should just Google what he had to say,
Lex Fridman (1:33:44.420)
and Google the scientific reviews of his book,
Lex Fridman (1:33:49.740)
The Mismeasure of Man,
Lex Fridman (1:33:51.500)
and they will take these statements apart.
Lex Fridman (1:33:54.740)
They were wrong, not only were they wrong,
Lex Fridman (1:33:57.940)
but when he asserted in his first book
Lex Fridman (1:34:00.380)
that there was no biological basis essentially to IQ,
Richard Haier (1:34:05.620)
by the time the second edition came around,
Lex Fridman (1:34:08.260)
there were studies of MRIs showing that brain size,
Richard Haier (1:34:13.980)
brain volume were correlated to IQ scores,
Lex Fridman (1:34:16.860)
which he declined to put in his book.
Lex Fridman (1:34:19.940)
So I'm learning a lot today.
Lex Fridman (1:34:21.900)
I didn't know actually the extent of his work.
Richard Haier (1:34:25.820)
I was just using a few little snippets of criticism.
Lex Fridman (1:34:28.940)
That's interesting.
Richard Haier (1:34:29.940)
There was a battle here.
Lex Fridman (1:34:30.940)
He wrote a book, Mismeasure of Man,
Richard Haier (1:34:32.820)
that's missing a lot of the scientific grounding.
Lex Fridman (1:34:36.220)
His book is highly popular in colleges today.
Richard Haier (1:34:39.700)
You can find it in any college bookstore
Lex Fridman (1:34:41.660)
under assigned reading.
Richard Haier (1:34:42.980)
It's highly popular.
Lex Fridman (1:34:44.220)
The Mismeasure of Man?
Richard Haier (1:34:45.220)
Yes, highly influential.
Lex Fridman (1:34:46.220)
Can you speak to the Mismeasure of Man?
Richard Haier (1:34:48.580)
I'm undereducated about this.
Lex Fridman (1:34:50.140)
So is this the book basically criticizing the ideas in the book?
Richard Haier (1:34:54.580)
Yeah, where those four things came from.
Lex Fridman (1:34:57.580)
And it is really a book that was really taken apart point by point
Richard Haier (1:35:04.580)
by a number of people who actually understood the data.
Lex Fridman (1:35:08.100)
And he didn't care.
Richard Haier (1:35:09.100)
Yeah.
Lex Fridman (1:35:10.100)
He didn't care.
Richard Haier (1:35:11.100)
He didn't modify anything.
Lex Fridman (1:35:12.100)
Listen, because this is such a sensitive topic,
Richard Haier (1:35:16.700)
like I said, I believe the impact of the work,
Lex Fridman (1:35:24.800)
because it is misinterpreted, has to be considered.
Richard Haier (1:35:28.620)
Because it's not just going to be scientific discourse,
Lex Fridman (1:35:31.380)
it's going to be political discourse,
Richard Haier (1:35:32.940)
there's going to be debates,
Lex Fridman (1:35:34.700)
there's going to be politically motivated people
Richard Haier (1:35:39.380)
that will use messages in each direction,
Lex Fridman (1:35:42.780)
make something like the bulk of the enemy
Richard Haier (1:35:45.060)
or the support for one's racist beliefs.
Lex Fridman (1:35:52.180)
And so I think you have to consider that.
Lex Fridman (1:35:55.260)
But it's difficult because Nietzsche was used by Hitler
Lex Fridman (1:35:59.300)
to justify a lot of his beliefs.
Lex Fridman (1:36:02.380)
And it's not exactly on Nietzsche to anticipate Hitler
Lex Fridman (1:36:09.060)
or how his ideas will be misinterpreted and used for evil.
Lex Fridman (1:36:12.820)
But there is a balance there.
Lex Fridman (1:36:14.680)
So I understand.
Richard Haier (1:36:15.680)
This is really interesting.
Lex Fridman (1:36:16.680)
I didn't know.
Richard Haier (1:36:17.680)
Is there any criticism of the book you find compelling
Lex Fridman (1:36:20.740)
or interesting or challenging to you from a scientific perspective?
Richard Haier (1:36:23.740)
There were factual criticisms about the nature of the statistics
Lex Fridman (1:36:29.900)
that were used, the statistical analyses.
Richard Haier (1:36:32.620)
These are more technical criticisms.
Lex Fridman (1:36:34.940)
And they were addressed by Murray in a couple of articles
Richard Haier (1:36:38.020)
where he took all the criticisms and spoke to them.
Lex Fridman (1:36:41.940)
And people listening to this podcast
Richard Haier (1:36:44.940)
can certainly find all those online.
Lex Fridman (1:36:47.940)
And it's very interesting.
Richard Haier (1:36:48.980)
Murray went on to write some additional books,
Lex Fridman (1:36:52.740)
two in the last couple of years, one about human diversity
Richard Haier (1:36:57.860)
where he goes through the data refuting the idea that race
Lex Fridman (1:37:02.980)
is only a social construct with no biological meaning.
Richard Haier (1:37:07.980)
He discusses the data.
Lex Fridman (1:37:09.820)
It's a very good discussion.
Richard Haier (1:37:11.020)
You don't have to agree with it.
Lex Fridman (1:37:12.500)
But he presents data in a cogent way.
Lex Fridman (1:37:16.420)
And he talks about the critics of that.
Lex Fridman (1:37:19.060)
And he talks about their data in a cogent, nonpersonal way.
Richard Haier (1:37:23.460)
It's a very informative discussion.
Lex Fridman (1:37:26.860)
The book is called Human Diversity.
Richard Haier (1:37:28.940)
He talks about race.
Lex Fridman (1:37:29.900)
And he talks about gender, same thing, about sex differences.
Lex Fridman (1:37:34.620)
And more recently, he's written what
Lex Fridman (1:37:37.180)
might be his final say on this, a book called Facing Reality
Richard Haier (1:37:43.060)
where he talks about this again.
Lex Fridman (1:37:46.220)
So he can certainly defend himself.
Richard Haier (1:37:49.580)
He doesn't need me to do that.
Lex Fridman (1:37:52.100)
But I would urge people who have heard
Richard Haier (1:37:55.060)
about him and the bell curve and who
Lex Fridman (1:37:58.500)
think they know what's in it, you are likely incorrect.
Lex Fridman (1:38:03.500)
And you need to read it for yourself.
Lex Fridman (1:38:06.300)
But it is, scientifically, it's a serious subject.
Richard Haier (1:38:12.220)
It's a difficult subject.
Lex Fridman (1:38:13.580)
Ethically, it's a difficult subject.
Richard Haier (1:38:15.980)
Everything you said here, calmly and thoughtfully,
Lex Fridman (1:38:19.740)
is difficult. It's difficult for me
Richard Haier (1:38:21.620)
to even consider that G factor exists.
Lex Fridman (1:38:26.420)
I don't mean from like that somehow G factor is inherently
Richard Haier (1:38:29.860)
racist or sexist or whatever.
Lex Fridman (1:38:32.260)
It's just it's difficult in the way
Richard Haier (1:38:35.260)
that considering the fact that we die one day is difficult.
Lex Fridman (1:38:38.220)
That we are limited by our biology is difficult.
Lex Fridman (1:38:42.820)
And at least from an American perspective,
Lex Fridman (1:38:47.740)
you would like to believe that everything
Richard Haier (1:38:49.300)
is possible in this world.
Lex Fridman (1:38:51.380)
Well, that leads us to what I think
Richard Haier (1:38:56.220)
we should do with this information.
Lex Fridman (1:38:59.980)
And what I think we should do with this information
Richard Haier (1:39:03.300)
is unusual.
Lex Fridman (1:39:07.660)
Because I think what we need to do
Richard Haier (1:39:09.540)
is fund more neuroscience research on the molecular
Lex Fridman (1:39:13.420)
biology of learning and memory.
Richard Haier (1:39:16.780)
Because one definition of intelligence
Lex Fridman (1:39:22.060)
is based on how much you can learn
Lex Fridman (1:39:24.580)
and how much you can remember.
Lex Fridman (1:39:27.140)
And if you accept that definition of intelligence,
Richard Haier (1:39:30.860)
then there are molecular studies going on now,
Lex Fridman (1:39:35.580)
and Nobel Prizes being won on molecular biology
Richard Haier (1:39:40.820)
or molecular neurobiology of learning and memory.
Lex Fridman (1:39:45.420)
Now, the step those researchers, those scientists
Richard Haier (1:39:49.500)
need to take when it comes to intelligence
Lex Fridman (1:39:53.180)
is to focus on the concept of individual differences.
Richard Haier (1:39:58.260)
Intelligence research has individual differences
Lex Fridman (1:40:03.740)
as its heart because it assumes that people
Richard Haier (1:40:08.900)
differ on this variable.
Lex Fridman (1:40:10.700)
And those differences are meaningful
Lex Fridman (1:40:13.020)
and need understanding.
Lex Fridman (1:40:15.900)
Cognitive psychologists who have morphed
Richard Haier (1:40:19.340)
into molecular biologists studying learning and memory
Lex Fridman (1:40:23.420)
hate the concept of individual differences historically.
Richard Haier (1:40:27.460)
Some now are coming around to it.
Lex Fridman (1:40:30.180)
I once sat next to a Nobel Prize winner
Richard Haier (1:40:34.740)
for his work on memory.
Lex Fridman (1:40:37.860)
And I asked him about individual differences.
Lex Fridman (1:40:41.340)
And he said, don't go there.
Lex Fridman (1:40:42.820)
It'll set us back 50 years.
Lex Fridman (1:40:46.580)
But I said, don't you think they're
Lex Fridman (1:40:48.660)
the key, though, to understand?
Lex Fridman (1:40:50.420)
Why can some people remember more than others?
Lex Fridman (1:40:53.980)
He said, you don't want to go there.
Richard Haier (1:40:55.980)
I think the 21st century will be remembered
Lex Fridman (1:40:58.700)
by the technology and the science that
Richard Haier (1:41:02.140)
goes to individual differences.
Lex Fridman (1:41:04.100)
Because we have now data.
Richard Haier (1:41:05.540)
We have now the tools to much, much better
Lex Fridman (1:41:07.380)
to start to measure, start to estimate,
Richard Haier (1:41:10.100)
not just on the sort of through tests and IQ test type
Lex Fridman (1:41:13.940)
of things, sort of outside the body kind of things,
Lex Fridman (1:41:18.220)
but measuring all kinds of stuff about the body.
Lex Fridman (1:41:20.380)
So yeah, truly go into the molecular biology,
Richard Haier (1:41:23.140)
to the neurobiology, to the neuroscience.
Lex Fridman (1:41:27.260)
Let me ask you about life.
Lex Fridman (1:41:31.940)
How does intelligence correlate with or lead to
Lex Fridman (1:41:36.820)
or has anything to do with career success?
Richard Haier (1:41:39.060)
You've mentioned these kinds of things.
Lex Fridman (1:41:40.660)
And is there any data?
Richard Haier (1:41:43.380)
You had an excellent conversation
Lex Fridman (1:41:44.780)
with Jordan Peterson, for example.
Richard Haier (1:41:46.740)
Is there any data on what intelligent
Lex Fridman (1:41:49.820)
means for success in life?
Richard Haier (1:41:53.380)
Success in life.
Lex Fridman (1:41:54.540)
There is a tremendous amount of validity data
Richard Haier (1:42:00.620)
that looked at intelligence test scores and various measures
Lex Fridman (1:42:08.260)
of life success.
Richard Haier (1:42:11.380)
Now, of course, life success is a pretty broad topic.
Lex Fridman (1:42:17.420)
And not everybody agrees on what success means.
Lex Fridman (1:42:22.100)
But there's general agreement on certain aspects of success
Lex Fridman (1:42:28.460)
that can be measured.
Richard Haier (1:42:33.180)
Including life expectancy, like you said.
Lex Fridman (1:42:35.060)
Life expectancy.
Richard Haier (1:42:36.500)
Now, there's life success.
Lex Fridman (1:42:42.020)
Life expectancy, I mean, that is such an interesting finding.
Lex Fridman (1:42:47.460)
But IQ scores are also correlated to things like income.
Lex Fridman (1:42:54.020)
Now, OK, so who thinks income means you're successful?
Richard Haier (1:42:59.900)
That's not the point.
Lex Fridman (1:43:01.340)
The point is that income is one empirical measure
Richard Haier (1:43:06.540)
in this culture that says something
Lex Fridman (1:43:09.300)
about your level of success.
Richard Haier (1:43:11.500)
You can define success in ways that
Lex Fridman (1:43:13.620)
have nothing to do with income.
Richard Haier (1:43:15.980)
You can define success based on your evolutionary natural
Lex Fridman (1:43:21.100)
selection success.
Lex Fridman (1:43:25.220)
But for variables, and even that, by the way,
Lex Fridman (1:43:29.780)
is correlated to IQ in some studies.
Lex Fridman (1:43:33.940)
So however you want to define success, IQ is important.
Lex Fridman (1:43:42.300)
It's not the only determinant.
Lex Fridman (1:43:44.260)
People get hung up on, well, what about personality?
Lex Fridman (1:43:46.780)
What about so called emotional intelligence?
Richard Haier (1:43:49.300)
Yes, all those things matter.
Lex Fridman (1:43:51.980)
The thing that matters empirically,
Richard Haier (1:43:54.460)
the single thing that matters the most
Lex Fridman (1:43:56.380)
is your general ability, your general mental intellectual
Richard Haier (1:44:01.180)
ability, your reasoning ability.
Lex Fridman (1:44:03.580)
And the more complex your vocation,
Richard Haier (1:44:07.100)
the more complex your job, the more G matters.
Lex Fridman (1:44:11.500)
G doesn't matter in a lot of occupations
Richard Haier (1:44:14.580)
don't require complex thinking.
Lex Fridman (1:44:17.020)
And there are occupations like that, and G doesn't matter.
Richard Haier (1:44:20.940)
Within an occupation, the G might not matter so much.
Lex Fridman (1:44:28.300)
So that if you look at all the professors at MIT
Lex Fridman (1:44:36.260)
and had a way to rank order them,
Lex Fridman (1:44:40.220)
there's a ceiling effect is what I'm saying.
Richard Haier (1:44:43.060)
That, you know.
Lex Fridman (1:44:45.140)
Also, when you get past a certain threshold,
Richard Haier (1:44:47.140)
then there's impact on wealth, for example,
Lex Fridman (1:44:49.900)
or career success, however that's
Richard Haier (1:44:52.460)
defined in each individual discipline.
Lex Fridman (1:44:54.220)
But after a certain point, it doesn't matter.
Richard Haier (1:44:56.780)
Actually, it does matter in certain things.
Lex Fridman (1:44:59.380)
So for example, there is a very classic study
Richard Haier (1:45:04.500)
that was started at Johns Hopkins when
Lex Fridman (1:45:07.220)
I was a graduate student there.
Lex Fridman (1:45:08.780)
And I actually worked on this study at the very beginning.
Lex Fridman (1:45:11.260)
It's the study of mathematically and scientifically
Richard Haier (1:45:13.700)
precocious youth.
Lex Fridman (1:45:15.660)
And they gave junior high school students
Richard Haier (1:45:20.140)
age 11 and 12 the standard SAT math exam.
Lex Fridman (1:45:27.140)
And they found a very large number of students
Richard Haier (1:45:31.420)
scored very high on this exam.
Lex Fridman (1:45:33.900)
Not a large number.
Richard Haier (1:45:35.100)
I mean, they found many students when
Lex Fridman (1:45:37.700)
they cast the net to all of Baltimore.
Richard Haier (1:45:40.700)
They found a number of students who
Lex Fridman (1:45:42.900)
scored as high on the SAT math when
Richard Haier (1:45:45.460)
they were 12 years old as incoming Hopkins freshmen.
Lex Fridman (1:45:50.220)
And they said, gee, now this is interesting.
Lex Fridman (1:45:53.580)
What shall we do now?
Lex Fridman (1:45:56.860)
And on a case by case basis, they
Richard Haier (1:46:00.140)
got some of those kids into their local community college
Lex Fridman (1:46:03.860)
math programs.
Richard Haier (1:46:06.660)
Many of those kids went on to be very successful.
Lex Fridman (1:46:10.260)
And now there's a 50 year follow up of those kids.
Lex Fridman (1:46:14.940)
And it turns out these kids were in the top 1%.
Lex Fridman (1:46:21.380)
So everybody in this study is in the top 1%.
Richard Haier (1:46:24.820)
If you take that group, that rarefied group,
Lex Fridman (1:46:28.100)
and divide them into quartiles so that you have the top 25%
Richard Haier (1:46:33.100)
of the top 1% and the bottom 25% of the top 1%,
Lex Fridman (1:46:39.780)
you can find unmeasurable variables of success.
Richard Haier (1:46:48.020)
The top quartile does better than the bottom quartile
Lex Fridman (1:46:51.740)
in the top 1%.
Richard Haier (1:46:53.780)
They have more patents.
Lex Fridman (1:46:54.920)
They have more publications.
Richard Haier (1:46:56.360)
They have more tenure at universities.
Lex Fridman (1:46:59.520)
And this is based on, you're dividing them
Richard Haier (1:47:03.140)
based on their score at age 12.
Lex Fridman (1:47:05.980)
I wonder how much interesting data
Richard Haier (1:47:10.220)
is in the variability in the differences.
Lex Fridman (1:47:12.700)
So but that's really, oh, boy.
Richard Haier (1:47:16.460)
That's very interesting.
Lex Fridman (1:47:17.620)
But it's also, I don't know, somehow painful.
Richard Haier (1:47:21.020)
I don't know why it's so painful that that G
Lex Fridman (1:47:25.020)
factor is so determinant of even in the nuanced top percent.
Richard Haier (1:47:31.140)
This is interesting that you find that painful.
Lex Fridman (1:47:32.780)
Do you find it painful that people with charisma
Richard Haier (1:47:37.180)
are very successful, can be very successful in life,
Lex Fridman (1:47:40.740)
even though having no other attributes other than they're
Lex Fridman (1:47:43.900)
famous and people like them?
Lex Fridman (1:47:45.780)
Do you find that painful?
Richard Haier (1:47:47.380)
Yes, if that charisma is untrainable.
Lex Fridman (1:47:51.100)
So one of the things, again, this
Richard Haier (1:47:53.500)
is like I learned psychology from the Johnny Depp trial.
Lex Fridman (1:47:56.740)
But one of the things the psychologist, the personality
Richard Haier (1:48:01.700)
psychologist, he can maybe speak to this
Lex Fridman (1:48:03.380)
because he had an interest in this for a time,
Richard Haier (1:48:07.220)
is she was saying that personality, technically
Lex Fridman (1:48:12.580)
speaking, is the thing that doesn't change over a lifetime.
Richard Haier (1:48:16.980)
It's the thing you're, I don't know if she was actually
Lex Fridman (1:48:20.140)
implying that you're born with it.
Richard Haier (1:48:21.660)
Well, it's a trait.
Lex Fridman (1:48:22.780)
It's a trait that's state.
Richard Haier (1:48:24.580)
It's a trait that's relatively stable over time.
Lex Fridman (1:48:27.220)
I think that's generally correct.
Lex Fridman (1:48:28.940)
So to the degree your personality
Lex Fridman (1:48:31.060)
is stable over time, yes, that too is painful.
Richard Haier (1:48:36.820)
Because what's not painful is the thing,
Lex Fridman (1:48:40.700)
if I'm fat and out of shape, I can
Richard Haier (1:48:42.660)
exercise and become healthier in that way.
Lex Fridman (1:48:47.900)
If my diet is a giant mess and that's
Richard Haier (1:48:51.020)
resulting in some kind of conditions
Lex Fridman (1:48:53.860)
that my body is experiencing, I can fix that
Richard Haier (1:48:55.980)
by having a better diet.
Lex Fridman (1:48:58.100)
That sort of my actions, my willed actions
Richard Haier (1:49:02.100)
can make a change.
Lex Fridman (1:49:03.860)
If charisma is part of the personality that's,
Richard Haier (1:49:07.940)
the part of the charisma that is part of the personality that
Lex Fridman (1:49:10.900)
is stable, yeah, yeah, that's painful too.
Richard Haier (1:49:15.380)
Because it's like, oh shit, I'm stuck with this.
Lex Fridman (1:49:18.420)
I'm stuck with this.
Richard Haier (1:49:19.860)
Well, and this pretty much generalizes
Lex Fridman (1:49:22.860)
to every aspect of your being.
Richard Haier (1:49:24.980)
This is who you are.
Lex Fridman (1:49:26.460)
You've got to deal with it.
Lex Fridman (1:49:27.540)
And what it undermines, of course,
Lex Fridman (1:49:29.340)
is a realistic appreciation for this,
Richard Haier (1:49:32.580)
undermines the fairly recent idea prevalent in this country
Lex Fridman (1:49:40.740)
that if you work hard, you can be anything you want to be,
Richard Haier (1:49:44.020)
which has morphed from the original idea
Lex Fridman (1:49:47.420)
that if you work hard, you can be successful.
Richard Haier (1:49:50.860)
Those are two different things.
Lex Fridman (1:49:53.900)
And now we have if you work hard,
Richard Haier (1:49:57.860)
you can be anything you want to be.
Lex Fridman (1:50:00.860)
This is completely unrealistic.
Richard Haier (1:50:03.460)
Sorry.
Lex Fridman (1:50:04.500)
It just is.
Richard Haier (1:50:05.180)
Now, you can work hard and be successful.
Lex Fridman (1:50:06.900)
There's no question.
Lex Fridman (1:50:08.500)
But you know what?
Lex Fridman (1:50:09.980)
I could work very hard, and I am not
Richard Haier (1:50:12.420)
going to be a successful theoretical physicist.
Lex Fridman (1:50:16.820)
I'm just not.
Richard Haier (1:50:18.740)
That said, I mean, we should, because we
Lex Fridman (1:50:21.020)
had this conversation already, but it's good to repeat.
Richard Haier (1:50:26.340)
The fact that you're not going to be
Lex Fridman (1:50:27.700)
a theoretical physicist is not judgment
Richard Haier (1:50:31.100)
on your basic humanity.
Lex Fridman (1:50:32.820)
We're turning again to the all men, which means
Richard Haier (1:50:36.180)
men and women are created equal.
Lex Fridman (1:50:39.100)
So again, some of the differences
Richard Haier (1:50:40.620)
we're talking about in quote unquote success, wealth,
Lex Fridman (1:50:47.020)
number of whether you win a Nobel Prize or not,
Richard Haier (1:50:50.620)
that doesn't put a measure on your basic humanity
Lex Fridman (1:50:55.820)
and basic value and even goodness of you
Richard Haier (1:51:00.940)
as a human being.
Lex Fridman (1:51:02.500)
Because your basic role and value in society
Richard Haier (1:51:06.900)
is largely within your control.
Lex Fridman (1:51:11.940)
It's some of these measures that we're talking about.
Richard Haier (1:51:16.740)
It's good to remember this.
Lex Fridman (1:51:19.380)
One question about the Flynn effect.
Lex Fridman (1:51:22.260)
What is it?
Lex Fridman (1:51:23.260)
Are humans getting smarter over the years, over the decades,
Lex Fridman (1:51:27.140)
over the centuries?
Lex Fridman (1:51:28.820)
The Flynn effect is James Flynn, who passed away about a year
Richard Haier (1:51:33.500)
ago, published a set of analyses going back
Lex Fridman (1:51:42.420)
a couple of decades when he first noticed this,
Richard Haier (1:51:46.020)
that IQ scores, when you looked over the years,
Lex Fridman (1:51:51.460)
seemed to be drifting up.
Richard Haier (1:51:54.420)
Now, this was not unknown to the people who make the test
Lex Fridman (1:51:58.820)
because they renorm the test periodically
Lex Fridman (1:52:02.220)
and they have to renorm the test periodically
Lex Fridman (1:52:05.260)
because what 10 items correct meant
Richard Haier (1:52:09.700)
relative to other people 50 years ago
Lex Fridman (1:52:13.700)
is not the same as what 10 items mean relative today.
Richard Haier (1:52:18.580)
People are getting more things correct.
Lex Fridman (1:52:21.220)
Now, the scores have been drifting up about three points.
Richard Haier (1:52:24.980)
IQ scores have been drifting up about three points per decade.
Lex Fridman (1:52:30.180)
This is not a personal effect.
Richard Haier (1:52:31.700)
This is a cohort effect.
Lex Fridman (1:52:34.020)
Well, it's not for an individual, but.
Lex Fridman (1:52:37.100)
The world, how do you explain?
Lex Fridman (1:52:38.700)
So what's that?
Lex Fridman (1:52:39.500)
And this has presented intelligence researchers
Lex Fridman (1:52:42.420)
with a great mystery.
Richard Haier (1:52:44.580)
Two questions.
Lex Fridman (1:52:46.220)
First, is it effect on the 50% of the variance that's
Lex Fridman (1:52:51.140)
the G factor or on the other 50%?
Lex Fridman (1:52:55.060)
And there's evidence that it is a G factor effect.
Lex Fridman (1:52:59.620)
And second, what on earth causes this?
Lex Fridman (1:53:02.660)
And doesn't this mean intelligence and G factor
Richard Haier (1:53:05.780)
cannot be genetic because the scale of natural selection
Lex Fridman (1:53:10.220)
is much, much longer than a couple of decades ago?
Lex Fridman (1:53:15.660)
And so it's been used to try to undermine the idea
Lex Fridman (1:53:19.700)
that there can be a genetic influence on intelligence.
Lex Fridman (1:53:24.420)
But certainly, it can be the Flynn effect
Lex Fridman (1:53:28.540)
can affect the nongenetic aspects of intelligence
Richard Haier (1:53:32.060)
because genes account for maybe 50% of the variance.
Lex Fridman (1:53:37.700)
Maybe higher, could be as high as 80% for adults,
Lex Fridman (1:53:40.900)
but let's just say 50% for discussion.
Lex Fridman (1:53:46.420)
So the Flynn effect, it's still a mystery.
Richard Haier (1:53:50.260)
It's still a mystery.
Lex Fridman (1:53:51.180)
That's interesting.
Richard Haier (1:53:51.780)
It's still a mystery, although the evidence is coming out.
Lex Fridman (1:53:54.060)
I told you before I edited a journal on intelligence,
Lex Fridman (1:53:56.700)
and we're doing a special issue in honor of James Flynn.
Lex Fridman (1:54:00.460)
So I'm starting to see papers now on really
Richard Haier (1:54:03.060)
the latest research on this.
Lex Fridman (1:54:06.300)
I think most people who specialize
Richard Haier (1:54:08.820)
in this area of trying to understand the Flynn effect
Lex Fridman (1:54:12.460)
are coming to the view based on data
Richard Haier (1:54:16.460)
that it has to do with advances in nutrition and health care.
Lex Fridman (1:54:20.180)
And there's also evidence that the effect is slowing down
Lex Fridman (1:54:25.100)
and possibly reversing.
Lex Fridman (1:54:27.500)
Oh, boy.
Lex Fridman (1:54:28.340)
So how would nutrition and health,
Lex Fridman (1:54:30.580)
so nutrition would still be connected to the G factor.
Lex Fridman (1:54:36.300)
So nutrition as it relates to the G factor,
Lex Fridman (1:54:38.660)
so the biology that leads to the intelligence.
Richard Haier (1:54:42.060)
Yes.
Lex Fridman (1:54:42.580)
That would be the claim.
Richard Haier (1:54:43.820)
Like the hypothesis being tested by the research.
Lex Fridman (1:54:50.300)
Yes.
Lex Fridman (1:54:50.820)
And there's some evidence from infants
Lex Fridman (1:54:54.460)
that nutrition has made a difference.
Lex Fridman (1:54:58.540)
So it's not an unreasonable connection.
Lex Fridman (1:55:02.140)
But does it negate the idea that there's a genetic influence?
Richard Haier (1:55:05.740)
Not logically at all.
Lex Fridman (1:55:07.980)
But it is very interesting.
Lex Fridman (1:55:09.860)
So that if you take an IQ test today but you take the score
Lex Fridman (1:55:17.460)
and use the tables that were available in 1940,
Richard Haier (1:55:22.900)
you're going to wind up with a much higher IQ number.
Lex Fridman (1:55:27.020)
So are we really smarter than a couple of generations ago?
Richard Haier (1:55:32.700)
No, but we might be able to solve problems a little better.
Lex Fridman (1:55:38.300)
And make use of our G because of things like Sesame Street
Lex Fridman (1:55:43.140)
and other curricula in school.
Lex Fridman (1:55:45.380)
More people are going to school.
Lex Fridman (1:55:49.540)
So there are a lot of factors here to disentangle.
Lex Fridman (1:55:53.820)
It's fascinating though.
Richard Haier (1:55:55.060)
It's fascinating that there's not clear answers yet.
Lex Fridman (1:55:58.860)
That as a population, we're getting smarter.
Richard Haier (1:56:02.620)
When you just zoom out, that's what it looks like.
Lex Fridman (1:56:04.500)
As a population, we're getting smarter.
Lex Fridman (1:56:06.020)
And it's interesting to see what the effects of that are.
Lex Fridman (1:56:08.740)
I mean, this raises the question.
Richard Haier (1:56:10.060)
We've mentioned it many times but haven't clearly addressed it,
Lex Fridman (1:56:14.420)
which is nature versus nurture question.
Lex Fridman (1:56:17.260)
So how much of intelligence is nature?
Lex Fridman (1:56:20.060)
How much of it is nurture?
Lex Fridman (1:56:21.860)
How much of it is determined by genetics versus environment?
Lex Fridman (1:56:25.700)
All of it.
Richard Haier (1:56:26.700)
All of it is genetics.
Lex Fridman (1:56:28.100)
No, all of it is nature and nurture.
Lex Fridman (1:56:32.620)
So yes.
Lex Fridman (1:56:33.620)
Yes.
Richard Haier (1:56:34.620)
Okay.
Lex Fridman (1:56:36.620)
But how much of the variance can you apportion to either?
Richard Haier (1:56:44.940)
Most of the people who work in this field say that the framing of that, if the question
Richard Haier (1:56:51.580)
is framed that way, it can't be answered because nature and nurture are not two independent
Richard Haier (1:56:57.500)
influences.
Lex Fridman (1:56:59.460)
They interact with each other.
Lex Fridman (1:57:01.420)
And understanding those interactions is so complex that many behavioral geneticists say
Richard Haier (1:57:10.220)
it is today impossible and always will be impossible to disentangle that, no matter
Lex Fridman (1:57:17.380)
what kind of advances there are in DNA technology and genomic informatics.
Lex Fridman (1:57:24.700)
But there's still, to push back on that, that same intuition from behavioral geneticists
Richard Haier (1:57:31.580)
would lead me to believe that there cannot possibly be a stable G factor because it's
Lex Fridman (1:57:37.340)
super complex.
Richard Haier (1:57:39.300)
Many of them would assert that as a logical outcome.
Lex Fridman (1:57:45.300)
But because I believe there is a stable G factor from lots of sources of data, not just
Richard Haier (1:57:51.820)
one study, but lots of sources of data over decades, I am more amenable to the idea that
Richard Haier (1:58:01.380)
whatever interactions between genes and environment exist, they can be explicated, they can be
Richard Haier (1:58:09.940)
studied, and that information can be used as a basis for molecular biology of intelligence.
Richard Haier (1:58:19.300)
Yeah, and we'll do this exact question because doesn't the stability of the G factor give
Lex Fridman (1:58:27.300)
you at least a hint that there is a biological basis for intelligence?
Richard Haier (1:58:33.780)
Yes, I think it's clear that the fact that an IQ score is correlated to things like thickness
Richard Haier (1:58:42.820)
of your cortex, that it's correlated to glucose metabolic rate in your brain, that identical
Lex Fridman (1:58:55.340)
twins reared apart are highly similar in their IQ scores.
Richard Haier (1:59:02.060)
These are all important observations that indicate, not just suggest, but indicate that
Lex Fridman (1:59:12.140)
there's a biological basis.
Lex Fridman (1:59:13.540)
And does anyone believe intelligence has nothing to do with the brain?
Lex Fridman (1:59:18.620)
I mean, it's so obvious.
Richard Haier (1:59:21.220)
Well indirectly definitely has to do with it, but the question is environment interacting
Lex Fridman (1:59:26.620)
with the brain or is it the actual raw hardware of the brain?
Richard Haier (1:59:34.920)
Well some would say that the raw hardware of the brain as it develops from conception
Richard Haier (1:59:45.500)
through adulthood, or at least through the childhood, that that so called hardware that
Richard Haier (1:59:52.700)
you are assuming is mostly genetic, in fact, is not as deterministic as you might think,
Lex Fridman (20:00.620)
It really, it's a fascinating group of people
Richard Haier (20:04.420)
who are studied.
Lex Fridman (20:08.100)
Not to get ahead of the story,
Lex Fridman (20:09.300)
but one of the most interesting things they found
Lex Fridman (20:12.540)
is a very strong relationship
Richard Haier (20:14.660)
between IQ measured at age 11 and mortality.
Lex Fridman (20:21.680)
So that, you know,
Richard Haier (20:24.380)
in the 70 years later, they looked at the survival rates
Lex Fridman (20:30.940)
and they could get death records from everybody.
Lex Fridman (20:33.300)
And Scotland has universal healthcare for everybody.
Lex Fridman (20:37.140)
And it turned out if you divide the people
Richard Haier (20:40.020)
by their age 11 IQ score into quartiles
Lex Fridman (20:44.020)
and then look at how many people are alive 70 years later,
Richard Haier (20:49.460)
the, I know this is in the book,
Lex Fridman (20:52.420)
I have the graph in the book,
Lex Fridman (20:54.660)
but there are essentially twice as many people alive
Lex Fridman (20:57.820)
in the highest IQ quartile than in the lowest IQ quartile.
Richard Haier (21:01.820)
It's true in men and women.
Lex Fridman (21:05.300)
Interesting.
Lex Fridman (21:06.940)
So it makes a big difference.
Lex Fridman (21:08.100)
Now, why this is the case is not so clear
Richard Haier (21:12.920)
since everyone had access to healthcare.
Lex Fridman (21:15.620)
Well, there's a lot, and we'll talk about it, you know,
Richard Haier (21:18.340)
just the sentences you used now
Lex Fridman (21:22.100)
could be explained by nature or nurture.
Richard Haier (21:25.780)
We don't know.
Lex Fridman (21:26.980)
Now, there's a lot of science that starts to then dig in
Lex Fridman (21:29.700)
and investigate that question.
Lex Fridman (21:31.700)
But let me linger on the IQ test.
Lex Fridman (21:33.700)
How are the test design, IQ test design, how do they work?
Lex Fridman (21:37.380)
Maybe some examples for people who are not aware.
Lex Fridman (21:39.980)
What makes a good IQ test question
Lex Fridman (21:44.020)
that sneaks up on this G factor measure?
Richard Haier (21:48.080)
Well, your question is interesting
Lex Fridman (21:49.740)
because you want me to give examples of items
Richard Haier (21:53.340)
that make good items.
Lex Fridman (21:55.220)
And what makes a good item is not so much its content,
Lex Fridman (21:59.420)
but its empirical relationship to the total score
Lex Fridman (22:03.180)
that turns out to be valid by other means.
Lex Fridman (22:07.740)
So for example, let me give you an odd example
Lex Fridman (22:12.500)
from personality testing.
Richard Haier (22:14.300)
Nice.
Lex Fridman (22:15.560)
So there's a personality test
Richard Haier (22:18.020)
called the Minnesota Multiphasic Personality Inventory, MMPI.
Lex Fridman (22:22.740)
Been around for decades.
Richard Haier (22:24.100)
I've heard about this test recently
Lex Fridman (22:25.980)
because of the Johnny Depp and Amber Heard trial.
Richard Haier (22:29.240)
I don't know if you've been paying attention to that.
Lex Fridman (22:31.180)
But they had psychologists.
Richard Haier (22:32.020)
I have not been paying attention to it.
Lex Fridman (22:33.540)
They had psychologists on the stand,
Lex Fridman (22:35.820)
and they were talking, apparently those psychologists did,
Lex Fridman (22:39.860)
again, I'm learning so much from this trial.
Richard Haier (22:42.020)
They did different battery of tests
Lex Fridman (22:45.820)
to diagnose personality disorders.
Richard Haier (22:50.420)
Apparently there's that systematic way of doing so,
Lex Fridman (22:53.300)
and the Minnesota one is one of the ones
Richard Haier (22:55.740)
that there's the most science on.
Lex Fridman (22:59.020)
There's a lot of great papers,
Richard Haier (23:00.440)
which were all continuously cited on the stand,
Lex Fridman (23:03.820)
which is fascinating to watch.
Richard Haier (23:05.060)
Sorry, a little bit of attention.
Lex Fridman (23:06.540)
It's okay.
Richard Haier (23:07.380)
I mean, this is interesting because you're right.
Lex Fridman (23:08.220)
It's been around for decades.
Richard Haier (23:09.540)
There's a lot of scientific research
Lex Fridman (23:11.220)
on the psychometric properties of the test,
Richard Haier (23:14.820)
including what it predicts with respect
Lex Fridman (23:18.000)
to different categories of personality disorder.
Lex Fridman (23:22.320)
But what I wanna mention is the content
Lex Fridman (23:24.840)
of the items on that test.
Richard Haier (23:26.860)
All of the items are essentially true false items.
Lex Fridman (23:32.440)
True or false, I prefer a shower to a bath.
Richard Haier (23:36.860)
True or false, I think Lincoln
Lex Fridman (23:39.900)
was a better president than Washington.
Lex Fridman (23:42.260)
But what of all these, what does that have to do?
Lex Fridman (23:47.560)
And the point is the content of these items,
Richard Haier (23:49.720)
nobody knows why these items in aggregate predict anything,
Lex Fridman (23:55.360)
but empirically they do.
Richard Haier (23:57.900)
It's a technique of choosing items for a test
Lex Fridman (24:01.880)
that is called dust bowl empiricism.
Richard Haier (24:05.800)
That the content doesn't matter,
Lex Fridman (24:07.440)
but for some reason when you get a criterion group
Richard Haier (24:10.640)
of people with this disorder and you compare them
Lex Fridman (24:13.600)
to people without that disorder,
Richard Haier (24:16.120)
these are the items that distinguish,
Lex Fridman (24:19.200)
irrespective of content.
Richard Haier (24:20.640)
It's a hard concept to grasp.
Lex Fridman (24:22.700)
Well, first of all, it's fascinating.
Lex Fridman (24:25.260)
But from, because I consider myself part psychologist
Lex Fridman (24:33.000)
because I love human robot interaction,
Lex Fridman (24:35.360)
and that's a problem.
Lex Fridman (24:36.600)
Half of that problem is a psychology problem
Richard Haier (24:39.600)
because there's a human.
Lex Fridman (24:41.800)
So designing these tests to get at the questions
Richard Haier (24:45.200)
is the fascinating part.
Lex Fridman (24:46.440)
Like how do you get to,
Lex Fridman (24:50.000)
like what does dust bowl empiricism refer to?
Lex Fridman (24:52.700)
Does it refer to the final result?
Richard Haier (24:57.460)
Yeah, so it's the test is dust bowl empiricism.
Lex Fridman (25:01.540)
But how do you arrive at the battery of questions?
Richard Haier (25:04.960)
I presume one of the things,
Lex Fridman (25:07.560)
now again, I'm going to the excellent testimony
Richard Haier (25:10.080)
in that trial, they explain it,
Lex Fridman (25:12.600)
because they also, they explain the tests.
Richard Haier (25:16.720)
That a bunch of the questions are kind of
Lex Fridman (25:20.320)
make you forget that you're taking a test.
Richard Haier (25:24.040)
Like it makes it very difficult for you
Lex Fridman (25:26.720)
to somehow figure out what you're supposed to answer.
Richard Haier (25:31.640)
Yes, it's called social desirability.
Lex Fridman (25:34.120)
But we're getting a little far afield
Richard Haier (25:35.580)
because I only wanted to give that example
Lex Fridman (25:37.400)
of dust bowl empiricism.
Richard Haier (25:40.500)
When we talk about the items on an IQ test,
Lex Fridman (25:44.400)
many of those items in the dust bowl empiricism method
Richard Haier (25:50.640)
have no face validity.
Lex Fridman (25:52.960)
In other words, they don't look like they measure anything.
Richard Haier (25:56.440)
Yes.
Lex Fridman (25:57.540)
Whereas most intelligence tests,
Richard Haier (25:59.920)
the items actually look like they're measuring
Lex Fridman (26:02.220)
some mental ability.
Lex Fridman (26:03.880)
So here's one of the.
Lex Fridman (26:05.320)
So you were bringing that up as an example
Richard Haier (26:07.000)
as what it is not.
Lex Fridman (26:08.200)
Yes.
Richard Haier (26:09.040)
Got it.
Lex Fridman (26:09.860)
Okay.
Lex Fridman (26:10.700)
So I don't want to go too far afield on it.
Lex Fridman (26:12.880)
Too far afield is actually one of the names of this podcast.
Lex Fridman (26:16.120)
So I should mention that.
Lex Fridman (26:19.080)
Far afield.
Richard Haier (26:20.000)
Far afield.
Lex Fridman (26:21.120)
Yeah, so anyway, sorry.
Lex Fridman (26:22.240)
So they feel the questions look like
Lex Fridman (26:25.000)
they pass the face validity test.
Lex Fridman (26:28.040)
And some more than others.
Lex Fridman (26:29.720)
So for example, let me give you a couple of things here.
Richard Haier (26:32.680)
If I, one of the subtests on a standard IQ test
Lex Fridman (26:37.320)
is general information.
Richard Haier (26:40.860)
Let me just think a little bit
Lex Fridman (26:41.880)
because I don't want to give you the actual item.
Lex Fridman (26:44.360)
But if I said, how far is it between Washington DC
Lex Fridman (26:49.820)
and Miami, Florida?
Richard Haier (26:52.320)
Within 500 miles plus or minus.
Lex Fridman (26:55.700)
Well, you know, it's not a fact most people memorize,
Lex Fridman (27:00.080)
but you know something about geography.
Lex Fridman (27:02.720)
You say, well, I flew there once.
Richard Haier (27:04.480)
I know planes fly for 500 miles.
Lex Fridman (27:06.600)
You know, you can kind of make an estimate.
Lex Fridman (27:10.360)
But it's also seems like it would be very cultural,
Lex Fridman (27:15.280)
you know, so there's that kind of general information.
Richard Haier (27:20.240)
Then there's vocabulary test.
Lex Fridman (27:22.520)
What does regatta mean?
Lex Fridman (27:27.520)
And I choose that word because that word was removed
Lex Fridman (27:31.320)
from the IQ test because people complained
Richard Haier (27:33.760)
that disadvantaged people would not know that word
Lex Fridman (27:38.280)
just from their everyday life.
Richard Haier (27:41.440)
Okay, here's another example
Lex Fridman (27:43.920)
from a different kind of subtest on.
Lex Fridman (27:46.520)
What's regatta, by the way?
Lex Fridman (27:48.200)
Regatta is a.
Richard Haier (27:50.400)
I think I'm disadvantaged.
Lex Fridman (27:51.640)
A sailing competition, a competition with boats.
Richard Haier (27:54.800)
Not necessarily sailing, but a competition with boats.
Lex Fridman (27:58.760)
Yep, yep, I'm probably disadvantaged in that way.
Richard Haier (28:02.200)
Okay, excellent, so that was removed anyway you were saying.
Lex Fridman (28:04.960)
Okay, so here's another subtest.
Richard Haier (28:07.840)
I'm gonna repeat a string of numbers,
Lex Fridman (28:09.840)
and when I'm done, I want you to repeat them back to me.
Lex Fridman (28:12.680)
Ready?
Lex Fridman (28:13.720)
Okay, seven, four, two, eight, one, six.
Richard Haier (28:21.540)
That's way too many.
Lex Fridman (28:22.560)
Seven, four, two, eight, one, six.
Richard Haier (28:25.160)
Okay, you get the idea.
Lex Fridman (28:26.280)
Now the actual test starts with a smaller number,
Richard Haier (28:30.800)
like two numbers, and then as people get it right,
Lex Fridman (28:33.500)
you keep going, adding to the string of numbers
Richard Haier (28:36.520)
until they can't do it anymore.
Lex Fridman (28:38.600)
Okay, but now try this.
Richard Haier (28:40.720)
I'm gonna say some numbers, and when I'm done,
Lex Fridman (28:43.760)
I want you to repeat them to me backwards.
Richard Haier (28:46.640)
I quit.
Lex Fridman (28:47.800)
Okay, now, so I gave you some examples
Richard Haier (28:51.520)
of the kind of items on an IQ test.
Lex Fridman (28:53.600)
General information, I can't even remember all,
Richard Haier (28:58.640)
general information, vocabulary, digit span forward
Lex Fridman (29:03.680)
and digit span backward.
Richard Haier (29:06.760)
Well, you said I can't even remember them.
Lex Fridman (29:08.960)
That's a good question for me.
Lex Fridman (29:11.480)
What does memory have to do with GFactor?
Lex Fridman (29:13.600)
Okay, well, let's hold on.
Richard Haier (29:15.200)
Okay, all right.
Lex Fridman (29:16.280)
Let's just talk about these examples.
Richard Haier (29:19.720)
Now, some of those items seem very cultural,
Lex Fridman (29:26.760)
and others seem less cultural.
Richard Haier (29:31.960)
Which ones do you think, scores on which subtest
Lex Fridman (29:35.400)
are most highly correlated with the GFactor?
Richard Haier (29:39.640)
Well, the intuitive answer is less cultural.
Lex Fridman (29:42.560)
Well, it turns out vocabulary is highly correlated,
Lex Fridman (29:49.860)
and it turns out that digit span backwards
Lex Fridman (29:54.220)
is highly correlated.
Lex Fridman (29:55.740)
How do you figure?
Lex Fridman (29:58.620)
Now you have decades of research to answer the question,
Lex Fridman (2:00:01.220)
but it is probabilistic and what affects the probabilities are things like in uterine environment
Lex Fridman (2:00:09.740)
and other factors like that, including chance.
Richard Haier (2:00:15.140)
That chance affects the way the neurons are connecting during gestation.
Lex Fridman (2:00:22.680)
It's not, hey, it's pre programmed.
Lex Fridman (2:00:26.820)
So there is push back on the concept that genes provide a blueprint, that it's a lot
Lex Fridman (2:00:35.500)
more fluid.
Richard Haier (2:00:36.500)
Well, but also, yeah, so there's a lot, a lot, a lot happens in the first few months
Lex Fridman (2:00:44.020)
of development.
Lex Fridman (2:00:47.340)
So in nine months inside the mother's body and in the few months afterwards, there's
Richard Haier (2:00:57.180)
a lot of fascinating stuff, like including chance and luck, like you said, how things
Richard Haier (2:01:02.120)
connect up.
Richard Haier (2:01:04.220)
The question is afterwards in your plasticity of the brain, how much adjustment there is
Richard Haier (2:01:08.460)
relative to the environment, how much that affects the G factor, but that's where the
Richard Haier (2:01:14.140)
whole conclusions of the studies that we've been talking about is that seems to have less
Lex Fridman (2:01:19.140)
and less and less of an effect as pretty quickly.
Richard Haier (2:01:23.620)
As yes, and I do think there is more of a genetic, by my view, and I'm not an expert
Richard Haier (2:01:30.060)
on this, I mean, genetics is a highly technical and complex subject.
Richard Haier (2:01:34.180)
I am not a geneticist, not a behavioral geneticist, but my reading of this, my interpretation
Richard Haier (2:01:41.900)
of this is that there is a genetic blueprint, more or less, and that has a profound influence
Lex Fridman (2:01:50.880)
on your subsequent intellectual development, including the G factor.
Lex Fridman (2:01:56.700)
And that's not to say things can't happen to, I mean, if you think of that genes provide
Richard Haier (2:02:03.900)
a potential, fine, and then various variables impact that potential, and every parent of
Richard Haier (2:02:12.060)
a newborn, implicitly or explicitly, wants to maximize that potential.
Lex Fridman (2:02:19.520)
This is why you buy educational toys.
Richard Haier (2:02:21.980)
This is why you pay attention to organic baby food.
Richard Haier (2:02:25.660)
This is why you do all these things, because you want your baby to be as healthy and as
Richard Haier (2:02:31.620)
smart as possible, and every parent will say that.
Richard Haier (2:02:36.000)
Is there a case to be made, can you steel man the case, that genetics is a very tiny
Lex Fridman (2:02:45.220)
component of all of this, and the environment is essential?
Lex Fridman (2:02:49.420)
I don't think the data supports that genetics is a tiny component.
Richard Haier (2:02:53.460)
I think the data support the idea that the genetics is a very important, and I don't
Richard Haier (2:02:58.260)
say component, I say influence, a very important influence, and the environment is a lot less
Richard Haier (2:03:05.740)
than people believe.
Lex Fridman (2:03:07.900)
Most people believe environment plays a big role.
Richard Haier (2:03:10.700)
I'm not so sure.
Lex Fridman (2:03:11.700)
I guess what I'm asking you is, can you see where what you just said, it might be wrong?
Lex Fridman (2:03:19.420)
Can you imagine a world, and what kind of evidence would you need to see to say, you
Lex Fridman (2:03:27.140)
know what, the intuition, the studies so far, like reversing the directions.
Lex Fridman (2:03:31.700)
So one of the cool things we have now more and more is we're getting more and more data,
Lex Fridman (2:03:36.140)
and the rate of the data is escalating because of the digital world.
Lex Fridman (2:03:41.860)
So when you start to look at a very large scale of data, both on the biology side and
Richard Haier (2:03:48.700)
the social side, we might be discovering some very counterintuitive things about society.
Richard Haier (2:03:53.860)
We might see the edge cases that reveal that if we actually scale those edge cases and
Richard Haier (2:04:00.300)
they become like the norm, that we'll have a complete shift in our, like you'll see G
Richard Haier (2:04:08.080)
factor be able to be modified throughout life in the teens and in later life.
Lex Fridman (2:04:15.300)
So is it any case you can make or for where your current intuitions are wrong?
Richard Haier (2:04:20.700)
Yes, and it's a good question because I think everyone should always be asked what evidence
Lex Fridman (2:04:25.300)
would change your mind.
Richard Haier (2:04:28.020)
It's certainly not only a fair question, it is really the key question for anybody working
Lex Fridman (2:04:32.860)
on any aspect of science.
Richard Haier (2:04:36.580)
I think that if environment was very important, we would have seen it clearly by now.
Richard Haier (2:04:45.900)
It would have been obvious that school interventions, compensatory education, early childhood education,
Richard Haier (2:04:53.660)
all these things that have been earnestly tried in well funded, well designed studies
Lex Fridman (2:04:59.340)
would show some effect, and they don't.
Lex Fridman (2:05:02.940)
What if the school, the way we've tried school, compensatory school sucks and we need to do
Lex Fridman (2:05:08.260)
better?
Richard Haier (2:05:09.260)
That's what everybody said at the beginning.
Lex Fridman (2:05:10.260)
That's what everybody said to Jensen.
Richard Haier (2:05:11.760)
He said, well, maybe we need to start earlier.
Lex Fridman (2:05:15.720)
Maybe we need not do prekindergarten, but pre, prekindergarten.
Richard Haier (2:05:20.320)
It's always an infinite, well, maybe we didn't get it right.
Lex Fridman (2:05:24.620)
But after decades of trying, 50 years, 50 or 60 years of trying, surely something would
Richard Haier (2:05:33.580)
have worked to the point where you could actually see a result and not need a probability level
Lex Fridman (2:05:39.700)
at 0.05 on some means.
Lex Fridman (2:05:42.540)
So that's why I, that's the kind of evidence that would change my mind.
Richard Haier (2:05:49.380)
Population level interventions like schooling that you would see like this actually has
Richard Haier (2:05:56.940)
an effect.
Lex Fridman (2:05:57.940)
Yes.
Lex Fridman (2:05:58.940)
And when you take adopted kids and they grow up in another family and you find out when
Richard Haier (2:06:04.940)
those adopted kids are adults, their IQ scores don't correlate with the IQ scores of their
Richard Haier (2:06:09.980)
adoptive parents, but they do correlate with their IQ scores of their biological parents
Lex Fridman (2:06:15.820)
whom they've never met.
Richard Haier (2:06:18.860)
I mean, these are important, these are powerful observations.
Lex Fridman (2:06:22.100)
And it would be convincing to you if the reverse was true.
Richard Haier (2:06:26.500)
Yes.
Lex Fridman (2:06:27.500)
That would be more.
Lex Fridman (2:06:28.500)
And there is some data on adoption that indicates that the adopted children are moving a little
Lex Fridman (2:06:35.180)
bit more toward their adoptive parents.
Lex Fridman (2:06:40.220)
But it's to me the overwhelming, I have this concept called the weight of evidence where
Lex Fridman (2:06:47.500)
I don't interpret any one study too much.
Richard Haier (2:06:50.500)
The weight of evidence tells me genes are important.
Lex Fridman (2:06:53.540)
But what does that mean?
Lex Fridman (2:06:54.780)
What does it mean that genes are important?
Richard Haier (2:06:57.100)
Knowing that gene expression, genes don't express themselves in a vacuum, they express
Richard Haier (2:07:03.180)
themselves in an environment.
Lex Fridman (2:07:05.900)
So the environment has to have something to do with it, especially if the best genetic
Richard Haier (2:07:10.300)
estimates of the amount of variants are around 50 or even if it's as high as 80%, it still
Lex Fridman (2:07:17.740)
leaves 20% of non genetic.
Richard Haier (2:07:21.540)
Now maybe that is all luck.
Lex Fridman (2:07:24.300)
Maybe that's all chance.
Richard Haier (2:07:25.580)
I could believe that, I could easily believe that.
Lex Fridman (2:07:31.320)
But I do think after 50 years of trying various interventions and nothing works, including
Richard Haier (2:07:39.060)
memory training, including listening to Mozart, including playing computer games, none of
Lex Fridman (2:07:44.340)
that has shown any impact on intelligence test scores.
Lex Fridman (2:07:49.060)
Is there data on the intelligence, the IQ of parents as it relates to the children?
Richard Haier (2:07:57.140)
Yes, and there is some genetic evidence of an interaction between the parents IQ and
Richard Haier (2:08:07.020)
the environment.
Richard Haier (2:08:08.760)
High IQ parents provide an enriched environment, which then can impact the child in addition
Richard Haier (2:08:17.020)
to the genes, it's that environment.
Lex Fridman (2:08:20.020)
So there are all these interactions that, think about the number of books in a household.
Lex Fridman (2:08:29.060)
This was a variable that's correlated with IQ and, well, why?
Richard Haier (2:08:37.140)
Especially if the kid never reads any of the books, it's because more intelligent people
Richard Haier (2:08:42.900)
have more books in their house.
Lex Fridman (2:08:45.620)
And if you're more intelligent and there's a genetic component to that, the child will
Richard Haier (2:08:52.500)
get those genes or some of those genes as well as the environment.
Lex Fridman (2:08:57.480)
But it's not the number of books in the house that actually directly impacts the child.
Lex Fridman (2:09:04.140)
So the two scenarios on this are you find that, and this was used to get rid of the
Richard Haier (2:09:11.620)
SAT test, oh, the SAT score is highly correlated with the social economic status of the parents.
Lex Fridman (2:09:18.300)
So all you're really measuring is how rich the parents are.
Lex Fridman (2:09:21.540)
Okay, well, why are the parents rich?
Lex Fridman (2:09:27.620)
And so the opposite kind of syllogism is that people who are very bright make more money,
Lex Fridman (2:09:37.940)
they can afford homes in better neighborhoods so their kids get better schools.
Richard Haier (2:09:44.540)
Now the kids grow up bright.
Lex Fridman (2:09:47.780)
Where in that chain of events does that come from?
Richard Haier (2:09:51.340)
Well, unless you have a genetically informative research design where you look at siblings
Richard Haier (2:09:58.720)
that have the same biological parents and so on, you can't really disentangle all that.
Richard Haier (2:10:05.940)
Most studies of social economic status and intelligence do not have a genetically informed
Lex Fridman (2:10:12.820)
design.
Lex Fridman (2:10:13.940)
So any conclusions they make about the causality of the social economic status being the cause
Lex Fridman (2:10:20.580)
of the IQ is a stretch.
Lex Fridman (2:10:25.540)
And where you do find genetically informative designs, you find most of the variance in
Lex Fridman (2:10:33.140)
your outcome measures are due to the genetic component.
Lex Fridman (2:10:38.980)
And sometimes the SES adds a little, but the weight of evidence is it doesn't add very
Lex Fridman (2:10:46.480)
much variance to predict what's going on beyond the genetic variance.
Lex Fridman (2:10:52.660)
So when you actually look at it in some, and there aren't that many studies that have genetically
Richard Haier (2:10:58.780)
informed designs, but when you do see those, the genes seem to have an advantage.
Richard Haier (2:11:05.060)
Sorry for the strange questions, but is there a connection between fertility or the number
Lex Fridman (2:11:13.620)
of kids that you have and G factor?
Lex Fridman (2:11:16.860)
So you know, the kind of conventional wisdom is people of maybe higher economic status
Lex Fridman (2:11:25.600)
or something like that are having fewer children.
Richard Haier (2:11:28.180)
I just loosely hear these kinds of things.
Lex Fridman (2:11:30.900)
Is there data that you're aware of in one direction or another on this?
Richard Haier (2:11:36.580)
Strange questions always get strange answers.
Lex Fridman (2:11:39.060)
Yes.
Richard Haier (2:11:40.060)
All right.
Lex Fridman (2:11:41.060)
Do you have a strange answer for that strange question?
Richard Haier (2:11:44.740)
The answer is there were some studies that indicated the more children in a family, the
Lex Fridman (2:11:54.540)
firstborn children would be more intelligent than the fourth or fifth or sixth.
Richard Haier (2:12:00.740)
It's not clear that those studies hold up over time.
Lex Fridman (2:12:05.500)
And of course what you see also is that families where there are multiple children, four, five,
Richard Haier (2:12:14.020)
six, seven, you know, really big families, the social economic status of those families
Lex Fridman (2:12:23.580)
usually in the modern age is not that high.
Richard Haier (2:12:28.700)
Maybe it used to be the aristocracy used to have a lot of kids, I'm not sure exactly.
Lex Fridman (2:12:33.500)
But there have been reports of correlations between IQ and fertility, but I'm not sure
Richard Haier (2:12:44.020)
that the data are very strong that the firstborn child is always the smartest.
Lex Fridman (2:12:50.500)
It seems like there's some data to that, but I'm not current on that.
Lex Fridman (2:12:54.980)
How would that be explained?
Lex Fridman (2:12:55.980)
That would be in a nurture.
Richard Haier (2:12:58.260)
Well, it could be nurture, it could be in uterine environment, I mean, and this is why
Richard Haier (2:13:08.500)
this, you know, like many areas of science, you said earlier that there are a lot of gray
Richard Haier (2:13:14.740)
areas and no definitive answers.
Richard Haier (2:13:21.660)
This is not uncommon in science that the closer you look at a problem, the more questions
Richard Haier (2:13:28.900)
you get, not the fewer questions, because the universe is complicated.
Lex Fridman (2:13:35.820)
And the idea that we have people on this planet who can study the first nanoseconds of the
Richard Haier (2:13:42.980)
Big Bang, that's pretty amazing.
Lex Fridman (2:13:48.260)
And I've always said that if they can study the first nanoseconds of the Big Bang, we
Richard Haier (2:13:53.340)
can certainly figure out something about intelligence that allows that.
Richard Haier (2:13:58.900)
I'm not sure what's more complicated, the human mind or the physics of the universe.
Richard Haier (2:14:06.020)
It's unclear to me.
Lex Fridman (2:14:08.220)
I think we overemphasize.
Richard Haier (2:14:09.220)
Well, that's a very humbling statement.
Richard Haier (2:14:13.220)
Maybe it's a very human centric, egotistical statement that our mind is somehow super complicated,
Lex Fridman (2:14:18.100)
but biology is a tricky one to unravel.
Lex Fridman (2:14:22.860)
Consciousness, what is that?
Richard Haier (2:14:27.100)
I've always believed that consciousness and intelligence are the two real fundamental
Lex Fridman (2:14:34.180)
problems of the human brain, and therefore I think they must be related.
Richard Haier (2:14:41.380)
Yeah, heart problems like walk together, holding hands kind of idea.
Richard Haier (2:14:49.060)
You may not know this, but I did some of the early research on anesthetic drugs with brain
Richard Haier (2:14:54.180)
imaging trying to answer the question, what part of the brain is the last to turn off
Lex Fridman (2:14:58.900)
when someone loses consciousness?
Lex Fridman (2:15:01.520)
And is that the first part of the brain to turn on when consciousness is regained?
Lex Fridman (2:15:07.060)
And I was working with an anesthesiologist named Mike Alkire, who was really brilliant
Richard Haier (2:15:11.060)
at this.
Richard Haier (2:15:12.060)
These were really the first studies of brain imaging using positron emission tomography
Richard Haier (2:15:18.300)
long before fMRI.
Lex Fridman (2:15:21.700)
And you would inject a radioactive sugar that labeled the brain, and the harder the brain
Richard Haier (2:15:28.220)
was working, the more sugar it would take up, and then you could make a picture of glucose
Lex Fridman (2:15:33.980)
use in the brain.
Lex Fridman (2:15:36.420)
And he was amazing.
Richard Haier (2:15:38.560)
He managed to do this in normal volunteers he brought in and anesthetized as if they
Richard Haier (2:15:44.860)
were going into surgery.
Richard Haier (2:15:49.060)
He managed all the human subjects requirements on this research, and he was brilliant at
Richard Haier (2:15:55.380)
this.
Lex Fridman (2:15:57.100)
And what we did is we had these normal volunteers come in on three occasions.
Richard Haier (2:16:05.140)
On one occasion, he gave them enough anesthetic drug so they were a little drowsy.
Lex Fridman (2:16:14.800)
And on another occasion, they came in and he fully anesthetized them.
Lex Fridman (2:16:20.780)
And he would say, Mike, can you hear me, and the person would say, uh, yeah.
Lex Fridman (2:16:31.820)
And then we would scan people under no anesthetic condition.
Lex Fridman (2:16:37.540)
So same person.
Lex Fridman (2:16:39.980)
And we were looking to see if we could see the part of the brain turn off.
Richard Haier (2:16:46.780)
He subsequently tried to do this with fMRI, which has a faster time resolution, and you
Richard Haier (2:16:51.980)
could do it in real time as the person went under and then regain consciousness where
Richard Haier (2:16:56.780)
you couldn't do that with PET.
Lex Fridman (2:16:57.980)
You had to have three different occasions.
Lex Fridman (2:17:00.780)
And the results were absolutely fascinating.
Richard Haier (2:17:03.380)
We did this with different anesthetic drugs, and different drugs impacted different parts
Richard Haier (2:17:08.600)
of the brain.
Lex Fridman (2:17:09.780)
So we were naturally looking for the common one, and it seemed to have something to do
Richard Haier (2:17:15.820)
with the thalamus.
Lex Fridman (2:17:18.300)
And consciousness, this was actual data on consciousness, actual consciousness.
Lex Fridman (2:17:25.740)
What part of the brain turns on?
Lex Fridman (2:17:28.420)
What part of the brain turns off?
Richard Haier (2:17:30.900)
It's not so clear.
Lex Fridman (2:17:33.220)
But maybe has something to do with the thalamus.
Richard Haier (2:17:35.940)
The sequence of events seemed to have the thalamus in it.
Lex Fridman (2:17:41.180)
Now here's the question.
Lex Fridman (2:17:42.700)
Are some people more conscious than others?
Lex Fridman (2:17:45.780)
Are there individual differences in consciousness?
Lex Fridman (2:17:49.660)
And I don't mean it in the psychedelic sense.
Lex Fridman (2:17:53.020)
I don't mean it in the political consciousness sense.
Richard Haier (2:17:55.780)
I just mean it in everyday life.
Lex Fridman (2:17:57.300)
Do some people go through everyday life more conscious than others?
Lex Fridman (2:18:01.900)
And are those the people we might actually label more intelligent?
Richard Haier (2:18:06.700)
Now the other thing I was looking for is whether the parts of the brain we were seeing in the
Richard Haier (2:18:11.700)
anesthesia studies were the same parts of the brain we were seeing in the intelligence
Lex Fridman (2:18:16.820)
studies.
Richard Haier (2:18:17.820)
Now, this was very complicated, expensive research.
Lex Fridman (2:18:22.580)
We didn't really have funding to do this.
Richard Haier (2:18:24.500)
We were trying to do it on the fly.
Lex Fridman (2:18:26.260)
I'm not sure anybody has pursued this.
Richard Haier (2:18:29.660)
I'm retired now.
Lex Fridman (2:18:31.740)
He's gone on to other things.
Lex Fridman (2:18:34.100)
But I think it's an area of research that would be fascinating to see the parts, a lot
Lex Fridman (2:18:41.020)
more imaging studies now of consciousness.
Richard Haier (2:18:43.140)
I'm just not up on them.
Lex Fridman (2:18:45.260)
But basically the question is which imaging, so newer imaging studies to see in high resolution,
Richard Haier (2:18:52.460)
spatial and temporal way, which part of the brain lights up when you're doing intelligence
Richard Haier (2:18:59.260)
tasks and which parts of the brain lights up when you're doing consciousness tasks and
Richard Haier (2:19:03.700)
see the interplay between them, try to infer, that's the challenge of neuroscience, without
Richard Haier (2:19:09.460)
understanding deeply, looking from the outside, try to infer something about how the whole
Richard Haier (2:19:18.260)
thing works.
Lex Fridman (2:19:19.260)
Well, imagine this.
Richard Haier (2:19:21.220)
Here's a simple question.
Richard Haier (2:19:23.160)
Does it take more anesthetic drug to have a person lose consciousness if their IQ is
Lex Fridman (2:19:33.460)
140 than a person with an IQ of 70?
Lex Fridman (2:19:39.060)
That's an interesting way to study it.
Richard Haier (2:19:40.660)
Yeah.
Lex Fridman (2:19:41.660)
I mean, if the answer to that is a stable yes, that's very interesting.
Lex Fridman (2:19:48.060)
So I tried to find out and I went to some anesthesiology textbooks about how you dose
Lex Fridman (2:19:55.700)
and they dose by weight.
Lex Fridman (2:19:59.620)
And what I also learned, this is a little bit off subject, anesthesiologists are never
Lex Fridman (2:20:07.440)
sure how deep you are.
Lex Fridman (2:20:10.980)
And they usually tell by poking you with a needle and if you don't jump, they tell the
Lex Fridman (2:20:14.780)
surgeon to go ahead.
Richard Haier (2:20:17.100)
I'm not sure that's literally true, but it's...
Lex Fridman (2:20:20.460)
Well, it might be very difficult to know precisely how deep you are.
Richard Haier (2:20:26.380)
It has to do with the same kind of measurements that you were doing with the consciousness.
Lex Fridman (2:20:31.860)
It's difficult to know.
Lex Fridman (2:20:34.740)
So I don't lose my train of thought.
Lex Fridman (2:20:35.940)
I couldn't find in the textbooks anything about dosing by intelligence.
Richard Haier (2:20:40.980)
I asked my friend, the anesthesiologist, he said, no, he doesn't know.
Richard Haier (2:20:45.620)
I said, can we do a chart review and look at people using their years of education as
Lex Fridman (2:20:52.220)
a proxy for IQ?
Lex Fridman (2:20:54.420)
Because if someone's gone to graduate school, that tells you something.
Richard Haier (2:20:58.340)
You can make some inference as opposed to someone who didn't graduate high school.
Lex Fridman (2:21:02.780)
Can we do a chart review?
Lex Fridman (2:21:03.980)
And he says, no, they never really put down the exact dose.
Lex Fridman (2:21:08.460)
And no, he said, no.
Lex Fridman (2:21:10.620)
So to this day, the simple question, does it take more anesthetic drug to put someone
Lex Fridman (2:21:18.820)
under if they have a high IQ or less, or less?
Richard Haier (2:21:23.440)
It could go either way.
Richard Haier (2:21:24.440)
Because by the way, our early PET scan studies of intelligence found the unexpected result
Richard Haier (2:21:33.780)
of an inverse correlation between glucose metabolic rate and intelligence.
Lex Fridman (2:21:38.540)
It wasn't how much a brain area lit up.
Lex Fridman (2:21:43.020)
How much it lit up was negatively correlated to how well they did on the test, which led
Lex Fridman (2:21:48.380)
to the brain efficiency hypothesis, which is still being studied today.
Lex Fridman (2:21:54.140)
And there's more and more evidence that the efficiency of brain information processing
Lex Fridman (2:22:00.280)
is more related to intelligence than just more activity.
Richard Haier (2:22:08.780)
Yeah, and it'll be interesting, again, this is the total hypothesis, how much in the relationship
Richard Haier (2:22:14.180)
between intelligence and consciousness, it's not obvious that those two, if there's correlation,
Richard Haier (2:22:22.060)
they could be inversely correlated.
Lex Fridman (2:22:23.860)
Wouldn't that be funny?
Richard Haier (2:22:26.060)
If you, the consciousness factor, the C factor plus the G factor equals one.
Richard Haier (2:22:38.340)
It's a nice trade off, you get a trade off, how deeply you experience the world versus
Lex Fridman (2:22:43.420)
how deeply you're able to reason through the world.
Lex Fridman (2:22:48.140)
What a great hypothesis.
Richard Haier (2:22:51.060)
Certainly somebody listening to this can do this study.
Richard Haier (2:22:54.140)
Even if it's the aliens analyzing humans a few centuries from now, let me ask you from
Richard Haier (2:22:59.740)
an AI perspective, I don't know how much you've thought about machines, but there's the famous
Richard Haier (2:23:08.600)
Turing test, test of intelligence for machines, which is a beautiful, almost like a cute formulation
Richard Haier (2:23:17.220)
of intelligence that Alan Turing proposed.
Richard Haier (2:23:24.220)
Basically conversation being, if you can fool a human to think that a machine is a human
Richard Haier (2:23:33.420)
that passes the test, I suppose you could do a similar thing for humans.
Richard Haier (2:23:40.620)
If I can fool you that I'm intelligent, then that's a good test of intelligence.
Richard Haier (2:23:48.700)
You're talking to two people, and the test is saying who has a higher IQ.
Richard Haier (2:24:02.500)
It's an interesting test, because maybe charisma can be very useful there, and you're only
Richard Haier (2:24:07.960)
allowed to use conversation, which is the formulation of the Turing test.
Lex Fridman (2:24:11.340)
Anyway, all that to say is what are good tests of intelligence for machines?
Lex Fridman (2:24:18.700)
What do you think it takes to achieve human level intelligence for machines?
Richard Haier (2:24:23.700)
I have thought a little bit about this, but every time I think about these things, I rapidly
Richard Haier (2:24:30.760)
reach the limits of my knowledge and imagination.
Richard Haier (2:24:37.580)
When Alexa first came out, and I think there was a competing one, well, there was Siri
Richard Haier (2:24:47.460)
with Apple, and Google had Alexa.
Lex Fridman (2:24:50.100)
No, no, Amazon had Alexa.
Richard Haier (2:24:52.660)
Amazon had Alexa.
Lex Fridman (2:24:53.660)
Google has Google Home.
Richard Haier (2:24:54.660)
Google has something.
Richard Haier (2:24:55.660)
I proposed to one of my colleagues that he buy one of these, one of each, and then ask
Richard Haier (2:25:04.020)
it questions from the IQ test.
Lex Fridman (2:25:09.020)
But it became apparent that they all searched the internet, so they all can find answers
Richard Haier (2:25:15.860)
to questions like how far is it between Washington and Miami, and repeat after me.
Richard Haier (2:25:22.540)
Now, I don't know if you said to Alexa, I'm going to repeat these numbers backwards to
Richard Haier (2:25:29.620)
me.
Lex Fridman (2:25:30.620)
I don't know what would happen.
Richard Haier (2:25:31.620)
I've never done it.
Lex Fridman (2:25:33.620)
So one answer to your question is you're going to try it right now.
Richard Haier (2:25:38.060)
Let's try it.
Lex Fridman (2:25:39.060)
No.
Richard Haier (2:25:40.060)
Let's try it.
Lex Fridman (2:25:41.060)
No, no, no.
Richard Haier (2:25:42.060)
Yes, Siri.
Lex Fridman (2:25:43.060)
So it would actually probably go to Google search, and it will be all confusing kind
Richard Haier (2:25:47.500)
of stuff.
Lex Fridman (2:25:49.500)
It would fail.
Richard Haier (2:25:50.500)
Well, then I guess there was a test that it would fail.
Richard Haier (2:25:53.980)
Well, but that's not, that has to do more with the language of communication versus
Richard Haier (2:26:02.560)
the content.
Lex Fridman (2:26:03.740)
So if you did an IQ test to a person who doesn't speak English, and the test was administered
Richard Haier (2:26:09.300)
in English, that's not really the test of...
Lex Fridman (2:26:11.660)
Well, let's think about the computers that beat the Jeopardy champions.
Richard Haier (2:26:15.740)
Yeah, so that, because I happen to know how those are programmed, those are very hard
Lex Fridman (2:26:21.760)
coded, and there's definitely a lack of intelligence there.
Richard Haier (2:26:25.660)
There's something like IQ tests, there's a guy, an artificial intelligence researcher,
Richard Haier (2:26:36.060)
Francois Chollet, he's at Google, he's one of the seminal people in machine learning.
Richard Haier (2:26:40.980)
He also, as a fun aside thing, developed an IQ test for machines.
Lex Fridman (2:26:45.860)
Oh, I haven't heard that.
Richard Haier (2:26:47.660)
I'd just like to know about that.
Lex Fridman (2:26:49.580)
I'll actually email you this, because it'd be very interesting for you.
Richard Haier (2:26:53.340)
It doesn't get much attention, because people don't know what to do with it, but it deserves
Richard Haier (2:26:59.900)
a lot of attention, which is, it basically does a pattern type of tests, where you have
Richard Haier (2:27:06.180)
to do, you know, one standard one is, you're given three things, and you have to do a fourth
Lex Fridman (2:27:12.740)
one, that kind of thing, so you have to understand the pattern here.
Lex Fridman (2:27:17.120)
And for that, it really simplifies to, so the interesting thing is, he's trying not
Lex Fridman (2:27:28.740)
to achieve high IQ, he's trying to achieve like, pretty low bar for IQ.
Richard Haier (2:27:35.300)
Things that are kind of trivial for humans, and they're actually really tough for machines.
Richard Haier (2:27:42.060)
It's just seeing, playing with these concepts of symmetry, of counting, like if I give you
Richard Haier (2:27:49.420)
one object, two objects, three objects, you'll know the last one is four objects, you can
Richard Haier (2:27:54.980)
like count them, you can cluster objects together, it's both visually and conceptually, we can
Richard Haier (2:28:01.300)
do all these things with our mind, that we take for granted, the objectness of things.
Richard Haier (2:28:07.420)
You can like, figure out what spatially is an object and isn't, and we can play with
Richard Haier (2:28:14.740)
those ideas, and machines really struggle with that, so he really cleanly formulated
Richard Haier (2:28:21.540)
these IQ tests, I wonder what like, that would equate to for humans with IQ, but it'd be
Richard Haier (2:28:27.340)
a very low IQ, but that's exactly the kind of formulation, like okay, we want to be able
Richard Haier (2:28:33.940)
to solve this, how do we solve this, and he does it as a challenge, and nobody's been
Richard Haier (2:28:38.060)
able to, it's similar to the Alexa prize, which is Amazon is hosting a conversational
Richard Haier (2:28:44.260)
challenge, nobody's been able to do well on his, but that's an interesting, those kinds
Richard Haier (2:28:51.440)
of tests are interesting, because we take for granted all the ability of the human mind
Richard Haier (2:28:58.500)
to play with concepts, and to formulate concepts out of novel things, so like, things we've
Richard Haier (2:29:08.620)
never seen before, we're able to use that, I mean that's, I've talked to a few people
Richard Haier (2:29:14.440)
that design IQ tests, sort of online, they write IQ tests, and I was trying to get some
Richard Haier (2:29:20.540)
questions from them, and they spoke to the fact that we can't really share questions
Richard Haier (2:29:25.180)
with you, because part of the, like first of all, it's really hard work to come up with
Richard Haier (2:29:31.980)
questions, it's really, really hard work, it takes a lot of research, but it also takes
Richard Haier (2:29:37.240)
a lot, it's novelty generating, you're constantly coming up with really new things, and part
Richard Haier (2:29:46.060)
of the point is that they're not supposed to be public, they're supposed to be new
Richard Haier (2:29:51.380)
to you when you look at them, it's interesting that the novelty is fundamental to the hardness
Richard Haier (2:29:56.700)
of the problem, at least a part of what makes the problem hard is that you've never seen
Lex Fridman (2:30:02.320)
it before.
Richard Haier (2:30:03.320)
Right, that's called fluid intelligence, as opposed to what's called crystallized intelligence,
Richard Haier (2:30:08.860)
which is your knowledge of facts, you know things, but can you use those things to solve
Richard Haier (2:30:16.200)
a problem, those are two different things.
Lex Fridman (2:30:19.500)
Do you think we'll be able to, because we spoke, I don't want to miss opportunity to
Richard Haier (2:30:25.180)
talk about this, we spoke about the neurobiology, about the molecular biology of intelligence,
Lex Fridman (2:30:30.300)
do you think one day we'll be able to modify the biology of, or the genetics of a person
Richard Haier (2:30:39.460)
to modify their intelligence, to increase their intelligence, we started this conversation
Lex Fridman (2:30:45.040)
by talking about a pill you could take, do you think that such a pill would exist?
Richard Haier (2:30:49.020)
Metaphorically, I do, and I am supremely confident that it's possible because I am supremely
Richard Haier (2:30:57.900)
ignorant of the complexities of neurobiology, and so I have written that the nightmares
Richard Haier (2:31:07.220)
of neurobiologists, understanding the complexities, this cascade of events that happens at the
Lex Fridman (2:31:15.860)
synaptic level, that these nightmares are what fuel some people to solve.
Lex Fridman (2:31:25.060)
So some people, you have to be undaunted, I mean yeah, this is not easy, look we're
Richard Haier (2:31:31.520)
still trying to figure out cancer, it was only recently that they figured out why aspirin
Richard Haier (2:31:38.660)
works, you know, these are not easy problems, but I also have the perspective of the history
Richard Haier (2:31:47.980)
of science, is the history of solving problems that are extraordinarily complex.
Lex Fridman (2:31:57.580)
And seem impossible at the time.
Lex Fridman (2:31:58.740)
And seem impossible at the time.
Lex Fridman (2:32:01.140)
And so one of the things you look at, at companies like Neuralink, you have brain computer interfaces,
Richard Haier (2:32:08.460)
you start to delve into the human mind and start to talk about machines measuring but
Richard Haier (2:32:12.500)
also sending signals to the human mind, and you start to wonder what that has, what impact
Lex Fridman (2:32:19.180)
that has on the G factor.
Richard Haier (2:32:23.700)
Modifying in small ways or in large ways the functioning, the mechanical, electrical, chemical
Lex Fridman (2:32:32.420)
functioning of the brain.
Richard Haier (2:32:34.140)
I look at everything about the brain, there are different levels of explanation.
Richard Haier (2:32:39.860)
On one hand you have a behavioral level, but then you have brain circuitry, and then you
Richard Haier (2:32:46.420)
have neurons, and then you have dendrites, and then you have synapses, and then you have
Richard Haier (2:32:57.380)
the neurotransmitters, and the presynaptic and the postsynaptic terminals, and then you
Richard Haier (2:33:06.580)
have all the things that influence neurotransmitters, and then you have the individual differences
Lex Fridman (2:33:13.980)
among people.
Richard Haier (2:33:15.460)
Yeah, it's complicated, but 51 million people in the United States have IQs under 85 and
Lex Fridman (2:33:27.540)
struggle with everyday life.
Lex Fridman (2:33:32.540)
Shouldn't that motivate people to take a look at this?
Richard Haier (2:33:37.140)
Yeah, but I just want to linger one more time that you have to remember that the science
Richard Haier (2:33:46.220)
of intelligence, the measure of intelligence is only a part of the human condition.
Richard Haier (2:33:54.740)
The thing that makes life beautiful and the creation of beautiful things in this world
Richard Haier (2:33:59.260)
is perhaps loosely correlated, but is not dependent entirely on intelligence.
Lex Fridman (2:34:08.860)
Absolutely, I certainly agree with that.
Lex Fridman (2:34:12.220)
So for anyone sort of listening, I'm still not convinced that more intelligence is always
Lex Fridman (2:34:22.660)
better if you want to create beauty in this world.
Richard Haier (2:34:26.060)
I don't know.
Richard Haier (2:34:27.060)
Well, I didn't say more intelligence is always better if you want to create beauty.
Richard Haier (2:34:31.340)
I just said all things being equal, more is better than less.
Lex Fridman (2:34:36.460)
That's all I mean.
Richard Haier (2:34:37.460)
Yeah, but that's sort of that I just want to sort of say because a lot to me, one of
Richard Haier (2:34:42.980)
the things that makes life great is the opportunity to create beautiful things, and so I just
Richard Haier (2:34:50.380)
want to sort of empower people to do that no matter what some IQ test says.
Richard Haier (2:34:56.380)
At the population level, we do need to look at IQ tests to help people and to also inspire
Richard Haier (2:35:02.420)
us to take on some of these extremely difficult scientific questions.
Lex Fridman (2:35:07.900)
Do you have advice for young people in high school, in college, whether they're thinking
Lex Fridman (2:35:16.860)
about career or they're thinking about a life they can be proud of?
Richard Haier (2:35:20.740)
Is there advice you can give whether they want to pursue psychology or biology or engineering
Lex Fridman (2:35:29.760)
or they want to be artists and musicians and poets?
Richard Haier (2:35:33.700)
I can't advise anybody on that level of what their passion is, but I can say if you're
Richard Haier (2:35:43.320)
interested in psychology or if you're interested in science and the science around the big
Richard Haier (2:35:52.840)
questions of consciousness and intelligence and psychiatric illness, we haven't really
Richard Haier (2:36:00.640)
talked about brain illnesses and what we might learn from.
Richard Haier (2:36:07.020)
If you are trying to develop a drug to treat Alzheimer's disease, you are trying to develop
Richard Haier (2:36:12.540)
a drug to impact learning and memory, which are core to intelligence.
Lex Fridman (2:36:20.420)
So it could well be that the so called IQ pill will come from a pharmaceutical company
Richard Haier (2:36:26.100)
trying to develop a drug for Alzheimer's disease.
Richard Haier (2:36:29.460)
Because that's exactly what you're trying to do, right, yeah, just like you said.
Lex Fridman (2:36:33.540)
What will that drug do in a college student that doesn't have Alzheimer's disease?
Lex Fridman (2:36:38.460)
So I would encourage people who are interested in psychology, who are interested in science
Richard Haier (2:36:47.740)
to pursue a scientific career and address the big questions.
Lex Fridman (2:36:54.300)
And the most important thing I can tell you if you're going to be in kind of a research
Richard Haier (2:37:03.080)
environment is you got to follow the data where the data take you.
Lex Fridman (2:37:07.420)
You can't decide in advance where you want the data to go.
Lex Fridman (2:37:10.700)
And if the data take you to places that you don't have the technical expertise to follow,
Richard Haier (2:37:16.300)
like you know, I would like to understand more about molecular biology, but I'm not
Richard Haier (2:37:21.940)
going to become a molecular biologist now.
Lex Fridman (2:37:24.940)
But I know people who are, and my job is to get them interested to take their expertise
Richard Haier (2:37:31.140)
into this direction.
Lex Fridman (2:37:33.640)
And that it's not so easy.
Lex Fridman (2:37:36.580)
And if the data takes you to a place that's controversial, that's counterintuitive in
Richard Haier (2:37:41.820)
this world, no, I would say it's probably a good idea to still push forward boldly,
Lex Fridman (2:37:52.460)
but to communicate the interpretation of the results with skill, with compassion, with
Richard Haier (2:38:01.380)
the greater breadth of understanding of humanity, not just the science, of the impact of the
Richard Haier (2:38:07.700)
results.
Richard Haier (2:38:08.900)
One famous psychologist wrote about this issue that somehow a balance has to be found between
Richard Haier (2:38:16.340)
pursuing the science and communicating it with respect to people's sensitivities, the
Lex Fridman (2:38:22.260)
legitimate sensitivities, somehow.
Richard Haier (2:38:26.020)
He didn't say how.
Lex Fridman (2:38:27.020)
Somehow.
Richard Haier (2:38:28.020)
Somehow.
Lex Fridman (2:38:29.020)
And this is...
Richard Haier (2:38:30.020)
This sense, somehow, and balance is left up to the interpretation of the reader.
Richard Haier (2:38:37.500)
Let me ask you, you said big questions, the biggest, or one of the biggest, we already
Richard Haier (2:38:44.220)
talked about consciousness and intelligence, one of the most fascinating, one of the biggest
Lex Fridman (2:38:48.060)
questions.
Lex Fridman (2:38:49.060)
But let's talk about the why.
Lex Fridman (2:38:51.420)
Why are we here?
Lex Fridman (2:38:53.420)
What's the meaning of life?
Lex Fridman (2:38:54.420)
I'm not going to tell you.
Lex Fridman (2:38:55.420)
You know you're not going to tell me?
Lex Fridman (2:38:56.420)
This is very...
Richard Haier (2:39:00.940)
I'm going to have to wait for your next book.
Lex Fridman (2:39:03.420)
The meaning of life.
Richard Haier (2:39:07.260)
We do the best we can to get through the day.
Lex Fridman (2:39:12.700)
And then there's just a finite number of the days.
Lex Fridman (2:39:16.140)
Are you afraid of the finiteness of it?
Lex Fridman (2:39:17.820)
I think about it more and more as I get older.
Richard Haier (2:39:21.140)
Yeah, I do.
Lex Fridman (2:39:23.940)
And it's one of these human things, that it is finite, we all know it.
Richard Haier (2:39:30.900)
Most of us deny it and don't want to think about it.
Richard Haier (2:39:35.980)
Sometimes you think about it in terms of estate planning, you try to do the rational thing.
Richard Haier (2:39:42.460)
Sometimes it makes you work harder because you know your time is more and more limited
Lex Fridman (2:39:46.900)
and you want to get things done.
Richard Haier (2:39:50.740)
I don't know where I am on that.
Lex Fridman (2:39:53.700)
It is just one of those things that's always in the back of my mind.
Lex Fridman (2:40:00.860)
And I don't think that's uncommon.
Lex Fridman (2:40:02.540)
Well it's just like G factor and intelligence, it's a hard truth that's there.
Lex Fridman (2:40:09.420)
And sometimes you kind of walk past it and you don't want to look at it, but it's still
Lex Fridman (2:40:15.100)
there.
Richard Haier (2:40:16.100)
Yeah.
Lex Fridman (2:40:17.100)
Yes, you can't escape it.
Lex Fridman (2:40:20.180)
And the thing about the G factor and intelligence is everybody knows this is true on a personal
Lex Fridman (2:40:28.220)
daily basis.
Richard Haier (2:40:31.020)
Even if you think back to when you were in school, you know who the smart kids were.
Richard Haier (2:40:39.060)
When you are on the phone talking to a customer service representative, that in response to
Richard Haier (2:40:44.820)
your detailed question is reading a script back to you and you get furious at this.
Lex Fridman (2:40:52.860)
Have you ever called this person a moron or wanted to call this person a moron?
Richard Haier (2:40:56.920)
You're not listening to me.
Richard Haier (2:40:58.820)
Everybody has had the experience of dealing with people who they think are not at their
Richard Haier (2:41:03.380)
level.
Lex Fridman (2:41:05.260)
It's just common because that's the way human beings are.
Richard Haier (2:41:09.460)
That's the way life is.
Lex Fridman (2:41:11.140)
But we also have a poor estimation of our own intelligence.
Richard Haier (2:41:16.380)
We have a poor, and we're not always a great, our judgment of human character of other people
Lex Fridman (2:41:22.980)
is not as good as a battery of tests.
Richard Haier (2:41:29.100)
That's where bias comes in.
Lex Fridman (2:41:31.780)
That's where our history, our emotions, all of that comes in.
Richard Haier (2:41:35.660)
So, you know, people on the internet, you know, there's such a thing as the internet
Lex Fridman (2:41:39.860)
and people on the internet will call each other dumb all the time.
Richard Haier (2:41:45.940)
You know, that's the worry here is that we give up on people.
Richard Haier (2:41:53.060)
We put them in a bin just because of one interaction or some small number of interactions as if
Richard Haier (2:42:00.140)
that's it.
Lex Fridman (2:42:01.140)
They're hopeless.
Richard Haier (2:42:02.140)
That's just in their genetics.
Lex Fridman (2:42:03.640)
But I think no matter what the science here says, once again, that does not mean we should
Richard Haier (2:42:11.140)
not have compassion for our fellow man.
Lex Fridman (2:42:15.020)
That's exactly what the science does say.
Richard Haier (2:42:17.700)
It's not opposite of what the science says.
Richard Haier (2:42:22.540)
Everything I know about psychology, everything I've learned about intelligence, everything
Richard Haier (2:42:30.120)
points to the inexorable conclusion that you have to treat people as individuals respectfully
Lex Fridman (2:42:38.220)
and with compassion.
Richard Haier (2:42:40.060)
Because through no fault of their own, some people are not as capable as others.
Lex Fridman (2:42:46.780)
And you want to turn a blind eye to it, you want to come up with theories about why that
Richard Haier (2:42:52.220)
might be true, fine.
Lex Fridman (2:42:54.300)
I would like to fix some of it as best I can.
Lex Fridman (2:42:58.060)
And everybody is deserving of love.
Lex Fridman (2:43:01.860)
Richard, this is a good way to end it, I think.
Richard Haier (2:43:05.340)
I'm just getting warmed up here.
Lex Fridman (2:43:07.500)
I know.
Richard Haier (2:43:08.500)
I know you can go for another many hours, but to respect your extremely valuable time,
Lex Fridman (2:43:15.220)
this is an amazing conversation.
Richard Haier (2:43:16.780)
Thank you for the teaching company, the lectures you've given with the New York Science of
Lex Fridman (2:43:23.900)
Intelligence.
Richard Haier (2:43:24.900)
Thank you for everything you're doing, it's a difficult topic, it's a topic that's controversial
Lex Fridman (2:43:29.700)
and sensitive to people and to push forward boldly and in that nuanced way, just thank
Richard Haier (2:43:35.820)
you for everything you do.
Lex Fridman (2:43:37.060)
And thank you for asking the big questions of intelligence, of consciousness.
Richard Haier (2:43:42.260)
Well thank you for asking me.
Lex Fridman (2:43:43.500)
I mean, there's nothing like good conversation on these topics.
Richard Haier (2:43:47.660)
Thanks for listening to this conversation with Richard Haier.
Lex Fridman (2:43:50.100)
To support this podcast, please check out our sponsors in the description.
Lex Fridman (2:43:54.540)
And now, let me leave you with some words from Albert Einstein.
Lex Fridman (2:43:57.620)
It is not that I'm so smart, but I stay with the questions much longer.
Richard Haier (2:44:04.460)
Thank you for listening and hope to see you next time.
Lex Fridman (30:03.300)
how do you figure?
Richard Haier (30:04.580)
Right, so now there's good research that gives you
Lex Fridman (30:08.980)
intuition about what kind of questions get at it,
Richard Haier (30:11.940)
just like there's something I've done,
Lex Fridman (30:18.100)
I've actually used for research in semi autonomous vehicle,
Richard Haier (30:21.980)
like whether humans are paying attention,
Lex Fridman (30:24.360)
there's a body of literature that does end back test,
Richard Haier (30:28.460)
for example, we have to put workload on the brain
Lex Fridman (30:35.620)
to do recall, memory recall, and that helps you
Richard Haier (30:38.900)
kind of put some work onto the brain
Lex Fridman (30:42.080)
while the person is doing some other task,
Lex Fridman (30:44.260)
and does some interesting research with that.
Lex Fridman (30:47.700)
But that's loading the memory,
Lex Fridman (30:48.940)
so there's like research around stably
Lex Fridman (30:52.260)
what that means about the human mind,
Lex Fridman (30:54.100)
and here you're saying recall backwards
Lex Fridman (30:58.420)
is a good protector.
Richard Haier (31:00.020)
It's a transformation.
Lex Fridman (31:01.740)
Yeah, so you have to do some,
Richard Haier (31:05.700)
like you have to load that into your brain,
Lex Fridman (31:07.820)
and not just remember it, but do something with it.
Richard Haier (31:11.220)
Right, here's another example of a different kind of test
Lex Fridman (31:14.460)
called the Hick paradigm, and it's not verbal at all.
Richard Haier (31:18.360)
It's a little box, and there are a series of lights
Lex Fridman (31:23.140)
arranged in a semi circle at the top of the box,
Lex Fridman (31:27.420)
and then there's a home button that you press,
Lex Fridman (31:31.300)
and when one of the lights goes on,
Richard Haier (31:34.740)
there's a button next to each of those lights,
Lex Fridman (31:37.780)
you take your finger off the home button,
Lex Fridman (31:39.940)
and you just press the button
Lex Fridman (31:41.980)
next to the light that goes on,
Lex Fridman (31:44.100)
and so it's a very simple reaction time.
Lex Fridman (31:46.500)
Light goes on, as quick as you can, you press the button,
Lex Fridman (31:49.220)
and you get a reaction time
Lex Fridman (31:50.540)
from the moment you lift your finger off the button
Richard Haier (31:53.700)
to when you press the button where the light is.
Lex Fridman (31:58.620)
That reaction time doesn't really correlate
Richard Haier (32:02.160)
with IQ very much, but if you change the instructions,
Lex Fridman (32:07.160)
and you say three lights are gonna come on simultaneously,
Richard Haier (32:13.120)
I want you to press the button next to the light
Lex Fridman (32:15.600)
that's furthest from the other two.
Lex Fridman (32:19.000)
So maybe lights one and two go on,
Lex Fridman (32:21.260)
and light six goes on simultaneously.
Richard Haier (32:24.320)
You take your finger off,
Lex Fridman (32:25.560)
and you would press the button by light six.
Richard Haier (32:28.880)
That's that reaction time to a more complex task.
Lex Fridman (32:34.360)
It's not really hard.
Richard Haier (32:36.360)
Almost everybody gets it all right,
Lex Fridman (32:38.360)
but your reaction time to that
Richard Haier (32:41.040)
is highly correlated with the G factor.
Lex Fridman (32:43.760)
This is fascinating.
Lex Fridman (32:45.040)
So reaction time, so there's a temporal aspect to this.
Lex Fridman (32:48.560)
So what role does time?
Richard Haier (32:50.040)
Speed of processing.
Lex Fridman (32:50.880)
It's the speed of processing.
Richard Haier (32:53.020)
Is this also true for ones that take longer,
Lex Fridman (32:55.740)
like five, 10, 30 seconds?
Lex Fridman (32:58.640)
Is time part of the measure with some of these things?
Lex Fridman (33:01.200)
Yes, and that is why some of the best IQ tests
Richard Haier (33:05.640)
have a time limit, because if you have no time limit,
Lex Fridman (33:10.640)
people can do better,
Lex Fridman (33:12.960)
but it doesn't distinguish among people that well.
Lex Fridman (33:17.760)
So that adding the time element is important.
Lex Fridman (33:21.440)
So speed of information processing,
Lex Fridman (33:25.020)
and reaction time is a measure
Richard Haier (33:26.680)
of speed of information processing,
Lex Fridman (33:29.320)
turns out to be related to the G factor.
Lex Fridman (33:31.920)
But the G factor only accounts for maybe half
Lex Fridman (33:35.040)
or some amount on the test performance.
Richard Haier (33:37.540)
For example, I get pretty bad test anxiety.
Lex Fridman (33:42.040)
Like I was never, I mean,
Richard Haier (33:46.280)
I just don't enjoy tests.
Lex Fridman (33:47.800)
I enjoy going back into my cave and working.
Richard Haier (33:51.280)
Like I've always enjoyed homework way more than tests,
Lex Fridman (33:56.080)
no matter how hard the homework is,
Richard Haier (33:57.800)
because I can go back to the cave
Lex Fridman (33:59.360)
and hide away and think deeply.
Richard Haier (34:00.760)
There's something about being watched
Lex Fridman (34:02.700)
and having a time limit that really makes me anxious,
Lex Fridman (34:06.040)
and I can just see the mind not operating optimally at all.
Lex Fridman (34:10.200)
But you're saying underneath there,
Richard Haier (34:11.640)
there's still a G factor, there's still.
Lex Fridman (34:13.720)
No question, there's no question.
Richard Haier (34:16.400)
Boy.
Lex Fridman (34:17.240)
And if you get anxious taking the test,
Richard Haier (34:19.280)
many people say, oh, I didn't do well,
Lex Fridman (34:20.880)
because I'm anxious.
Richard Haier (34:23.560)
I hear that a lot.
Lex Fridman (34:24.760)
Say, well, fine, if you're really anxious during the test,
Richard Haier (34:28.360)
the score will be a bad estimate of your G factor.
Lex Fridman (34:32.040)
It doesn't mean the G factor isn't there.
Richard Haier (34:34.040)
That's right.
Lex Fridman (34:34.880)
And by the way, standardized tests like the SAT,
Richard Haier (34:40.600)
they're essentially intelligence tests.
Lex Fridman (34:43.060)
They are highly G loaded.
Richard Haier (34:45.240)
Now, the people who make the SAT don't wanna mention that.
Lex Fridman (34:50.680)
They have enough trouble justifying standardized testing,
Lex Fridman (34:54.020)
but to call it an intelligence test
Lex Fridman (34:56.020)
is really beyond the pale.
Lex Fridman (34:58.440)
But in fact, it's so highly correlated,
Lex Fridman (35:00.720)
because it's a reasoning test.
Richard Haier (35:03.200)
SAT is a reasoning test,
Lex Fridman (35:04.880)
a verbal reasoning, mathematical reasoning.
Lex Fridman (35:08.240)
And if it's a reasoning test, it has to be related to G.
Lex Fridman (35:14.040)
But if people go in and take a standardized test,
Richard Haier (35:17.560)
whether it's an IQ test or the SAT,
Lex Fridman (35:20.120)
and they happen to be sick that day with 102 fever,
Richard Haier (35:24.600)
the score is not going to be a good estimate of their G.
Lex Fridman (35:29.600)
If they retake the test when they're not anxious
Richard Haier (35:33.080)
or less anxious or don't have a fever,
Lex Fridman (35:36.640)
the score will go up, and that will be a better estimate.
Lex Fridman (35:39.960)
But you can't say their G factor increased
Lex Fridman (35:43.120)
between the two tests.
Richard Haier (35:45.160)
Well, it's interesting.
Lex Fridman (35:46.480)
So the question is how wide of a battery of tests
Lex Fridman (35:50.000)
is required to estimate the G factor well?
Lex Fridman (35:53.380)
Because I'll give you as my personal example,
Richard Haier (35:55.180)
I took the SAT in, I think it was called the ACT,
Lex Fridman (35:58.720)
where I was two, also, I took SAT many times.
Richard Haier (36:02.880)
Every single time, I got it perfect on math.
Lex Fridman (36:05.480)
And verbal, the time limit on the verbal
Richard Haier (36:08.760)
made me very anxious.
Lex Fridman (36:10.960)
I did not, I mean, part of it,
Richard Haier (36:12.480)
I didn't speak English very well.
Lex Fridman (36:14.200)
But honestly, it was like you're supposed to remember stuff,
Lex Fridman (36:17.440)
and I was so anxious.
Lex Fridman (36:18.760)
And as I'm reading, I'm sweating, I can't,
Richard Haier (36:21.640)
you know that feeling you have when you're reading a book
Lex Fridman (36:26.640)
and you just read a page and you know nothing
Richard Haier (36:30.100)
about what you've read because you zoned out.
Lex Fridman (36:32.600)
That's the same feeling of like, I can't, I have to,
Richard Haier (36:36.440)
you're like, nope, read and understand.
Lex Fridman (36:39.480)
And that anxiety is like, and you start seeing
Richard Haier (36:43.080)
like the typography versus the content of the words.
Lex Fridman (36:47.120)
Like that was, I don't, it's interesting
Richard Haier (36:50.300)
because I know that what they're measuring,
Lex Fridman (36:55.300)
I could see being correlated with something.
Lex Fridman (36:58.780)
But that anxiety or some aspect of the performance
Lex Fridman (37:04.660)
sure plays a factor.
Lex Fridman (37:07.020)
And I wonder how you sneak up in a stable way.
Lex Fridman (37:10.420)
I mean, this is a broader discussion
Richard Haier (37:11.900)
about like standardized testing, how you sneak up,
Lex Fridman (37:16.580)
how you get at the fact that I'm super anxious
Lex Fridman (37:19.860)
and still nevertheless measure some aspect
Lex Fridman (37:22.260)
of my ontology.
Richard Haier (37:23.100)
I wonder, I don't know.
Lex Fridman (37:24.860)
I don't know if you can say to that,
Richard Haier (37:26.640)
that time limit sure is a pain.
Lex Fridman (37:28.540)
Well, let me say this.
Richard Haier (37:30.540)
There are two ways to approach the very real problem
Lex Fridman (37:34.160)
that you say that some people just get anxious
Richard Haier (37:36.880)
or not good test takers.
Lex Fridman (37:38.940)
By the way, part of testing is you know the answer,
Richard Haier (37:45.620)
you can figure out the answer or you can't.
Lex Fridman (37:49.980)
If you don't know the answer, there are many reasons
Richard Haier (37:52.900)
you don't know the answer at that particular moment.
Lex Fridman (37:55.240)
You may have learned it once and forgotten it.
Richard Haier (37:58.460)
It may be on the tip of your tongue
Lex Fridman (38:00.580)
and you just can't get it
Richard Haier (38:01.860)
because you're anxious about the time limit.
Lex Fridman (38:03.820)
You may never have learned it.
Richard Haier (38:05.860)
You may have been exposed to it,
Lex Fridman (38:08.660)
but it was too complicated and you couldn't learn it.
Richard Haier (38:11.420)
I mean, there are all kinds of reasons here.
Lex Fridman (38:13.940)
But for an individual to interpret your scores
Richard Haier (38:18.820)
as an individual, whoever is interpreting the score
Lex Fridman (38:23.340)
has to take into account various things
Richard Haier (38:26.260)
that would affect your individual score.
Lex Fridman (38:29.220)
And that's why decisions about college admission
Richard Haier (38:32.740)
or anything else where tests are used
Lex Fridman (38:35.720)
are hardly ever the only criterion to make a decision.
Lex Fridman (38:42.340)
And I think people are, college admissions
Lex Fridman (38:45.160)
letting go of that very much.
Richard Haier (38:46.860)
Oh yes, yeah.
Lex Fridman (38:48.100)
But what does that even mean?
Richard Haier (38:51.060)
Because is it possible to design standardized tests
Lex Fridman (38:55.540)
that do get, that are useful to college admissions?
Richard Haier (38:58.420)
Well, they already exist.
Lex Fridman (38:59.740)
The SAT is highly correlated with many aspects
Richard Haier (39:03.820)
of success at college.
Lex Fridman (39:05.220)
Here's the problem.
Lex Fridman (39:06.380)
So maybe you could speak to this.
Lex Fridman (39:09.220)
The correlation across the population versus individuals.
Lex Fridman (39:13.360)
So our criminal justice system is designed to make sure,
Lex Fridman (39:23.420)
wow, it's still, there's tragic cases
Richard Haier (39:27.300)
where innocent people go to jail,
Lex Fridman (39:29.700)
but you try to avoid that.
Lex Fridman (39:31.340)
And the same way with testing,
Lex Fridman (39:34.460)
it just, it would suck for an SAT to miss genius.
Richard Haier (39:38.780)
Yes, and it's possible, but it's statistically unlikely.
Lex Fridman (39:43.260)
So it really comes down to which piece of information
Richard Haier (39:51.980)
maximizes your decision making ability.
Lex Fridman (39:58.640)
So if you just use high school grades, it's okay.
Lex Fridman (40:05.300)
But you will miss some people
Lex Fridman (40:07.100)
who just don't do well in high school,
Lex Fridman (40:09.140)
but who are actually pretty smart,
Lex Fridman (40:11.340)
smart enough to be bored silly in high school,
Lex Fridman (40:13.980)
and they don't care,
Lex Fridman (40:14.940)
and their high school GPA isn't that good.
Lex Fridman (40:17.780)
So you will miss them in the same sense
Lex Fridman (40:21.380)
that somebody who could be very able and ready for college
Richard Haier (40:25.980)
just doesn't do well on their SAT.
Lex Fridman (40:28.340)
This is why you make decisions
Richard Haier (40:31.660)
with taking in a variety of information.
Lex Fridman (40:36.140)
The other thing I wanted to say,
Richard Haier (40:38.080)
I talked about when you make a decision for an individual,
Lex Fridman (40:43.840)
statistically for groups,
Richard Haier (40:46.720)
there are many people who have a disparity
Lex Fridman (40:50.060)
between their math score and their verbal score.
Richard Haier (40:53.080)
That disparity, or the other way around,
Lex Fridman (40:55.640)
that disparity is called tilt.
Richard Haier (40:58.680)
The score is tilted one way or the other.
Lex Fridman (41:01.760)
And that tilt has been studied empirically
Richard Haier (41:05.080)
to see what that predicts.
Lex Fridman (41:07.320)
And in fact, you can't make predictions
Richard Haier (41:09.440)
about college success based on tilt.
Lex Fridman (41:14.880)
And mathematics is a good example.
Richard Haier (41:16.760)
There are many people,
Lex Fridman (41:18.320)
especially non native speakers of English
Richard Haier (41:20.680)
who come to this country,
Lex Fridman (41:22.360)
take the SATs, do very well on the math
Lex Fridman (41:24.940)
and not so well on the verbal.
Lex Fridman (41:26.840)
Well, if they're applying to a math program,
Richard Haier (41:31.220)
the professors there who are making the decision
Lex Fridman (41:33.840)
or the admissions officers
Richard Haier (41:35.400)
don't wait so much to score on verbal,
Lex Fridman (41:39.600)
especially if it's a non native speaker.
Richard Haier (41:42.080)
Well, so yeah, you have to try to,
Lex Fridman (41:44.480)
in the admission process, bring in the context.
Lex Fridman (41:47.800)
But non native isn't really the problem.
Lex Fridman (41:50.760)
I mean, that was part of the problem for me.
Lex Fridman (41:53.720)
But it's the anxiety was, which it's interesting.
Lex Fridman (41:57.980)
It's interesting.
Richard Haier (41:59.000)
Oh boy, reducing yourself down to numbers.
Lex Fridman (42:06.560)
But it's still true.
Richard Haier (42:07.880)
It's still the truth.
Lex Fridman (42:09.240)
It's a painful truth.
Richard Haier (42:10.760)
That same anxiety that led me to be,
Lex Fridman (42:16.080)
to struggle with the SAT verbal tests
Richard Haier (42:20.600)
is still within me in all ways of life.
Lex Fridman (42:24.640)
So maybe that's not anxiety.
Richard Haier (42:26.880)
Maybe that's something, like personality
Lex Fridman (42:30.600)
is also pretty stable.
Richard Haier (42:32.380)
Personality is stable.
Lex Fridman (42:34.480)
Personality does impact the way you navigate life.
Richard Haier (42:41.040)
There's no question.
Lex Fridman (42:42.460)
Yeah, and we should say that the G factor in intelligence
Richard Haier (42:45.680)
is not just about some kind of number on a paper.
Lex Fridman (42:50.400)
It also has to do with how you navigate life.
Lex Fridman (42:53.940)
How easy life is for you in this very complicated world.
Lex Fridman (43:00.880)
So personality's all tied into that
Richard Haier (43:02.880)
in some deep fundamental way.
Lex Fridman (43:05.880)
But now you've hit the key point
Richard Haier (43:07.720)
about why we even want to study intelligence.
Lex Fridman (43:11.320)
And personality, I think, to a lesser extent.
Lex Fridman (43:13.360)
But that's my interest, is more on intelligence.
Lex Fridman (43:17.480)
I went to graduate school and wanted to study personality,
Lex Fridman (43:20.140)
but that's kind of another story
Lex Fridman (43:22.640)
how I got kind of shifted from personality research
Richard Haier (43:25.240)
over to intelligence research.
Lex Fridman (43:27.440)
Because it's not just a number.
Richard Haier (43:30.040)
Intelligence is not just an IQ score.
Lex Fridman (43:32.560)
It's not just an SAT score.
Richard Haier (43:34.740)
It's what those numbers reflect about your ability
Lex Fridman (43:39.740)
to navigate everyday life.
Richard Haier (43:43.000)
It has been said that life is one long intelligence test.
Lex Fridman (43:48.000)
And who can't relate to that?
Lex Fridman (43:55.400)
And if you doubt, see, another problem here
Lex Fridman (43:58.440)
is a lot of critics of intelligence research,
Richard Haier (44:00.800)
intelligence testing, tend to be academics
Lex Fridman (44:04.000)
who, by and large, are pretty smart people.
Lex Fridman (44:07.240)
And pretty smart people, by and large,
Lex Fridman (44:10.040)
have enormous difficulty understanding
Lex Fridman (44:12.900)
what the world is like for people with IQs of 80 or 75.
Lex Fridman (44:18.960)
It is a completely different everyday experience.
Richard Haier (44:23.040)
Even IQ scores of 85, 90.
Lex Fridman (44:27.900)
You know, there's a popular television program, Judge Judy.
Richard Haier (44:32.040)
Judge Judy deals with everyday people
Lex Fridman (44:35.340)
with everyday problems, and you can see the full range
Richard Haier (44:39.440)
of problem solving ability demonstrated there.
Lex Fridman (44:43.240)
And sometimes she does it for laughs,
Lex Fridman (44:45.300)
but it really isn't funny because people who are,
Lex Fridman (44:52.460)
there are people who are very limited
Richard Haier (44:54.500)
in their life navigation, let alone success,
Lex Fridman (45:00.040)
by not having good reasoning skills, which cannot be taught.
Richard Haier (45:05.040)
We know this, by the way, because there are many efforts.
Lex Fridman (45:07.720)
You know, the United States military,
Richard Haier (45:09.320)
which excels at training people,
Lex Fridman (45:12.160)
I mean, I don't know that there's a better organization
Richard Haier (45:14.560)
in the world for training diverse people,
Lex Fridman (45:18.360)
and they won't take people with IQs under,
Richard Haier (45:20.760)
I think, 83 is the cutoff, because they have found
Lex Fridman (45:24.960)
they are unable to train people with lower IQs
Richard Haier (45:30.520)
to do jobs in the military.
Lex Fridman (45:32.520)
So one of the things that G Factor has to do is learning.
Richard Haier (45:35.720)
Absolutely, some people learn faster than others.
Lex Fridman (45:40.720)
Some people learn more than others.
Richard Haier (45:43.520)
Now, faster, by the way, is not necessarily better,
Lex Fridman (45:47.320)
as long as you get to the same place eventually.
Richard Haier (45:51.720)
But, you know, there are professional schools
Lex Fridman (45:54.320)
that want students who can learn the fastest
Richard Haier (45:57.640)
because they can learn more or learn better.
Lex Fridman (46:01.520)
Or learn deeper, or all kinds of ideas
Richard Haier (46:06.880)
about why you select people with the highest scores.
Lex Fridman (46:09.560)
And there's nothing funnier, by the way,
Richard Haier (46:12.640)
to listen to a bunch of academics
Lex Fridman (46:15.120)
complain about the concept of intelligence
Lex Fridman (46:17.680)
and intelligence testing, and then you go
Lex Fridman (46:20.120)
to a faculty meeting where they're discussing
Richard Haier (46:22.480)
who to hire among the applicants.
Lex Fridman (46:24.760)
And all they talk about is how smart the person is.
Richard Haier (46:28.400)
We'll get to that, we'll sneak up to that in different ways,
Lex Fridman (46:31.200)
but there's something about reducing a person
Richard Haier (46:33.000)
to a number that in part is grounded
Lex Fridman (46:35.280)
to the person's genetics that makes people very uncomfortable.
Lex Fridman (46:38.800)
But nobody does that.
Lex Fridman (46:40.480)
Nobody in the field actually does that.
Richard Haier (46:43.760)
That is a worry that is a worry like,
Lex Fridman (46:54.040)
well, I don't wanna call it a conspiracy theory.
Richard Haier (46:55.880)
I mean, it's a legitimate worry,
Lex Fridman (46:58.360)
but it just doesn't happen.
Richard Haier (47:01.400)
Now, I had a professor in graduate school
Lex Fridman (47:03.600)
who was the only person I ever knew
Richard Haier (47:05.880)
who considered the students only by their test scores.
Lex Fridman (47:12.680)
And later in his life, he kind of backed off that.
Richard Haier (47:19.480)
Let me ask you this, so we'll jump around,
Lex Fridman (47:21.560)
I'll come back to it, but I tend to,
Richard Haier (47:26.560)
I've had like political discussions with people
Lex Fridman (47:29.520)
and actually my friend Michael Malice, he's an anarchist.
Richard Haier (47:36.240)
I disagree with him on basically everything
Lex Fridman (47:39.200)
except the fact that love is a beautiful thing in this world.
Lex Fridman (47:47.360)
And he says this test about left versus right,
Lex Fridman (47:50.600)
whatever, it doesn't matter what the test is,
Lex Fridman (47:52.280)
but he believes, the question is,
Lex Fridman (47:54.920)
do you believe that some people are better than others?
Richard Haier (48:00.360)
Question is ambiguous.
Lex Fridman (48:03.880)
Do you believe some people are better than others?
Lex Fridman (48:06.120)
And to me, sort of the immediate answer is no.
Lex Fridman (48:11.720)
It's a poetic question, it's an ambiguous question, right?
Richard Haier (48:15.680)
Like people wanna maybe the temptation
Lex Fridman (48:19.520)
to ask better at what, better at like sports and so on.
Richard Haier (48:23.440)
No, to me, I stand with the sort of defining documents
Lex Fridman (48:28.440)
of this country, which is all men are created equal.
Richard Haier (48:32.280)
There's a basic humanity.
Lex Fridman (48:34.320)
And there's something about tests of intelligence.
Richard Haier (48:39.840)
Just knowing that some people are different,
Lex Fridman (48:43.380)
like the science of intelligence that shows
Richard Haier (48:45.400)
that some people are genetically
Lex Fridman (48:49.200)
in some stable way across a lifetime,
Richard Haier (48:52.400)
have a greater intelligence than others,
Lex Fridman (48:56.160)
makes people feel like some people are better than others.
Lex Fridman (49:01.200)
And that makes them very uncomfortable.
Lex Fridman (49:03.600)
And I, maybe you can speak to that.
Richard Haier (49:06.960)
The fact that some people are more intelligent than others
Lex Fridman (49:09.560)
in a way that's, cannot be compensated
Richard Haier (49:14.560)
through education, through anything you do in life.
Lex Fridman (49:22.600)
What do we do with that?
Richard Haier (49:24.480)
Okay, there's a lot there.
Lex Fridman (49:26.800)
We haven't really talked about the genetics of it yet.
Lex Fridman (49:29.960)
But you are correct in that it is my interpretation
Lex Fridman (49:35.240)
of the data that genetics has a very important influence
Richard Haier (49:39.720)
on the G factor.
Lex Fridman (49:41.380)
And this is controversial, and we can talk about it,
Lex Fridman (49:44.440)
but if you think that genetics,
Lex Fridman (49:47.040)
that genes are deterministic, are always deterministic,
Richard Haier (49:50.800)
that leads to kind of the worry that you expressed.
Lex Fridman (49:55.320)
But we know now in the 21st century
Richard Haier (49:58.600)
that many genes are not deterministic,
Lex Fridman (50:00.920)
that are probabilistic,
Richard Haier (50:02.800)
meaning their gene expression can be influenced.
Lex Fridman (50:09.880)
Now, whether they're influenced only
Richard Haier (50:11.320)
by other biological variables or other genetic variables
Lex Fridman (50:16.440)
or environmental or cultural variables,
Richard Haier (50:19.160)
that's where the controversy comes in.
Lex Fridman (50:23.040)
And we can discuss that in more detail if you like.
Lex Fridman (50:27.200)
But to go to the question about better, are people better?
Lex Fridman (50:31.700)
There's zero evidence that smart people are better
Richard Haier (50:36.700)
with respect to important aspects of life,
Lex Fridman (50:43.300)
like honesty, even likability.
Richard Haier (50:47.460)
I'm sure you know many very intelligent people
Lex Fridman (50:50.340)
who are not terribly likable or terribly kind
Richard Haier (50:53.420)
or terribly honest.
Lex Fridman (50:55.480)
Is there something to be said?
Lex Fridman (50:56.760)
So one of the things I've recently reread
Lex Fridman (50:59.820)
for the second time, I guess that's what the word reread
Richard Haier (51:03.860)
means, the rise and fall of the Third Reich,
Lex Fridman (51:08.980)
which is, I think, the best telling
Richard Haier (51:12.060)
of the rise and fall of Hitler.
Lex Fridman (51:14.700)
And one of the interesting things about the people that,
Lex Fridman (51:20.900)
how should I say it?
Lex Fridman (51:27.260)
Justified or maybe propped up the ideas
Richard Haier (51:32.260)
that Hitler put forward is the fact
Lex Fridman (51:35.940)
that they were extremely intelligent.
Richard Haier (51:38.460)
They were the intellectual class.
Lex Fridman (51:41.780)
They were like, it was obvious that they thought
Richard Haier (51:46.260)
very deeply and rationally about the world.
Lex Fridman (51:49.540)
So what I would like to say is one of the things
Richard Haier (51:52.660)
that shows to me is some of the worst atrocities
Lex Fridman (51:56.100)
in the history of humanity have been committed
Richard Haier (51:58.860)
by very intelligent people.
Lex Fridman (52:00.780)
So that means that intelligence
Richard Haier (52:04.620)
doesn't make you a good person.
Lex Fridman (52:06.420)
I wonder if there's a G factor for intelligence.
Richard Haier (52:12.820)
I wonder if there's a G factor for goodness.
Lex Fridman (52:16.860)
The Nietzschean good and evil,
Richard Haier (52:19.180)
of course that's probably harder to measure
Lex Fridman (52:21.740)
because it's such a subjective thing
Lex Fridman (52:23.240)
what it means to be good.
Lex Fridman (52:25.340)
And even the idea of evil is a deeply uncomfortable thing
Lex Fridman (52:29.380)
because how do we know?
Lex Fridman (52:31.340)
But it's independent, whatever it is,
Richard Haier (52:33.500)
it's independent of intelligence.
Lex Fridman (52:35.900)
So I agree with you about that.
Lex Fridman (52:37.980)
But let me say this.
Lex Fridman (52:39.340)
I have also asserted my belief
Richard Haier (52:44.100)
that more intelligence is better than less.
Lex Fridman (52:49.220)
That doesn't mean more intelligent people are better people
Lex Fridman (52:54.400)
but all things being equal,
Lex Fridman (52:55.860)
would you like to be smarter or less smart?
Lex Fridman (52:58.740)
So if I had a pill, I have two pills.
Lex Fridman (53:01.260)
I said, this one will make you smarter,
Richard Haier (53:02.880)
this one will make you dumber.
Lex Fridman (53:04.500)
Which one would you like?
Richard Haier (53:06.460)
Are there any circumstances
Lex Fridman (53:07.940)
under which you would choose to be dumber?
Richard Haier (53:09.860)
Well, let me ask you this.
Lex Fridman (53:11.560)
That's a very nuanced and interesting question.
Lex Fridman (53:16.040)
There's been books written about this, right?
Lex Fridman (53:19.680)
Now we'll return to the hard questions,
Richard Haier (53:21.680)
the interesting questions,
Lex Fridman (53:22.760)
but let me ask about human happiness.
Lex Fridman (53:25.780)
Does intelligence lead to happiness?
Lex Fridman (53:29.180)
No.
Richard Haier (53:31.820)
So, okay, so back to the pill then.
Lex Fridman (53:34.980)
So when would you take the pill?
Lex Fridman (53:38.980)
So you said IQ 80, 90, 100, 110,
Lex Fridman (53:44.860)
you start going through the quartiles
Lex Fridman (53:46.580)
and is it obvious?
Lex Fridman (53:50.220)
Isn't there diminishing returns
Lex Fridman (53:54.500)
and then it starts becoming negative?
Lex Fridman (53:57.420)
This is an empirical question.
Lex Fridman (54:00.420)
And so that I have advocated in many forums
Lex Fridman (54:06.160)
more research on enhancing the G factor.
Richard Haier (54:11.820)
Right now there have been many claims
Lex Fridman (54:14.420)
about enhancing intelligence with,
Richard Haier (54:17.320)
you mentioned the NBAC training,
Lex Fridman (54:19.100)
it was a big deal a few years ago, it doesn't work.
Richard Haier (54:22.640)
Data is very clear, it does not work.
Lex Fridman (54:25.740)
Or doing like memory tests, like training and so on.
Richard Haier (54:28.580)
Yeah, it may give you a better memory in the short run,
Lex Fridman (54:32.700)
but it doesn't impact your G factor.
Richard Haier (54:38.140)
It was very popular a couple of decades ago
Lex Fridman (54:40.940)
that the idea that listening to Mozart
Richard Haier (54:44.560)
could make you more intelligent.
Lex Fridman (54:46.580)
There was a paper published on this
Richard Haier (54:48.180)
with somebody I knew published this paper,
Lex Fridman (54:50.300)
and intelligence researchers never believed it for a second.
Richard Haier (54:54.660)
Been hundreds of studies, all the meta analyses,
Lex Fridman (54:57.820)
all the summaries and so on,
Richard Haier (54:59.420)
show that there's nothing to it, nothing to it at all.
Lex Fridman (55:05.900)
But wouldn't it be something,
Richard Haier (55:08.780)
wouldn't it be world shaking
Lex Fridman (55:12.820)
if you could take the normal distribution of intelligence,
Richard Haier (55:15.940)
which we haven't really talked about yet,
Lex Fridman (55:17.500)
but IQ scores and the G factor
Richard Haier (55:20.400)
is thought to be a normal distribution,
Lex Fridman (55:23.940)
and shift it to the right so that everybody is smarter?
Richard Haier (55:30.140)
Even a half a standard deviation would be world shaking,
Lex Fridman (55:35.340)
because there are many social problems,
Richard Haier (55:38.700)
many, many social problems that are exacerbated
Lex Fridman (55:43.260)
by people with lower ability to reason stuff out
Lex Fridman (55:48.700)
and navigate everyday life.
Lex Fridman (55:51.860)
So.
Richard Haier (55:52.700)
I wonder if there's a threshold.
Lex Fridman (55:53.780)
So maybe I would push back and say universal shifting
Richard Haier (55:59.420)
of the normal distribution
Lex Fridman (56:02.240)
may not be the optimal way of shifting.
Richard Haier (56:05.060)
Maybe it's better to,
Lex Fridman (56:07.340)
whatever the asymmetric kind of distributions
Richard Haier (56:10.620)
is like really pushing the lower up
Lex Fridman (56:13.380)
versus trying to make the people
Richard Haier (56:17.580)
at the average more intelligent.
Lex Fridman (56:19.660)
So you're saying that if in fact
Richard Haier (56:21.060)
there was some way to increase G,
Lex Fridman (56:23.540)
let's just call it metaphorically a pill, an IQ pill,
Richard Haier (56:27.800)
we should only give it to people at the lower end.
Lex Fridman (56:30.580)
No, it's just intuitively I can see
Richard Haier (56:34.940)
that life becomes easier at the lower end if it's increased.
Lex Fridman (56:39.820)
It becomes less and less,
Richard Haier (56:41.820)
it is an empirical scientific question,
Lex Fridman (56:43.580)
but it becomes less and less obvious to me
Richard Haier (56:46.180)
that more intelligence is better.
Lex Fridman (56:50.500)
At the high end, not because it would make life easier,
Lex Fridman (56:56.500)
but it would make whatever problems you're working on
Lex Fridman (57:00.980)
more solvable.
Lex Fridman (57:02.700)
And if you are working on artificial intelligence,
Lex Fridman (57:06.740)
there's a tremendous potential for that to improve society.
Richard Haier (57:13.300)
I understand.
Lex Fridman (57:14.580)
So at whatever problems you're working on, yes.
Lex Fridman (57:19.000)
But there's also the problem of the human condition.
Lex Fridman (57:21.720)
There's love, there's fear,
Lex Fridman (57:24.340)
and all of those beautiful things
Lex Fridman (57:26.700)
that sometimes if you're good at solving problems,
Richard Haier (57:29.860)
you're going to create more problems for yourself.
Lex Fridman (57:32.100)
It's, I'm not exactly sure.
Lex Fridman (57:34.460)
So ignorance is bliss is a thing.
Lex Fridman (57:37.220)
So there might be a place,
Richard Haier (57:38.400)
there might be a sweet spot of intelligence
Lex Fridman (57:40.960)
given your environment, given your personality,
Richard Haier (57:43.680)
all of those kinds of things.
Lex Fridman (57:45.060)
And that becomes less beautifully complicated
Richard Haier (57:48.140)
the more and more intelligent you become.
Lex Fridman (57:50.500)
But that's a question for literature,
Richard Haier (57:53.160)
not for science perhaps.
Lex Fridman (57:54.700)
Well, imagine this.
Richard Haier (57:56.180)
Imagine there was an IQ pill
Lex Fridman (57:58.380)
and it was developed by a private company
Lex Fridman (58:01.580)
and they are willing to sell it to you.
Lex Fridman (58:05.060)
And whatever price they put on it,
Richard Haier (58:07.900)
you are willing to pay it
Lex Fridman (58:09.500)
because you would like to be smarter.
Lex Fridman (58:11.780)
But just before they give you a pill,
Lex Fridman (58:14.360)
they give you a disclaimer form to sign.
Richard Haier (58:18.380)
Yes.
Lex Fridman (58:20.020)
Don't hold us,
Richard Haier (58:22.620)
you understand that this pill has no guarantee
Lex Fridman (58:25.180)
that your life is going to be better
Lex Fridman (58:26.980)
and in fact it could be worse.
Lex Fridman (58:28.920)
Well, yes, that's how lawyers work.
Lex Fridman (58:32.200)
But I would love for science to answer the question
Lex Fridman (58:35.160)
to try to predict if your life
Richard Haier (58:36.640)
is going to be better or worse
Lex Fridman (58:38.680)
when you become more or less intelligent.
Richard Haier (58:41.200)
It's a fascinating question
Lex Fridman (58:43.260)
about what is the sweet spot for the human condition.
Richard Haier (58:47.600)
Some of the things we see as bugs
Lex Fridman (58:49.820)
might be actually features,
Richard Haier (58:51.900)
may be crucial to our overall happiness
Lex Fridman (58:55.360)
as our limitations might lead to more happiness than less.
Lex Fridman (58:59.120)
But again, more intelligence is better at the lower end.
Lex Fridman (59:02.720)
That's more, that's something that's less arguable
Lex Fridman (59:06.480)
and fascinating if possible to increase.
Lex Fridman (59:10.080)
But you know, there's virtually no research
Richard Haier (59:12.960)
that's based on a neuroscience approach
Lex Fridman (59:15.320)
to solving that problem.
Richard Haier (59:17.520)
All the solutions that have been proposed
Lex Fridman (59:20.800)
to solve that problem or to ameliorate that problem
Richard Haier (59:25.400)
are essentially based on the blank slate assumption
Lex Fridman (59:29.680)
that enriching the environment, removing barriers,
Richard Haier (59:34.680)
all good things by the way,
Lex Fridman (59:36.000)
I'm not against any of those things.
Lex Fridman (59:38.160)
But there's no empirical evidence
Lex Fridman (59:39.700)
that they're going to improve the general reasoning ability
Richard Haier (59:45.260)
or make people more employable.
Lex Fridman (59:47.920)
Have you read Flowers of Algernon? Yes.
Richard Haier (59:51.840)
That's to the question of intelligence and happiness.
Lex Fridman (59:56.240)
There are many profound aspects of that story.
Richard Haier (59:59.440)
It was a film that was very good.
🔗 相关节目