Charles Isbell and Michael Littman: Machine Learning and Education
技术与编程音乐与艺术AI 与机器学习心理与人性生物与进化
📋 章节目录
暂无章节信息
🔑 关键词
donlearninggotmichaelcharlesstatisticsgoingdidnmachineanywaygeorgiabettertruestudentsexperiencecomputerdoingsaidclasseducation
💬 精彩语录
暂无语录
🎙️ 完整对话（3333 条）
Lex Fridman (00:00.000)
The following is a conversation with Charles Isbell
以下是与查尔斯·伊斯贝尔的对话
Lex Fridman (00:02.480)
and Michael Whitman.
和迈克尔·惠特曼。
Lex Fridman (00:03.840)
Charles is the Dean of the College of Computing
查尔斯是计算机学院院长
Lex Fridman (00:06.280)
at Georgia Tech and Michael is a computer science professor
乔治亚理工学院的迈克尔是一名计算机科学教授
Lex Fridman (00:09.600)
at Brown University.
在布朗大学。
Charles Isbell and Michael Littman (00:10.840)
I've spoken with each of them individually on this podcast
我在这个播客上分别与他们每个人交谈过
Lex Fridman (00:14.880)
and since they are good friends in real life,
而且由于他们在现实生活中是好朋友，
Charles Isbell and Michael Littman (00:17.360)
we all thought it would be fun
我们都认为这会很有趣
Lex Fridman (00:19.160)
to have a conversation together.
一起交谈。
Charles Isbell and Michael Littman (00:21.760)
Quick mention of each sponsor,
快速提及每个赞助商，
Lex Fridman (00:23.400)
followed by some thoughts related to the episode.
接下来是一些与这一集相关的想法。
Charles Isbell and Michael Littman (00:26.040)
Thank you to Athletic Greens,
感谢运动绿党，
Lex Fridman (00:28.440)
the all in one drink that I start every day with
我每天开始喝的多合一饮料
Charles Isbell and Michael Littman (00:30.920)
to cover all my nutritional bases.
涵盖我所有的营养基础。
Lex Fridman (00:33.240)
Eight Sleep, a mattress that cools itself
八号睡眠，一个可以自动冷却的床垫
Lex Fridman (00:35.960)
and gives me yet another reason to enjoy sleep.
给了我另一个享受睡眠的理由。
Lex Fridman (00:38.920)
Masterclass, online courses
大师班、在线课程
Charles Isbell and Michael Littman (00:40.760)
from some of the most amazing humans in history
来自历史上一些最了不起的人
Lex Fridman (00:43.640)
and Cash App, the app I use to send money to friends.
还有 Cash App，我用来给朋友汇款的应用程序。
Charles Isbell and Michael Littman (00:47.460)
Please check out the sponsors in the description
请查看说明中的赞助商
Lex Fridman (00:49.840)
to get a discount and to support this podcast.
Charles Isbell and Michael Littman (00:53.020)
As a side note, let me say that having two guests
Lex Fridman (00:55.800)
on the podcast is an experiment
Charles Isbell and Michael Littman (00:57.800)
that I've been meaning to do for a while.
Lex Fridman (01:00.040)
In particular, because down the road,
Charles Isbell and Michael Littman (01:02.580)
I would like to occasionally be a kind of moderator
Lex Fridman (01:05.680)
for debates between people that may disagree
Charles Isbell and Michael Littman (01:09.120)
in some interesting ways.
Lex Fridman (01:10.720)
If you have suggestions for who you would like to see debate
Charles Isbell and Michael Littman (01:13.780)
on this podcast, let me know.
Lex Fridman (01:16.220)
As with all experiments of this kind,
Charles Isbell and Michael Littman (01:18.620)
it is a learning process.
Lex Fridman (01:20.600)
Both the video and the audio might need improvement.
Charles Isbell and Michael Littman (01:23.720)
I realized I think I should probably do
Lex Fridman (01:25.800)
three or more cameras next time
Charles Isbell and Michael Littman (01:27.400)
as opposed to just two.
Lex Fridman (01:28.960)
And also try different ways to mount the microphone
Charles Isbell and Michael Littman (01:31.720)
for the third person.
Lex Fridman (01:34.680)
Also, after recording this intro,
Charles Isbell and Michael Littman (01:36.900)
I'm going to have to go figure out the thumbnail
Lex Fridman (01:40.200)
for the video version of the podcast
Charles Isbell and Michael Littman (01:42.140)
since I usually put the guest's head on the thumbnail.
Lex Fridman (01:45.680)
And now there's two heads and two names
Charles Isbell and Michael Littman (01:49.800)
to try to fit into the thumbnail.
Lex Fridman (01:52.540)
It's a kind of a bin packing problem
Charles Isbell and Michael Littman (01:55.160)
which in theoretical computer science
Lex Fridman (01:58.420)
happens to be an NP hard problem.
Charles Isbell and Michael Littman (02:02.320)
Whatever I come up with, if you have better ideas
Lex Fridman (02:04.720)
for the thumbnail, let me know as well.
Lex Fridman (02:06.760)
And in general, I always welcome ideas
Lex Fridman (02:08.680)
how this thing can be improved.
Charles Isbell and Michael Littman (02:10.360)
If you enjoy it, subscribe on YouTube,
Lex Fridman (02:12.680)
review it with Five Stars and Apple Podcast,
Charles Isbell and Michael Littman (02:15.040)
follow on Spotify, support on Patreon,
Lex Fridman (02:17.600)
or connect with me on Twitter at Lex Friedman.
Lex Fridman (02:21.120)
And now here's my conversation with Charles Isbell
Lex Fridman (02:24.880)
and Michael Littman.
Charles Isbell and Michael Littman (02:27.220)
You'll probably disagree about this question,
Lex Fridman (02:30.360)
but what is your biggest, would you say, disagreement
Charles Isbell and Michael Littman (02:33.560)
about either something profound and very important
Lex Fridman (02:37.200)
or something completely not important at all?
Charles Isbell and Michael Littman (02:39.760)
I don't think we have any disagreements at all.
Lex Fridman (02:42.080)
I'm not sure that's true.
Lex Fridman (02:44.280)
We walked into that one, didn't we?
Lex Fridman (02:45.680)
So one thing that you sometimes mention is that,
Lex Fridman (02:48.840)
and we did this one on air too, as it were,
Lex Fridman (02:51.920)
whether or not machine learning
Charles Isbell and Michael Littman (02:52.960)
is computational statistics.
Lex Fridman (02:55.280)
It's not.
Lex Fridman (02:57.000)
But it is.
Lex Fridman (02:57.840)
Well, it's not.
Lex Fridman (02:58.680)
And in particular, and more importantly,
Lex Fridman (03:00.520)
it is not just computational statistics.
Lex Fridman (03:02.820)
So what's missing in the picture?
Lex Fridman (03:04.520)
All the rest of it.
Lex Fridman (03:05.520)
What's missing?
Lex Fridman (03:07.560)
That which is missing.
Charles Isbell and Michael Littman (03:08.520)
Oh, yes, well, you can't be wrong now.
Lex Fridman (03:10.360)
Well, it's not just the statistics.
Charles Isbell and Michael Littman (03:11.560)
He doesn't even believe this.
Lex Fridman (03:12.700)
We've had this conversation before.
Charles Isbell and Michael Littman (03:14.260)
If it were just the statistics,
Lex Fridman (03:15.720)
then we would be happy with where we are.
Lex Fridman (03:18.920)
But it's not just the statistics.
Lex Fridman (03:19.760)
That's why it's computational statistics.
Charles Isbell and Michael Littman (03:21.440)
Or if it were just the computational.
Lex Fridman (03:22.720)
I agree that machine learning is not just statistics.
Charles Isbell and Michael Littman (03:24.720)
It is not just statistics.
Lex Fridman (03:25.560)
We can agree on that.
Charles Isbell and Michael Littman (03:26.440)
Nor is it just computational statistics.
Lex Fridman (03:28.040)
It's computational statistics.
Charles Isbell and Michael Littman (03:29.360)
It is computational.
Lex Fridman (03:30.180)
What is the computational and computational statistics?
Lex Fridman (03:33.100)
Does this take us into the realm of computing?
Lex Fridman (03:35.120)
It does, but I think perhaps the way I can get him
Charles Isbell and Michael Littman (03:37.800)
to admit that he's wrong is that it's about rules.
Lex Fridman (03:43.280)
It's about rules.
Charles Isbell and Michael Littman (03:44.160)
It's about symbols.
Lex Fridman (03:45.040)
It's about all these other things.
Lex Fridman (03:45.880)
But statistics is not about rules?
Lex Fridman (03:47.520)
I'm gonna say statistics is about rules.
Lex Fridman (03:48.920)
But it's not just the statistics, right?
Lex Fridman (03:50.160)
It's not just a random variable that you choose
Lex Fridman (03:51.840)
and you have a probability.
Lex Fridman (03:52.680)
I think you have a narrow view of statistics.
Charles Isbell and Michael Littman (03:54.520)
Okay, well then what would be the broad view of statistics
Lex Fridman (03:56.960)
that would still allow it to be statistics
Lex Fridman (03:58.440)
and not say history that would make
Lex Fridman (04:01.360)
computational statistics okay?
Charles Isbell and Michael Littman (04:03.000)
Well, okay, so I had my first sort of research mentor,
Lex Fridman (04:07.200)
a guy named Tom Landauer,
Lex Fridman (04:09.500)
taught me to do some statistics, right?
Lex Fridman (04:12.400)
And I was annoyed all the time
Charles Isbell and Michael Littman (04:14.940)
because the statistics would say
Lex Fridman (04:16.080)
that what I was doing was not statistically significant.
Lex Fridman (04:19.280)
And I was like, but, but, and basically what he said to me
Lex Fridman (04:23.360)
is statistics is how you're gonna keep
Charles Isbell and Michael Littman (04:25.120)
from lying to yourself, which I thought was really deep.
Lex Fridman (04:29.640)
It is a way to keep yourself honest in a particular way.
Charles Isbell and Michael Littman (04:33.280)
I agree with that.
Lex Fridman (04:34.120)
Yeah, and so you're trying to find rules.
Charles Isbell and Michael Littman (04:36.960)
I'm just gonna bring it back to rules.
Lex Fridman (04:38.600)
Wait, wait, wait, could you possibly try to define rules?
Charles Isbell and Michael Littman (04:44.000)
Even regular statisticians, noncomputational statisticians,
Lex Fridman (04:47.680)
do spend some of their time evaluating rules, right?
Charles Isbell and Michael Littman (04:51.200)
Applying statistics to try to understand
Lex Fridman (04:52.760)
does this rule capture this?
Lex Fridman (04:54.760)
Does this not capture that?
Lex Fridman (04:55.600)
You mean like hypothesis testing kind of thing?
Lex Fridman (04:57.560)
Or like confidence intervals?
Lex Fridman (04:59.360)
I think more like hypothesis.
Charles Isbell and Michael Littman (05:01.600)
Like I feel like the word statistic
Lex Fridman (05:03.080)
literally means like a summary,
Charles Isbell and Michael Littman (05:04.680)
like a number that summarizes other numbers.
Lex Fridman (05:06.880)
But I think the field of statistics
Charles Isbell and Michael Littman (05:08.280)
actually applies that idea to things like rules,
Lex Fridman (05:11.420)
to understand whether or not a rule is valid.
Lex Fridman (05:15.320)
Does software engineering statistics?
Lex Fridman (05:18.160)
No.
Lex Fridman (05:19.000)
Programming languages statistics?
Lex Fridman (05:20.820)
No.
Charles Isbell and Michael Littman (05:21.660)
Because I think there's a very,
Lex Fridman (05:22.480)
it's useful to think about a lot of what AI
Lex Fridman (05:24.640)
and machine learning is or certainly should be
Lex Fridman (05:26.520)
as software engineering, as programming languages.
Charles Isbell and Michael Littman (05:29.520)
Just to put it in language that you might understand,
Lex Fridman (05:33.480)
the hyperparameters beyond the problem itself.
Charles Isbell and Michael Littman (05:35.680)
The hyperparameters is too many syllables
Lex Fridman (05:37.000)
for me to understand.
Charles Isbell and Michael Littman (05:37.840)
The hyperparameters.
Lex Fridman (05:39.720)
That's better.
Lex Fridman (05:40.560)
That goes around it, right?
Lex Fridman (05:41.380)
It's the decisions you choose to make.
Charles Isbell and Michael Littman (05:42.840)
It's the metrics you choose to use.
Lex Fridman (05:44.880)
It's the loss function.
Charles Isbell and Michael Littman (05:45.720)
You wanna say the practice of machine learning
Lex Fridman (05:48.160)
is different than the practice of statistics.
Charles Isbell and Michael Littman (05:50.080)
Like the things you have to worry about
Lex Fridman (05:51.560)
and how you worry about them are different,
Charles Isbell and Michael Littman (05:53.480)
therefore they're different.
Lex Fridman (05:54.640)
Right.
Charles Isbell and Michael Littman (05:55.560)
At a very little, I mean, at the very least.
Lex Fridman (05:57.920)
It's that much is true.
Charles Isbell and Michael Littman (05:59.080)
It doesn't mean that statistics,
Lex Fridman (06:00.280)
computational or otherwise aren't important.
Charles Isbell and Michael Littman (06:02.400)
I think they are.
Lex Fridman (06:03.240)
I mean, I do a lot of that, for example.
Lex Fridman (06:05.680)
But I think it goes beyond that.
Lex Fridman (06:06.920)
I think that we could think about game theory
Charles Isbell and Michael Littman (06:09.200)
in terms of statistics,
Lex Fridman (06:10.100)
but I don't think it's very as useful to do.
Charles Isbell and Michael Littman (06:12.600)
I mean, the way I would think about it
Lex Fridman (06:14.040)
or a way I would think about it is this way.
Charles Isbell and Michael Littman (06:17.380)
Chemistry is just physics.
Lex Fridman (06:19.980)
But I don't think it's as useful to think about chemistry
Charles Isbell and Michael Littman (06:22.180)
as being just physics.
Lex Fridman (06:23.320)
It's useful to think about it as chemistry.
Charles Isbell and Michael Littman (06:25.560)
The level of abstraction really matters here.
Lex Fridman (06:27.180)
So I think it is,
Charles Isbell and Michael Littman (06:28.520)
there are contexts in which it is useful.
Lex Fridman (06:30.420)
Yes.
Lex Fridman (06:31.260)
I think of it that way, right?
Lex Fridman (06:32.080)
So finding that connection is actually helpful.
Lex Fridman (06:33.160)
And I think that's when I emphasize
Lex Fridman (06:34.760)
the computational statistics thing.
Charles Isbell and Michael Littman (06:36.120)
I think I want to befriend statistics and not absorb them.
Lex Fridman (06:41.120)
Here's the A way to think about it
Lex Fridman (06:43.760)
beyond what I just said, right?
Lex Fridman (06:44.800)
So what would you say,
Lex Fridman (06:47.200)
and I want you to think back to a conversation
Lex Fridman (06:48.440)
we had a very long time ago.
Lex Fridman (06:49.520)
What would you say is the difference between,
Lex Fridman (06:52.120)
say, the early 2000s, ICML
Lex Fridman (06:54.560)
and what we used to call NIPS, NeurIPS?
Lex Fridman (06:57.240)
Is there a difference?
Charles Isbell and Michael Littman (06:58.400)
A lot of, particularly on the machine learning
Lex Fridman (06:59.960)
that was done there?
Charles Isbell and Michael Littman (07:00.960)
ICML was around that long.
Lex Fridman (07:02.960)
Oh, yeah.
Lex Fridman (07:03.800)
So iClear is the new conference, newish.
Lex Fridman (07:06.880)
Yeah, I guess so.
Lex Fridman (07:07.720)
And ICML was around the 2000.
Lex Fridman (07:10.120)
So ICML predates that.
Charles Isbell and Michael Littman (07:12.280)
I think my most cited ICML paper is from 94.
Lex Fridman (07:15.520)
Michael knows this better than me
Charles Isbell and Michael Littman (07:16.480)
because, of course, he's significantly older than I am.
Lex Fridman (07:18.000)
But the point is, what is the difference
Lex Fridman (07:20.600)
between ICML and NeurIPS in the late 90s, early 2000s?
Lex Fridman (07:24.440)
I don't know what everyone else's perspective would be,
Lex Fridman (07:26.500)
but I had a particular perspective at that time,
Lex Fridman (07:28.920)
which is I felt like ICML was more
Charles Isbell and Michael Littman (07:31.120)
of a computer science place
Lex Fridman (07:33.560)
and that NIPS, NeurIPS was more of an engineering place,
Charles Isbell and Michael Littman (07:37.140)
like the kind of math that happened at the two places.
Lex Fridman (07:40.560)
As a computer scientist,
Charles Isbell and Michael Littman (07:41.520)
I felt more comfortable with the ICML math.
Lex Fridman (07:44.480)
And the NeurIPS people would say
Charles Isbell and Michael Littman (07:46.520)
that that's because I'm dumb.
Lex Fridman (07:48.880)
And that's such an engineering thing to say, so.
Charles Isbell and Michael Littman (07:51.720)
I agree with that part of it,
Lex Fridman (07:52.560)
but I do it a little differently.
Charles Isbell and Michael Littman (07:53.400)
We actually had a nice conversation
Lex Fridman (07:54.760)
with Tom Dietrich about this in public.
Charles Isbell and Michael Littman (07:57.320)
On Twitter just a couple of days ago.
Lex Fridman (07:58.560)
I put it a little differently,
Charles Isbell and Michael Littman (07:59.520)
which is that ICML was machine learning done
Lex Fridman (08:02.360)
by a computer scientist.
Lex Fridman (08:04.600)
And NeurIPS was machine learning done
Lex Fridman (08:07.960)
by a computer scientist trying to impress statisticians.
Charles Isbell and Michael Littman (08:12.920)
Which was weird because it was the same people,
Lex Fridman (08:15.080)
at least by the time I started paying attention.
Lex Fridman (08:17.260)
But it just felt very, very different.
Lex Fridman (08:18.940)
And I think that that perspective
Charles Isbell and Michael Littman (08:20.960)
of whether you're trying to impress the statisticians
Lex Fridman (08:22.760)
or you're trying to impress the programmers
Charles Isbell and Michael Littman (08:24.300)
is actually very different and has real impact
Lex Fridman (08:26.720)
on what you choose to worry about
Lex Fridman (08:29.600)
and what kind of outcomes you come to.
Lex Fridman (08:31.440)
So I think it really matters.
Charles Isbell and Michael Littman (08:32.680)
I think computational statistics is a means to an end.
Lex Fridman (08:34.480)
It is not an end in some sense.
Lex Fridman (08:36.980)
And I think that really matters here
Lex Fridman (08:39.120)
in the same way that I don't think computer science
Charles Isbell and Michael Littman (08:40.760)
is just engineering or just science
Lex Fridman (08:42.120)
or just math or whatever.
Charles Isbell and Michael Littman (08:43.160)
Okay, so I'd have to now agree
Lex Fridman (08:44.880)
that now we agree on everything.
Charles Isbell and Michael Littman (08:46.280)
Yes, yes.
Lex Fridman (08:47.780)
The important thing here is that
Charles Isbell and Michael Littman (08:50.560)
my opinions may have changed,
Lex Fridman (08:51.760)
but not the fact that I'm right,
Charles Isbell and Michael Littman (08:53.000)
I think is what we just came to.
Lex Fridman (08:54.600)
Right, and my opinions may have changed
Lex Fridman (08:55.920)
and not the fact that I'm wrong.
Lex Fridman (08:57.380)
That's right.
Charles Isbell and Michael Littman (08:59.120)
You lost me.
Lex Fridman (08:59.960)
I'm not even.
Charles Isbell and Michael Littman (09:00.780)
I think I lost myself there too.
Lex Fridman (09:01.760)
But anyway, we're back.
Charles Isbell and Michael Littman (09:04.320)
This happens to us sometimes.
Lex Fridman (09:05.440)
We're sorry.
Lex Fridman (09:06.280)
How does neural networks change this,
Lex Fridman (09:08.840)
just to even linger on this topic,
Charles Isbell and Michael Littman (09:11.120)
change this idea of statistics,
Lex Fridman (09:15.600)
how big of a pie statistics is
Lex Fridman (09:17.720)
within the machine learning thing?
Lex Fridman (09:19.220)
Like, because it sounds like hyperparameters
Lex Fridman (09:22.000)
and also just the role of data.
Lex Fridman (09:24.280)
You know, people are starting to use
Charles Isbell and Michael Littman (09:25.920)
this terminology of software 2.0,
Lex Fridman (09:28.440)
which is like the act of programming
Charles Isbell and Michael Littman (09:31.140)
as a, like you're a designer
Lex Fridman (09:35.800)
in the hyperparameter space of neural networks,
Lex Fridman (09:38.240)
and you're also the collector and the organizer
Lex Fridman (09:40.760)
and the cleaner of the data,
Lex Fridman (09:44.080)
and that's part of the programming.
Lex Fridman (09:47.040)
So how did, on the NeurIPS versus ICML topic,
Charles Isbell and Michael Littman (09:52.040)
what's the role of neural networks
Lex Fridman (09:54.000)
in redefining the size and the role of machine learning?
Charles Isbell and Michael Littman (09:57.320)
I can't wait to hear what Michael thinks about this,
Lex Fridman (10:00.120)
but I would add one.
Lex Fridman (10:01.280)
But you will.
Lex Fridman (10:02.120)
That's true, I will, I'll force myself to.
Charles Isbell and Michael Littman (10:04.320)
I think there's one other thing
Lex Fridman (10:06.320)
I would add to your description,
Charles Isbell and Michael Littman (10:07.400)
which is the kind of software engineering part
Lex Fridman (10:09.120)
of what does it mean to debug, for example.
Lex Fridman (10:10.920)
But this is a difference between
Lex Fridman (10:13.080)
the kind of computational statistics view
Charles Isbell and Michael Littman (10:14.800)
of machine learning and the computational view
Lex Fridman (10:16.840)
of machine learning, which is, I think,
Charles Isbell and Michael Littman (10:18.600)
one is worried about the equation, as it were.
Lex Fridman (10:20.800)
And by the way, this is not a value judgment.
Charles Isbell and Michael Littman (10:23.120)
I just think it's about perspective.
Lex Fridman (10:24.960)
But the kind of questions you would ask
Charles Isbell and Michael Littman (10:26.440)
when you start asking yourself,
Lex Fridman (10:27.320)
well, what does it mean to program
Lex Fridman (10:28.320)
and develop and build the system,
Lex Fridman (10:29.720)
is a very computer sciencey view of the problem.
Charles Isbell and Michael Littman (10:33.080)
I mean, if you get on data science Twitter
Lex Fridman (10:35.960)
and econ Twitter, you actually hear this a lot
Charles Isbell and Michael Littman (10:39.620)
with the economist and the data scientist
Lex Fridman (10:43.000)
complaining about the machine learning people.
Charles Isbell and Michael Littman (10:44.600)
Well, it's just statistics,
Lex Fridman (10:46.280)
and I don't know why they don't see this.
Lex Fridman (10:47.800)
But they're not even asking the same questions.
Lex Fridman (10:49.360)
They're not thinking about it
Charles Isbell and Michael Littman (10:50.780)
as a kind of programming problem.
Lex Fridman (10:53.160)
And I think that that really matters,
Charles Isbell and Michael Littman (10:54.620)
just asking this question.
Lex Fridman (10:55.980)
I actually think it's a little different
Charles Isbell and Michael Littman (10:57.020)
from programming in hyperparameter space
Lex Fridman (11:00.640)
and sort of collecting the data.
Lex Fridman (11:03.120)
But I do think that that immersion really matters.
Lex Fridman (11:06.040)
So I'll give you a quick example
Charles Isbell and Michael Littman (11:07.920)
of the way I think about this.
Lex Fridman (11:08.760)
So I teach machine learning.
Charles Isbell and Michael Littman (11:09.840)
Michael and I have co taught a machine learning class,
Lex Fridman (11:12.040)
which has now reached, I don't know, 10,000 people at least
Charles Isbell and Michael Littman (11:14.960)
over the last several years, or somewhere there's abouts.
Lex Fridman (11:17.040)
And my machine learning assignments are of this form.
Lex Fridman (11:21.400)
So the first one is something like,
Lex Fridman (11:23.820)
implement these five algorithms,
Charles Isbell and Michael Littman (11:25.400)
KNN and SVMs and boosting and decision trees
Lex Fridman (11:29.680)
and neural networks, and maybe that's it, I can't remember.
Lex Fridman (11:32.560)
And when I say implement, I mean steal the code.
Lex Fridman (11:34.500)
I am completely uninterested.
Charles Isbell and Michael Littman (11:36.000)
You get zero points for getting the thing to work.
Lex Fridman (11:38.800)
I don't want you spending your time worrying about
Charles Isbell and Michael Littman (11:41.640)
getting the corner case right of what happens
Lex Fridman (11:44.280)
when you are trying to normalize distances
Lex Fridman (11:46.320)
and the points on the thing.
Lex Fridman (11:47.480)
And so you divide by zero.
Lex Fridman (11:48.320)
I'm not interested in that, right?
Lex Fridman (11:50.020)
Steal the code.
Charles Isbell and Michael Littman (11:51.480)
However, you're going to run those algorithms
Lex Fridman (11:54.080)
on two data sets.
Charles Isbell and Michael Littman (11:55.840)
The data sets have to be interesting.
Lex Fridman (11:57.960)
What does it mean to be interesting?
Charles Isbell and Michael Littman (11:58.880)
Well, data sets interesting if it reveals differences
Lex Fridman (12:01.060)
between algorithms, which presumably are all the same
Charles Isbell and Michael Littman (12:03.760)
because they can represent whatever they can represent.
Lex Fridman (12:06.160)
And two data sets are interesting together
Charles Isbell and Michael Littman (12:07.680)
if they show different differences, as it were.
Lex Fridman (12:10.720)
And you have to analyze them.
Charles Isbell and Michael Littman (12:11.920)
You have to justify their interestingness
Lex Fridman (12:13.280)
and you have to analyze them in a whole bunch of ways.
Lex Fridman (12:15.180)
But all I care about is the data in your analysis,
Lex Fridman (12:17.480)
not the programming.
Lex Fridman (12:18.340)
And I occasionally end up in these long discussions
Lex Fridman (12:20.720)
with students, well, I don't really,
Charles Isbell and Michael Littman (12:22.880)
I copy and paste the things that I've said
Lex Fridman (12:24.440)
the other 15,000 times it's come up,
Charles Isbell and Michael Littman (12:26.320)
which is they go, but the only way to learn,
Lex Fridman (12:29.020)
really understand is to code them up,
Charles Isbell and Michael Littman (12:31.320)
which is a very programmer,
Lex Fridman (12:33.480)
software engineering view of the world.
Charles Isbell and Michael Littman (12:35.220)
If you don't program it, you don't understand it,
Lex Fridman (12:37.360)
which is, by the way, I think is wrong
Charles Isbell and Michael Littman (12:39.020)
in a very specific way.
Lex Fridman (12:40.600)
But it is a way that you come to understand
Charles Isbell and Michael Littman (12:42.880)
because then you have to wrestle with the algorithm.
Lex Fridman (12:44.820)
But the thing about machine learning
Charles Isbell and Michael Littman (12:45.960)
is it's not just sorting numbers
Lex Fridman (12:47.660)
where in some sense the data doesn't matter.
Lex Fridman (12:49.180)
What matters is, well, does the algorithm work
Lex Fridman (12:50.980)
on these abstract things, one less than the other?
Charles Isbell and Michael Littman (12:53.020)
In machine learning, the data matters.
Lex Fridman (12:54.780)
It matters more than almost anything.
Lex Fridman (12:57.200)
And not everything, but almost anything.
Lex Fridman (12:59.780)
And so as a result, you have to live with the data
Lex Fridman (13:02.120)
and don't get distracted by the algorithm per se.
Lex Fridman (13:04.960)
And I think that that focus on the data
Lex Fridman (13:07.840)
and what it can tell you
Lex Fridman (13:09.040)
and what question it's actually answering for you
Charles Isbell and Michael Littman (13:11.840)
as opposed to the question you thought you were asking
Lex Fridman (13:14.000)
is a key and important thing about machine learning
Lex Fridman (13:16.360)
and is a way that computationalists
Lex Fridman (13:18.960)
as opposed to statisticians bring a particular view
Charles Isbell and Michael Littman (13:21.800)
about how to think about the process.
Lex Fridman (13:23.320)
The statisticians, by contrast, bring,
Charles Isbell and Michael Littman (13:25.600)
I think I'd be willing to say,
Lex Fridman (13:27.880)
a better view about the kind of formal math that's behind it
Lex Fridman (13:31.360)
and what an actual number ultimately is saying
Lex Fridman (13:35.040)
about the data.
Lex Fridman (13:35.960)
And those are both important, but they're also different.
Lex Fridman (13:38.520)
I didn't really think of it this way
Charles Isbell and Michael Littman (13:40.360)
is to build intuition about the role of data,
Lex Fridman (13:44.480)
the different characteristics of data
Charles Isbell and Michael Littman (13:45.940)
by having two data sets that are different
Lex Fridman (13:48.160)
and they reveal the differences in the differences.
Charles Isbell and Michael Littman (13:50.880)
That's a really fascinating,
Lex Fridman (13:52.160)
that's a really interesting educational approach.
Charles Isbell and Michael Littman (13:55.000)
The students love it, but not right away.
Lex Fridman (13:57.500)
No, they love it at the end.
Charles Isbell and Michael Littman (13:58.340)
They love it later.
Lex Fridman (13:59.160)
They love it at the end.
Charles Isbell and Michael Littman (14:00.100)
Not at the beginning.
Lex Fridman (14:02.520)
Not even immediately after.
Charles Isbell and Michael Littman (14:04.360)
I feel like there's a deep profound lesson
Lex Fridman (14:06.280)
about education there.
Charles Isbell and Michael Littman (14:07.560)
Yeah.
Lex Fridman (14:08.400)
That you can't listen to students
Charles Isbell and Michael Littman (14:10.920)
about whether what you're doing is the right
Lex Fridman (14:14.080)
or the wrong thing.
Charles Isbell and Michael Littman (14:15.140)
Yeah, well, as a wise, Michael Lippmann once said to me
Lex Fridman (14:19.220)
about children, which I think applies to teaching,
Charles Isbell and Michael Littman (14:22.060)
is you have to give them what they need
Lex Fridman (14:24.940)
without bending to their will.
Lex Fridman (14:27.120)
And students are like that.
Lex Fridman (14:28.060)
You have to figure out what they need.
Charles Isbell and Michael Littman (14:29.060)
You're a curator.
Lex Fridman (14:29.900)
Your whole job is to curate and to present
Charles Isbell and Michael Littman (14:32.060)
because on their own,
Lex Fridman (14:33.240)
they're not gonna necessarily know where to search.
Lex Fridman (14:35.060)
So you're providing pushes in some direction
Lex Fridman (14:37.160)
and learn space and you have to give them what they need
Charles Isbell and Michael Littman (14:42.300)
in a way that keeps them engaged enough
Lex Fridman (14:44.500)
so that they eventually discover what they want
Lex Fridman (14:46.860)
and they get the tools they need to go
Lex Fridman (14:48.100)
and learn other things off of.
Lex Fridman (14:50.060)
What's your view?
Lex Fridman (14:52.180)
Let me put on my Russian hat,
Charles Isbell and Michael Littman (14:54.260)
which believes that life is suffering.
Lex Fridman (14:55.780)
I like Russian hats, by the way.
Charles Isbell and Michael Littman (14:56.820)
If you have one, I would like this.
Lex Fridman (14:58.100)
Those are ridiculous, yes.
Lex Fridman (14:59.420)
But in a delightful way.
Lex Fridman (15:01.460)
But sure.
Lex Fridman (15:04.040)
What do you think is the role of,
Lex Fridman (15:06.300)
we talked about balance a little bit.
Lex Fridman (15:08.140)
What do you think is the role of hardship in education?
Lex Fridman (15:11.740)
Like I think the biggest things I've learned,
Charles Isbell and Michael Littman (15:16.540)
like what made me fall in love with math, for example,
Lex Fridman (15:20.260)
is by being bad at it until I got good at it.
Lex Fridman (15:24.700)
So like struggling with a problem,
Lex Fridman (15:28.400)
which increased the level of joy I felt
Charles Isbell and Michael Littman (15:31.160)
when I finally figured it out.
Lex Fridman (15:33.440)
And it always felt with me, with teachers,
Charles Isbell and Michael Littman (15:37.220)
especially modern discussions of education,
Lex Fridman (15:39.760)
how can we make education more fun,
Lex Fridman (15:42.420)
more engaging, more all those things?
Lex Fridman (15:44.900)
Or from my perspective, it's like,
Charles Isbell and Michael Littman (15:46.940)
you're maybe missing the point
Lex Fridman (15:49.220)
that education, that life is suffering.
Charles Isbell and Michael Littman (15:52.740)
Education is supposed to be hard
Lex Fridman (15:54.840)
and that actually what increases the joy you feel
Charles Isbell and Michael Littman (15:57.860)
when you actually learn something.
Lex Fridman (15:59.540)
Is that ridiculous?
Lex Fridman (16:02.340)
Do you like to see your students suffer?
Lex Fridman (16:04.380)
Okay, so this may be a point where we differ.
Charles Isbell and Michael Littman (16:07.460)
I suspect not.
Lex Fridman (16:08.620)
I'm gonna do go on.
Lex Fridman (16:10.180)
Well, what would your answer be?
Lex Fridman (16:11.220)
I wanna hear you first.
Charles Isbell and Michael Littman (16:12.060)
Okay, well, I was gonna not answer the question.
Lex Fridman (16:14.420)
You don't want the students to know you enjoy them suffering?
Charles Isbell and Michael Littman (16:18.180)
No, no, no, no, no, no.
Lex Fridman (16:19.020)
I was gonna say that there's,
Charles Isbell and Michael Littman (16:21.700)
I think there's a distinction that you can make
Lex Fridman (16:23.700)
in the kind of suffering, right?
Lex Fridman (16:25.020)
So I think you can be in a mode
Lex Fridman (16:27.060)
where you're suffering in a hopeless way
Lex Fridman (16:30.300)
versus you're suffering in a hopeful way, right?
Lex Fridman (16:33.400)
Where you're like, you can see that if you,
Charles Isbell and Michael Littman (16:37.260)
that you still have,
Lex Fridman (16:39.200)
you can still imagine getting to the end, right?
Lex Fridman (16:41.940)
And as long as people are in that mindset
Lex Fridman (16:43.640)
where they're struggling,
Lex Fridman (16:44.480)
but it's not a hopeless kind of struggling,
Lex Fridman (16:47.460)
that's productive.
Charles Isbell and Michael Littman (16:49.060)
I think that's really helpful.
Lex Fridman (16:50.540)
But it's struggling, like if you break their will,
Charles Isbell and Michael Littman (16:53.700)
if you leave them hopeless.
Lex Fridman (16:56.140)
No, that don't, sure, some people are gonna,
Charles Isbell and Michael Littman (16:58.660)
whatever, lift themselves up by their bootstraps,
Lex Fridman (17:00.500)
but like mostly you give up
Lex Fridman (17:01.940)
and certainly it takes the joy out of it.
Lex Fridman (17:03.520)
And you're not gonna spend a lot of time
Charles Isbell and Michael Littman (17:05.740)
on something that brings you no joy.
Lex Fridman (17:07.860)
So it is a bit of a delicate balance, right?
Charles Isbell and Michael Littman (17:10.380)
You have to thwart people in a way
Lex Fridman (17:12.920)
that they still believe that there's a way through.
Charles Isbell and Michael Littman (17:17.020)
Right, so that's a, we strongly agree actually.
Lex Fridman (17:20.020)
So I think, well, first off,
Lex Fridman (17:21.140)
struggling and suffering aren't the same thing, right?
Lex Fridman (17:24.060)
Yeah, just being poetic.
Charles Isbell and Michael Littman (17:25.340)
Oh, no, no, I actually appreciate the poetry.
Lex Fridman (17:27.660)
And one of the reasons I appreciate it
Charles Isbell and Michael Littman (17:29.720)
is that they are often the same thing
Lex Fridman (17:31.860)
and often quite different, right?
Lex Fridman (17:32.900)
So you can struggle without suffering,
Lex Fridman (17:34.580)
you can certainly suffer pretty easily.
Charles Isbell and Michael Littman (17:37.220)
You don't necessarily have to struggle to suffer.
Lex Fridman (17:38.820)
So I think that you want people to struggle,
Lex Fridman (17:41.860)
but that hope matters.
Lex Fridman (17:42.940)
You have to, they have to understand
Charles Isbell and Michael Littman (17:44.620)
that they're gonna get through it on the other side.
Lex Fridman (17:46.500)
And it's very easy to confuse the two.
Charles Isbell and Michael Littman (17:50.060)
I actually think Brown University has a very,
Lex Fridman (17:52.580)
just philosophically has a very different take
Charles Isbell and Michael Littman (17:55.420)
on the relationship with their students,
Lex Fridman (17:56.780)
particularly undergrads from say a place like Georgia Tech,
Charles Isbell and Michael Littman (17:59.300)
which is.
Lex Fridman (18:00.140)
Which university is better?
Charles Isbell and Michael Littman (18:01.780)
Well, I have my opinions on that.
Lex Fridman (18:03.340)
I mean, remember, Charles said,
Charles Isbell and Michael Littman (18:05.260)
it doesn't matter what the facts are, I'm always right.
Lex Fridman (18:07.500)
The correct answer is that it doesn't matter,
Charles Isbell and Michael Littman (18:09.780)
they're different.
Lex Fridman (18:10.980)
But clearly, clearly the answer is different.
Charles Isbell and Michael Littman (18:14.460)
He went to a school like the school
Lex Fridman (18:16.900)
where he is as an undergrad.
Charles Isbell and Michael Littman (18:18.500)
I went to a school, specifically the same school,
Lex Fridman (18:21.060)
though it was changed a bit in the intervening years.
Lex Fridman (18:23.500)
Brown or Georgia Tech?
Lex Fridman (18:24.340)
No, I was talking about Georgia Tech.
Lex Fridman (18:25.260)
And I went to an undergrad place
Lex Fridman (18:28.180)
that's a lot like the place where I work now.
Lex Fridman (18:29.780)
And so it does seem like we're more familiar
Lex Fridman (18:32.360)
with these models.
Lex Fridman (18:33.280)
So there's a similarity between Brown and Yale?
Lex Fridman (18:35.500)
Yeah, I think they're quite similar, yeah.
Lex Fridman (18:38.380)
And Duke.
Lex Fridman (18:39.260)
Duke has some similarities too,
Lex Fridman (18:40.900)
but it's got a little Southern draw.
Lex Fridman (18:42.940)
You've kind of worked your,
Charles Isbell and Michael Littman (18:43.860)
you've sort of worked at universities
Lex Fridman (18:45.540)
that are like the places where you learned.
Lex Fridman (18:50.000)
And the same would be true for me.
Lex Fridman (18:52.180)
Are you uncomfortable venturing outside the box?
Lex Fridman (18:56.100)
Is that what you're saying?
Lex Fridman (18:57.300)
Journeying out?
Charles Isbell and Michael Littman (18:58.120)
That's not what I'm saying.
Lex Fridman (18:58.960)
Yeah, Charles is definitely.
Charles Isbell and Michael Littman (19:00.060)
He only goes to places
Lex Fridman (19:01.100)
that have institute in the name, right?
Charles Isbell and Michael Littman (19:02.960)
It has worked out that way.
Lex Fridman (19:04.060)
Well, academic places anyway.
Charles Isbell and Michael Littman (19:06.180)
Well, no, I was a visiting scientist at UPenn
Lex Fridman (19:08.260)
or visiting something at UPenn.
Charles Isbell and Michael Littman (19:11.120)
Oh, wow, I just understood your joke.
Lex Fridman (19:14.120)
Which one?
Charles Isbell and Michael Littman (19:14.960)
Five minutes later.
Lex Fridman (19:18.020)
I like to set the sort of time bomb.
Charles Isbell and Michael Littman (19:20.020)
The institute is in the,
Lex Fridman (19:22.060)
that Charles only goes to places
Charles Isbell and Michael Littman (19:23.620)
that have institute in the name.
Lex Fridman (19:25.780)
So I guess Georgia,
Charles Isbell and Michael Littman (19:27.560)
I forget that Georgia Tech
Lex Fridman (19:28.860)
is Georgia Institute of Technology.
Charles Isbell and Michael Littman (19:30.740)
The number of people who refer to it
Lex Fridman (19:32.440)
as Georgia Tech University is large
Lex Fridman (19:34.300)
and incredibly irritating.
Lex Fridman (19:35.620)
It's one of the few things
Charles Isbell and Michael Littman (19:37.820)
that genuinely gets under my skin.
Lex Fridman (19:39.220)
But like schools like Georgia Tech and MIT
Charles Isbell and Michael Littman (19:41.180)
have as part of the ethos,
Lex Fridman (19:42.780)
like there is,
Charles Isbell and Michael Littman (19:43.900)
I wanna say there's an abbreviation
Lex Fridman (19:45.900)
that someone taught me,
Charles Isbell and Michael Littman (19:47.660)
like IHTFP, something like that.
Lex Fridman (19:49.700)
Like there's an expression
Charles Isbell and Michael Littman (19:51.300)
which is basically I hate being here,
Lex Fridman (19:53.460)
which they say so proudly.
Lex Fridman (19:55.580)
And that is definitely not the ethos at Brown.
Lex Fridman (19:57.980)
Like Brown is,
Charles Isbell and Michael Littman (19:58.820)
there's a little more pampering
Lex Fridman (1:00:02.300)
in some subtle way, it creates a lot of problems.
Lex Fridman (1:00:05.620)
And those problems are dramatized when you're in space,
Lex Fridman (1:00:08.460)
because you don't have a way to walk away.
Charles Isbell and Michael Littman (1:00:11.340)
Well, as the man said,
Lex Fridman (1:00:12.820)
once we started making the decisions for you,
Lex Fridman (1:00:15.300)
it stopped being your world, right?
Lex Fridman (1:00:17.220)
That's the matrix, Michael, in case you don't remember.
Lex Fridman (1:00:20.420)
But on the other hand, I could say no,
Lex Fridman (1:00:23.180)
because isn't that what we do with people anyway?
Charles Isbell and Michael Littman (1:00:25.620)
You know, just kind of the shared intelligence
Lex Fridman (1:00:27.120)
that is humanity is relying on other people constantly.
Lex Fridman (1:00:30.220)
I mean, we hyper specialize, right?
Lex Fridman (1:00:32.340)
As individuals, we're still generally intelligent.
Charles Isbell and Michael Littman (1:00:34.620)
We make our own decisions in a lot of ways,
Lex Fridman (1:00:36.040)
but we leave most of this up to other people.
Lex Fridman (1:00:37.620)
And that's perfectly fine.
Lex Fridman (1:00:39.860)
And by the way, everyone doesn't necessarily share our goals.
Charles Isbell and Michael Littman (1:00:43.220)
Sometimes they seem to be quite against us.
Lex Fridman (1:00:45.100)
Sometimes we make decisions that others would see
Charles Isbell and Michael Littman (1:00:47.900)
as against our own interests.
Lex Fridman (1:00:49.080)
And yet we somehow manage it, manage to survive.
Charles Isbell and Michael Littman (1:00:51.340)
I'm not entirely sure why an AI
Lex Fridman (1:00:54.240)
would actually make that worse or even different, really.
Charles Isbell and Michael Littman (1:01:00.260)
You mentioned the matrix.
Lex Fridman (1:01:02.140)
Do you think we're living in a simulation?
Charles Isbell and Michael Littman (1:01:04.420)
It does feel like a thought game
Lex Fridman (1:01:08.060)
more than a real scientific question.
Charles Isbell and Michael Littman (1:01:10.740)
Well, I'll tell you why I think
Lex Fridman (1:01:12.140)
it's an interesting thought experiment.
Charles Isbell and Michael Littman (1:01:13.500)
Let's see what you think.
Lex Fridman (1:01:14.340)
From a computer science perspective,
Charles Isbell and Michael Littman (1:01:16.140)
it's a good experiment of how difficult would it be
Lex Fridman (1:01:20.180)
to create a sufficiently realistic world
Charles Isbell and Michael Littman (1:01:22.860)
that us humans would enjoy being in.
Lex Fridman (1:01:26.700)
That's almost like a competition.
Charles Isbell and Michael Littman (1:01:27.660)
If we're living in a simulation,
Lex Fridman (1:01:29.100)
then I don't believe that we were put in the simulation.
Charles Isbell and Michael Littman (1:01:31.600)
I believe that it's just physics playing out
Lex Fridman (1:01:34.180)
and we came out of that.
Charles Isbell and Michael Littman (1:01:36.400)
Like, I don't think.
Lex Fridman (1:01:39.220)
So you think you have to build the universe
Lex Fridman (1:01:40.900)
and have all the fun in the world?
Lex Fridman (1:01:41.740)
I think that the universe itself,
Charles Isbell and Michael Littman (1:01:42.560)
we can think of that as a simulation.
Lex Fridman (1:01:43.820)
And in fact, sometimes I try to think about,
Charles Isbell and Michael Littman (1:01:46.820)
to understand what it's like for a computer
Lex Fridman (1:01:49.560)
to start to think about the world.
Charles Isbell and Michael Littman (1:01:52.660)
I try to think about the world.
Lex Fridman (1:01:55.060)
Things like quantum mechanics,
Charles Isbell and Michael Littman (1:01:56.780)
where it doesn't feel very natural to me at all.
Lex Fridman (1:01:59.940)
And it really strikes me as,
Charles Isbell and Michael Littman (1:02:02.840)
I don't understand this thing that we're living in.
Lex Fridman (1:02:05.220)
It has, there's weird things happening in it
Charles Isbell and Michael Littman (1:02:07.620)
that don't feel natural to me at all.
Lex Fridman (1:02:09.660)
Now, if you want to call that as the result of a simulator,
Charles Isbell and Michael Littman (1:02:13.060)
okay, I'm fine with that.
Lex Fridman (1:02:14.280)
But like, I don't.
Charles Isbell and Michael Littman (1:02:15.120)
There's the bugs in the simulation.
Lex Fridman (1:02:16.940)
There's the bugs.
Charles Isbell and Michael Littman (1:02:17.780)
I mean, the interesting thing about the simulation
Lex Fridman (1:02:19.540)
is that it might have bugs.
Charles Isbell and Michael Littman (1:02:21.260)
I mean, that's the thing that I,
Lex Fridman (1:02:23.060)
But there would be bugs for the people in the simulation.
Charles Isbell and Michael Littman (1:02:25.540)
That's just reality.
Lex Fridman (1:02:27.100)
Unless you were aware enough to know that there was a bug.
Lex Fridman (1:02:29.300)
But I think.
Lex Fridman (1:02:30.300)
Back to the matrix.
Charles Isbell and Michael Littman (1:02:31.420)
Yeah, the way you put the question though.
Lex Fridman (1:02:32.260)
I don't think that we live in a simulation created for us.
Charles Isbell and Michael Littman (1:02:35.300)
Okay, I would say that.
Lex Fridman (1:02:36.340)
I think that's interesting.
Charles Isbell and Michael Littman (1:02:37.180)
I've actually never thought about it that way.
Lex Fridman (1:02:38.100)
I mean, the way you asked the question though,
Lex Fridman (1:02:40.140)
could you create a world that is enough for us humans?
Lex Fridman (1:02:43.020)
It's an interestingly sort of self referential question
Charles Isbell and Michael Littman (1:02:45.460)
because the beings that created the simulation
Lex Fridman (1:02:49.920)
probably have not created the simulation
Charles Isbell and Michael Littman (1:02:51.460)
that's realistic for them.
Lex Fridman (1:02:53.300)
But we're in the simulation and so it's realistic for us.
Lex Fridman (1:02:56.180)
So we could create a simulation
Lex Fridman (1:02:58.220)
that is fine for the people in the simulation, as it were.
Charles Isbell and Michael Littman (1:03:02.140)
That would not necessarily be fine for us
Lex Fridman (1:03:03.660)
as the creators of the simulation.
Charles Isbell and Michael Littman (1:03:05.140)
But, well, you can forget.
Lex Fridman (1:03:07.460)
I mean, if you play video games in virtual reality,
Charles Isbell and Michael Littman (1:03:11.320)
you can, if some suspension of disbelief or whatever.
Lex Fridman (1:03:16.340)
It becomes a world.
Charles Isbell and Michael Littman (1:03:17.420)
It becomes a world.
Lex Fridman (1:03:18.500)
Even like in brief moments,
Charles Isbell and Michael Littman (1:03:20.100)
you forget that another world exists.
Lex Fridman (1:03:22.300)
I mean, that's what like good stories do.
Charles Isbell and Michael Littman (1:03:24.220)
They pull you in.
Lex Fridman (1:03:25.060)
And the question is, is it possible to pull,
Charles Isbell and Michael Littman (1:03:28.440)
our brains are limited.
Lex Fridman (1:03:29.380)
Is it possible to pull the brain in
Charles Isbell and Michael Littman (1:03:31.280)
to where we actually stay in that world
Lex Fridman (1:03:32.700)
longer and longer and longer and longer?
Lex Fridman (1:03:34.780)
And like, not only that, but we don't wanna leave.
Lex Fridman (1:03:39.000)
And so, especially this is the key thing
Charles Isbell and Michael Littman (1:03:41.500)
about the developing brain,
Lex Fridman (1:03:43.860)
is if we journey into that world early on in life, often.
Lex Fridman (1:03:48.140)
How would you even know, yeah.
Lex Fridman (1:03:49.740)
Yeah, so I, but like from a video game design perspective,
Charles Isbell and Michael Littman (1:03:53.200)
from a Westworld perspective,
Lex Fridman (1:03:54.420)
it's, I think it's an important thing
Charles Isbell and Michael Littman (1:03:57.540)
for even computer scientists to think about
Lex Fridman (1:04:00.820)
because it's clear that video games are getting much better.
Lex Fridman (1:04:04.580)
And virtual reality,
Lex Fridman (1:04:06.500)
although it's been ups and downs
Charles Isbell and Michael Littman (1:04:08.460)
just like artificial intelligence,
Lex Fridman (1:04:09.860)
it feels like virtual reality will be here
Charles Isbell and Michael Littman (1:04:14.780)
in a very impressive form
Lex Fridman (1:04:16.340)
if we were to fast forward 100 years into the future
Charles Isbell and Michael Littman (1:04:19.060)
in a way that might change society fundamentally.
Lex Fridman (1:04:22.100)
Like if I were to,
Charles Isbell and Michael Littman (1:04:23.180)
I'm very limited in predicting the future as all of us are,
Lex Fridman (1:04:26.540)
but if I were to try to predict,
Charles Isbell and Michael Littman (1:04:28.620)
like in which way I'd be surprised
Lex Fridman (1:04:32.540)
to see the world 100 years from now,
Charles Isbell and Michael Littman (1:04:35.780)
it'd be that, or impressed,
Lex Fridman (1:04:39.540)
it'd be that we're all no longer living
Charles Isbell and Michael Littman (1:04:42.280)
in this physical world,
Lex Fridman (1:04:43.340)
that we're all living in a virtual world.
Charles Isbell and Michael Littman (1:04:45.020)
You really need to read Calculating God by Sawyer.
Lex Fridman (1:04:51.140)
It's a, he'll read it in the night.
Charles Isbell and Michael Littman (1:04:53.080)
It's a very easy read,
Lex Fridman (1:04:54.300)
but it's, assuming you're that kind of reader,
Lex Fridman (1:04:56.180)
but it's a good story.
Lex Fridman (1:04:58.380)
And it's kind of about this,
Lex Fridman (1:04:59.680)
but not in a way that it appears.
Lex Fridman (1:05:01.420)
And I really enjoyed the thought experiment.
Lex Fridman (1:05:07.020)
And I think it's pretty sure it's Robert Sawyer.
Lex Fridman (1:05:08.260)
But anyway, he's apparently
Charles Isbell and Michael Littman (1:05:10.140)
Canadian's top science fiction writer,
Lex Fridman (1:05:12.420)
which is why the story mostly takes place in Toronto.
Lex Fridman (1:05:14.980)
But it's a very good sort of story
Lex Fridman (1:05:18.520)
that sort of imagines this.
Charles Isbell and Michael Littman (1:05:21.260)
Very different kind of simulation hypothesis sort of thing
Lex Fridman (1:05:25.020)
from say, The Egg, for example.
Charles Isbell and Michael Littman (1:05:28.340)
You know, I'm talking about the short story.
Lex Fridman (1:05:32.140)
By the guy who did The Martian.
Lex Fridman (1:05:34.900)
Who wrote The Martian?
Lex Fridman (1:05:36.300)
You know what I'm talking about.
Charles Isbell and Michael Littman (1:05:37.140)
The Martian. Matt Damon.
Lex Fridman (1:05:38.980)
The book.
Lex Fridman (1:05:39.820)
So we had this whole discussion
Lex Fridman (1:05:41.500)
that Michael doesn't partake in this exercise of reading.
Charles Isbell and Michael Littman (1:05:45.740)
He doesn't seem to like it,
Lex Fridman (1:05:46.780)
which seems very strange to me,
Charles Isbell and Michael Littman (1:05:48.120)
considering how much he has to read.
Lex Fridman (1:05:50.020)
I read all the time.
Charles Isbell and Michael Littman (1:05:50.980)
I used to read 10 books every week
Lex Fridman (1:05:53.340)
when I was in sixth grade or whatever.
Charles Isbell and Michael Littman (1:05:55.260)
I was, a lot of it's science fiction,
Lex Fridman (1:05:57.040)
a lot of it's history, but I love to read.
Lex Fridman (1:05:59.940)
But anyway, you should read Calculating God.
Lex Fridman (1:06:01.740)
I think you'll, it's very easy to read, like I said,
Lex Fridman (1:06:04.980)
and I think you'll enjoy sort of the ideas that it presents.
Lex Fridman (1:06:08.660)
Yeah, I think the thought experiment is quite interesting.
Charles Isbell and Michael Littman (1:06:12.740)
One thing I've noticed about people growing up now,
Lex Fridman (1:06:15.660)
I mean, we talk about social media,
Lex Fridman (1:06:17.240)
but video games is a much bigger,
Lex Fridman (1:06:19.580)
bigger and bigger and bigger part of their lives.
Lex Fridman (1:06:21.700)
And the video games have become much more realistic.
Lex Fridman (1:06:24.180)
I think it's possible that the three of us are not,
Charles Isbell and Michael Littman (1:06:31.380)
maybe the two of you are not familiar exactly
Lex Fridman (1:06:33.420)
with the numbers we're talking about here.
Charles Isbell and Michael Littman (1:06:36.060)
The number of people.
Lex Fridman (1:06:37.140)
It's bigger than movies, right?
Charles Isbell and Michael Littman (1:06:38.460)
It's huge.
Lex Fridman (1:06:39.900)
I used to do a lot of the computational narrative stuff.
Charles Isbell and Michael Littman (1:06:42.900)
I understand that economists can actually see
Lex Fridman (1:06:45.660)
the impact of video games on the labor market.
Charles Isbell and Michael Littman (1:06:48.700)
That there's fewer young men of a certain age
Lex Fridman (1:06:54.460)
participating in like paying jobs than you'd expect.
Lex Fridman (1:06:59.260)
And that they trace it back to video games.
Lex Fridman (1:07:01.300)
I mean, the problem with Star Trek
Charles Isbell and Michael Littman (1:07:02.900)
was not warp drive or teleportation.
Lex Fridman (1:07:06.320)
It was the holodeck.
Charles Isbell and Michael Littman (1:07:07.980)
Like if you have the holodeck, that's it.
Lex Fridman (1:07:12.020)
That's it, you go in the holodeck, you never come out.
Charles Isbell and Michael Littman (1:07:13.580)
I mean, it just never made, once I saw that,
Lex Fridman (1:07:16.860)
I thought, okay, well, so this is the end of humanity
Lex Fridman (1:07:19.260)
as we know it, right?
Lex Fridman (1:07:20.100)
They've invented the holodeck.
Charles Isbell and Michael Littman (1:07:21.700)
Because that feels like the singularity,
Lex Fridman (1:07:23.140)
not some AGI or whatever.
Charles Isbell and Michael Littman (1:07:25.020)
It's some possibility to go into another world
Lex Fridman (1:07:28.140)
that can be artificially made better than this one.
Lex Fridman (1:07:32.380)
And slowing it down so you live forever.
Lex Fridman (1:07:34.140)
Or speeding it up so you appear to live forever.
Charles Isbell and Michael Littman (1:07:35.700)
Or making the decision of when to die.
Lex Fridman (1:07:39.020)
And then most of us will just be old people on the porch
Charles Isbell and Michael Littman (1:07:42.300)
yelling at the kids these days in their virtual reality.
Lex Fridman (1:07:47.300)
But they won't hear us because they've got headphones on.
Charles Isbell and Michael Littman (1:07:49.860)
So, I mean, rewinding back to Mook's,
Lex Fridman (1:07:53.540)
is there lessons that you've, speaking to kids these days?
Charles Isbell and Michael Littman (1:07:58.660)
That was a transition.
Lex Fridman (1:07:59.500)
That was fantastic.
Charles Isbell and Michael Littman (1:08:01.860)
I'll fix it in post.
Lex Fridman (1:08:04.620)
That's Charles's favorite phrase.
Lex Fridman (1:08:06.400)
Fix it in post?
Lex Fridman (1:08:07.240)
Fix it in post.
Charles Isbell and Michael Littman (1:08:08.060)
Fix it in post.
Lex Fridman (1:08:08.900)
When we were recording all the time,
Charles Isbell and Michael Littman (1:08:10.600)
whenever the editor didn't like something or whatever,
Lex Fridman (1:08:12.900)
I would say, we'll fix it in post.
Charles Isbell and Michael Littman (1:08:14.420)
He hated that.
Lex Fridman (1:08:15.860)
He hated that more than anything.
Charles Isbell and Michael Littman (1:08:16.860)
Because it's Charles's way of saying,
Lex Fridman (1:08:17.980)
I'm not gonna do it again.
Charles Isbell and Michael Littman (1:08:20.700)
You're on your own for this one.
Lex Fridman (1:08:22.300)
But it always got fixed in post.
Charles Isbell and Michael Littman (1:08:24.060)
Exactly right.
Lex Fridman (1:08:24.880)
So is there something you've learned about,
Charles Isbell and Michael Littman (1:08:28.340)
I mean, it's interesting to talk about Mook's.
Lex Fridman (1:08:29.820)
Is there something you've learned
Charles Isbell and Michael Littman (1:08:30.700)
about the process of education,
Lex Fridman (1:08:32.140)
about thinking about the present?
Charles Isbell and Michael Littman (1:08:35.820)
I think there's two lines of conversation to be had here.
Lex Fridman (1:08:38.780)
There's the future of education in general
Charles Isbell and Michael Littman (1:08:41.540)
that you've learned about.
Lex Fridman (1:08:42.820)
And more passionately is the education
Charles Isbell and Michael Littman (1:08:49.060)
in the times of COVID.
Lex Fridman (1:08:50.420)
Yeah.
Charles Isbell and Michael Littman (1:08:51.260)
The second thing in some ways matters more than the first,
Lex Fridman (1:08:54.020)
for at least in my head,
Charles Isbell and Michael Littman (1:08:55.860)
not just because it's happening now,
Lex Fridman (1:08:57.060)
but because I think it's reminded us of a lot of things.
Charles Isbell and Michael Littman (1:09:00.620)
Coincidentally, today, there's an article out
Lex Fridman (1:09:02.920)
by a good friend of mine,
Charles Isbell and Michael Littman (1:09:04.560)
who's also a professor at Georgia Tech,
Lex Fridman (1:09:06.120)
but more importantly, a writer and editor
Charles Isbell and Michael Littman (1:09:07.900)
at the Atlantic, a guy named Ian Bogost.
Lex Fridman (1:09:10.740)
And the title is something like,
Charles Isbell and Michael Littman (1:09:13.060)
Americans Will Sacrifice Anything
Lex Fridman (1:09:15.380)
for the College Experience.
Lex Fridman (1:09:17.500)
And it's about why we went back to college
Lex Fridman (1:09:20.300)
and why people wanted us to go back to college.
Lex Fridman (1:09:22.420)
And it's not greedy presidents
Lex Fridman (1:09:24.580)
trying to get the last dollar from someone.
Charles Isbell and Michael Littman (1:09:26.300)
It's because they want to go to college.
Lex Fridman (1:09:28.000)
And what they're paying for is not the classes.
Lex Fridman (1:09:29.880)
What they're paying for is the college experience.
Lex Fridman (1:09:32.200)
It's not the education that's being there.
Charles Isbell and Michael Littman (1:09:33.580)
I've believed this for a long time,
Lex Fridman (1:09:35.580)
that we continually make this mistake of,
Charles Isbell and Michael Littman (1:09:39.080)
people want to go back to college
Lex Fridman (1:09:40.660)
as being people want to go back to class.
Charles Isbell and Michael Littman (1:09:42.340)
They don't.
Lex Fridman (1:09:43.180)
They want to go back to campus.
Charles Isbell and Michael Littman (1:09:44.000)
They want to move away from home.
Lex Fridman (1:09:44.840)
They want to do all those things that people experience.
Charles Isbell and Michael Littman (1:09:47.240)
It's a rite of passage.
Lex Fridman (1:09:48.140)
It's an identity, if I can steal some of Ian's words here.
Lex Fridman (1:09:53.740)
And I think that's right.
Lex Fridman (1:09:54.760)
And I think what we've learned through COVID
Charles Isbell and Michael Littman (1:09:57.140)
is it has made it,
Lex Fridman (1:09:59.780)
the disaggregation was not the disaggregation
Charles Isbell and Michael Littman (1:10:02.300)
of the education from the place, the university place,
Lex Fridman (1:10:05.140)
and that you can get the best anywhere you want to.
Charles Isbell and Michael Littman (1:10:07.080)
Turns out there's lots of reasons
Lex Fridman (1:10:08.100)
why that is not necessarily true.
Charles Isbell and Michael Littman (1:10:10.220)
The disaggregation is having it shoved in our faces
Lex Fridman (1:10:13.220)
that the reason to go, again,
Charles Isbell and Michael Littman (1:10:14.960)
that the reason to go to college
Lex Fridman (1:10:16.700)
is not necessarily to learn.
Charles Isbell and Michael Littman (1:10:18.280)
It's to have the college experience.
Lex Fridman (1:10:20.180)
And that's very difficult for us to accept,
Charles Isbell and Michael Littman (1:10:21.780)
even though we behave that way,
Lex Fridman (1:10:23.820)
most of us, when we were undergrads.
Charles Isbell and Michael Littman (1:10:26.620)
A lot of us didn't go to every single class.
Lex Fridman (1:10:28.740)
We learned and we got it and we look back on it
Lex Fridman (1:10:30.660)
and we're happy we had the learning experience as well,
Lex Fridman (1:10:32.320)
obviously, particularly us,
Charles Isbell and Michael Littman (1:10:33.420)
because this is the kind of thing that we do.
Lex Fridman (1:10:35.380)
And my guess is that's true
Charles Isbell and Michael Littman (1:10:36.860)
of the vast majority of your audience.
Lex Fridman (1:10:39.380)
But that doesn't mean the,
Charles Isbell and Michael Littman (1:10:41.600)
I'm standing in front of you telling you this,
Lex Fridman (1:10:43.380)
is the thing that people are excited about.
Lex Fridman (1:10:47.380)
And that's why they want to be there,
Lex Fridman (1:10:49.060)
primarily why they want to be there.
Lex Fridman (1:10:50.860)
So to me, that's what COVID has forced us to deal with,
Lex Fridman (1:10:54.380)
even though I think we're still all in deep denial about it
Lex Fridman (1:10:57.020)
and hoping that it'll go back to that.
Lex Fridman (1:10:59.900)
And I think about 85% of it will.
Charles Isbell and Michael Littman (1:11:01.520)
We'll be able to pretend
Lex Fridman (1:11:02.360)
that that's really the way it is, again,
Lex Fridman (1:11:03.620)
and we'll forget the lessons of this.
Lex Fridman (1:11:05.340)
But technically what'll come out of it,
Charles Isbell and Michael Littman (1:11:07.500)
or technologically what'll come out of it
Lex Fridman (1:11:09.060)
is a way of providing a more dispersed experience
Charles Isbell and Michael Littman (1:11:12.700)
through online education
Lex Fridman (1:11:13.820)
and these kinds of remote things that we've learned.
Lex Fridman (1:11:16.020)
And we'll have to come up with new ways to engage them
Lex Fridman (1:11:19.060)
in the experience of college,
Charles Isbell and Michael Littman (1:11:20.540)
which includes not just the parties
Lex Fridman (1:11:22.060)
or the whatever kids do,
Lex Fridman (1:11:23.580)
but the learning part of it
Lex Fridman (1:11:25.460)
so that they actually come out four or five
Charles Isbell and Michael Littman (1:11:27.060)
or six years later with having actually learned something.
Lex Fridman (1:11:30.900)
So I think the world
Charles Isbell and Michael Littman (1:11:32.700)
will be radically different afterwards.
Lex Fridman (1:11:34.080)
And I think technology will matter for that,
Charles Isbell and Michael Littman (1:11:36.200)
just not in the way that the people
Lex Fridman (1:11:38.460)
who were building the technology originally
Charles Isbell and Michael Littman (1:11:40.700)
imagined it would be.
Lex Fridman (1:11:42.120)
And I think this would have been true even without COVID,
Lex Fridman (1:11:45.260)
but COVID has accelerated that reality.
Lex Fridman (1:11:47.840)
So it's happening in two or three years or five years,
Charles Isbell and Michael Littman (1:11:50.200)
as opposed to 10 or 15.
Lex Fridman (1:11:52.180)
That was an amazing answer that I did not understand.
Charles Isbell and Michael Littman (1:11:56.180)
It was passionate and meaningful.
Lex Fridman (1:11:58.180)
Shots fired.
Lex Fridman (1:11:59.180)
But I don't, no, I just didn't,
Lex Fridman (1:12:00.380)
no, I'm not trying to criticize it.
Charles Isbell and Michael Littman (1:12:01.420)
I just think, I don't think I'm getting it.
Lex Fridman (1:12:03.220)
So you mentioned disaggregation.
Lex Fridman (1:12:05.420)
So what's that?
Lex Fridman (1:12:06.380)
Well, so the power of technology
Charles Isbell and Michael Littman (1:12:09.480)
that if you go on the West Coast and hang out long enough
Lex Fridman (1:12:11.580)
is all about we're gonna disaggregate these things together.
Charles Isbell and Michael Littman (1:12:13.580)
The books from the bookstore, that kind of a thing.
Lex Fridman (1:12:15.900)
And then suddenly Amazon controls the universe, right?
Lex Fridman (1:12:17.820)
And technology is a disruptor, right?
Lex Fridman (1:12:19.460)
And people have been predicting that
Charles Isbell and Michael Littman (1:12:20.820)
for higher education for a long time,
Lex Fridman (1:12:22.860)
but certainly in the age of moves.
Lex Fridman (1:12:23.700)
So is this the sort of idea like
Lex Fridman (1:12:26.700)
students can aggregate on a campus someplace
Lex Fridman (1:12:30.120)
and then take classes over the network anywhere?
Lex Fridman (1:12:33.500)
Yeah, this is what people thought was gonna happen,
Lex Fridman (1:12:34.900)
or at least people claimed it was gonna happen, right?
Lex Fridman (1:12:37.060)
Because my daughter is essentially doing that now.
Charles Isbell and Michael Littman (1:12:38.940)
She's on one campus, but learning in a different campus.
Lex Fridman (1:12:41.060)
Sure, and COVID makes that possible, right?
Lex Fridman (1:12:43.580)
COVID makes that legal, all but avoidable, right?
Lex Fridman (1:12:47.540)
But the idea originally was that,
Charles Isbell and Michael Littman (1:12:49.480)
you and I were gonna create this machine learning class
Lex Fridman (1:12:51.180)
and it was gonna be great,
Lex Fridman (1:12:52.020)
and then no one else would,
Lex Fridman (1:12:52.860)
there'd be the machine learning class everyone takes, right?
Charles Isbell and Michael Littman (1:12:54.860)
That was never gonna happen, but something like that,
Lex Fridman (1:12:57.300)
you can see happening. But I feel like
Charles Isbell and Michael Littman (1:12:58.140)
you didn't address that.
Lex Fridman (1:12:58.960)
Why, why, why is it that, why, why?
Charles Isbell and Michael Littman (1:13:02.220)
I don't think that will be the thing that happens.
Lex Fridman (1:13:04.220)
So the college experience,
Charles Isbell and Michael Littman (1:13:05.340)
maybe I missed what the college experience was.
Lex Fridman (1:13:07.220)
I thought it was peers, like people hanging around.
Charles Isbell and Michael Littman (1:13:10.140)
A large part of it is peers.
Lex Fridman (1:13:11.660)
Well, it's peers and independence.
Charles Isbell and Michael Littman (1:13:13.600)
Yeah, but none of that,
Lex Fridman (1:13:15.040)
you can do classes online for all of that.
Lex Fridman (1:13:17.320)
No, no, no, no, because we're social people, right?
Lex Fridman (1:13:20.780)
So you wanna be in the same room.
Lex Fridman (1:13:21.620)
So when we take the classes,
Lex Fridman (1:13:22.440)
that also has to be part of an experience.
Charles Isbell and Michael Littman (1:13:25.200)
It's in a context, and the context is the university.
Lex Fridman (1:13:27.340)
And by the way, it actually matters
Charles Isbell and Michael Littman (1:13:29.580)
that Georgia Tech really is different from Brown.
Lex Fridman (1:13:33.260)
I see, because then students can choose
Charles Isbell and Michael Littman (1:13:36.120)
the kind of experience they think
Lex Fridman (1:13:37.220)
is gonna be best for them.
Charles Isbell and Michael Littman (1:13:38.300)
Okay, I think we're giving too much agency to the students
Lex Fridman (1:13:41.060)
in making an informed decision.
Charles Isbell and Michael Littman (1:13:42.460)
Okay. But the truth,
Lex Fridman (1:13:43.300)
but yes, they will make choices
Lex Fridman (1:13:45.480)
and they will have different experiences.
Lex Fridman (1:13:46.780)
And some of those choices will be made for them.
Charles Isbell and Michael Littman (1:13:48.620)
Some of them will be choices they're making
Lex Fridman (1:13:49.980)
because they think it's this, that, or the other.
Charles Isbell and Michael Littman (1:13:51.620)
I just don't want to say,
Lex Fridman (1:13:52.700)
I don't want to give the idea.
Charles Isbell and Michael Littman (1:13:53.540)
It's not homogenous.
Lex Fridman (1:13:55.100)
Yes, it's certainly not homogenous, right?
Charles Isbell and Michael Littman (1:13:56.940)
I mean, Georgia Tech is different from Brown.
Lex Fridman (1:13:59.420)
Brown is different from pick your favorite state school
Lex Fridman (1:14:03.340)
in Iowa, Iowa State, okay?
Lex Fridman (1:14:05.780)
Which I guess is my favorite state school in Iowa.
Lex Fridman (1:14:07.900)
But these are all different.
Lex Fridman (1:14:09.700)
They have different contexts.
Lex Fridman (1:14:10.700)
And a lot of those contexts are,
Lex Fridman (1:14:12.180)
they're about history, yes,
Lex Fridman (1:14:13.560)
but they're also about the location of where you are.
Lex Fridman (1:14:15.980)
They're about the larger group of people who are around you,
Charles Isbell and Michael Littman (1:14:18.240)
whether you're in Athens, Georgia,
Lex Fridman (1:14:20.560)
and you're basically the only thing that's there
Charles Isbell and Michael Littman (1:14:23.120)
as a university, you're responsible for all the jobs,
Lex Fridman (1:14:25.340)
or whether you're at Georgia State University,
Charles Isbell and Michael Littman (1:14:27.100)
which is an urban campus,
Lex Fridman (1:14:28.980)
where you're surrounded by six million people
Charles Isbell and Michael Littman (1:14:31.860)
in your campus where it ends and begins in the city,
Lex Fridman (1:14:33.920)
ends and begins, we don't know.
Charles Isbell and Michael Littman (1:14:35.540)
It actually matters whether you're a small campus
Lex Fridman (1:14:37.300)
or a large campus.
Charles Isbell and Michael Littman (1:14:38.140)
I mean, these things matter.
Lex Fridman (1:14:38.960)
Why is it that if you go to Georgia Tech,
Charles Isbell and Michael Littman (1:14:41.580)
you're forever proud of that,
Lex Fridman (1:14:44.860)
and you say that to people at dinners,
Charles Isbell and Michael Littman (1:14:47.380)
like bars and whatever,
Lex Fridman (1:14:49.400)
and if you get a degree at an online university somewhere,
Charles Isbell and Michael Littman (1:14:56.660)
that's not a thing that comes up at a bar.
Lex Fridman (1:14:58.940)
Well, it's funny you say that.
Lex Fridman (1:14:59.800)
So the students who take our online masters
Lex Fridman (1:15:03.700)
by several measures are more loyal
Charles Isbell and Michael Littman (1:15:06.580)
than the students who come on campus,
Lex Fridman (1:15:07.820)
certainly for the master's degree.
Charles Isbell and Michael Littman (1:15:09.300)
The reason for that, I think,
Lex Fridman (1:15:10.920)
and you'd have to ask them,
Lex Fridman (1:15:11.840)
but based on my conversations with them,
Lex Fridman (1:15:13.920)
I feel comfortable saying this,
Charles Isbell and Michael Littman (1:15:15.460)
is because this didn't exist before.
Lex Fridman (1:15:18.100)
I mean, we talk about this online masters
Lex Fridman (1:15:19.740)
and that it's reaching 11,000 students,
Lex Fridman (1:15:22.060)
and that's an amazing thing,
Lex Fridman (1:15:22.980)
and we're admitting everyone we believe who can succeed.
Lex Fridman (1:15:25.060)
We got a 60% acceptance rate.
Lex Fridman (1:15:26.620)
It's amazing, right?
Lex Fridman (1:15:27.820)
It's also a $6,600 degree.
Charles Isbell and Michael Littman (1:15:29.660)
The entire degree costs $6,600 or $7,000,
Lex Fridman (1:15:32.020)
depending on how long you take.
Charles Isbell and Michael Littman (1:15:33.440)
A dollar degree, as opposed to $46,000
Lex Fridman (1:15:35.580)
it would cost you to come on campus.
Lex Fridman (1:15:37.860)
So that feels, and I can do it while I'm working full time,
Lex Fridman (1:15:40.540)
and I've got a family and a mortgage
Lex Fridman (1:15:42.180)
and all these other things.
Lex Fridman (1:15:43.340)
So it's an opportunity to do something you wanted to do,
Lex Fridman (1:15:46.220)
but you didn't think was possible
Lex Fridman (1:15:47.700)
without giving up two years of your life,
Charles Isbell and Michael Littman (1:15:50.300)
as well as all the money
Lex Fridman (1:15:51.300)
and everything else in the life that you had built.
Lex Fridman (1:15:53.140)
So I think we created something that's had an impact,
Lex Fridman (1:15:56.940)
but importantly, we gave a set of people opportunities
Charles Isbell and Michael Littman (1:15:59.640)
they otherwise didn't feel they had.
Lex Fridman (1:16:00.840)
So I think people feel very loyal about that.
Lex Fridman (1:16:02.860)
And my biggest piece of evidence for that,
Lex Fridman (1:16:04.180)
besides the surveys,
Charles Isbell and Michael Littman (1:16:05.320)
is that we have somewhere north of 80 students,
Lex Fridman (1:16:08.660)
might be 100 at this point,
Charles Isbell and Michael Littman (1:16:09.980)
who graduated, but come back in TA for this class,
Lex Fridman (1:16:15.020)
for basically minimum wage,
Charles Isbell and Michael Littman (1:16:16.780)
even though they're working full time,
Lex Fridman (1:16:17.980)
because they believe in sort of having that opportunity
Lex Fridman (1:16:21.940)
and they wanna be a part of something.
Lex Fridman (1:16:23.360)
Now, will generation three feel this way?
Lex Fridman (1:16:25.980)
15 years from now, will people have that same sense?
Lex Fridman (1:16:28.020)
I don't know, but right now they kind of do.
Lex Fridman (1:16:31.100)
And so it's not the online,
Lex Fridman (1:16:32.900)
it's a matter of feeling as if you're a part of something.
Lex Fridman (1:16:36.140)
Right, we're all very tribal, right?
Lex Fridman (1:16:39.020)
And I think there's something very tribal
Charles Isbell and Michael Littman (1:16:42.100)
about being a part of something like that.
Lex Fridman (1:16:44.280)
Being on campus makes that easier,
Charles Isbell and Michael Littman (1:16:45.880)
going through a shared experience makes that easier.
Lex Fridman (1:16:48.220)
It's harder to have that shared experience
Charles Isbell and Michael Littman (1:16:49.720)
if you're alone looking at a computer screen.
Lex Fridman (1:16:52.020)
We can create ways to make that true.
Lex Fridman (1:16:53.340)
But is it possible?
Lex Fridman (1:16:54.340)
It is possible.
Charles Isbell and Michael Littman (1:16:55.220)
The question is, it still is the intuition to me,
Lex Fridman (1:16:58.360)
and it was at the beginning when I saw something
Charles Isbell and Michael Littman (1:17:01.420)
like the online master's program,
Lex Fridman (1:17:04.580)
is that this is gonna replace universities.
Charles Isbell and Michael Littman (1:17:07.980)
No, it won't replace universities.
Lex Fridman (1:17:09.200)
But like why?
Charles Isbell and Michael Littman (1:17:11.080)
Because it's living
Lex Fridman (1:17:11.920)
in a different part of the ecosystem, right?
Charles Isbell and Michael Littman (1:17:13.920)
The people who are taking it are already adults,
Lex Fridman (1:17:15.660)
they've gone through their undergrad experience.
Charles Isbell and Michael Littman (1:17:18.720)
I think their goals have shifted from when they were 17.
Lex Fridman (1:17:21.900)
They have other things that are going on.
Lex Fridman (1:17:23.380)
But it does do something really important,
Lex Fridman (1:17:25.540)
something very social and very important, right?
Charles Isbell and Michael Littman (1:17:28.380)
You know this whole thing about,
Lex Fridman (1:17:30.220)
don't build the sidewalks, just leave the grass
Lex Fridman (1:17:32.060)
and the students or the people will walk
Lex Fridman (1:17:33.640)
and you put the sidewalks where they create paths,
Charles Isbell and Michael Littman (1:17:35.340)
this kind of thing.
Lex Fridman (1:17:36.180)
That's interesting, yeah.
Charles Isbell and Michael Littman (1:17:37.580)
Their architects apparently believe
Lex Fridman (1:17:39.060)
that's the right way to do things.
Charles Isbell and Michael Littman (1:17:40.740)
The metaphor here is that we created this environment,
Lex Fridman (1:17:45.260)
we didn't quite know how to think about the social aspect,
Lex Fridman (1:17:48.660)
but we didn't have time to solve all,
Lex Fridman (1:17:51.100)
do all the social engineering, right?
Charles Isbell and Michael Littman (1:17:53.100)
The students did it themselves,
Lex Fridman (1:17:54.220)
they created these groups, like on Google Plus,
Charles Isbell and Michael Littman (1:17:58.400)
there were like 30 something groups created
Lex Fridman (1:18:00.160)
in the first year because somebody had used Google Plus.
Lex Fridman (1:18:04.220)
And they created these groups
Lex Fridman (1:18:05.680)
and they divided up in ways that made sense.
Charles Isbell and Michael Littman (1:18:07.380)
We live in the same state or we're working
Lex Fridman (1:18:08.620)
on the same things or we have the same background
Charles Isbell and Michael Littman (1:18:10.060)
or whatever and they created these social things.
Lex Fridman (1:18:12.060)
We sent them T shirts and they wear,
Charles Isbell and Michael Littman (1:18:14.100)
we have all these great pictures of students
Lex Fridman (1:18:16.540)
putting on their T shirts as they travel around the world.
Charles Isbell and Michael Littman (1:18:18.340)
I climbed this mountain top, I'm putting this T shirt on,
Lex Fridman (1:18:20.560)
I'm a part of this, they were a part of them.
Charles Isbell and Michael Littman (1:18:22.060)
They created the social environment
Lex Fridman (1:18:24.660)
on top of the social network and the social media
Charles Isbell and Michael Littman (1:18:26.740)
that existed to create this sense of belonging
Lex Fridman (1:18:29.380)
and being a part of something.
Lex Fridman (1:18:30.500)
They found a way to do it, right?
Lex Fridman (1:18:32.700)
And I think they had other,
Charles Isbell and Michael Littman (1:18:36.180)
it scratched an itch that they had,
Lex Fridman (1:18:38.100)
but they had scratched some of that itch
Charles Isbell and Michael Littman (1:18:40.140)
that might've required they'd be physically
Lex Fridman (1:18:41.500)
in the same place long before, right?
Lex Fridman (1:18:44.420)
So I think, yes, it's possible
Lex Fridman (1:18:47.220)
and it's more than possible, it's necessary.
Lex Fridman (1:18:49.640)
But I don't think it's going to replace the university
Lex Fridman (1:18:54.220)
as we know it.
Charles Isbell and Michael Littman (1:18:55.180)
The university as we know it will change.
Lex Fridman (1:18:57.620)
But there's just a lot of power
Charles Isbell and Michael Littman (1:18:59.180)
in the kind of rite of passage
Lex Fridman (1:19:00.440)
kind of going off to yourself.
Charles Isbell and Michael Littman (1:19:01.500)
Now, maybe there'll be some other rite of passage
Lex Fridman (1:19:03.140)
that'll happen.
Charles Isbell and Michael Littman (1:19:03.980)
That'll drive us somewhere else, it's possible.
Lex Fridman (1:19:06.100)
So the university is such a fascinating mess of things.
Lex Fridman (1:19:11.100)
So just even the faculty position is a fascinating mess.
Lex Fridman (1:19:14.660)
Like it doesn't make any sense.
Charles Isbell and Michael Littman (1:19:15.900)
It's stabilized itself,
Lex Fridman (1:19:18.260)
but like why are the world class researchers
Charles Isbell and Michael Littman (1:19:22.080)
spending a huge amount of time or their time teaching
Lex Fridman (1:19:26.660)
and service?
Charles Isbell and Michael Littman (1:19:27.980)
Like you're doing like three jobs.
Lex Fridman (1:19:29.540)
And I mean, it turns out it's maybe an accident of history
Charles Isbell and Michael Littman (1:19:34.980)
or human evolution, I don't know.
Lex Fridman (1:19:36.340)
It seems like the people who are really good at teaching
Charles Isbell and Michael Littman (1:19:38.460)
are often really good at research.
Lex Fridman (1:19:40.420)
There seems to be a parallel there,
Lex Fridman (1:19:42.780)
but like it doesn't make any sense
Lex Fridman (1:19:44.480)
that you should be doing that.
Charles Isbell and Michael Littman (1:19:45.460)
At the same time, it also doesn't seem to make sense
Lex Fridman (1:19:48.740)
that your place where you party
Charles Isbell and Michael Littman (1:19:53.840)
is the same place where you go to learn calculus
Lex Fridman (1:19:56.340)
or whatever.
Lex Fridman (1:19:57.540)
But it's a safe space.
Lex Fridman (1:19:59.300)
Safe space for everything.
Charles Isbell and Michael Littman (1:20:00.540)
Yeah, relatively speaking, it's a safe space.
Lex Fridman (1:20:02.380)
Now, by the way, I feel the need very strongly
Charles Isbell and Michael Littman (1:20:05.300)
to point out that we are living
Lex Fridman (1:20:07.180)
in a very particular weird bubble, right?
Charles Isbell and Michael Littman (1:20:09.500)
Most people don't go to college.
Lex Fridman (1:20:10.860)
And by the way, the ones who do go to college,
Lex Fridman (1:20:12.880)
they're not 18 years old, right?
Lex Fridman (1:20:14.340)
They're like 25 or something.
Charles Isbell and Michael Littman (1:20:15.500)
I forget the numbers.
Lex Fridman (1:20:17.120)
The places where we've been, where we are,
Charles Isbell and Michael Littman (1:20:20.940)
they look like whatever we think
Lex Fridman (1:20:22.380)
the traditional movie version of universities are.
Lex Fridman (1:20:25.520)
But for most people, it's not that way at all.
Lex Fridman (1:20:27.460)
By the way, most people who drop out of college,
Lex Fridman (1:20:28.940)
it's entirely for financial reasons, right?
Lex Fridman (1:20:32.320)
So we were talking about a particular experience.
Lex Fridman (1:20:36.700)
And so for that set of people,
Lex Fridman (1:20:38.700)
which is very small, but larger than it was a decade
Charles Isbell and Michael Littman (1:20:42.940)
or two or three or four, certainly, ago,
Lex Fridman (1:20:45.940)
I don't think that will change.
Charles Isbell and Michael Littman (1:20:47.180)
My concern, which I think is kind of implicit
Lex Fridman (1:20:50.460)
in some of these questions,
Charles Isbell and Michael Littman (1:20:51.540)
is that somehow we will divide the world up further
Lex Fridman (1:20:55.440)
into the people who get to have this experience
Lex Fridman (1:20:57.060)
and get to have the network
Lex Fridman (1:20:57.900)
and they sort of benefit from it,
Lex Fridman (1:20:59.220)
and everyone else, while increasingly requiring
Lex Fridman (1:21:01.900)
that they have more and more credentials
Lex Fridman (1:21:03.180)
in order to get a job as a barista, right?
Lex Fridman (1:21:05.920)
You gotta have a master's degree
Charles Isbell and Michael Littman (1:21:07.020)
in order to work at Starbucks.
Lex Fridman (1:21:08.780)
I mean, we're gonna force people to do these things,
Lex Fridman (1:21:10.740)
but they're not gonna get to have that experience,
Lex Fridman (1:21:12.460)
and there'll be a small group of people who do
Charles Isbell and Michael Littman (1:21:13.860)
who will continue to, you know, positive feedback,
Lex Fridman (1:21:15.740)
look, et cetera, et cetera, et cetera.
Charles Isbell and Michael Littman (1:21:16.940)
I worry a lot about that, which is why, for me,
Lex Fridman (1:21:21.060)
and by the way, here's an answer
Charles Isbell and Michael Littman (1:21:21.980)
to your question about faculty,
Lex Fridman (1:21:22.940)
which is why, to me, that you have to focus
Charles Isbell and Michael Littman (1:21:24.400)
on access and the mission.
Lex Fridman (1:21:26.060)
I think the reason, whether it's good, bad, or strange,
Charles Isbell and Michael Littman (1:21:28.180)
I mean, I agree, it's strange,
Lex Fridman (1:21:29.740)
but I think it's useful to have the faculty member,
Charles Isbell and Michael Littman (1:21:32.180)
particularly at large R1 universities
Lex Fridman (1:21:33.860)
where we've all had experiences,
Charles Isbell and Michael Littman (1:21:36.580)
that you tie what they get to do
Lex Fridman (1:21:41.060)
and with the fundamental mission of the university
Lex Fridman (1:21:43.900)
and let the mission drive.
Lex Fridman (1:21:45.060)
What I hear when I talk to faculty is,
Charles Isbell and Michael Littman (1:21:47.080)
they love their PhD students
Lex Fridman (1:21:48.340)
because they're reproducing, basically, right?
Lex Fridman (1:21:51.100)
And it lets them do their research and multiply.
Lex Fridman (1:21:53.700)
But they understand that the mission is the undergrads,
Lex Fridman (1:21:57.300)
and so they will do it without complaint, mostly,
Lex Fridman (1:22:00.380)
because it's a part of the mission and why they're here,
Lex Fridman (1:22:02.420)
and they have experiences with it themselves,
Lex Fridman (1:22:04.140)
and it was important to get them
Charles Isbell and Michael Littman (1:22:06.340)
where they were going.
Lex Fridman (1:22:07.420)
The people who tend to get squeezed in that, by the way,
Charles Isbell and Michael Littman (1:22:09.020)
are the master's students, right,
Lex Fridman (1:22:10.380)
who are neither the PhDs who are like us
Charles Isbell and Michael Littman (1:22:12.220)
nor the undergrads we have already bought into the idea
Lex Fridman (1:22:14.820)
that we have to teach, though.
Charles Isbell and Michael Littman (1:22:16.460)
That's increasingly changing.
Lex Fridman (1:22:18.180)
Anyway, I think tying that mission in really matters,
Lex Fridman (1:22:21.020)
and it gives you a way to unify people
Lex Fridman (1:22:23.260)
around making it an actual higher calling.
Charles Isbell and Michael Littman (1:22:26.100)
Education feels like more of a higher calling to me
Lex Fridman (1:22:28.100)
than even research,
Charles Isbell and Michael Littman (1:22:30.780)
because education, you cannot treat it as a hobby
Lex Fridman (1:22:33.860)
if you're going to do it well.
Lex Fridman (1:22:34.900)
But that's the pushback on this whole system
Lex Fridman (1:22:38.380)
is that education should be a full time job, right?
Lex Fridman (1:22:44.420)
And it's almost like research is a distraction from that.
Lex Fridman (1:22:49.020)
Yes, although I think most of our colleagues,
Charles Isbell and Michael Littman (1:22:51.300)
many of our colleagues would say that research is the job
Lex Fridman (1:22:53.340)
and education is the distraction.
Charles Isbell and Michael Littman (1:22:55.100)
Right, but that's the beautiful dance.
Lex Fridman (1:22:56.620)
It seems to be that tension in itself seems to work,
Charles Isbell and Michael Littman (1:23:01.360)
seems to bring out the best in the faculty.
Lex Fridman (1:23:07.220)
But I will point out two things.
Charles Isbell and Michael Littman (1:23:08.300)
One thing I'm going to point out,
Lex Fridman (1:23:09.120)
and the other thing I want Michael to point out,
Charles Isbell and Michael Littman (1:23:10.740)
because I think Michael is much closer
Lex Fridman (1:23:11.920)
to sort of the ideal professor in some sense than I am.
Charles Isbell and Michael Littman (1:23:17.260)
Well, he is a dean.
Lex Fridman (1:23:18.100)
You're the platonic sense of a professor.
Charles Isbell and Michael Littman (1:23:19.620)
I don't know what he meant by that,
Lex Fridman (1:23:20.860)
but he is a dean, so he has a different experience.
Charles Isbell and Michael Littman (1:23:23.340)
I'm giving him time to think of the profound thing
Lex Fridman (1:23:26.200)
he's going to say.
Charles Isbell and Michael Littman (1:23:27.040)
That was good.
Lex Fridman (1:23:27.860)
But let me point this out,
Charles Isbell and Michael Littman (1:23:28.700)
which is that we have lecturers
Lex Fridman (1:23:31.500)
in the College of Computing where I am.
Charles Isbell and Michael Littman (1:23:33.900)
There's 10 or 12 of them, depending on how you count,
Lex Fridman (1:23:35.640)
as opposed to the 90 or so tenure track faculty.
Charles Isbell and Michael Littman (1:23:39.140)
Those 10 lecturers who only teach,
Lex Fridman (1:23:41.140)
well, they don't only teach, they also do service.
Charles Isbell and Michael Littman (1:23:42.860)
Some of them do research as well, but primarily they teach.
Lex Fridman (1:23:46.560)
They teach 50%, over 50% of our credit hours,
Lex Fridman (1:23:49.620)
and we teach everybody, right?
Lex Fridman (1:23:51.260)
So they're doing not just,
Charles Isbell and Michael Littman (1:23:54.100)
they're doing more than eight times the work
Lex Fridman (1:23:56.300)
of the tenure track faculty,
Charles Isbell and Michael Littman (1:23:59.180)
just more closer to nine or 10.
Lex Fridman (1:24:01.620)
And that's including our grad courses, right?
Lex Fridman (1:24:03.020)
So they're doing this, they're teaching more,
Lex Fridman (1:24:05.100)
they're touching more than anyone,
Lex Fridman (1:24:07.100)
and they're beloved for it.
Lex Fridman (1:24:08.860)
I mean, so we recently had a survey.
Charles Isbell and Michael Littman (1:24:11.660)
Everyone does these alumni surveys.
Lex Fridman (1:24:12.900)
You hire someone from the outside to do whatever,
Lex Fridman (1:24:14.260)
and I was really struck by something.
Lex Fridman (1:24:15.900)
You saw all these really cool numbers.
Charles Isbell and Michael Littman (1:24:17.380)
I'm not going to talk about it
Lex Fridman (1:24:18.260)
because it's all internal, confidential stuff.
Lex Fridman (1:24:19.900)
But one thing I will talk about
Lex Fridman (1:24:21.140)
is there was a single question we asked our alum,
Lex Fridman (1:24:23.100)
and these are people who graduated,
Lex Fridman (1:24:24.420)
born in the 30s and 40s,
Lex Fridman (1:24:25.860)
all the way up to people who graduated last week, right?
Lex Fridman (1:24:29.520)
Well, last semester.
Charles Isbell and Michael Littman (1:24:30.540)
Okay, good.
Lex Fridman (1:24:32.480)
Time flies.
Charles Isbell and Michael Littman (1:24:33.320)
Yeah, time flies.
Lex Fridman (1:24:34.680)
And it was the question,
Charles Isbell and Michael Littman (1:24:36.420)
name a single person who had a strong positive impact on you,
Lex Fridman (1:24:40.900)
something like that.
Lex Fridman (1:24:42.100)
I think it was special impact?
Lex Fridman (1:24:44.340)
Yeah, special impact on you.
Lex Fridman (1:24:45.580)
And then, so they got all the answers from people,
Lex Fridman (1:24:47.220)
and they created a word cloud.
Charles Isbell and Michael Littman (1:24:49.040)
It was clearly a word cloud created by people
Lex Fridman (1:24:50.540)
who don't do word clouds for a living
Charles Isbell and Michael Littman (1:24:52.220)
because they had one person whose name appeared
Lex Fridman (1:24:54.780)
like nine different times,
Charles Isbell and Michael Littman (1:24:56.220)
like Philip, Phil, Dr. Phil, you know, but whatever.
Lex Fridman (1:24:59.340)
But they got all this.
Lex Fridman (1:25:00.180)
And I looked at it, and I noticed something really cool.
Lex Fridman (1:25:02.340)
The five people from the College of Computing,
Charles Isbell and Michael Littman (1:25:06.700)
I recognized, were in that cloud.
Lex Fridman (1:25:09.220)
And four of them were lecturers,
Charles Isbell and Michael Littman (1:25:13.780)
the people who teach.
Lex Fridman (1:25:15.220)
Two of them, relatively modern,
Charles Isbell and Michael Littman (1:25:17.320)
both were chairs of our division of computing instruction.
Lex Fridman (1:25:19.840)
One just, one retired, one is going to retire soon.
Lex Fridman (1:25:22.180)
And the other two were lecturers,
Lex Fridman (1:25:23.620)
I remembered, from the 1980s.
Charles Isbell and Michael Littman (1:25:26.460)
Two of those four actually have.
Lex Fridman (1:25:28.020)
By the way, the fifth person was Charles.
Charles Isbell and Michael Littman (1:25:29.700)
That's not important.
Lex Fridman (1:25:30.540)
The thing is, I don't tell people that.
Lex Fridman (1:25:32.920)
But the two of those people
Lex Fridman (1:25:34.540)
our teaching awards are named after.
Charles Isbell and Michael Littman (1:25:36.000)
Thank you, Michael.
Lex Fridman (1:25:36.840)
Two of those our teaching awards are named after, right?
Lex Fridman (1:25:39.680)
So when you ask students, alumni,
Lex Fridman (1:25:41.720)
people who are now 60, 70 years old even,
Lex Fridman (1:25:44.220)
you know, who touched them?
Lex Fridman (1:25:45.280)
They say the Dean of Students.
Charles Isbell and Michael Littman (1:25:46.660)
They say the big teachers who taught
Lex Fridman (1:25:48.260)
the big introductory classes that got me into it.
Charles Isbell and Michael Littman (1:25:50.220)
There's a guy named Richard Park who's on there,
Lex Fridman (1:25:52.300)
who's, you know, who's known as a great teacher.
Charles Isbell and Michael Littman (1:25:55.380)
The Phil Adler guy who,
Lex Fridman (1:25:58.500)
I probably just said his last name wrong,
Lex Fridman (1:26:00.060)
but I know the first name's Phil
Lex Fridman (1:26:01.100)
because he kept showing up over and over again.
Charles Isbell and Michael Littman (1:26:03.140)
Famous.
Lex Fridman (1:26:03.960)
Adler is what it said.
Charles Isbell and Michael Littman (1:26:04.800)
Okay, good.
Lex Fridman (1:26:05.640)
But different people spelled it differently.
Lex Fridman (1:26:06.820)
So he appeared multiple times.
Lex Fridman (1:26:07.940)
Right.
Lex Fridman (1:26:08.780)
So he was a, clearly,
Lex Fridman (1:26:10.500)
he was a professor in the business school.
Lex Fridman (1:26:14.180)
But when you read about him,
Lex Fridman (1:26:15.860)
I went to read about him because I was curious who he was.
Charles Isbell and Michael Littman (1:26:17.420)
You know, it's all about his teaching
Lex Fridman (1:26:18.660)
and the students that he touched, right?
Lex Fridman (1:26:20.060)
So whatever it is that we're doing
Lex Fridman (1:26:22.320)
and we think we're doing that's important
Charles Isbell and Michael Littman (1:26:23.420)
or why we think the universities function,
Lex Fridman (1:26:25.340)
the people who go through it,
Charles Isbell and Michael Littman (1:26:27.740)
they remember the people who were kind to them,
Lex Fridman (1:26:29.500)
the people who taught them something,
Lex Fridman (1:26:31.380)
and they do remember it.
Lex Fridman (1:26:32.460)
They remember it later.
Charles Isbell and Michael Littman (1:26:33.500)
I think that's important.
Lex Fridman (1:26:35.820)
That's why the mission matters.
Charles Isbell and Michael Littman (1:26:37.200)
Yeah.
Lex Fridman (1:26:38.040)
Not to completely lose track of the fundamental problem
Charles Isbell and Michael Littman (1:26:41.940)
of how do we replace the party aspect of universities
Lex Fridman (1:26:46.940)
before we go to the what makes the platonic professor.
Lex Fridman (1:26:51.420)
Do you think, like, what in your sense is the role of MOOCs
Lex Fridman (1:26:57.960)
in this whole picture during COVID?
Charles Isbell and Michael Littman (1:27:00.360)
Like, should we desperately be clamoring
Lex Fridman (1:27:04.160)
to get back on campus?
Lex Fridman (1:27:05.880)
Or is this a stable place to be for a little while?
Lex Fridman (1:27:08.880)
I don't know.
Charles Isbell and Michael Littman (1:27:09.720)
I know that the online teaching experience
Lex Fridman (1:27:12.560)
and learning experience has been really rough.
Charles Isbell and Michael Littman (1:27:15.920)
I think that people find it to be a struggle
Lex Fridman (1:27:18.240)
in a way that's not a happy, positive struggle,
Charles Isbell and Michael Littman (1:27:21.960)
that when you got through it,
Lex Fridman (1:27:23.080)
you just feel like glad that it's over
Charles Isbell and Michael Littman (1:27:24.760)
as opposed to I've achieved something.
Lex Fridman (1:27:27.780)
So, you know, I worry about that.
Charles Isbell and Michael Littman (1:27:29.640)
But, you know, I worry about just even before this happened,
Lex Fridman (1:27:33.800)
I worry about lecture teaching,
Lex Fridman (1:27:35.640)
how well is that actually really working
Lex Fridman (1:27:38.700)
as far as a way to do education,
Charles Isbell and Michael Littman (1:27:40.640)
as a way to inspire people.
Lex Fridman (1:27:43.560)
I mean, all the data that I'm aware of seems to indicate,
Lex Fridman (1:27:47.040)
and this kind of fits, I think, with Charles's story,
Lex Fridman (1:27:49.640)
is that people respond to connection, right?
Charles Isbell and Michael Littman (1:27:54.480)
They actually feel, if they feel connected
Lex Fridman (1:27:57.400)
to the person teaching the class,
Charles Isbell and Michael Littman (1:27:59.120)
they're more likely to go along with it.
Lex Fridman (1:28:00.680)
They're more able to retain information.
Charles Isbell and Michael Littman (1:28:02.960)
They're more motivated to be involved
Lex Fridman (1:28:05.040)
in the class in some way.
Lex Fridman (1:28:06.840)
And that really matters.
Lex Fridman (1:28:09.840)
People...
Charles Isbell and Michael Littman (1:28:10.680)
You mean to the human themselves.
Lex Fridman (1:28:12.160)
Yeah.
Charles Isbell and Michael Littman (1:28:13.000)
Okay, can't you do that actually
Lex Fridman (1:28:14.600)
perhaps more effectively online?
Charles Isbell and Michael Littman (1:28:18.360)
Like you mentioned, science communication.
Lex Fridman (1:28:20.220)
So I literally, I think, learned linear algebra
Charles Isbell and Michael Littman (1:28:24.920)
from Gilbert Strang by watching MIT OpenCourseWare
Lex Fridman (1:28:28.280)
when I was in track.
Charles Isbell and Michael Littman (1:28:29.320)
Like, and he was a personality,
Lex Fridman (1:28:31.040)
he was a bit like a tiny...
Charles Isbell and Michael Littman (1:28:33.160)
In this tiny little world of math,
Lex Fridman (1:28:35.160)
he's a bit of a rockstar, right?
Lex Fridman (1:28:36.560)
So you kind of look up to that person.
Lex Fridman (1:28:40.720)
Can't that replace the in person education?
Charles Isbell and Michael Littman (1:28:44.880)
It can help.
Lex Fridman (1:28:45.720)
I will point out something, I can't share the numbers,
Lex Fridman (1:28:47.520)
but we have surveyed our students,
Lex Fridman (1:28:50.000)
and even though they have feelings
Charles Isbell and Michael Littman (1:28:51.440)
about what I would interpret as connection,
Lex Fridman (1:28:54.400)
I like that word, in the different modes of classrooms,
Charles Isbell and Michael Littman (1:28:58.440)
there's no difference between how well
Lex Fridman (1:29:00.860)
they think they're learning.
Charles Isbell and Michael Littman (1:29:02.560)
For them, the thing that makes them unhappy
Lex Fridman (1:29:05.680)
is the situation they're in.
Lex Fridman (1:29:06.960)
And I think the lack of connection,
Lex Fridman (1:29:08.700)
it's not whether they're learning anything.
Lex Fridman (1:29:10.560)
They seem to think they're learning something anyway, right?
Lex Fridman (1:29:13.360)
In fact, they seem to think
Charles Isbell and Michael Littman (1:29:14.360)
they're learning it equally well,
Lex Fridman (1:29:16.920)
presumably because the faculty are putting in,
Charles Isbell and Michael Littman (1:29:20.640)
or the instructors, more generally speaking,
Lex Fridman (1:29:22.960)
are putting in the energy and effort
Charles Isbell and Michael Littman (1:29:25.840)
to try to make certain that what they've curated
Lex Fridman (1:29:28.920)
can be expressed to them in a useful way.
Lex Fridman (1:29:30.640)
But the connection is missing.
Lex Fridman (1:29:31.800)
And so there's huge differences in what they prefer.
Lex Fridman (1:29:34.160)
And as far as I can tell,
Lex Fridman (1:29:35.000)
what they prefer is more connection, not less.
Charles Isbell and Michael Littman (1:29:37.360)
That connection just doesn't have to be physically
Lex Fridman (1:29:39.440)
in a classroom.
Charles Isbell and Michael Littman (1:29:40.280)
I mean, look, I used to teach 348 students
Lex Fridman (1:29:43.520)
in my machine learning class on campus.
Lex Fridman (1:29:44.840)
Do you know why?
Lex Fridman (1:29:45.680)
That was the biggest classroom on campus.
Charles Isbell and Michael Littman (1:29:48.640)
They're sitting in theater seats.
Lex Fridman (1:29:50.840)
I'm literally on a stage looking down on them
Lex Fridman (1:29:54.400)
and talking to them, right?
Lex Fridman (1:29:56.320)
There's no, I mean, we're not sitting down,
Charles Isbell and Michael Littman (1:29:59.640)
having a one on one conversation,
Lex Fridman (1:30:01.380)
reading each other's body language,
Charles Isbell and Michael Littman (1:30:02.760)
trying to communicate and going,
Lex Fridman (1:30:04.080)
we're not doing any of that.
Lex Fridman (1:30:05.400)
So if you're past the third row,
Lex Fridman (1:30:07.520)
it might as well be online anyway
Charles Isbell and Michael Littman (1:30:08.800)
is the kind of thing that people have said.
Lex Fridman (1:30:10.280)
Daphne has actually said some version of this
Charles Isbell and Michael Littman (1:30:12.920)
that online starts on the third row or something like that.
Lex Fridman (1:30:15.600)
And I think that's not, yeah, I like it.
Charles Isbell and Michael Littman (1:30:18.560)
I think it captures something important.
Lex Fridman (1:30:20.320)
But people still came, by the way.
Charles Isbell and Michael Littman (1:30:22.120)
Even the people who had access to our material
Lex Fridman (1:30:23.880)
would still come to class.
Charles Isbell and Michael Littman (1:30:25.240)
I mean, there's a certain element
Lex Fridman (1:30:26.420)
about looking to the person next to you.
Charles Isbell and Michael Littman (1:30:28.560)
It's just like their presence there, their boredom.
Lex Fridman (1:30:32.320)
And like when the parts are boring
Lex Fridman (1:30:34.960)
and their excitement when the parts are exciting,
Lex Fridman (1:30:37.960)
like in sharing in that,
Charles Isbell and Michael Littman (1:30:39.560)
like unspoken kind of, yeah, communication.
Lex Fridman (1:30:43.680)
In part, the connection is with the other people
Charles Isbell and Michael Littman (1:30:45.840)
in the room.
Lex Fridman (1:30:46.680)
Yeah, watching the circus on TV alone is not really.
Charles Isbell and Michael Littman (1:30:52.360)
Ever been to a movie theater
Lex Fridman (1:30:53.440)
and been the only one there at a comedy?
Charles Isbell and Michael Littman (1:30:55.280)
It's not as funny as when you're in a room
Lex Fridman (1:30:58.200)
full of people all laughing.
Charles Isbell and Michael Littman (1:31:00.080)
Well, you need, maybe you need just another person.
Lex Fridman (1:31:02.480)
It's like, as opposed to many.
Charles Isbell and Michael Littman (1:31:04.640)
Maybe there's some kind of.
Lex Fridman (1:31:06.120)
Well, there's different kinds of connection, right?
Lex Fridman (1:31:07.800)
And there's different kinds of comedy.
Lex Fridman (1:31:11.720)
Well, in the sense that.
Charles Isbell and Michael Littman (1:31:12.560)
As we're learning today.
Lex Fridman (1:31:15.000)
I wasn't sure if that was gonna land.
Lex Fridman (1:31:16.200)
But just the idea that different jokes,
Lex Fridman (1:31:21.640)
I've now done a little bit of standup.
Lex Fridman (1:31:23.120)
And so different jokes work in different size crowds too.
Lex Fridman (1:31:26.560)
No, it's true.
Charles Isbell and Michael Littman (1:31:27.400)
Where sometimes if it's a big enough crowd,
Lex Fridman (1:31:30.320)
then even a really subtle joke can take root someplace
Lex Fridman (1:31:33.600)
and then that cues other people.
Lex Fridman (1:31:34.920)
And it kind of,
Charles Isbell and Michael Littman (1:31:36.320)
there's a whole statistics of.
Lex Fridman (1:31:38.640)
I did this terrible thing to my brother.
Lex Fridman (1:31:40.160)
So when I was really young,
Lex Fridman (1:31:41.360)
I decided that my brother was only laughing
Charles Isbell and Michael Littman (1:31:44.720)
as it comes when I laughed.
Lex Fridman (1:31:46.480)
Like he was taking cues from me.
Lex Fridman (1:31:48.160)
So I like purposely didn't laugh
Lex Fridman (1:31:50.100)
just to see if I was right.
Lex Fridman (1:31:50.940)
And did you laugh at non funny things?
Lex Fridman (1:31:52.320)
Yes.
Charles Isbell and Michael Littman (1:31:53.160)
You really wanna do both sides.
Lex Fridman (1:31:54.000)
I did both sides.
Lex Fridman (1:31:54.960)
And at the end of it, I told him what I did.
Lex Fridman (1:31:58.400)
He was very upset about this.
Lex Fridman (1:32:00.000)
And from that day on.
Lex Fridman (1:32:01.920)
He lost his sense of humor.
Charles Isbell and Michael Littman (1:32:03.000)
No, no, no, no.
Lex Fridman (1:32:03.840)
Well, yes.
Lex Fridman (1:32:04.660)
But from that day on, he laughed on his own.
Lex Fridman (1:32:07.680)
He stopped taking cues from me.
Charles Isbell and Michael Littman (1:32:08.760)
I see.
Lex Fridman (1:32:09.600)
So I wanna say that it was a good thing that I did.
Charles Isbell and Michael Littman (1:32:11.720)
Yes, yes.
Lex Fridman (1:32:12.560)
You saved that man's life.
Charles Isbell and Michael Littman (1:32:14.040)
Yes, but it was mostly mean.
Lex Fridman (1:32:15.160)
But it's true though.
Lex Fridman (1:32:15.980)
It's true, right?
Lex Fridman (1:32:16.820)
That people, I think you're right.
Lex Fridman (1:32:19.320)
But okay, so where does that get us?
Lex Fridman (1:32:20.860)
That gets us the idea that,
Lex Fridman (1:32:23.040)
I mean, certainly movie theaters are a thing, right?
Lex Fridman (1:32:26.220)
Where people like to be watching together,
Charles Isbell and Michael Littman (1:32:28.260)
even though the people on the screen
Lex Fridman (1:32:30.800)
aren't really co present with the people in the audience.
Charles Isbell and Michael Littman (1:32:33.040)
The audience is co present with themselves.
Lex Fridman (1:32:35.000)
By the way, and that point,
Charles Isbell and Michael Littman (1:32:36.200)
it's an open question that's being raised by this,
Lex Fridman (1:32:38.600)
whether movies will no longer be a thing
Charles Isbell and Michael Littman (1:32:40.960)
because Netflix's audience is growing.
Lex Fridman (1:32:43.720)
So that's, it's a very parallel question for education.
Lex Fridman (1:32:47.160)
Will movie theaters still be a thing in 2021?
Lex Fridman (1:32:50.080)
No, but I think the argument is
Charles Isbell and Michael Littman (1:32:52.000)
that there is a feeling of being in the crowd
Lex Fridman (1:32:54.680)
that isn't replicated by being at home watching it
Lex Fridman (1:32:57.480)
and that there's value in that.
Lex Fridman (1:32:59.280)
And then I think just.
Charles Isbell and Michael Littman (1:33:00.120)
But, but.
Lex Fridman (1:33:02.040)
It scales better online.
Lex Fridman (1:33:03.280)
But I feel like we're having a conversation
Lex Fridman (1:33:06.840)
about whether concerts will still exist
Charles Isbell and Michael Littman (1:33:09.320)
after the invention of the record or the CD
Lex Fridman (1:33:13.040)
or wherever it is, right?
Charles Isbell and Michael Littman (1:33:13.960)
They won't.
Lex Fridman (1:33:14.920)
You're right, concerts are dead.
Charles Isbell and Michael Littman (1:33:16.880)
Well, okay, I think the joke is only funny
Lex Fridman (1:33:19.580)
if you say it before now.
Charles Isbell and Michael Littman (1:33:21.520)
Right, yeah, that's true.
Lex Fridman (1:33:23.480)
Like three years ago.
Charles Isbell and Michael Littman (1:33:24.440)
It's like, well, no, obviously concerts are still a big thing.
Lex Fridman (1:33:25.920)
I'll wait to publish this until we have a vaccine.
Charles Isbell and Michael Littman (1:33:27.920)
No, you know, we'll fix it in post.
Lex Fridman (1:33:30.360)
But I think the important thing is.
Charles Isbell and Michael Littman (1:33:33.320)
Fix the virus post.
Lex Fridman (1:33:34.400)
Concerts changed, right?
Charles Isbell and Michael Littman (1:33:36.960)
Concerts changed.
Lex Fridman (1:33:37.800)
First of all, movie theaters weren't this way, right?
Charles Isbell and Michael Littman (1:33:39.560)
In like the 60s and 70s, they weren't like this.
Lex Fridman (1:33:41.920)
Like blockbusters were basically what?
Lex Fridman (1:33:44.000)
Well, Jaws and Star Wars created blockbusters, right?
Lex Fridman (1:33:47.000)
Before then, there weren't.
Charles Isbell and Michael Littman (1:33:47.840)
Like the whole shared summer experience
Lex Fridman (1:33:49.740)
didn't exist in our lifetimes, right?
Charles Isbell and Michael Littman (1:33:52.320)
Certainly you were well into adulthood
Lex Fridman (1:33:53.760)
by the time this was true, right?
Lex Fridman (1:33:54.940)
So it's just a very different.
Lex Fridman (1:33:56.520)
It's very different.
Lex Fridman (1:33:57.360)
So what we've been experiencing in the last 10 years
Lex Fridman (1:33:59.480)
is not like the majority of human history,
Lex Fridman (1:34:01.760)
but more importantly, concerts, right?
Lex Fridman (1:34:03.480)
Concerts mean something different.
Charles Isbell and Michael Littman (1:34:04.680)
Most people don't go to concerts anymore.
Lex Fridman (1:34:07.640)
Like there's an age where you care about it.
Charles Isbell and Michael Littman (1:34:09.640)
You sort of stop doing it,
Lex Fridman (1:34:10.480)
but you keep listening to music or whatever
Lex Fridman (1:34:12.240)
and da, da, da, da, da, da, da.
Lex Fridman (1:34:13.800)
So I think that's a painful way of saying that
Charles Isbell and Michael Littman (1:34:21.100)
it will change.
Lex Fridman (1:34:22.520)
It was not the same thing as it going away.
Charles Isbell and Michael Littman (1:34:23.840)
Replace is too strong of a word, but it will change.
Lex Fridman (1:34:27.000)
It has to.
Charles Isbell and Michael Littman (1:34:27.840)
Actually, like to push back, I wonder,
Lex Fridman (1:34:29.760)
because I think you're probably just throwing
Charles Isbell and Michael Littman (1:34:31.880)
that your intuition now.
Lex Fridman (1:34:33.160)
Oh, I wasn't.
Lex Fridman (1:34:34.200)
And it's possible that concerts,
Lex Fridman (1:34:37.240)
more people go to concerts now,
Lex Fridman (1:34:39.620)
but obviously much more people listen to,
Lex Fridman (1:34:42.720)
well, that's dumb, than before there was records.
Charles Isbell and Michael Littman (1:34:46.920)
It's possible to argue that if you look at the data,
Lex Fridman (1:34:51.660)
that it just expanded the pie of what music listening means.
Lex Fridman (1:34:55.960)
So it's possible that universities grow in the parallel
Lex Fridman (1:34:59.400)
or the theaters grow,
Lex Fridman (1:35:00.680)
but also more people get to watch movies,
Lex Fridman (1:35:02.600)
more people get to be educated.
Charles Isbell and Michael Littman (1:35:05.640)
Yeah, I hope that is true.
Lex Fridman (1:35:07.040)
Yeah, and to the extent that we can grow the pie
Lex Fridman (1:35:09.680)
and have education be not just something you do
Lex Fridman (1:35:11.960)
for four years when you're done with your other education,
Lex Fridman (1:35:16.400)
but it be a more lifelong thing,
Lex Fridman (1:35:19.080)
that would have tremendous benefits,
Charles Isbell and Michael Littman (1:35:20.600)
especially as the economy and the world change rapidly.
Lex Fridman (1:35:24.480)
People need opportunities to stay abreast of these changes.
Lex Fridman (1:35:28.880)
And so, I don't know,
Lex Fridman (1:35:31.720)
that's all part of the ecosystem.
Charles Isbell and Michael Littman (1:35:33.480)
It's all to the good.
Lex Fridman (1:35:34.320)
I mean, I'm not gonna have an argument
Charles Isbell and Michael Littman (1:35:36.880)
about whether we lost fidelity
Lex Fridman (1:35:38.920)
when we went from Laserdisc to DVDs
Charles Isbell and Michael Littman (1:35:40.740)
or record players to CDs.
Lex Fridman (1:35:43.000)
I mean, I'm willing to grant that that is true,
Lex Fridman (1:35:45.580)
but convenience matters and the ability to do something
Lex Fridman (1:35:50.800)
that you couldn't do otherwise
Charles Isbell and Michael Littman (1:35:51.720)
because that convenience matters.
Lex Fridman (1:35:53.760)
And you can tell me I'm only getting 90% of the experience,
Lex Fridman (1:35:56.120)
but I'm getting the experience.
Lex Fridman (1:35:57.720)
I wasn't getting it before or it wasn't lasting as long
Charles Isbell and Michael Littman (1:36:00.080)
or it wasn't as easy.
Lex Fridman (1:36:00.920)
I mean, this just seems straightforward to me.
Charles Isbell and Michael Littman (1:36:03.600)
It's gonna, it's going to change.
Lex Fridman (1:36:05.500)
It is for the good that more people get access
Lex Fridman (1:36:08.240)
and it is our job to do two separate things.
Lex Fridman (1:36:10.480)
One, to educate them and make access available.
Charles Isbell and Michael Littman (1:36:13.440)
That's our mission.
Lex Fridman (1:36:14.700)
But also for very simple selfish reasons,
Charles Isbell and Michael Littman (1:36:17.040)
we need to figure out how to do it better
Lex Fridman (1:36:18.280)
so that we individually stay in business.
Charles Isbell and Michael Littman (1:36:20.100)
We can do both of those things at the same time.
Lex Fridman (1:36:21.840)
They are not in, they may be intention,
Lex Fridman (1:36:24.360)
but they are not mutually exclusive.
Lex Fridman (1:36:28.020)
So you've educated some scary number of people.
Lex Fridman (1:36:34.960)
So you've seen a lot of people succeed,
Lex Fridman (1:36:37.320)
find their path through life.
Charles Isbell and Michael Littman (1:36:39.520)
Is there a device that you can give to a young person today
Lex Fridman (1:36:45.280)
about computer science education,
Charles Isbell and Michael Littman (1:36:48.960)
about education in general, about life,
Lex Fridman (1:36:53.320)
about whatever the journey that one takes in there,
Charles Isbell and Michael Littman (1:36:59.480)
maybe in their teens, in their early 20s,
Lex Fridman (1:37:02.600)
sort of in those underground years
Charles Isbell and Michael Littman (1:37:05.080)
as you try to go through the essential process of partying
Lex Fridman (1:37:09.120)
and not going to classes
Lex Fridman (1:37:10.740)
and yet somehow trying to get a degree?
Lex Fridman (1:37:12.920)
If you get to the point where you're far enough up
Charles Isbell and Michael Littman (1:37:16.420)
in the hierarchy of needs that you can actually
Lex Fridman (1:37:20.240)
make decisions like this,
Charles Isbell and Michael Littman (1:37:21.880)
then find the thing that you're passionate about
Lex Fridman (1:37:24.460)
and pursue it.
Lex Fridman (1:37:25.680)
And sometimes it's the thing that drives your life
Lex Fridman (1:37:27.820)
and sometimes it's secondary.
Lex Fridman (1:37:29.040)
And you'll do other things because you've got to eat, right?
Lex Fridman (1:37:31.600)
You've got a family, you've got to feed,
Charles Isbell and Michael Littman (1:37:32.800)
you've got people you have to help or whatever.
Lex Fridman (1:37:34.480)
And I understand that and it's not easy for everyone,
Lex Fridman (1:37:36.360)
but always take a moment or two
Lex Fridman (1:37:39.760)
to pursue the things that you love,
Charles Isbell and Michael Littman (1:37:42.520)
the things that bring passion and happiness to your life.
Lex Fridman (1:37:45.400)
And if you don't, I know that sounds corny,
Lex Fridman (1:37:46.780)
but I genuinely believe it.
Lex Fridman (1:37:47.960)
And if you don't have such a thing,
Charles Isbell and Michael Littman (1:37:49.880)
then you're lying to yourself.
Lex Fridman (1:37:51.380)
You have such a thing.
Charles Isbell and Michael Littman (1:37:52.400)
You just have to find it.
Lex Fridman (1:37:53.480)
And it's okay if it takes you a long time to get there.
Charles Isbell and Michael Littman (1:37:56.000)
Rodney Dangerfield became a comedian in his 50s, I think.
Lex Fridman (1:38:00.360)
Certainly wasn't his 20s.
Lex Fridman (1:38:01.760)
And lots of people failed for a very long time
Lex Fridman (1:38:03.660)
before getting to where they were going.
Charles Isbell and Michael Littman (1:38:06.680)
I try to have hope and it wasn't obvious.
Lex Fridman (1:38:09.740)
I mean, you and I talked about the experience that I had
Charles Isbell and Michael Littman (1:38:13.760)
a long time ago with a particular police officer.
Lex Fridman (1:38:17.480)
Was it my first one and was it my last one?
Lex Fridman (1:38:20.920)
But in my view, I wasn't supposed to be here after that
Lex Fridman (1:38:24.160)
and I'm here.
Lex Fridman (1:38:25.000)
So it's all gravy.
Lex Fridman (1:38:25.920)
So you might as well go ahead and grab life as you can
Charles Isbell and Michael Littman (1:38:29.000)
because of that.
Lex Fridman (1:38:29.840)
That's sort of how I see it.
Lex Fridman (1:38:31.200)
While recognizing, again, the delusion matters, right?
Lex Fridman (1:38:34.480)
Allow yourself to be deluded.
Charles Isbell and Michael Littman (1:38:35.880)
Allow yourself to believe that it's all gonna work out.
Lex Fridman (1:38:38.060)
Just don't be so deluded that you miss the obvious.
Lex Fridman (1:38:41.700)
And you're gonna be fine.
Lex Fridman (1:38:43.440)
It's gonna be there.
Charles Isbell and Michael Littman (1:38:44.860)
It's gonna be there.
Lex Fridman (1:38:45.700)
It's gonna work out.
Lex Fridman (1:38:46.600)
What do you think?
Lex Fridman (1:38:47.980)
I like to say choose your parents wisely
Charles Isbell and Michael Littman (1:38:51.120)
because that has a big impact on your life.
Lex Fridman (1:38:53.120)
It's different.
Charles Isbell and Michael Littman (1:38:54.560)
Yeah, I mean, there's a whole lot of things
Lex Fridman (1:38:57.360)
that you don't get to pick.
Lex Fridman (1:38:58.440)
And whether you get to have one kind of life
Lex Fridman (1:39:02.920)
or a different kind of life can depend a lot
Charles Isbell and Michael Littman (1:39:05.080)
on things out of your control.
Lex Fridman (1:39:06.840)
But I really do believe in the passion, excitement thing.
Charles Isbell and Michael Littman (1:39:09.960)
My, I was talking to my mom on the phone the other day
Lex Fridman (1:39:11.920)
and essentially what came out is that computer science
Charles Isbell and Michael Littman (1:39:19.840)
is really popular right now.
Lex Fridman (1:39:22.080)
And I get to be a professor teaching something
Charles Isbell and Michael Littman (1:39:25.880)
that's very attractive to people.
Lex Fridman (1:39:28.840)
And she was like trying to give me some appreciation
Charles Isbell and Michael Littman (1:39:33.560)
for how foresightful I was for choosing this line of work
Lex Fridman (1:39:37.680)
as if somehow I knew that this is what was gonna happen
Charles Isbell and Michael Littman (1:39:40.000)
in 2020, but that's not how it went for me at all.
Lex Fridman (1:39:44.020)
Like I studied computer science
Charles Isbell and Michael Littman (1:39:45.540)
because I was just interested.
Lex Fridman (1:39:47.900)
It was just so interesting to me.
Charles Isbell and Michael Littman (1:39:49.600)
I didn't think it would be particularly lucrative.
Lex Fridman (1:39:54.600)
And I've done everything I've can to keep it
Charles Isbell and Michael Littman (1:39:56.480)
as unlucrative as possible.
Lex Fridman (1:39:59.480)
Some of my friends and colleagues have not done that.
Lex Fridman (1:40:03.600)
And I pride myself on my ability to remain unrich.
Lex Fridman (1:40:07.880)
But I do believe that, like I'm glad.
Charles Isbell and Michael Littman (1:40:13.300)
I mean, I'm glad that it worked out for me.
Lex Fridman (1:40:15.240)
It could have been like, oh, what I was really fascinated by
Charles Isbell and Michael Littman (1:40:17.920)
is this particular kind of engraving
Lex Fridman (1:40:19.300)
that nobody cares about.
Lex Fridman (1:40:20.680)
But so I got lucky and the thing that I cared about
Lex Fridman (1:40:22.800)
happened to be a thing that other people
Charles Isbell and Michael Littman (1:40:24.120)
eventually cared about.
Lex Fridman (1:40:26.300)
But I don't think I would have had a fun time
Charles Isbell and Michael Littman (1:40:28.080)
choosing anything else.
Lex Fridman (1:40:29.180)
Like this was the thing that kept me interested and engaged.
Charles Isbell and Michael Littman (1:40:32.720)
Well, one thing that people tell me,
Lex Fridman (1:40:34.720)
especially around the early undergraduate,
Lex Fridman (1:40:38.200)
and the internet is part of the problem here,
Lex Fridman (1:40:41.560)
is they say they're passionate about so many things.
Lex Fridman (1:40:44.840)
How do I choose a thing?
Lex Fridman (1:40:46.520)
Which is a harder thing for me to know what to do with.
Lex Fridman (1:40:50.280)
Is there any?
Lex Fridman (1:40:51.360)
I mean, don't you know which, I mean, you know, look.
Charles Isbell and Michael Littman (1:40:55.680)
A long time ago, I walked down a hallway
Lex Fridman (1:40:57.720)
and I took a left turn.
Charles Isbell and Michael Littman (1:40:59.020)
Yeah.
Lex Fridman (1:40:59.860)
I could have taken a right turn.
Lex Fridman (1:41:01.320)
And my world could be better or it could be worse.
Lex Fridman (1:41:03.800)
I have no idea.
Charles Isbell and Michael Littman (1:41:04.640)
I have no way of knowing.
Lex Fridman (1:41:05.460)
Is there anything about this particular hallway
Lex Fridman (1:41:07.080)
that's relevant or you're just in general choices?
Lex Fridman (1:41:09.080)
Yeah, you were on the left.
Charles Isbell and Michael Littman (1:41:09.920)
It sounds like you regret not taking the right turn.
Lex Fridman (1:41:11.880)
Oh no, not at all.
Charles Isbell and Michael Littman (1:41:12.800)
You brought it up.
Lex Fridman (1:41:13.920)
Well, because there was a turn there.
Lex Fridman (1:41:16.460)
On the left was Michael Newman's office, right?
Lex Fridman (1:41:18.080)
I mean, these sorts of things happen, right?
Lex Fridman (1:41:20.080)
But here's the thing.
Lex Fridman (1:41:20.920)
On the right, by the way, there was just a blank wall.
Charles Isbell and Michael Littman (1:41:22.640)
It wasn't a huge choice.
Lex Fridman (1:41:24.480)
It would have really hurt.
Charles Isbell and Michael Littman (1:41:25.320)
He tried first.
Lex Fridman (1:41:26.280)
No, but it's true, right?
Lex Fridman (1:41:27.840)
You know, I think about Ron Brockman, right?
Lex Fridman (1:41:29.840)
I went, I took a trip I wasn't supposed to take
Lex Fridman (1:41:33.320)
and I ended up talking to Ron about this
Lex Fridman (1:41:38.100)
and I ended up going down this entire path
Charles Isbell and Michael Littman (1:41:40.720)
that allowed me to, I think, get tenure.
Lex Fridman (1:41:42.840)
But by the way, I decided to say yes to something
Charles Isbell and Michael Littman (1:41:45.840)
that didn't make any sense
Lex Fridman (1:41:46.740)
and I went down this educational path.
Lex Fridman (1:41:48.260)
But it would have been, you know, who knows, right?
Lex Fridman (1:41:50.480)
Maybe if I hadn't done that,
Charles Isbell and Michael Littman (1:41:52.120)
I would be a billionaire right now.
Lex Fridman (1:41:54.320)
I'd be Elon Musk.
Charles Isbell and Michael Littman (1:41:55.320)
My life could be so much better.
Lex Fridman (1:41:57.020)
My life could also be so much worse.
Charles Isbell and Michael Littman (1:41:59.560)
You know, you just gotta feel that sometimes
Lex Fridman (1:42:01.600)
you have decisions you're gonna make.
Charles Isbell and Michael Littman (1:42:03.020)
You cannot know what's gonna do.
Lex Fridman (1:42:04.200)
You should think about it, right?
Charles Isbell and Michael Littman (1:42:05.600)
Some things are clearly smarter than other things.
Lex Fridman (1:42:07.520)
You gotta play the odds a little bit.
Lex Fridman (1:42:09.560)
But in the end, if you've got multiple choices,
Lex Fridman (1:42:11.600)
there are lots of things you think you might love.
Charles Isbell and Michael Littman (1:42:12.840)
Go with the thing that you actually love,
Lex Fridman (1:42:14.420)
the thing that jumps out at you
Lex Fridman (1:42:15.920)
and sort of pursue it for a little while.
Lex Fridman (1:42:17.280)
The worst thing that'll happen is you took a left turn
Charles Isbell and Michael Littman (1:42:18.960)
instead of a right turn and you ended up merely happy.
Lex Fridman (1:42:22.640)
Beautiful.
Charles Isbell and Michael Littman (1:42:23.580)
So, so accepting, so taking the step
Lex Fridman (1:42:26.620)
and just accepting, accepting that,
Charles Isbell and Michael Littman (1:42:28.480)
that don't like question, question the choice.
Lex Fridman (1:42:31.200)
Life is long and there's time to actually pursue.
Charles Isbell and Michael Littman (1:42:36.600)
Every once in a while, you have to put on a leather suit
Lex Fridman (1:42:41.160)
and make a thriller video.
Charles Isbell and Michael Littman (1:42:43.040)
Every once in a while.
Lex Fridman (1:42:44.280)
If I ever get the chance again, I'm doing it.
Charles Isbell and Michael Littman (1:42:47.360)
Yeah.
Lex Fridman (1:42:49.080)
I was told that you actually dance,
Lex Fridman (1:42:50.960)
but that part was edited out.
Lex Fridman (1:42:53.800)
I don't dance.
Charles Isbell and Michael Littman (1:42:55.760)
There was a thing where we did do the zombie thing.
Lex Fridman (1:42:59.160)
We did do the zombie thing.
Charles Isbell and Michael Littman (1:43:00.640)
That wasn't edited out.
Lex Fridman (1:43:01.920)
It just wasn't put into the final thing.
Charles Isbell and Michael Littman (1:43:05.540)
I'm quite happy.
Lex Fridman (1:43:06.380)
There was a reason for that too, right?
Charles Isbell and Michael Littman (1:43:07.520)
Like I wasn't wearing something right.
Lex Fridman (1:43:09.480)
There was a reason for that.
Charles Isbell and Michael Littman (1:43:10.320)
I can't remember what it was.
Lex Fridman (1:43:11.160)
No leather suit.
Lex Fridman (1:43:12.360)
Is that what it was?
Lex Fridman (1:43:13.200)
I can't remember.
Charles Isbell and Michael Littman (1:43:14.020)
Anyway, the right thing happened.
Lex Fridman (1:43:16.040)
Exactly.
Charles Isbell and Michael Littman (1:43:16.880)
You took the left turn and ended up being the right thing.
Lex Fridman (1:43:19.600)
So a lot of people ask me that are a little bit
Charles Isbell and Michael Littman (1:43:23.120)
tangential to the programming and the computing world
Lex Fridman (1:43:26.480)
and they're interested to learn programming,
Charles Isbell and Michael Littman (1:43:28.280)
like all kinds of disciplines that are outside
Lex Fridman (1:43:30.360)
of the particular discipline of computer science.
Lex Fridman (1:43:33.240)
What advice do you have for people
Lex Fridman (1:43:36.080)
that want to learn how to program
Charles Isbell and Michael Littman (1:43:38.080)
or want to either taste this little skill set
Lex Fridman (1:43:43.600)
or discipline or try to see if it can be used somehow
Lex Fridman (1:43:47.240)
in their own life?
Lex Fridman (1:43:48.880)
What stage of life are they in?
Charles Isbell and Michael Littman (1:43:53.040)
One of the magic things about the internet
Lex Fridman (1:43:55.040)
of the people that write me is I don't know.
Charles Isbell and Michael Littman (1:43:58.160)
Because my answer's different for, my daughter
Lex Fridman (1:44:00.520)
is taking AP computer science right now.
Charles Isbell and Michael Littman (1:44:02.480)
Hi, Joni.
Lex Fridman (1:44:03.560)
She's amazing and doing amazing things
Lex Fridman (1:44:06.340)
and my son's beginning to get interested
Lex Fridman (1:44:08.040)
and I'll be really curious where he takes it.
Charles Isbell and Michael Littman (1:44:10.280)
I think his mind actually works very well
Lex Fridman (1:44:12.400)
for this sort of thing and she's doing great.
Lex Fridman (1:44:14.680)
But one of the things I have to tell her all the time,
Lex Fridman (1:44:17.280)
she points, well, I want to make a rhythm game.
Lex Fridman (1:44:19.880)
So I want to go for two weeks and then build a rhythm game.
Lex Fridman (1:44:23.320)
Show me how to build a rhythm game.
Charles Isbell and Michael Littman (1:44:25.240)
Start small, learn the building blocks
Lex Fridman (1:44:27.800)
and how to take the time.
Charles Isbell and Michael Littman (1:44:28.920)
Have patience, eventually you'll build a rhythm game.
Lex Fridman (1:44:31.100)
I was in grad school when I suddenly woke up one day
Charles Isbell and Michael Littman (1:44:34.100)
over the Royal East and I thought, wait a minute,
Lex Fridman (1:44:37.200)
I'm a computer scientist.
Charles Isbell and Michael Littman (1:44:38.040)
I should be able to write Pac Man in an afternoon.
Lex Fridman (1:44:39.740)
And I did, not with great graphics.
Charles Isbell and Michael Littman (1:44:42.040)
It was actually a very cool game.
Lex Fridman (1:44:43.120)
I had to figure out how the ghost moved and everything
Lex Fridman (1:44:45.120)
and I did it in an afternoon in Pascal
Lex Fridman (1:44:47.320)
on an old Apple 2GS.
Lex Fridman (1:44:49.920)
But if I had started out trying to build Pac Man,
Lex Fridman (1:44:52.440)
I think it probably would have ended very poorly for me.
Charles Isbell and Michael Littman (1:44:55.040)
Luckily back then, there weren't
Lex Fridman (1:44:57.160)
these magical devices we call phones
Lex Fridman (1:44:58.920)
and software everywhere to give me this illusion
Lex Fridman (1:45:01.160)
that I could create something by myself
Charles Isbell and Michael Littman (1:45:03.480)
from the basics inside of a weekend like that.
Lex Fridman (1:45:05.760)
I mean, that was a culmination of years and years and years
Charles Isbell and Michael Littman (1:45:09.440)
right before I decided, oh, I should be able to write this
Lex Fridman (1:45:11.200)
and I could.
Lex Fridman (1:45:12.140)
So my advice if you're early on is you've got the internet.
Lex Fridman (1:45:16.920)
There are lots of people there to give you the information.
Charles Isbell and Michael Littman (1:45:18.840)
Find someone who cares about this.
Lex Fridman (1:45:20.540)
Remember, they've been doing it for a very long time.
Charles Isbell and Michael Littman (1:45:22.640)
Take it slow, learn the little pieces, get excited about it
Lex Fridman (1:45:25.660)
and then keep the big project you want to build in mind.
Charles Isbell and Michael Littman (1:45:28.520)
You'll get there soon enough.
Lex Fridman (1:45:29.640)
Because as a wise man once said, life is long.
Charles Isbell and Michael Littman (1:45:32.820)
Sometimes it doesn't seem that long, but it is long
Lex Fridman (1:45:35.640)
and you'll have enough time to build it all out.
Charles Isbell and Michael Littman (1:45:39.120)
All the information is out there, but start small.
Lex Fridman (1:45:43.280)
Generate Fibonacci numbers.
Charles Isbell and Michael Littman (1:45:44.720)
That's not exciting, but it'll get you the language.
Lex Fridman (1:45:48.680)
Well, there's only one programming language, it's Lisp.
Lex Fridman (1:45:50.960)
But if you have to pick a programming language,
Lex Fridman (1:45:53.560)
I guess in today's day, what would I do?
Charles Isbell and Michael Littman (1:45:55.480)
I guess I'd do.
Lex Fridman (1:45:56.800)
Python is basically Lisp, but with better syntax.
Charles Isbell and Michael Littman (1:46:00.600)
Blasphemy.
Lex Fridman (1:46:01.440)
Yeah, with C syntax, how about that?
Lex Fridman (1:46:03.800)
So you're gonna argue that C syntax
Lex Fridman (1:46:05.280)
is better than anything?
Charles Isbell and Michael Littman (1:46:07.080)
Anyway, also I'm gonna answer Python despite what he said.
Lex Fridman (1:46:10.200)
Tell your story about somebody's dissertation
Charles Isbell and Michael Littman (1:46:12.560)
that had a Lisp program in it.
Lex Fridman (1:46:14.640)
It was so funny.
Charles Isbell and Michael Littman (1:46:15.480)
This is Dave's, Dave's dissertation was like,
Lex Fridman (1:46:17.920)
Dave McAllister, who was a professor at MIT for a while
Lex Fridman (1:46:20.120)
and then he came to Bell Labs and now he's at
Lex Fridman (1:46:24.120)
Technology Technical Institute of Chicago.
Charles Isbell and Michael Littman (1:46:26.360)
A brilliant guy.
Lex Fridman (1:46:27.920)
Such an interesting guy.
Charles Isbell and Michael Littman (1:46:28.880)
Anyway, his thesis, it was a theorem prover
Lex Fridman (1:46:33.920)
and he decided to have as an appendix his actual code,
Charles Isbell and Michael Littman (1:46:38.400)
which of course was all written in Lisp
Lex Fridman (1:46:39.640)
because of course it was.
Lex Fridman (1:46:40.920)
And like the last 20 pages are just right parenthesis.
Lex Fridman (1:46:43.680)
It's just wonderful.
Charles Isbell and Michael Littman (1:46:47.120)
That's programming right there.
Lex Fridman (1:46:48.720)
Pages upon pages of right parenthesis.
Charles Isbell and Michael Littman (1:46:51.000)
Anyway, Lisp is the only real language,
Lex Fridman (1:46:52.860)
but I understand that that's not necessarily
Charles Isbell and Michael Littman (1:46:54.400)
the place where you start.
Lex Fridman (1:46:56.000)
Python is just fine.
Charles Isbell and Michael Littman (1:46:57.720)
Python is good.
Lex Fridman (1:46:59.000)
If you're, you know, of a certain age,
Charles Isbell and Michael Littman (1:47:00.680)
if you're really young and trying to figure it out,
Lex Fridman (1:47:02.260)
graphical languages that let you kind of see
Lex Fridman (1:47:04.000)
how the thing works and that's fine too.
Lex Fridman (1:47:05.920)
They're all fine.
Charles Isbell and Michael Littman (1:47:06.760)
It almost doesn't matter.
Lex Fridman (1:47:07.580)
But there are people who spend a lot of time
Charles Isbell and Michael Littman (1:47:09.000)
thinking about how to build languages that get people in.
Lex Fridman (1:47:13.040)
The question is, are you trying to get in
Lex Fridman (1:47:14.840)
and figure out what it is?
Lex Fridman (1:47:15.880)
Or do you already know what you want?
Lex Fridman (1:47:18.160)
And that's why I asked you what stage of life people are in
Lex Fridman (1:47:19.920)
because if you're different stages of life,
Charles Isbell and Michael Littman (1:47:21.200)
you would attack it differently.
Lex Fridman (1:47:23.360)
The answer to that question of which language
Charles Isbell and Michael Littman (1:47:25.360)
keeps changing, I mean, there's some value
Lex Fridman (1:47:27.960)
to exploring, a lot of people write to me about Julia.
Charles Isbell and Michael Littman (1:47:33.080)
There's these like more modern languages
Lex Fridman (1:47:35.240)
that keep being invented, Rust and Kotlin.
Charles Isbell and Michael Littman (1:47:39.080)
There's stuff that, for people who love
Lex Fridman (1:47:42.280)
functional languages like Lisp,
Charles Isbell and Michael Littman (1:47:44.880)
that apparently there's echoes of that,
Lex Fridman (1:47:46.920)
but much better in the modern languages.
Lex Fridman (1:47:49.400)
And it's worthwhile to,
Lex Fridman (1:47:51.720)
especially when you're learning languages,
Charles Isbell and Michael Littman (1:47:53.360)
it feels like it's okay to try one
Lex Fridman (1:47:55.320)
that's not like the popular one.
Charles Isbell and Michael Littman (1:47:57.840)
Oh yeah, but you want something simple.
Lex Fridman (1:47:59.160)
And I think you get that way of thinking
Charles Isbell and Michael Littman (1:48:02.180)
almost no matter what language.
Lex Fridman (1:48:04.500)
And if you push far enough,
Charles Isbell and Michael Littman (1:48:06.520)
like it can be assembly language,
Lex Fridman (1:48:08.040)
but you need to push pretty far
Charles Isbell and Michael Littman (1:48:09.560)
before you start to hit the really deep concepts
Lex Fridman (1:48:11.520)
that you would get sooner in other languages.
Lex Fridman (1:48:13.400)
But like, I don't know, computation is kind of computation,
Lex Fridman (1:48:16.600)
is kind of Turing equivalent, is kind of computation.
Lex Fridman (1:48:19.520)
And so it matters how you express things,
Lex Fridman (1:48:22.220)
but you have to build out that mental structure
Charles Isbell and Michael Littman (1:48:24.440)
in your mind.
Lex Fridman (1:48:25.440)
And I don't think it's super matters which language.
Charles Isbell and Michael Littman (1:48:28.800)
I mean, it matters a little,
Lex Fridman (1:48:29.940)
because some things are just
Charles Isbell and Michael Littman (1:48:31.000)
at the wrong level of abstraction.
Lex Fridman (1:48:32.240)
I think assembly is at the wrong level of abstraction
Charles Isbell and Michael Littman (1:48:33.840)
for someone coming in new.
Lex Fridman (1:48:35.920)
I think that if you start.
Charles Isbell and Michael Littman (1:48:37.360)
For someone coming in new.
Lex Fridman (1:48:38.400)
Yes, for frameworks, big frameworks are quite a bit.
Charles Isbell and Michael Littman (1:48:42.640)
You know, you've got to get to the point
Lex Fridman (1:48:43.640)
where I want to learn a new language,
Charles Isbell and Michael Littman (1:48:44.940)
means I just pick up a reference book
Lex Fridman (1:48:46.180)
and I think of a project and I go through it in a weekend.
Charles Isbell and Michael Littman (1:48:49.240)
Right, you got to get there.
Lex Fridman (1:48:50.320)
You're right though, the languages that are designed
Charles Isbell and Michael Littman (1:48:52.400)
for that are, it almost doesn't matter.
Lex Fridman (1:48:54.920)
Pick the ones that people have built tutorials
Lex Fridman (1:48:57.800)
and infrastructure around to help you get kind of,
Lex Fridman (1:48:59.760)
kind of ease into it.
Charles Isbell and Michael Littman (1:49:00.840)
Because it's hard.
Lex Fridman (1:49:01.680)
I mean, I did this little experiment once.
Charles Isbell and Michael Littman (1:49:05.300)
I was teaching intro to CS in the summer as a favor.
Lex Fridman (1:49:11.080)
Which is, anyway.
Charles Isbell and Michael Littman (1:49:11.920)
I was teaching.
Lex Fridman (1:49:12.760)
I was teaching intro to CS as a favor.
Lex Fridman (1:49:15.120)
And it was very funny because I'd go in every single time
Lex Fridman (1:49:17.000)
and I would think to myself,
Lex Fridman (1:49:18.800)
how am I possibly going to fill up an hour and a half
Lex Fridman (1:49:21.560)
talking about for loops, right?
Lex Fridman (1:49:23.120)
And there wasn't enough time.
Lex Fridman (1:49:25.240)
Took me a while to realize this, right?
Lex Fridman (1:49:26.660)
There are only three things, right?
Lex Fridman (1:49:27.920)
There's reading from a variable,
Charles Isbell and Michael Littman (1:49:29.120)
writing to a variable and conditional branching.
Lex Fridman (1:49:31.580)
Everything else is syntactic sugar, right?
Charles Isbell and Michael Littman (1:49:34.100)
The syntactic sugar matters, but that's it.
Lex Fridman (1:49:36.080)
And when I say that's it, I don't mean it's simple.
Charles Isbell and Michael Littman (1:49:38.960)
I mean, it's hard.
Lex Fridman (1:49:40.120)
Like conditional branching, loops, variable.
Charles Isbell and Michael Littman (1:49:43.480)
Those are really hard concepts.
Lex Fridman (1:49:45.200)
So you shouldn't be discouraged by this.
Charles Isbell and Michael Littman (1:49:47.240)
Here's a simple experiment.
Lex Fridman (1:49:48.080)
I'm gonna ask you a question now.
Lex Fridman (1:49:49.280)
You ready?
Lex Fridman (1:49:50.720)
X equals three.
Charles Isbell and Michael Littman (1:49:51.680)
Okay.
Lex Fridman (1:49:53.440)
Y equals four.
Charles Isbell and Michael Littman (1:49:54.960)
Okay.
Lex Fridman (1:49:55.800)
What is X?
Charles Isbell and Michael Littman (1:49:57.080)
Three.
Lex Fridman (1:49:57.920)
What is Y?
Charles Isbell and Michael Littman (1:49:59.040)
Four.
Lex Fridman (1:49:59.880)
Y equals X.
Charles Isbell and Michael Littman (1:50:00.700)
I'm gonna mess this up.
Lex Fridman (1:50:01.540)
No, it's easy.
Charles Isbell and Michael Littman (1:50:02.620)
Y equals X.
Lex Fridman (1:50:04.020)
Y equals X.
Lex Fridman (1:50:04.860)
What is Y?
Lex Fridman (1:50:07.880)
Three.
Charles Isbell and Michael Littman (1:50:08.720)
That's right.
Lex Fridman (1:50:09.560)
X equals seven.
Lex Fridman (1:50:11.360)
What is Y?
Lex Fridman (1:50:12.520)
That's one of the trickiest things to get for programmers,
Charles Isbell and Michael Littman (1:50:15.540)
that there's a memory and the variables are pointing
Lex Fridman (1:50:19.240)
to a particular thing in memory,
Lex Fridman (1:50:21.040)
and sometimes the languages hide that from you
Lex Fridman (1:50:23.240)
and they bring it closer
Charles Isbell and Michael Littman (1:50:24.360)
to the way you think mathematics works.
Lex Fridman (1:50:26.640)
Right, so in fact, Mark Guzdal,
Charles Isbell and Michael Littman (1:50:28.480)
who worries about these sorts of things,
Lex Fridman (1:50:30.040)
or used to worry about these sorts of things anyway,
Charles Isbell and Michael Littman (1:50:32.440)
had this kind of belief that actually,
Lex Fridman (1:50:35.560)
people when they see these statements,
Charles Isbell and Michael Littman (1:50:36.960)
X equals something, Y equals something, Y equals X,
Lex Fridman (1:50:39.560)
that you have now made a mathematical statement
Charles Isbell and Michael Littman (1:50:42.900)
that Y and X are the same.
Lex Fridman (1:50:45.580)
Which you can if you just put like an anchor in front of it.
Lex Fridman (1:50:48.080)
Yes, but people, that's not what you're doing, right?
Lex Fridman (1:50:51.400)
I thought, and I kind of asked the question,
Lex Fridman (1:50:54.040)
and I think I had some evidence for this,
Lex Fridman (1:50:55.480)
it's hardly a study,
Charles Isbell and Michael Littman (1:50:56.600)
is that most of the people who didn't know the answer,
Lex Fridman (1:50:59.240)
weren't sure about the answer, they had used spreadsheets.
Charles Isbell and Michael Littman (1:51:02.240)
Ah, interesting.
Lex Fridman (1:51:03.480)
And so it's, you know,
Lex Fridman (1:51:06.380)
it's by reference, or by name really, right?
Lex Fridman (1:51:10.920)
And so depending upon what you think they are,
Charles Isbell and Michael Littman (1:51:13.280)
you get completely different answers.
Lex Fridman (1:51:14.720)
The fact that I could go, or one could go,
Charles Isbell and Michael Littman (1:51:17.860)
two thirds of the way through a semester,
Lex Fridman (1:51:20.120)
and people still hadn't figured out in their heads,
Charles Isbell and Michael Littman (1:51:22.560)
when you say Y equals X, what that meant,
Lex Fridman (1:51:25.080)
tells you it's actually hard.
Charles Isbell and Michael Littman (1:51:27.100)
Because all those answers are possible,
Lex Fridman (1:51:29.080)
and in fact, when you said,
Charles Isbell and Michael Littman (1:51:30.100)
oh, if you just put an ampersand in front of it,
Lex Fridman (1:51:31.800)
I mean, that doesn't make any sense for an intro class,
Lex Fridman (1:51:33.680)
and of course a lot of languages
Lex Fridman (1:51:34.720)
don't even give you the ability
Charles Isbell and Michael Littman (1:51:35.720)
to think about it in terms of ampersand.
Lex Fridman (1:51:37.320)
Do we want to have a 45 minute discussion
Lex Fridman (1:51:38.920)
about the difference between equal EQ and equal in Lisp?
Lex Fridman (1:51:42.360)
Yeah.
Charles Isbell and Michael Littman (1:51:43.200)
I know you do.
Lex Fridman (1:51:44.020)
No.
Lex Fridman (1:51:44.860)
But you know, you could do that.
Lex Fridman (1:51:47.440)
This is actually really hard stuff.
Lex Fridman (1:51:49.160)
So you shouldn't be, it's not too hard, we all do it,
Lex Fridman (1:51:52.360)
but you shouldn't be discouraged.
Charles Isbell and Michael Littman (1:51:53.920)
It's why you should start small,
Lex Fridman (1:51:55.320)
so that you can figure out these things,
Lex Fridman (1:51:56.600)
so you have the right model in your head,
Lex Fridman (1:51:58.380)
so that when you write the language,
Charles Isbell and Michael Littman (1:51:59.960)
you can execute it, and build the machine
Lex Fridman (1:52:02.020)
that you want to build, right?
Charles Isbell and Michael Littman (1:52:03.280)
Yeah, the funny thing about programming,
Lex Fridman (1:52:05.200)
and those very basic things,
Charles Isbell and Michael Littman (1:52:06.960)
is the very basics are not often made explicit,
Lex Fridman (1:52:11.160)
which is actually what drives everybody away
Charles Isbell and Michael Littman (1:52:13.880)
from basically any discipline,
Lex Fridman (1:52:15.320)
but programming is just another one.
Charles Isbell and Michael Littman (1:52:17.040)
Like even a simpler version of the equal sign
Lex Fridman (1:52:19.360)
that I kind of forget, is in mathematics,
Charles Isbell and Michael Littman (1:52:23.740)
equals is not assignment.
Lex Fridman (1:52:25.480)
Yeah.
Charles Isbell and Michael Littman (1:52:26.520)
Like, I think basically every single programming language
Lex Fridman (1:52:30.260)
with just a few handful of exceptions,
Charles Isbell and Michael Littman (1:52:33.080)
equals is assignment.
Lex Fridman (1:52:35.160)
And you have some other operator for equality.
Lex Fridman (1:52:38.840)
And even that, like everyone kind of knows it,
Lex Fridman (1:52:42.640)
once you started doing it,
Lex Fridman (1:52:45.020)
but like you need to say that explicitly,
Lex Fridman (1:52:47.120)
or you just realize it, like yourself.
Charles Isbell and Michael Littman (1:52:51.000)
Otherwise you might be stuck for,
Lex Fridman (1:52:53.240)
you said like half a semester,
Charles Isbell and Michael Littman (1:52:54.680)
you could be stuck for quite a long time.
Lex Fridman (1:52:57.160)
And I think also part of the programming
Charles Isbell and Michael Littman (1:53:00.320)
is being okay in that state of confusion for a while.
Lex Fridman (1:53:04.520)
It's to the debugging point.
Charles Isbell and Michael Littman (1:53:06.720)
It's like, I just wrote two lines of code,
Lex Fridman (1:53:09.800)
why doesn't this work?
Lex Fridman (1:53:10.980)
And staring at that for like hours,
Lex Fridman (1:53:14.640)
and trying to figure out.
Lex Fridman (1:53:15.800)
And then every once in a while,
Lex Fridman (1:53:16.960)
you just have to restart your computer
Lex Fridman (1:53:18.480)
and everything works again.
Lex Fridman (1:53:19.540)
And then you just kind of stare into the void
Charles Isbell and Michael Littman (1:53:24.200)
with the tear slowly rolling down your eye.
Lex Fridman (1:53:26.600)
By the way, the fact that they didn't get this
Charles Isbell and Michael Littman (1:53:28.200)
actually had no impact on,
Lex Fridman (1:53:30.000)
I mean, they were still able to do their assignments.
Charles Isbell and Michael Littman (1:53:32.360)
Because it turns out their misunderstanding
Lex Fridman (1:53:35.080)
wasn't being revealed to them
Charles Isbell and Michael Littman (1:53:37.600)
by the problem sets we were giving them.
Lex Fridman (1:53:39.840)
It's pretty profound actually, yeah.
Charles Isbell and Michael Littman (1:53:41.360)
I wrote a program a long time ago,
Lex Fridman (1:53:44.400)
actually for my master's thesis,
Lex Fridman (1:53:46.360)
and in C++ I think, or C, I guess it was C.
Lex Fridman (1:53:49.400)
And it was all memory management and terrible.
Lex Fridman (1:53:52.720)
And it wouldn't work for a while.
Lex Fridman (1:53:56.320)
And it was some kind of,
Charles Isbell and Michael Littman (1:53:57.480)
it was clear to me that it was overriding memory.
Lex Fridman (1:53:59.680)
And I just couldn't, I was like,
Charles Isbell and Michael Littman (1:54:01.400)
look, I got to pay for this time for this.
Lex Fridman (1:54:03.360)
So I basically declared a variable
Charles Isbell and Michael Littman (1:54:06.560)
at the front in the main that was like 400K,
Lex Fridman (1:54:10.160)
just an array, and it worked.
Charles Isbell and Michael Littman (1:54:12.480)
Because wherever I was scribbling over memory,
Lex Fridman (1:54:14.340)
it would scribble into that space and it didn't matter.
Lex Fridman (1:54:17.120)
And so I never figured out what the bug was.
Lex Fridman (1:54:19.640)
But I did create something to sort of deal with it.
Charles Isbell and Michael Littman (1:54:21.840)
To work around it.
Lex Fridman (1:54:22.720)
And it, you know, that's crazy, that's crazy.
Charles Isbell and Michael Littman (1:54:25.640)
It was okay, because that's what I wanted.
Lex Fridman (1:54:27.240)
But I knew enough about memory managed to go,
Charles Isbell and Michael Littman (1:54:29.320)
you know, management to go, you know,
Lex Fridman (1:54:30.900)
I'm just going to create an empty array here
Lex Fridman (1:54:32.240)
and hope that that deals with this scribbling memory problem.
Lex Fridman (1:54:34.760)
And it did.
Charles Isbell and Michael Littman (1:54:35.600)
That takes a long time to figure out.
Lex Fridman (1:54:36.960)
And by the way, the language you first learned
Charles Isbell and Michael Littman (1:54:38.520)
probably just garbage collection anyway,
Lex Fridman (1:54:39.800)
so you're not even going to come up across,
Charles Isbell and Michael Littman (1:54:41.080)
you're not going to come across that problem.
Lex Fridman (1:54:43.500)
So we talked about the Minsky idea
Charles Isbell and Michael Littman (1:54:46.080)
of hating everything you do and hating yourself.
Lex Fridman (1:54:49.580)
So let's end on a question
Charles Isbell and Michael Littman (1:54:52.600)
that's going to make both of you very uncomfortable.
Lex Fridman (1:54:54.760)
Okay.
Charles Isbell and Michael Littman (1:54:55.600)
Which is, what is your, Charles,
Lex Fridman (1:54:58.000)
what's your favorite thing that you're grateful for
Lex Fridman (1:55:01.880)
about Michael?
Lex Fridman (1:55:04.280)
And Michael, what is your favorite thing
Lex Fridman (1:55:06.680)
that you're grateful for about Charles?
Lex Fridman (1:55:09.640)
Well, that answer is actually quite easy.
Charles Isbell and Michael Littman (1:55:12.120)
His friendship.
Lex Fridman (1:55:14.480)
He stole the easy answer.
Charles Isbell and Michael Littman (1:55:15.320)
I did.
Lex Fridman (1:55:16.160)
Yeah, I can tell you what I hate about Charles,
Charles Isbell and Michael Littman (1:55:17.400)
he steals my good answers.
Lex Fridman (1:55:19.440)
The thing I like most about Charles,
Charles Isbell and Michael Littman (1:55:21.120)
he sees the world in a similar enough,
Lex Fridman (1:55:24.080)
but different way that I,
Charles Isbell and Michael Littman (1:55:25.720)
it's sort of like having another life.
Lex Fridman (1:55:28.880)
It's sort of like I get to experience things
Charles Isbell and Michael Littman (1:55:31.400)
that I wouldn't otherwise get to experience
Lex Fridman (1:55:32.920)
because I would not naturally gravitate to them that way.
Lex Fridman (1:55:36.040)
And so he just, he just shows me a whole other world.
Lex Fridman (1:55:39.080)
It's awesome.
Charles Isbell and Michael Littman (1:55:39.920)
Yeah, the inner product is not zero for sure.
Lex Fridman (1:55:44.080)
It's not quite one, 0.7 maybe.
Charles Isbell and Michael Littman (1:55:47.540)
Just enough that you can learn.
Lex Fridman (1:55:50.720)
Just enough that you can learn.
Charles Isbell and Michael Littman (1:55:53.060)
That's the definition of friendship.
Lex Fridman (1:55:54.320)
The inner product is 0.7.
Charles Isbell and Michael Littman (1:55:55.720)
Yeah, I think so.
Lex Fridman (1:55:56.680)
That's the answer to life really.
Charles Isbell and Michael Littman (1:55:58.120)
Charles sometimes believes in me
Lex Fridman (1:55:59.360)
when I have not believed in me.
Charles Isbell and Michael Littman (1:56:01.360)
He also sometimes works as an outward confidence
Lex Fridman (1:56:04.360)
that he has so much, so much confidence and self,
Charles Isbell and Michael Littman (1:56:08.760)
I don't know, comfortableness.
Lex Fridman (1:56:11.360)
Okay, let's go with that.
Charles Isbell and Michael Littman (1:56:13.000)
That I feel better a little bit.
Lex Fridman (1:56:16.080)
If he thinks I'm okay,
Charles Isbell and Michael Littman (1:56:17.680)
then maybe I'm not as bad as I think I am.
Lex Fridman (1:56:20.240)
At the end of the day, luck favors the Charles.
Charles Isbell and Michael Littman (1:56:24.620)
It's a huge honor to talk with you.
Lex Fridman (1:56:26.720)
Thank you so much for taking this time,
Charles Isbell and Michael Littman (1:56:29.320)
wasting your time with me.
Lex Fridman (1:56:30.740)
It was an awesome conversation.
Charles Isbell and Michael Littman (1:56:32.160)
You guys are an inspiration to a huge number of people
Lex Fridman (1:56:35.240)
and to me, so really enjoyed this.
Charles Isbell and Michael Littman (1:56:37.280)
Thanks for talking to me.
Lex Fridman (1:56:38.120)
I enjoyed it as well.
Charles Isbell and Michael Littman (1:56:38.940)
Thank you so much.
Lex Fridman (1:56:39.780)
And by the way, if luck favors the Charles,
Charles Isbell and Michael Littman (1:56:40.600)
then it's certainly the case
Lex Fridman (1:56:41.440)
that I've been very lucky to know you.
Charles Isbell and Michael Littman (1:56:43.840)
I'm gonna edit that part out.
Lex Fridman (1:56:47.840)
Thanks for listening to this conversation
Charles Isbell and Michael Littman (1:56:49.360)
with Charles Isbell and Michael Littman.
Lex Fridman (1:56:51.520)
And thank you to our sponsors,
Charles Isbell and Michael Littman (1:56:53.940)
Athletic Greens, Super Nutritional Drink,
Lex Fridman (1:56:57.200)
Eight Sleep, Self Cooling Mattress,
Charles Isbell and Michael Littman (1:57:00.360)
Masterclass Online Courses
Lex Fridman (1:57:02.980)
from some of the most amazing humans in history,
Lex Fridman (1:57:05.880)
and Cash App, the app I use to send money to friends.
Lex Fridman (1:57:09.880)
Please check out the sponsors in the description
Charles Isbell and Michael Littman (1:57:12.480)
to get a discount and to support this podcast.
Lex Fridman (1:57:16.160)
If you enjoy this thing, subscribe on YouTube,
Charles Isbell and Michael Littman (1:57:18.520)
review it with Five Stars Napa Podcast,
Lex Fridman (1:57:20.760)
follow on Spotify, support it on Patreon,
Charles Isbell and Michael Littman (1:57:23.520)
or connect with me on Twitter at Lex Friedman.
Lex Fridman (1:57:26.800)
And now, let me leave you with some words from Desmond Tutu.
Charles Isbell and Michael Littman (1:57:30.760)
Don't raise your voice, improve your argument.
Lex Fridman (1:57:34.160)
Thank you for listening and hope to see you next time.
Lex Fridman (20:01.140)
and empowerment and stuff.
Lex Fridman (20:02.260)
And it's not like we're gonna crush you
Lex Fridman (20:03.780)
and you're gonna love it.
Lex Fridman (20:04.980)
So yeah, I think there's a,
Charles Isbell and Michael Littman (20:06.300)
I think the ethoses are different.
Lex Fridman (20:09.300)
That's interesting, yeah.
Charles Isbell and Michael Littman (20:10.300)
We had Drown Proofing.
Lex Fridman (20:12.100)
What's that?
Charles Isbell and Michael Littman (20:12.940)
In order to graduate from Georgia Tech,
Lex Fridman (20:14.500)
this is a true thing.
Charles Isbell and Michael Littman (20:15.320)
Feel free to look it up.
Lex Fridman (20:16.940)
If you.
Charles Isbell and Michael Littman (20:17.780)
A lot of schools have this by the way.
Lex Fridman (20:19.220)
No, actually Georgia Tech was barely the first.
Charles Isbell and Michael Littman (20:20.900)
Brandeis has it.
Lex Fridman (20:21.940)
Had it.
Charles Isbell and Michael Littman (20:23.260)
I feel like Georgia Tech was the first
Lex Fridman (20:25.020)
in a lot of things.
Charles Isbell and Michael Littman (20:27.460)
It was the first in a lot of things.
Lex Fridman (20:28.860)
Had the first master's degree.
Charles Isbell and Michael Littman (20:29.700)
First Bumblebee mascot.
Lex Fridman (20:30.600)
Stop that.
Charles Isbell and Michael Littman (20:32.300)
First master's in computer science actually.
Lex Fridman (20:34.300)
Right, online master's.
Charles Isbell and Michael Littman (20:35.620)
Well that too, but way back in the 60s.
Lex Fridman (20:37.800)
NSF grant.
Charles Isbell and Michael Littman (20:38.640)
Yeah, yeah.
Lex Fridman (20:39.480)
You're the first information
Lex Fridman (20:40.300)
and computer science master's degree in the country.
Lex Fridman (20:42.820)
But the Georgia Tech,
Charles Isbell and Michael Littman (20:45.660)
it used to be the case
Lex Fridman (20:46.580)
that in order to graduate from Georgia Tech,
Charles Isbell and Michael Littman (20:48.300)
you had to take a Drown Proofing class.
Lex Fridman (20:49.980)
Where effectively, they threw you in the water
Lex Fridman (20:52.340)
and tied you up.
Lex Fridman (20:53.180)
If you didn't drown, you got to graduate.
Lex Fridman (20:54.620)
Tied you up?
Lex Fridman (20:55.780)
I believe so.
Charles Isbell and Michael Littman (20:56.620)
No.
Lex Fridman (20:57.440)
There were certainly versions of it,
Lex Fridman (20:58.500)
but I mean luckily they ended it
Lex Fridman (21:00.460)
just before I had to graduate
Charles Isbell and Michael Littman (21:01.620)
because otherwise I would have never graduated.
Lex Fridman (21:03.220)
It wasn't going to happen.
Charles Isbell and Michael Littman (21:04.300)
I want to say 84, 83,
Lex Fridman (21:06.460)
somewhere around then they ended it.
Lex Fridman (21:08.100)
But yeah, you used to have to prove
Lex Fridman (21:10.420)
you could tread water for some ridiculous amount of time
Charles Isbell and Michael Littman (21:13.220)
or you couldn't graduate.
Lex Fridman (21:14.060)
Two minutes.
Charles Isbell and Michael Littman (21:14.900)
No, it was more than two minutes.
Lex Fridman (21:15.720)
I bet it was two minutes.
Charles Isbell and Michael Littman (21:16.560)
Okay, well we'll look at it.
Lex Fridman (21:17.400)
And it was in a bathtub.
Charles Isbell and Michael Littman (21:18.340)
Yeah, right.
Lex Fridman (21:19.180)
You could just stare.
Charles Isbell and Michael Littman (21:20.020)
It was in a pool.
Lex Fridman (21:20.900)
But it was a real thing.
Lex Fridman (21:21.740)
But that idea that, you know, push you.
Lex Fridman (21:23.500)
Fully clothed.
Charles Isbell and Michael Littman (21:24.380)
Yeah, fully clothed.
Lex Fridman (21:25.300)
I bet it was that and not tied up.
Lex Fridman (21:27.460)
Because who needs to learn how to swim when you're tied?
Lex Fridman (21:30.420)
Nobody.
Lex Fridman (21:31.260)
But who needs to learn to swim
Lex Fridman (21:32.540)
when you're actually falling into the water dressed?
Charles Isbell and Michael Littman (21:34.220)
That's a real thing.
Lex Fridman (21:35.060)
I think your facts are getting in the way
Charles Isbell and Michael Littman (21:36.700)
with a good story.
Lex Fridman (21:37.540)
Oh, that's fair.
Charles Isbell and Michael Littman (21:38.360)
That's fair.
Lex Fridman (21:39.200)
I didn't mean to.
Charles Isbell and Michael Littman (21:40.040)
All right, so they tie you up.
Lex Fridman (21:40.860)
Sometimes the narrative matters.
Lex Fridman (21:41.980)
But whatever it was, you had to,
Lex Fridman (21:43.300)
it was called drown proofing for a reason.
Charles Isbell and Michael Littman (21:44.820)
The point of the story, Michael, is that it's,
Lex Fridman (21:49.500)
well, no, but that's good.
Charles Isbell and Michael Littman (21:50.700)
It doesn't bring it back to struggle.
Lex Fridman (21:52.340)
That's a part of what Georgia Tech has always been.
Lex Fridman (21:54.980)
And we struggle with that, by the way,
Lex Fridman (21:56.740)
about what we want to be, particularly as things go.
Lex Fridman (21:59.860)
But you sort of,
Lex Fridman (22:02.140)
how much can you be pushed without breaking?
Lex Fridman (22:06.660)
And you come out of the other end stronger, right?
Lex Fridman (22:08.940)
There's a saying we used to have
Charles Isbell and Michael Littman (22:09.780)
when I was an undergrad there.
Lex Fridman (22:10.620)
It was just Georgia Tech,
Charles Isbell and Michael Littman (22:11.660)
building tomorrow the night before.
Lex Fridman (22:13.700)
Right?
Lex Fridman (22:14.540)
And it was just kind of idea that,
Lex Fridman (22:17.780)
give me something impossible to do
Lex Fridman (22:19.460)
and I'll do it in a couple of days
Lex Fridman (22:20.900)
because that's what I just spent
Charles Isbell and Michael Littman (22:21.900)
the last four or five or six years with.
Lex Fridman (22:24.140)
That ethos definitely stuck to you.
Charles Isbell and Michael Littman (22:26.860)
Having now done a number of projects with you,
Lex Fridman (22:28.980)
you definitely will do it the night before.
Charles Isbell and Michael Littman (22:30.420)
That's not entirely true.
Lex Fridman (22:31.340)
There's nothing wrong with waiting until the last minute.
Charles Isbell and Michael Littman (22:33.620)
The secret is knowing when the last minute is.
Lex Fridman (22:35.660)
Right, that's brilliantly put.
Charles Isbell and Michael Littman (22:38.260)
Yeah, that is a definite Charles statement
Lex Fridman (22:41.700)
that I am trying not to embrace.
Lex Fridman (22:44.860)
And I appreciate that
Lex Fridman (22:45.700)
because you helped move my last minute up.
Charles Isbell and Michael Littman (22:47.700)
That's the social construct
Lex Fridman (22:49.300)
the way you converge together
Lex Fridman (22:50.780)
what the definition of last minute is.
Lex Fridman (22:53.140)
We figure that all out together.
Charles Isbell and Michael Littman (22:54.660)
In fact, MIT, I'm sure a lot of universities have this,
Lex Fridman (22:58.580)
but MIT has like MIT time
Charles Isbell and Michael Littman (23:00.380)
that everyone has always agreed together
Lex Fridman (23:03.740)
that there is such a concept
Lex Fridman (23:05.500)
and everyone just keeps showing up like 10 to 15 to 20,
Lex Fridman (23:08.740)
depending on the department, late to everything.
Lex Fridman (23:11.420)
So there's like a weird drift that happens.
Lex Fridman (23:13.940)
It's kind of fascinating.
Charles Isbell and Michael Littman (23:14.900)
Yeah, we're five minutes.
Lex Fridman (23:15.980)
We're five minutes.
Charles Isbell and Michael Littman (23:16.820)
In fact, the classes will say,
Lex Fridman (23:18.620)
well, this is no longer true actually,
Lex Fridman (23:20.380)
but it used to be a class that started at eight,
Lex Fridman (23:22.540)
but actually it started at eight oh five,
Charles Isbell and Michael Littman (23:24.180)
it ends at nine, actually it ends at eight fifty five.
Lex Fridman (23:26.460)
Everything's five minutes off
Lex Fridman (23:27.460)
and nobody expects anything to start until five minutes
Lex Fridman (23:29.420)
after the half hour, whatever it is.
Charles Isbell and Michael Littman (23:31.580)
It still exists.
Lex Fridman (23:32.420)
It hurts my head.
Charles Isbell and Michael Littman (23:33.260)
Well, let's rewind the clock back to the fifties and sixties
Lex Fridman (23:37.860)
when you guys met, how did you,
Charles Isbell and Michael Littman (23:39.580)
I'm just kidding, I don't know.
Lex Fridman (23:40.820)
But what, can you tell the story of how you met?
Lex Fridman (23:43.100)
So you've, like the internet and the world
Lex Fridman (23:45.420)
kind of knows you as connected in some ways
Charles Isbell and Michael Littman (23:50.180)
in terms of education of teaching the world.
Lex Fridman (23:53.100)
That's like the public facing thing,
Lex Fridman (23:54.740)
but how did you as human beings
Lex Fridman (23:56.740)
and as collaborators meet?
Charles Isbell and Michael Littman (24:00.700)
I think there's two stories.
Lex Fridman (24:01.780)
One is how we met and the other is how we
Charles Isbell and Michael Littman (24:05.260)
got to know each other.
Lex Fridman (24:06.220)
I'm not gonna say fell in love.
Charles Isbell and Michael Littman (24:08.260)
I'm gonna say that we came to understand that we
Lex Fridman (24:11.180)
Had some common something.
Charles Isbell and Michael Littman (24:13.660)
Yeah, it's funny.
Lex Fridman (24:14.620)
Cause on the surface, I think we're different
Charles Isbell and Michael Littman (24:16.660)
in a lot of ways, but there's something
Lex Fridman (24:18.820)
Yeah, I mean, now we complete each other's
Charles Isbell and Michael Littman (24:21.740)
There you go.
Lex Fridman (24:22.580)
Afternoon.
Lex Fridman (24:23.580)
So I will tell the story of how we met
Lex Fridman (24:25.980)
and I'll let Michael tell the story of how we met.
Charles Isbell and Michael Littman (24:27.940)
Okay, all right.
Lex Fridman (24:28.780)
Okay, so here's how we met.
Charles Isbell and Michael Littman (24:30.140)
I was already at that point, it was AT&T labs.
Lex Fridman (24:32.980)
There's a long, interesting story there.
Lex Fridman (24:34.180)
But anyway, I was there and Michael was coming to interview.
Lex Fridman (24:38.780)
He was a professor at Duke at the time,
Lex Fridman (24:40.300)
but decided for reasons that he wanted to be in New Jersey.
Lex Fridman (24:45.140)
And so that would mean Bell Labs slash AT&T labs.
Lex Fridman (24:48.860)
And we were doing the interview.
Lex Fridman (24:49.700)
Interviews are very much like academic interviews.
Lex Fridman (24:51.540)
And so I had to be there.
Lex Fridman (24:53.220)
We all had to meet with him afterwards
Lex Fridman (24:54.860)
and so on, one on one.
Lex Fridman (24:56.180)
But it was obvious to me that he was going to be hired.
Charles Isbell and Michael Littman (24:59.460)
Like no matter what, because everyone loved him.
Lex Fridman (25:01.100)
They were just talking about all the great stuff he did.
Charles Isbell and Michael Littman (25:03.300)
Oh, he did this great thing.
Lex Fridman (25:04.300)
And you had just won something at AAAI, I think.
Charles Isbell and Michael Littman (25:06.500)
Or maybe you got 18 papers in AAAI that year.
Lex Fridman (25:08.580)
I got the best paper award at AAAI
Charles Isbell and Michael Littman (25:10.260)
for the crossword stuff.
Lex Fridman (25:11.380)
Right, exactly.
Lex Fridman (25:12.220)
So that had all happened and everyone was going on
Lex Fridman (25:14.020)
and on and on about it.
Charles Isbell and Michael Littman (25:14.860)
Actually, so Tinder was saying incredibly nice things
Lex Fridman (25:16.600)
about you.
Lex Fridman (25:17.440)
Really?
Lex Fridman (25:18.260)
Yes.
Charles Isbell and Michael Littman (25:19.100)
He can be very grumpy.
Lex Fridman (25:19.940)
Yes.
Charles Isbell and Michael Littman (25:20.760)
That's nice to hear.
Lex Fridman (25:21.600)
He was grumpily saying very nice things.
Charles Isbell and Michael Littman (25:22.500)
Oh, that makes sense.
Lex Fridman (25:23.340)
And that does make sense.
Charles Isbell and Michael Littman (25:24.180)
So, you know, it was going to come.
Lex Fridman (25:25.940)
So why was I meeting him?
Charles Isbell and Michael Littman (25:28.060)
I had something else I had to do.
Lex Fridman (25:29.060)
I can't remember what it was.
Charles Isbell and Michael Littman (25:29.900)
It probably involved comedy.
Lex Fridman (25:31.300)
So he remembers meeting me
Charles Isbell and Michael Littman (25:32.340)
as inconveniencing his afternoon.
Lex Fridman (25:34.180)
So he came.
Lex Fridman (25:35.020)
So I eventually came to my office.
Lex Fridman (25:36.140)
I was in the middle of trying to do something.
Charles Isbell and Michael Littman (25:37.080)
I can't remember what.
Lex Fridman (25:37.920)
And he came and he sat down.
Lex Fridman (25:38.980)
And for reasons that are purely accidental,
Lex Fridman (25:41.160)
despite what Michael thinks,
Charles Isbell and Michael Littman (25:42.560)
my desk at the time was set up
Lex Fridman (25:44.740)
in such a way that it had sort of an L shape.
Lex Fridman (25:46.660)
And the chair on the outside was always lower
Lex Fridman (25:48.740)
than the chair that I was in.
Charles Isbell and Michael Littman (25:50.380)
And, you know, the kind of point was to...
Lex Fridman (25:52.620)
The only reason I think that it was on purpose
Charles Isbell and Michael Littman (25:54.440)
is because you told me it was on purpose.
Lex Fridman (25:56.220)
I don't remember that.
Charles Isbell and Michael Littman (25:57.060)
Anyway, the thing is, is that, you know, it kind of gives...
Lex Fridman (25:59.020)
His guest chair was really low
Lex Fridman (26:00.260)
so that he could look down at everybody.
Lex Fridman (26:02.940)
The idea was just to simply create a nice environment
Charles Isbell and Michael Littman (26:04.940)
that you were asking for a mortgage
Lex Fridman (26:06.220)
and I was going to say no.
Charles Isbell and Michael Littman (26:07.120)
That was the point.
Lex Fridman (26:07.960)
It was a very simple idea here.
Charles Isbell and Michael Littman (26:09.500)
Anyway, so we sat there
Lex Fridman (26:10.980)
and we just talked for a little while.
Lex Fridman (26:12.260)
And I think he got the impression that I didn't like him.
Lex Fridman (26:14.460)
Which wasn't true.
Charles Isbell and Michael Littman (26:15.300)
I strongly got that impression.
Lex Fridman (26:16.120)
The talk was really good.
Charles Isbell and Michael Littman (26:16.960)
The talk, by the way, was terrible.
Lex Fridman (26:18.680)
And right after the talk,
Charles Isbell and Michael Littman (26:20.300)
I said to my host, Michael Kearns,
Lex Fridman (26:21.700)
who ultimately was my boss.
Charles Isbell and Michael Littman (26:23.340)
I'm a huge fan.
Lex Fridman (26:24.180)
I'm a friend and a huge fan of Michael, yeah.
Charles Isbell and Michael Littman (26:25.780)
Yeah, he is a remarkable person.
Lex Fridman (26:29.660)
After my talk, I went into the...
Charles Isbell and Michael Littman (26:30.980)
He went into basketball.
Lex Fridman (26:32.420)
I went...
Charles Isbell and Michael Littman (26:33.260)
Racquetball, he's good at everything.
Lex Fridman (26:34.080)
No, basketball.
Charles Isbell and Michael Littman (26:34.920)
No, but basketball and racquetball too.
Lex Fridman (26:36.020)
Squash.
Charles Isbell and Michael Littman (26:36.860)
Squash, squash, squash, not racquetball.
Lex Fridman (26:38.180)
Yes, squash, which is not...
Charles Isbell and Michael Littman (26:39.980)
Racquetball, yes.
Lex Fridman (26:41.420)
Squash, no.
Lex Fridman (26:42.240)
And I hope you hear that, Michael.
Lex Fridman (26:43.980)
Oh, Michael Kearns.
Charles Isbell and Michael Littman (26:45.980)
As a game, not his skill level,
Lex Fridman (26:47.580)
because I'm pretty sure he's...
Charles Isbell and Michael Littman (26:50.060)
All right, there's some competitiveness there,
Lex Fridman (26:51.600)
but the point is that it was like the middle of the day,
Charles Isbell and Michael Littman (26:54.360)
I had full day of interviews.
Lex Fridman (26:55.760)
I got met with people,
Lex Fridman (26:56.620)
but then in the middle of the day, I gave a job talk.
Lex Fridman (26:58.900)
And then there was gonna be more interviews,
Lex Fridman (27:01.540)
but I pulled Michael aside and I said,
Lex Fridman (27:04.860)
I think it's in both of our best interest
Charles Isbell and Michael Littman (27:07.240)
if I just leave now, because that was so bad
Lex Fridman (27:11.100)
that it'd just be embarrassing
Charles Isbell and Michael Littman (27:12.540)
if I have to talk to any more people.
Lex Fridman (27:14.300)
You look bad for having invited me.
Charles Isbell and Michael Littman (27:16.380)
It's just, let's just forget this ever happened.
Lex Fridman (27:19.580)
So I don't think the talk went well.
Charles Isbell and Michael Littman (27:21.780)
That's one of the most Michael Lipman set of sentences
Lex Fridman (27:23.620)
I think I've ever heard.
Charles Isbell and Michael Littman (27:24.540)
He did great, or at least everyone knew he was great,
Lex Fridman (27:27.220)
so maybe it didn't matter.
Charles Isbell and Michael Littman (27:28.220)
I was there, I remember the talk,
Lex Fridman (27:29.940)
and I remember him being very much the way
Charles Isbell and Michael Littman (27:31.980)
I remember him now, on any given week.
Lex Fridman (27:33.980)
So it was good.
Lex Fridman (27:34.800)
And we met and we talked about stuff.
Lex Fridman (27:36.620)
He thinks I didn't like him, but...
Charles Isbell and Michael Littman (27:37.940)
Because he was so grumpy.
Lex Fridman (27:39.340)
Must've been the chair thing.
Charles Isbell and Michael Littman (27:40.740)
The chair thing and the low voice, I think.
Lex Fridman (27:42.700)
But like, he obviously...
Lex Fridman (27:43.700)
And that slight skeptical look.
Lex Fridman (27:47.020)
Yes.
Charles Isbell and Michael Littman (27:48.540)
I have no idea what you're talking about.
Lex Fridman (27:50.540)
Well, I probably didn't have any idea
Lex Fridman (27:51.900)
what you were talking about.
Lex Fridman (27:53.820)
Anyway, I liked him.
Charles Isbell and Michael Littman (27:54.820)
He asked me questions, I answered questions.
Lex Fridman (27:56.420)
I felt bad about myself.
Charles Isbell and Michael Littman (27:57.380)
It was a normal day.
Lex Fridman (27:58.340)
It was a normal day.
Lex Fridman (28:00.540)
And then he left.
Lex Fridman (28:01.380)
And then he left, and that's how you met.
Charles Isbell and Michael Littman (28:03.140)
Can we take a...
Lex Fridman (28:03.980)
And then I got hired and I was in the group.
Charles Isbell and Michael Littman (28:05.860)
Can we take a slight tangent on this topic of,
Lex Fridman (28:09.100)
it sounds like, maybe you could speak
Charles Isbell and Michael Littman (28:11.500)
to the bigger picture.
Lex Fridman (28:12.580)
It sounds like you're quite self critical.
Lex Fridman (28:15.060)
Who, Charles?
Lex Fridman (28:15.900)
No, you.
Charles Isbell and Michael Littman (28:16.720)
Oh.
Lex Fridman (28:17.560)
I can do better.
Charles Isbell and Michael Littman (28:18.560)
I can do better.
Lex Fridman (28:19.400)
Try me again.
Charles Isbell and Michael Littman (28:20.220)
I'll do better.
Lex Fridman (28:21.060)
I'll be so self critical.
Charles Isbell and Michael Littman (28:21.900)
I won't.
Lex Fridman (28:22.740)
I won't.
Charles Isbell and Michael Littman (28:23.560)
I won't.
Lex Fridman (28:24.400)
Yeah, that was like a three out of 10 response.
Lex Fridman (28:26.420)
So let's try to work it up to five and six.
Lex Fridman (28:30.380)
Yeah, I remember Marvin Minsky said on a video interview,
Charles Isbell and Michael Littman (28:35.160)
something that the key to success in academic research
Lex Fridman (28:38.820)
is to hate everything you do.
Charles Isbell and Michael Littman (28:43.580)
For some reason...
Lex Fridman (28:44.420)
I think I followed that because I hate everything he's done.
Charles Isbell and Michael Littman (28:46.820)
That's a good line.
Lex Fridman (28:49.740)
That's a six out of 10.
Charles Isbell and Michael Littman (28:52.660)
Maybe that's a keeper.
Lex Fridman (28:53.500)
But do you find that resonates with you at all
Lex Fridman (28:57.660)
in how you think about talks and so on?
Lex Fridman (28:59.740)
I would say it differently.
Charles Isbell and Michael Littman (29:00.840)
It's not that.
Lex Fridman (29:01.680)
No, not really.
Charles Isbell and Michael Littman (29:02.500)
That's such an MIT view of the world though.
Lex Fridman (29:04.460)
So I remember talking about this when, as a student,
Charles Isbell and Michael Littman (29:08.480)
you were basically told I will clean it up
Lex Fridman (29:10.980)
for the purpose of the podcast.
Charles Isbell and Michael Littman (29:13.660)
My work is crap.
Lex Fridman (29:14.500)
My work is crap.
Charles Isbell and Michael Littman (29:15.320)
My work is crap.
Lex Fridman (29:16.160)
Then you go to a conference or something.
Charles Isbell and Michael Littman (29:17.660)
You're like, everybody else's work is crap.
Lex Fridman (29:18.980)
Everybody else's work is crap.
Lex Fridman (29:19.820)
And you feel better and better about it, relatively speaking.
Lex Fridman (29:23.260)
And then you sort of keep working on it.
Charles Isbell and Michael Littman (29:25.060)
I don't hate my work.
Lex Fridman (29:26.820)
That resonates with me.
Charles Isbell and Michael Littman (29:27.660)
Yes, I've never hated my work,
Lex Fridman (29:28.900)
but I have been dissatisfied with it.
Lex Fridman (29:33.460)
And I think being dissatisfied,
Lex Fridman (29:35.980)
being okay with the fact that you've taken a positive step,
Charles Isbell and Michael Littman (29:38.620)
the derivative's positive,
Lex Fridman (29:40.340)
maybe even the second derivative's positive,
Lex Fridman (29:42.300)
that's important because that's a part of the hope, right?
Lex Fridman (29:45.060)
But you have to, but I haven't gotten there yet.
Charles Isbell and Michael Littman (29:47.300)
If that's not there, that I haven't gotten there yet,
Lex Fridman (29:49.880)
then it's hard to move forward, I think.
Lex Fridman (29:53.460)
So I buy that, which is a little different
Lex Fridman (29:55.100)
from hating everything that you do.
Charles Isbell and Michael Littman (29:56.420)
Yeah, I mean, there's things that I've done
Lex Fridman (29:59.300)
that I like better than I like myself.
Lex Fridman (30:01.260)
So it's separating me from the work, essentially.
Lex Fridman (30:04.040)
So I think I am very critical of myself,
Lex Fridman (30:06.780)
but sometimes the work I'm really excited about.
Lex Fridman (30:08.700)
And sometimes I think it's kind of good.
Lex Fridman (30:10.540)
Does that happen right away?
Lex Fridman (30:11.380)
So I found the work that I've liked, that I've done,
Charles Isbell and Michael Littman (30:15.380)
most of it, I liked it in retrospect
Lex Fridman (30:18.420)
more when I was far away from it in time.
Charles Isbell and Michael Littman (30:21.220)
I have to be fairly excited about it to get done.
Lex Fridman (30:24.380)
No, excited at the time, but then happy with the result.
Lex Fridman (30:26.900)
But years later, or even I might go back,
Lex Fridman (30:28.780)
you know what, that actually turned out to matter.
Charles Isbell and Michael Littman (30:31.180)
That turned out to matter.
Lex Fridman (30:32.020)
Or, oh gosh, it turns out I've been thinking about that.
Charles Isbell and Michael Littman (30:34.220)
It's actually influenced all the work that I've done since
Lex Fridman (30:36.420)
without realizing it.
Charles Isbell and Michael Littman (30:37.860)
Boy, that guy was smart.
Lex Fridman (30:39.220)
Yeah, that guy had a future.
Charles Isbell and Michael Littman (30:41.300)
Yeah, I think there's something to it.
Lex Fridman (30:47.060)
I think there's something to the idea
Charles Isbell and Michael Littman (30:48.060)
you've got to hate what you do, but it's not quite hate.
Lex Fridman (30:50.340)
It's just being unsatisfied.
Lex Fridman (30:52.580)
And different people motivate themselves differently.
Lex Fridman (30:54.300)
I don't happen to motivate myself with self loathing.
Charles Isbell and Michael Littman (30:56.740)
I happen to motivate myself with something else.
Lex Fridman (30:58.860)
So you're able to sit back and be proud of,
Charles Isbell and Michael Littman (31:02.900)
in retrospect, of the work you've done.
Lex Fridman (31:04.780)
Well, and it's easier when you can connect it
Charles Isbell and Michael Littman (31:06.540)
with other people, because then you can be proud of them.
Lex Fridman (31:08.940)
Proud of the people, yeah.
Lex Fridman (31:10.180)
And then the question is.
Lex Fridman (31:11.020)
You can still safely hate yourself.
Charles Isbell and Michael Littman (31:12.420)
Yeah, that's right.
Lex Fridman (31:13.700)
It's win, win, Michael.
Charles Isbell and Michael Littman (31:15.220)
Or at least win, lose, which is what you're looking for.
Lex Fridman (31:18.420)
Oh, wow, there's so many brilliant minds in this.
Charles Isbell and Michael Littman (31:22.180)
There's levels.
Lex Fridman (31:23.580)
So how did you actually meet me?
Charles Isbell and Michael Littman (31:26.020)
Yeah, Michael.
Lex Fridman (31:26.860)
So the way I think about it is,
Charles Isbell and Michael Littman (31:28.540)
because we didn't do much research together at AT&T,
Lex Fridman (31:32.500)
but then we all got laid off.
Lex Fridman (31:34.540)
So that sucked.
Lex Fridman (31:36.540)
By the way, sorry to interrupt,
Lex Fridman (31:37.940)
but that was one of the most magical places
Lex Fridman (31:40.500)
historically speaking.
Charles Isbell and Michael Littman (31:42.340)
They did not appreciate what they had.
Lex Fridman (31:45.700)
And how do we,
Charles Isbell and Michael Littman (31:47.940)
I feel like there's a profound lesson in there too.
Lex Fridman (31:50.180)
How do we get it, like what was, why was it so magical?
Lex Fridman (31:53.100)
Is it just a coincidence of history?
Lex Fridman (31:54.940)
Or is there something special about?
Charles Isbell and Michael Littman (31:56.380)
There were some really good managers
Lex Fridman (31:57.740)
and people who really believed in machine learning
Charles Isbell and Michael Littman (32:00.460)
as this is gonna be important.
Lex Fridman (32:03.100)
Let's get the people who are thinking about this
Charles Isbell and Michael Littman (32:05.620)
in creative and insightful ways
Lex Fridman (32:08.060)
and put them in one place and stir.
Lex Fridman (32:10.260)
Yeah, but even beyond that, right?
Lex Fridman (32:11.500)
It was Bell Labs at its heyday.
Lex Fridman (32:15.540)
And even when we were there, which I think was past its heyday.
Lex Fridman (32:17.980)
And to be clear, he's gotten to be at Bell Labs.
Charles Isbell and Michael Littman (32:19.700)
I never got to be at Bell Labs.
Lex Fridman (32:21.100)
I joined after that.
Charles Isbell and Michael Littman (32:22.300)
Yeah, I showed up in 91 as a grad student.
Lex Fridman (32:24.620)
So I was there for a long time, every summer, except for two.
Lex Fridman (32:28.420)
So twice I worked for companies
Lex Fridman (32:29.580)
that had just stopped being Bell Labs.
Charles Isbell and Michael Littman (32:31.900)
Bellcore and then AT&T Labs.
Lex Fridman (32:33.620)
So Bell Labs was several locations or for the research
Lex Fridman (32:37.300)
or is it one?
Lex Fridman (32:38.140)
I don't know if Jersey's involved somehow.
Charles Isbell and Michael Littman (32:41.020)
They're all in Jersey.
Lex Fridman (32:41.900)
Yeah, they're all over the place.
Lex Fridman (32:42.740)
But they were in a couple of places in Jersey.
Lex Fridman (32:44.060)
Murray Hill was the Bell Labs place.
Lex Fridman (32:47.420)
So you had an office in Murray Hill
Lex Fridman (32:49.820)
at one point in your career.
Charles Isbell and Michael Littman (32:51.100)
Yeah, and I played Ultimate Frisbee
Lex Fridman (32:53.380)
on the cricket pitch at Bell Labs at Murray Hill.
Lex Fridman (32:56.340)
And then it became AT&T Labs when it split off
Lex Fridman (32:58.300)
with loose during what we called Trivestiture.
Lex Fridman (33:00.940)
Are you better than Michael Korn's at Ultimate Frisbee?
Lex Fridman (33:03.660)
Yeah. Oh, yeah.
Charles Isbell and Michael Littman (33:04.580)
Okay.
Lex Fridman (33:05.420)
But I think that one's not boasting.
Charles Isbell and Michael Littman (33:06.620)
I think Charles plays a lot of Ultimate
Lex Fridman (33:08.580)
and I don't think Michael does.
Charles Isbell and Michael Littman (33:10.500)
Yes, but that wasn't the point.
Lex Fridman (33:12.100)
The point is yes.
Charles Isbell and Michael Littman (33:12.940)
I'm finally better.
Lex Fridman (33:13.780)
Sorry.
Charles Isbell and Michael Littman (33:14.620)
Okay, I have played on a championship winning
Lex Fridman (33:17.420)
Ultimate Frisbee team or whatever,
Charles Isbell and Michael Littman (33:19.420)
Ultimate team with Charles.
Lex Fridman (33:20.780)
So I know how good he is.
Charles Isbell and Michael Littman (33:22.660)
He's really good.
Lex Fridman (33:23.500)
How good I was anyway, when I was younger.
Lex Fridman (33:24.620)
But the thing is.
Lex Fridman (33:25.460)
I know how young he was when he was younger.
Charles Isbell and Michael Littman (33:26.820)
That's true.
Lex Fridman (33:27.740)
So much younger than now.
Charles Isbell and Michael Littman (33:28.900)
He's older now.
Lex Fridman (33:29.820)
Yeah, I'm older.
Charles Isbell and Michael Littman (33:30.660)
Michael was a much better basketball player than I was.
Lex Fridman (33:33.140)
Michael Kearns.
Charles Isbell and Michael Littman (33:34.380)
Yes, no, not Michael.
Lex Fridman (33:36.220)
Let's be very clear about that.
Charles Isbell and Michael Littman (33:37.060)
To be clear, I've not played basketball with you.
Lex Fridman (33:38.820)
So you don't know how terrible I am,
Lex Fridman (33:40.460)
but you have a probably pretty good guess.
Lex Fridman (33:42.340)
And that you're not as good as Michael Kearns.
Charles Isbell and Michael Littman (33:44.300)
He's tall and athletic.
Lex Fridman (33:45.740)
And he cared about it.
Charles Isbell and Michael Littman (33:46.580)
He's very athletic.
Lex Fridman (33:47.420)
He's very good.
Lex Fridman (33:48.240)
And probably competitive.
Lex Fridman (33:49.080)
I love hanging out with Michael.
Charles Isbell and Michael Littman (33:50.300)
Anyway, but we were talking about something else,
Lex Fridman (33:51.660)
although I no longer remember what it was.
Lex Fridman (33:52.980)
What were we talking about?
Lex Fridman (33:53.820)
Oh, Bell Labs.
Charles Isbell and Michael Littman (33:54.660)
Oh, Bell Labs.
Lex Fridman (33:55.500)
But also Labs.
Lex Fridman (33:56.340)
So this was kind of cool about what was magical about it.
Lex Fridman (34:00.220)
The first thing you have to know
Lex Fridman (34:01.300)
is that Bell Labs was an arm of the government, right?
Lex Fridman (34:03.500)
Because AT&T was an arm of the government.
Charles Isbell and Michael Littman (34:05.300)
It was a monopoly.
Lex Fridman (34:07.420)
And every month you paid a little thing on your phone bill,
Charles Isbell and Michael Littman (34:10.780)
which turned out was a tax
Lex Fridman (34:12.000)
for all the research that Bell Labs was doing.
Lex Fridman (34:14.340)
And they invented transistors and the laser
Lex Fridman (34:16.700)
and whatever else is that they did.
Charles Isbell and Michael Littman (34:17.540)
The Big Bang or whatever, the cosmic background radiation.
Lex Fridman (34:20.580)
Yeah, they did all that stuff.
Charles Isbell and Michael Littman (34:21.420)
They had some amazing stuff with directional microphones,
Lex Fridman (34:23.380)
by the way.
Charles Isbell and Michael Littman (34:24.220)
I got to go in this room
Lex Fridman (34:25.520)
where they had all these panels and everything.
Lex Fridman (34:27.940)
And we would talk and one another,
Lex Fridman (34:29.460)
and he'd move some panels around.
Lex Fridman (34:30.740)
And then he would have me step two steps to the left.
Lex Fridman (34:33.700)
And I couldn't hear a thing he was saying
Charles Isbell and Michael Littman (34:35.100)
because nothing was bouncing off the walls.
Lex Fridman (34:37.100)
And then he would shut it all down
Lex Fridman (34:38.460)
and you could hear your heartbeat,
Lex Fridman (34:40.340)
which is deeply disturbing to hear your heartbeat.
Charles Isbell and Michael Littman (34:43.320)
You can feel it.
Lex Fridman (34:44.160)
I mean, you can feel it now.
Charles Isbell and Michael Littman (34:44.980)
There's just so much all this sort of noise around.
Lex Fridman (34:46.620)
Anyway, Bell Labs was about pure research.
Charles Isbell and Michael Littman (34:48.300)
It was a university, in some sense,
Lex Fridman (34:50.420)
the purest sense of a university, but without students.
Lex Fridman (34:53.660)
So it was all the faculty working with one another
Lex Fridman (34:56.300)
and students would come in to learn.
Charles Isbell and Michael Littman (34:57.860)
They would come in for three or four months
Lex Fridman (34:59.220)
during the summer and they would go away.
Lex Fridman (35:00.860)
But it was just this kind of wonderful experience.
Lex Fridman (35:02.820)
I could walk out my door.
Charles Isbell and Michael Littman (35:04.700)
In fact, I would often have to walk out my door
Lex Fridman (35:06.540)
and deal with Rich Sutton and Michael Kearns
Charles Isbell and Michael Littman (35:08.320)
yelling at each other about whatever it is
Lex Fridman (35:10.940)
they were yelling about the proper way
Charles Isbell and Michael Littman (35:13.380)
to prove something or another.
Lex Fridman (35:14.620)
And I could just do that.
Lex Fridman (35:15.580)
And Dave McAllister and Peter Stone
Lex Fridman (35:17.980)
and all of these other people,
Charles Isbell and Michael Littman (35:19.780)
including, it's a tender and then eventually Michael.
Lex Fridman (35:22.620)
And it was just a place where you could think thoughts.
Lex Fridman (35:25.260)
And it was okay because so long as once every 25 years or so
Lex Fridman (35:29.300)
somebody invented a transistor, it paid for everything else.
Charles Isbell and Michael Littman (35:31.780)
You could afford to take the risk.
Lex Fridman (35:34.140)
And then when that all went away,
Charles Isbell and Michael Littman (35:36.440)
it became harder and harder and harder to justify it
Lex Fridman (35:39.380)
as far as the folks who were very far away were concerned.
Lex Fridman (35:41.500)
And there was such a fast turnaround
Lex Fridman (35:43.540)
among mental management on the AT&T side
Charles Isbell and Michael Littman (35:46.380)
that you never had a chance to really build a relationship.
Lex Fridman (35:48.340)
At least people like us didn't have a chance
Charles Isbell and Michael Littman (35:49.820)
to build a relationship.
Lex Fridman (35:51.460)
So when the diaspora happened, it was amazing, right?
Charles Isbell and Michael Littman (35:55.500)
Everybody left and I think everybody ended up
Lex Fridman (35:57.660)
at a great place and made a huge,
Charles Isbell and Michael Littman (36:00.060)
continued to do really good work with machine learning.
Lex Fridman (36:02.700)
But it was a wonderful place.
Lex Fridman (36:03.740)
And people will ask me, what's the best job you've ever had?
Lex Fridman (36:07.060)
And as a professor, anyway, the answer that I would give is
Charles Isbell and Michael Littman (36:11.140)
well, probably Bell Labs in some very real sense.
Lex Fridman (36:16.100)
And I will never have a job like that again
Charles Isbell and Michael Littman (36:17.640)
because Bell Labs doesn't exist anymore.
Lex Fridman (36:19.380)
And Microsoft research is great and Google does good stuff.
Lex Fridman (36:22.280)
And you can pick IBM, you can tell if you want to,
Lex Fridman (36:24.180)
but Bell Labs was magical.
Charles Isbell and Michael Littman (36:25.860)
It was around for, it was an important time
Lex Fridman (36:28.020)
and it represents a high watermark
Charles Isbell and Michael Littman (36:30.540)
in basic research in the US.
Lex Fridman (36:32.820)
Is there something you could say about the physical proximity
Lex Fridman (36:35.380)
and the chance collisions?
Lex Fridman (36:36.660)
Like we live in this time of the pandemic
Charles Isbell and Michael Littman (36:39.380)
where everyone is maybe trying to see the silver lining
Lex Fridman (36:43.780)
and accepting the remote nature of things.
Charles Isbell and Michael Littman (36:46.940)
Is there one of the things that people like faculty
Lex Fridman (36:50.380)
that I talk to miss is the procrastination.
Charles Isbell and Michael Littman (36:57.020)
Like the chance to make everything is about meetings
Lex Fridman (36:59.980)
that are supposed to be,
Charles Isbell and Michael Littman (37:00.960)
there's not a chance to just talk about comic book
Lex Fridman (37:04.100)
or whatever, like go into discussion that's totally pointless.
Lex Fridman (37:07.180)
So it's funny you say this
Lex Fridman (37:08.180)
because that's how we met, met, it was exactly that.
Lex Fridman (37:11.100)
So I'll let Michael say that, but I'll just add one thing
Lex Fridman (37:12.780)
which is just that research is a social process
Lex Fridman (37:16.460)
and it helps to have random social interactions
Lex Fridman (37:20.140)
even if they don't feel social at the time,
Charles Isbell and Michael Littman (37:21.580)
that's how you get things done.
Lex Fridman (37:22.760)
One of the great things about the AI Lab when I was there,
Charles Isbell and Michael Littman (37:25.860)
I don't quite know what it looks like now
Lex Fridman (37:27.900)
once they moved buildings,
Lex Fridman (37:28.780)
but we had entire walls that were whiteboards
Lex Fridman (37:30.860)
and people would just get up there
Lex Fridman (37:31.860)
and they were just right and people would walk up
Lex Fridman (37:33.620)
and you'd have arguments
Lex Fridman (37:34.460)
and you'd explain things to one another
Lex Fridman (37:36.120)
and you got so much out of the freedom to do that.
Charles Isbell and Michael Littman (37:39.600)
You had to be okay with people challenging
Lex Fridman (37:42.940)
every fricking word you said,
Charles Isbell and Michael Littman (37:44.460)
which I would sometimes find deeply irritating,
Lex Fridman (37:47.220)
but most of the time it was quite useful.
Lex Fridman (37:49.620)
But the sort of pointlessness and the interaction
Lex Fridman (37:52.020)
was in some sense the point, at least for me.
Charles Isbell and Michael Littman (37:54.780)
Yeah, I think offline yesterday I mentioned
Lex Fridman (37:57.540)
Josh Tenenbaum and he's very much, he's a man,
Charles Isbell and Michael Littman (38:01.660)
he's such an inspiration in the childlike way
Lex Fridman (38:06.220)
that he pulls you in on any topic.
Charles Isbell and Michael Littman (38:07.820)
It doesn't even have to be about machine learning
Lex Fridman (38:10.380)
or the brain, he'll just pull you in
Charles Isbell and Michael Littman (38:12.860)
to a closest writable surface,
Lex Fridman (38:15.900)
which is still, you can find whiteboards
Charles Isbell and Michael Littman (38:18.260)
at MIT everywhere, and just like basically cancel
Lex Fridman (38:23.380)
all meetings and talk for a couple hours
Charles Isbell and Michael Littman (38:25.040)
about some aimless thing and it feels like
Lex Fridman (38:28.060)
the whole world, the time space continuum kind of warps
Lex Fridman (38:30.820)
and that becomes the most important thing.
Lex Fridman (38:32.740)
And then it's just, it's definitely something
Charles Isbell and Michael Littman (38:36.660)
worth missing in this world where everything's remote.
Lex Fridman (38:40.140)
There's some magic to the physical presence.
Charles Isbell and Michael Littman (38:42.980)
Whenever I wonder myself whether MIT really is
Lex Fridman (38:44.940)
as great as I remember it, I just go talk to Josh.
Charles Isbell and Michael Littman (38:48.220)
Yeah, you know, that's funny.
Lex Fridman (38:49.760)
There's a few people in this world that carry
Lex Fridman (38:52.780)
the best of what particular institutions stand for, right?
Lex Fridman (38:56.380)
And it's.
Charles Isbell and Michael Littman (38:57.220)
It's Josh.
Lex Fridman (38:58.060)
I mean, I don't, my guess is he's unaware of this.
Charles Isbell and Michael Littman (39:00.980)
That's the point.
Lex Fridman (39:02.020)
Yeah.
Charles Isbell and Michael Littman (39:02.860)
That the masters are not aware of their mastery.
Lex Fridman (39:06.680)
So.
Lex Fridman (39:07.520)
How did we meet?
Lex Fridman (39:09.100)
Yes, but first a tangent, no.
Lex Fridman (39:13.700)
How did you meet me?
Lex Fridman (39:14.700)
So I'm not sure what you were thinking,
Lex Fridman (39:16.100)
but when it started to dawn on me
Lex Fridman (39:19.220)
that maybe we had a longer term bond
Charles Isbell and Michael Littman (39:21.540)
was after we all got laid off.
Lex Fridman (39:23.980)
And you had decided at that point
Charles Isbell and Michael Littman (39:26.620)
that we were still paid.
Lex Fridman (39:28.460)
We were given an opportunity to like do a job search
Lex Fridman (39:30.980)
and kind of make a transition,
Lex Fridman (39:32.980)
but it was clear that we were done.
Lex Fridman (39:35.220)
And I would go to my office to work
Lex Fridman (39:38.400)
and you would go to my office to keep me from working.
Charles Isbell and Michael Littman (39:41.340)
That was my recollection of it.
Lex Fridman (39:43.500)
You had decided that there was no,
Charles Isbell and Michael Littman (39:44.580)
really no point in working for the company
Lex Fridman (39:46.540)
because our relationship with the company was done.
Charles Isbell and Michael Littman (39:49.720)
Yeah, but remember I felt that way beforehand.
Lex Fridman (39:51.260)
It wasn't about the company.
Charles Isbell and Michael Littman (39:52.180)
It was about the set of people there
Lex Fridman (39:53.420)
doing really cool things.
Lex Fridman (39:54.260)
And it always, always been that way.
Lex Fridman (39:55.780)
But we were working on something together.
Charles Isbell and Michael Littman (39:57.460)
Oh yeah, yeah, yeah.
Lex Fridman (39:58.300)
That's right.
Lex Fridman (39:59.120)
So at the very end, we all got laid off,
Lex Fridman (40:00.280)
but then our boss came to, our boss's boss came to us
Charles Isbell and Michael Littman (40:04.140)
because our boss was Michael Kearns
Lex Fridman (40:05.620)
and he had jumped ship brilliantly, like perfect timing.
Charles Isbell and Michael Littman (40:08.980)
Like things like right before the ship was about to sink,
Lex Fridman (40:12.060)
he was like, gotta go and landed perfectly
Charles Isbell and Michael Littman (40:16.820)
because Michael Kearns.
Lex Fridman (40:18.180)
Because Michael Kearns.
Lex Fridman (40:19.020)
And leaving the rest of us to go like, this is fine.
Lex Fridman (40:23.580)
And then it was clear that it wasn't fine
Lex Fridman (40:25.780)
and we were all toast.
Lex Fridman (40:27.440)
So we had this sort of long period of time.
Lex Fridman (40:29.220)
But then our boss figured out, okay, wait,
Lex Fridman (40:30.860)
maybe we can save a couple of these people
Charles Isbell and Michael Littman (40:33.660)
if we can have them do something really useful.
Lex Fridman (40:37.260)
And the useful thing was we were gonna make
Charles Isbell and Michael Littman (40:40.680)
basically an automated assistant
Lex Fridman (40:42.280)
that could help you with your calendar.
Charles Isbell and Michael Littman (40:43.900)
You could like tell it things
Lex Fridman (40:45.700)
and it would respond appropriately.
Charles Isbell and Michael Littman (40:47.860)
It would just kind of integrate across
Lex Fridman (40:49.700)
all sorts of your personal information.
Lex Fridman (40:53.600)
And so me and Charles and Peter Stone
Lex Fridman (40:56.700)
were set up as the crack team
Charles Isbell and Michael Littman (40:58.880)
to actually solve this problem.
Lex Fridman (41:00.860)
Other people maybe were too theoretical that they thought,
Lex Fridman (41:04.220)
but we could actually get something done.
Lex Fridman (41:05.660)
So we sat down to get something done
Lex Fridman (41:07.260)
and there wasn't time and it wouldn't have saved us anyway.
Lex Fridman (41:10.100)
And so it all kind of went downhill.
Lex Fridman (41:12.060)
But the interesting, I think, coda to that
Lex Fridman (41:15.380)
is that our boss's boss is a guy named Ron Brockman.
Lex Fridman (41:18.860)
And when he left AT&T,
Lex Fridman (41:22.100)
cause we were all laid off,
Charles Isbell and Michael Littman (41:23.820)
he went to DARPA, started up a program there
Lex Fridman (41:27.220)
that became KALO,
Charles Isbell and Michael Littman (41:28.960)
which is the program from which Siri sprung,
Lex Fridman (41:32.100)
which is a digital assistant
Charles Isbell and Michael Littman (41:34.060)
that helps you with your calendar
Lex Fridman (41:35.260)
and a bunch of other things.
Charles Isbell and Michael Littman (41:37.660)
It really, in some ways got its start
Lex Fridman (41:40.860)
with me and Charles and Peter trying to implement this vision
Charles Isbell and Michael Littman (41:44.440)
that Ron Brockman had,
Lex Fridman (41:45.580)
that he ultimately got implemented
Charles Isbell and Michael Littman (41:47.740)
through his role at DARPA.
Lex Fridman (41:49.380)
So when I'm trying to feel less bad
Charles Isbell and Michael Littman (41:51.380)
about having been laid off
Lex Fridman (41:52.420)
from what is possibly the greatest job of all time,
Charles Isbell and Michael Littman (41:56.020)
I think about, well, we kind of helped birth Siri.
Lex Fridman (42:00.060)
That's something.
Lex Fridman (42:01.860)
And then he did other things too.
Lex Fridman (42:03.120)
But we got to spend a lot of time in his office
Lex Fridman (42:06.620)
and talk about lots of things.
Lex Fridman (42:07.860)
We got to spend a lot of time in my office, yeah.
Charles Isbell and Michael Littman (42:10.340)
Yeah, yeah.
Lex Fridman (42:11.180)
And so then we went on our merry way.
Charles Isbell and Michael Littman (42:13.460)
Everyone went to different places.
Lex Fridman (42:15.340)
Charles landed at Georgia Tech,
Charles Isbell and Michael Littman (42:16.540)
which was what he always dreamed he would do.
Lex Fridman (42:20.300)
And so that worked out well.
Charles Isbell and Michael Littman (42:23.740)
I came up with a saying at the time,
Lex Fridman (42:25.380)
which is luck favors the Charles.
Charles Isbell and Michael Littman (42:27.980)
It's kind of like luck favors the prepared,
Lex Fridman (42:30.540)
but Charles, like he wished something
Lex Fridman (42:32.940)
and then it would basically happen just the way he wanted.
Lex Fridman (42:35.460)
It was inspirational to see things go that way.
Charles Isbell and Michael Littman (42:38.500)
Things worked out.
Lex Fridman (42:39.340)
And we stayed in touch.
Lex Fridman (42:40.180)
And then I think it really helped
Lex Fridman (42:43.740)
when you were working on,
Charles Isbell and Michael Littman (42:46.100)
I mean, you'd kept me in the loop for things like threads
Lex Fridman (42:48.300)
and the work that you were doing at Georgia Tech.
Lex Fridman (42:49.780)
But then when they were starting
Lex Fridman (42:50.980)
their online master's program,
Charles Isbell and Michael Littman (42:52.860)
he knew that I was really excited about MOOCs
Lex Fridman (42:55.020)
and online teaching.
Lex Fridman (42:56.140)
And he's like, I have a plan.
Lex Fridman (42:57.980)
And I'm like, tell me your plan.
Charles Isbell and Michael Littman (42:58.900)
He's like, I can't tell you the plan yet.
Lex Fridman (43:00.660)
Cause they were deep in negotiations
Charles Isbell and Michael Littman (43:02.980)
between Georgia Tech and Udacity to make this happen.
Lex Fridman (43:05.500)
And they didn't want it to leak.
Lex Fridman (43:07.340)
So Charles would kept teasing me about it,
Lex Fridman (43:09.260)
but wouldn't tell me what was actually going on.
Lex Fridman (43:10.740)
And eventually it was announced and he said,
Lex Fridman (43:13.020)
I would like you to teach the machine learning course
Charles Isbell and Michael Littman (43:15.060)
with me.
Lex Fridman (43:15.900)
I'm like, that can't possibly work.
Lex Fridman (43:18.340)
But it was a great idea.
Lex Fridman (43:19.340)
And it was super fun.
Charles Isbell and Michael Littman (43:20.940)
It was a lot of work to put together,
Lex Fridman (43:22.140)
but it was really great.
Charles Isbell and Michael Littman (43:23.980)
Was that the first time you thought about,
Lex Fridman (43:26.020)
first of all, was it the first time
Lex Fridman (43:27.680)
you got seriously into teaching?
Lex Fridman (43:30.140)
I mean, I was a professor.
Charles Isbell and Michael Littman (43:32.100)
This was already after you jumped to,
Lex Fridman (43:35.500)
so like there's a little bit of jumping around in time.
Charles Isbell and Michael Littman (43:38.740)
Yeah, sorry about that.
Lex Fridman (43:39.580)
There's a pretty big jump in time.
Lex Fridman (43:40.500)
So like the MOOCs thing.
Lex Fridman (43:42.540)
So Charles got to Georgia Tech and he,
Charles Isbell and Michael Littman (43:44.260)
I mean, maybe Charles, maybe this is a Charles story.
Lex Fridman (43:46.060)
I got to Georgia Tech in 2002.
Charles Isbell and Michael Littman (43:47.060)
He got to Georgia Tech in 2002.
Lex Fridman (43:49.300)
And worked on things like revamping the curriculum,
Charles Isbell and Michael Littman (43:52.820)
the undergraduate curriculum,
Lex Fridman (43:53.780)
so that it had some kind of semblance of modular structure
Charles Isbell and Michael Littman (43:57.800)
because computer science was at the time
Lex Fridman (44:00.060)
moving from a fairly narrow specific set of topics
Charles Isbell and Michael Littman (44:03.620)
to touching a lot of other parts of intellectual life.
Lex Fridman (44:08.340)
And the curriculum was supposed to reflect that.
Lex Fridman (44:10.860)
And so Charles played a big role in kind of redesigning that.
Lex Fridman (44:15.480)
And then the.
Lex Fridman (44:16.320)
And for my labors, I ended up as associate dean.
Lex Fridman (44:20.660)
Right, he got to become associate dean
Charles Isbell and Michael Littman (44:22.420)
of charge of educational stuff.
Lex Fridman (44:24.780)
Yeah, I was under.
Charles Isbell and Michael Littman (44:25.620)
This should be a valuable lesson.
Lex Fridman (44:26.700)
If you're good at something,
Charles Isbell and Michael Littman (44:30.260)
they will give you responsibility to do more of that thing.
Lex Fridman (44:33.740)
Well.
Charles Isbell and Michael Littman (44:34.580)
Until you.
Lex Fridman (44:35.400)
Don't show competence.
Charles Isbell and Michael Littman (44:36.240)
Don't show competence if you.
Lex Fridman (44:37.260)
Don't want responsibility.
Charles Isbell and Michael Littman (44:38.900)
Here's what they say.
Lex Fridman (44:40.700)
The reward for good work is more work.
Charles Isbell and Michael Littman (44:43.340)
The reward for bad work is less work.
Lex Fridman (44:47.060)
Which, I don't know.
Charles Isbell and Michael Littman (44:48.140)
Depending on what you're trying to do that week,
Lex Fridman (44:50.080)
one of those is better than the other.
Charles Isbell and Michael Littman (44:51.300)
Well, one of the problems with the word work,
Lex Fridman (44:52.820)
sorry to interrupt, is that it seems to be an antonym
Charles Isbell and Michael Littman (44:57.580)
in this particular language.
Lex Fridman (44:59.240)
We have the opposite of happiness.
Lex Fridman (45:01.780)
But it seems like they're.
Lex Fridman (45:02.980)
That's one of, you know, we talked about balance.
Charles Isbell and Michael Littman (45:07.460)
It's always like work life balance.
Lex Fridman (45:09.700)
It always rubbed me the wrong way as a terminology.
Charles Isbell and Michael Littman (45:12.940)
I know it's just words.
Lex Fridman (45:13.900)
Right, the opposite of work is play.
Lex Fridman (45:15.460)
But ideally, work is play.
Lex Fridman (45:17.900)
Oh, I can't tell you how much time I'd spend.
Charles Isbell and Michael Littman (45:20.340)
Certainly, when I was at Bell Labs,
Lex Fridman (45:21.780)
except for a few very key moments,
Charles Isbell and Michael Littman (45:23.740)
as a professor, I would do this too.
Lex Fridman (45:25.140)
I would just say, I cannot believe
Charles Isbell and Michael Littman (45:26.200)
they're paying me to do this.
Lex Fridman (45:28.700)
Because it's fun.
Charles Isbell and Michael Littman (45:29.520)
It's something that I would do for a hobby
Lex Fridman (45:32.140)
if I could anyway.
Lex Fridman (45:34.900)
So that sort of worked out.
Lex Fridman (45:35.740)
Are you sure you want to be saying that
Lex Fridman (45:37.020)
when this is being recorded?
Lex Fridman (45:38.860)
As a dean, that is not true at all.
Charles Isbell and Michael Littman (45:40.220)
I need a raise.
Lex Fridman (45:42.220)
But I think here with this,
Charles Isbell and Michael Littman (45:43.700)
even though a lot of time passed,
Lex Fridman (45:45.380)
Mike and I talked almost every, well, we texted,
Charles Isbell and Michael Littman (45:47.520)
almost every day during the period.
Lex Fridman (45:49.860)
Charles, at one point, took me,
Charles Isbell and Michael Littman (45:53.060)
the ICML conference, the machine learning conference
Lex Fridman (45:55.020)
was in Atlanta.
Charles Isbell and Michael Littman (45:57.320)
I was the chair, the general chair of the conference.
Lex Fridman (46:00.380)
Charles was my publicity chair or something like that,
Charles Isbell and Michael Littman (46:03.380)
or fundraising chair.
Lex Fridman (46:05.540)
Yeah, but he decided it'd be really funny
Charles Isbell and Michael Littman (46:08.020)
if he didn't actually show up for the conference
Lex Fridman (46:09.660)
in his own home city.
Lex Fridman (46:11.660)
So he didn't, but he did at one point
Lex Fridman (46:13.460)
pick me up at the conference in his Tesla
Lex Fridman (46:16.200)
and drove me to the Atlanta mall
Lex Fridman (46:19.140)
and forced me to buy an iPhone
Charles Isbell and Michael Littman (46:22.060)
because he didn't like how it was to text with me
Lex Fridman (46:25.660)
and thought it would be better for him
Charles Isbell and Michael Littman (46:27.220)
if I had an iPhone, the text would be somehow smoother.
Lex Fridman (46:30.380)
And it was.
Lex Fridman (46:31.220)
And it was.
Lex Fridman (46:32.060)
And it is, and his life is better.
Lex Fridman (46:32.880)
And my life is better.
Lex Fridman (46:33.720)
And so, yeah, but it was, yeah,
Charles Isbell and Michael Littman (46:36.340)
Charles forced me to get an iPhone
Lex Fridman (46:38.220)
so that he could text me more efficiently.
Charles Isbell and Michael Littman (46:40.340)
I thought that was an interesting moment.
Lex Fridman (46:42.020)
It works for me.
Charles Isbell and Michael Littman (46:42.840)
Anyway, so we kept talking the whole time
Lex Fridman (46:44.100)
and then eventually we did the teaching thing
Lex Fridman (46:46.340)
and it was great.
Lex Fridman (46:47.180)
And there's a couple of reasons for that, by the way.
Charles Isbell and Michael Littman (46:48.780)
One is I really wanted to do something different.
Lex Fridman (46:51.460)
Like you've got this medium here,
Charles Isbell and Michael Littman (46:53.200)
people claim it can change things.
Lex Fridman (46:54.500)
What's a thing that you could do in this medium
Lex Fridman (46:56.940)
that you could not do otherwise besides edit, right?
Lex Fridman (47:00.500)
I mean, what could you do?
Lex Fridman (47:01.340)
And being able to do something with another person
Lex Fridman (47:03.940)
was that kind of thing.
Charles Isbell and Michael Littman (47:04.780)
It's very hard.
Lex Fridman (47:05.600)
I mean, you can take turns,
Lex Fridman (47:06.440)
but teaching together, having conversations is very hard.
Lex Fridman (47:09.020)
So that was a cool thing.
Charles Isbell and Michael Littman (47:10.240)
The second thing, give me an excuse
Lex Fridman (47:11.340)
to do more stuff with him.
Charles Isbell and Michael Littman (47:12.420)
Yeah, I always thought, he makes it sound brilliant.
Lex Fridman (47:15.700)
And it is, I guess.
Lex Fridman (47:17.200)
But at the time it really felt like
Lex Fridman (47:20.120)
I've got a lot to do, Charles is saying,
Lex Fridman (47:22.720)
and it would be great if Michael could teach the course
Lex Fridman (47:25.740)
and I could just hang out.
Charles Isbell and Michael Littman (47:27.700)
Yeah, just kind of coast on that.
Lex Fridman (47:29.460)
Well, that's what the second class was more like that.
Charles Isbell and Michael Littman (47:31.540)
Because the second class was explicitly like that.
Lex Fridman (47:33.380)
But the first class, it was at least half.
Charles Isbell and Michael Littman (47:36.280)
Yeah, but I do all the stuff.
Lex Fridman (47:37.120)
So the structure that we came up with.
Charles Isbell and Michael Littman (47:37.940)
I think you're once again letting the facts
Lex Fridman (47:39.940)
get in the way of a good story.
Charles Isbell and Michael Littman (47:42.900)
I should just let Charles talk to us.
Lex Fridman (47:44.700)
But that's the facts that he saw.
Lex Fridman (47:46.380)
So that was kind of true for 7642.
Lex Fridman (47:48.540)
Yeah, that was sort of true for 7642,
Charles Isbell and Michael Littman (47:50.180)
which is the reinforcement learning class,
Lex Fridman (47:51.340)
because that was really his class.
Lex Fridman (47:52.540)
You started with reinforcement learning or machine learning?
Lex Fridman (47:55.260)
Intro machine learning, 7641,
Charles Isbell and Michael Littman (47:57.420)
which is supervised learning, unsupervised learning,
Lex Fridman (48:00.300)
and reinforcement learning and decision making,
Charles Isbell and Michael Littman (48:02.140)
cram all that in there,
Lex Fridman (48:03.060)
the kind of assignments that we talked about earlier.
Lex Fridman (48:04.860)
And then eventually, about a year later,
Lex Fridman (48:06.540)
we did a follow on 7642,
Charles Isbell and Michael Littman (48:08.540)
which is reinforcement learning and decision making.
Lex Fridman (48:10.860)
The first class was based on something
Charles Isbell and Michael Littman (48:12.380)
I'd been teaching at that point for well over a decade.
Lex Fridman (48:14.540)
And the second class was based on something
Charles Isbell and Michael Littman (48:15.940)
Michael had been teaching.
Lex Fridman (48:17.500)
Actually, I learned quite a bit
Charles Isbell and Michael Littman (48:18.860)
teaching that class with him, but he drove most of that.
Lex Fridman (48:21.620)
But the first one I drove most, it was all my material.
Charles Isbell and Michael Littman (48:23.860)
Although I had stolen that material originally
Lex Fridman (48:26.340)
from slides I found online from Michael,
Charles Isbell and Michael Littman (48:28.660)
who had originally stolen that material
Lex Fridman (48:30.540)
from, I guess, slides he found online,
Charles Isbell and Michael Littman (48:32.180)
probably from Andrew Moore,
Lex Fridman (48:33.220)
because the jokes were the same anyway.
Charles Isbell and Michael Littman (48:34.660)
At least some of the, at least when I found the slides,
Lex Fridman (48:36.660)
some of the stuff with it.
Lex Fridman (48:37.500)
Is that true?
Lex Fridman (48:38.340)
Yes, every machine learning class taught in the early 2000s
Charles Isbell and Michael Littman (48:40.540)
stole from Andrew Moore.
Lex Fridman (48:41.900)
A particular joke or two?
Charles Isbell and Michael Littman (48:43.980)
At least the structure.
Lex Fridman (48:44.940)
Now, I did, and he did, actually,
Charles Isbell and Michael Littman (48:46.620)
a lot more with reinforcement learning and such,
Lex Fridman (48:48.940)
and game theory and those kinds of things.
Charles Isbell and Michael Littman (48:50.380)
But, you know, we all sort of built in.
Lex Fridman (48:51.700)
You mean in the research world?
Charles Isbell and Michael Littman (48:52.860)
No, no, no, in that class.
Lex Fridman (48:54.060)
No, I mean in teaching that class.
Charles Isbell and Michael Littman (48:54.900)
The coverage was different than what we started.
Lex Fridman (48:57.540)
Most people were just doing supervised learning
Lex Fridman (48:58.980)
and maybe a little bit of clustering and whatnot,
Lex Fridman (49:01.740)
but we took it all the way to machine learning.
Charles Isbell and Michael Littman (49:03.060)
A lot of it just comes from Tom Mitchell's book.
Lex Fridman (49:04.820)
Oh, no, yeah, except, well,
Lex Fridman (49:06.060)
half of it comes from Tom Mitchell's book, right?
Lex Fridman (49:07.940)
I mean, the other half doesn't.
Lex Fridman (49:10.020)
This is why it's all readings, right?
Lex Fridman (49:12.500)
Because certain things weren't invented
Charles Isbell and Michael Littman (49:13.500)
when Tom wrote that stuff.
Lex Fridman (49:14.340)
Yeah, okay, that's true.
Charles Isbell and Michael Littman (49:15.160)
All right, but it was quite good.
Lex Fridman (49:17.660)
But there's a reason for that besides, you know,
Charles Isbell and Michael Littman (49:19.920)
just, I wanted to do it.
Lex Fridman (49:21.060)
I wanted to do something new,
Lex Fridman (49:21.900)
and I wanted to do something with him,
Lex Fridman (49:23.380)
which is a realization,
Charles Isbell and Michael Littman (49:24.620)
which is despite what you might believe,
Lex Fridman (49:27.380)
he's an introvert and I'm an introvert,
Charles Isbell and Michael Littman (49:29.460)
or I'm on the edge of being an introvert anyway.
Lex Fridman (49:32.140)
But both of us, I think, enjoy the energy of the crowd,
Lex Fridman (49:36.940)
right?
Lex Fridman (49:37.780)
There's something about talking to people
Lex Fridman (49:39.700)
and bringing them into whatever we find interesting
Lex Fridman (49:41.940)
that is empowering, energizing, or whatever.
Lex Fridman (49:45.440)
And I found the idea of staring alone at a computer screen
Lex Fridman (49:50.440)
and then talking off of materials
Charles Isbell and Michael Littman (49:52.720)
less inspiring than I wanted it to be.
Lex Fridman (49:55.360)
And I had in fact done a MOOC for Udacity on algorithms.
Lex Fridman (49:59.200)
And it was a week in a dark room talking at the screen,
Lex Fridman (50:05.480)
writing on the little pad.
Lex Fridman (50:07.040)
And I didn't know this was happening,
Lex Fridman (50:09.320)
but they had watched,
Charles Isbell and Michael Littman (50:10.160)
the crew had watched some of the videos
Lex Fridman (50:12.460)
while, you know, like in the middle of this,
Lex Fridman (50:13.720)
and they're like, something's wrong.
Lex Fridman (50:15.800)
You're sort of shutting down.
Lex Fridman (50:19.540)
And I think a lot of it was I'll make jokes
Lex Fridman (50:22.440)
and no one would laugh.
Lex Fridman (50:24.220)
And I felt like the crowd hated me.
Lex Fridman (50:26.480)
Now, of course, there was no crowd.
Lex Fridman (50:27.800)
So like, it wasn't rational.
Lex Fridman (50:29.960)
But each time I tried it and I got no reaction,
Charles Isbell and Michael Littman (50:32.900)
it just was taking the energy out of my performance,
Lex Fridman (50:37.280)
out of my presentation.
Charles Isbell and Michael Littman (50:38.720)
Such a fantastic metaphor for grad school.
Lex Fridman (50:40.360)
Anyway, by working together,
Charles Isbell and Michael Littman (50:42.600)
we could play off each other and have a good time.
Lex Fridman (50:44.600)
And keep the energy up,
Charles Isbell and Michael Littman (50:45.420)
because you can't let your guard down for a moment
Lex Fridman (50:48.480)
with Charles, he'll just overpower you.
Charles Isbell and Michael Littman (50:51.080)
I have no idea what you're talking about.
Lex Fridman (50:52.200)
But we would work really well together, I thought,
Lex Fridman (50:54.040)
and we knew each other,
Lex Fridman (50:54.880)
so I knew that we could sort of make it work.
Charles Isbell and Michael Littman (50:56.640)
Plus, I was the associate dean,
Lex Fridman (50:57.720)
so they had to do what I told them to do.
Charles Isbell and Michael Littman (51:00.080)
We had to make it work.
Lex Fridman (51:01.480)
And so it worked out very well, I thought,
Charles Isbell and Michael Littman (51:03.880)
well enough that we.
Lex Fridman (51:04.920)
With great power comes great power.
Charles Isbell and Michael Littman (51:06.560)
That's right.
Lex Fridman (51:07.380)
And we became smooth and curly.
Lex Fridman (51:09.320)
And that's when we did the overfitting thriller video.
Lex Fridman (51:15.680)
Yeah, that's a thing.
Lex Fridman (51:17.400)
So can we just, like, smooth and curly,
Lex Fridman (51:20.880)
where did that come from?
Charles Isbell and Michael Littman (51:21.720)
Okay, so it happened.
Lex Fridman (51:23.440)
It was completely spontaneous.
Charles Isbell and Michael Littman (51:24.680)
These are nicknames you go by.
Lex Fridman (51:25.880)
Yeah, so it's what the students call us.
Charles Isbell and Michael Littman (51:28.600)
He was lecturing.
Lex Fridman (51:30.300)
So the way that we structured the lectures
Charles Isbell and Michael Littman (51:32.040)
is one of us is the lecturer
Lex Fridman (51:33.400)
and one of us is basically the student.
Lex Fridman (51:35.360)
And so he was lecturing on.
Lex Fridman (51:37.440)
The lecturer prepares all the materials,
Charles Isbell and Michael Littman (51:39.040)
comes up with the quizzes,
Lex Fridman (51:40.440)
and then the student comes in not knowing anything.
Lex Fridman (51:43.120)
So it was just like being on campus.
Lex Fridman (51:45.440)
And I was doing game theory in particular,
Charles Isbell and Michael Littman (51:48.080)
the Prisoner's Dilemma.
Lex Fridman (51:48.900)
Prisoner's Dilemma.
Lex Fridman (51:49.740)
And so he needed to set up a little Prisoner's Dilemma grid.
Lex Fridman (51:52.200)
So he drew it and I could see what he was drawing.
Lex Fridman (51:54.320)
And the Prisoner's Dilemma consists of two players,
Lex Fridman (51:57.520)
two parties.
Lex Fridman (51:58.360)
So he decided he would make little cartoons
Lex Fridman (52:00.120)
of the two of us.
Lex Fridman (52:01.240)
And so there was two criminals, right,
Lex Fridman (52:04.860)
that were deciding whether or not to rat each other out.
Charles Isbell and Michael Littman (52:07.960)
One of them he drew as a circle with a smiley face
Lex Fridman (52:11.120)
and a kind of goatee thing, smooth head.
Lex Fridman (52:14.080)
And the other one with all sorts of curly hair.
Lex Fridman (52:16.320)
And he said, this is smooth and curly.
Lex Fridman (52:18.400)
I said, smooth and curly?
Lex Fridman (52:19.680)
He said, no, no, smooth with a V.
Charles Isbell and Michael Littman (52:21.560)
It's very important that it have a V.
Lex Fridman (52:23.600)
And then the students really took to that.
Charles Isbell and Michael Littman (52:27.220)
Like they found that relatable.
Lex Fridman (52:29.160)
He started singing Smooth Criminal by Michael Jackson.
Charles Isbell and Michael Littman (52:31.200)
Yeah, yeah, yeah.
Lex Fridman (52:32.040)
And those names stuck.
Lex Fridman (52:33.560)
So we now have a video series,
Lex Fridman (52:36.320)
an episode, our kind of first actual episode
Charles Isbell and Michael Littman (52:38.440)
should be coming out today,
Lex Fridman (52:39.900)
Smooth and Curly on video,
Charles Isbell and Michael Littman (52:43.260)
where the two of us discuss episodes of Westworld.
Lex Fridman (52:47.300)
We watch Westworld and we're like, huh,
Lex Fridman (52:49.360)
what does this say about computer science and AI?
Lex Fridman (52:51.900)
And we've never, we did not watch it.
Charles Isbell and Michael Littman (52:53.940)
I mean, no, it's on season three or whatever we have.
Lex Fridman (52:55.760)
As of this recording, it's on season three.
Charles Isbell and Michael Littman (52:57.660)
We've watched now two episodes total.
Lex Fridman (52:59.960)
Yeah, I think I watched three.
Lex Fridman (53:01.460)
What do you think about Westworld?
Lex Fridman (53:02.660)
Two episodes in.
Lex Fridman (53:03.500)
So I can tell you so far,
Lex Fridman (53:05.940)
I'm just guessing what's gonna happen next.
Charles Isbell and Michael Littman (53:08.160)
It seems like bad things are gonna happen
Lex Fridman (53:10.180)
with the robots uprising.
Charles Isbell and Michael Littman (53:11.260)
It's a lot of.
Lex Fridman (53:12.100)
Spoiler alert.
Lex Fridman (53:12.940)
So I have not, I have not,
Lex Fridman (53:13.780)
I mean, you know, I vaguely remember a movie existing.
Lex Fridman (53:16.200)
So I assume it's related to that, but.
Lex Fridman (53:18.600)
That was more my time than your time, Charles.
Charles Isbell and Michael Littman (53:20.220)
That's right, cause you're much older than I am.
Lex Fridman (53:21.540)
I think the important thing here is that
Lex Fridman (53:24.180)
it's narrative, right?
Lex Fridman (53:25.060)
It's all about telling a story.
Charles Isbell and Michael Littman (53:26.100)
That's the whole driving thing.
Lex Fridman (53:27.500)
But the idea that they would give these reveries,
Charles Isbell and Michael Littman (53:29.720)
that they would make people,
Lex Fridman (53:31.420)
they would make them.
Charles Isbell and Michael Littman (53:32.260)
Let them remember.
Lex Fridman (53:33.080)
Remember the awful things that happened.
Charles Isbell and Michael Littman (53:33.920)
The terrible things that happened.
Lex Fridman (53:35.460)
Who could possibly think that was gonna,
Charles Isbell and Michael Littman (53:36.860)
I gotta, I mean, I don't know.
Lex Fridman (53:38.380)
I've only seen the first two episodes
Charles Isbell and Michael Littman (53:39.660)
or maybe the third one.
Lex Fridman (53:40.500)
I think I've only seen the first one.
Lex Fridman (53:41.320)
You know what it was?
Lex Fridman (53:42.160)
You know what the problem is?
Charles Isbell and Michael Littman (53:43.000)
That the robots were actually designed by Hannibal Lecter.
Lex Fridman (53:45.520)
That's true.
Charles Isbell and Michael Littman (53:46.360)
They weren't.
Lex Fridman (53:47.200)
So like, what do you think is gonna happen?
Charles Isbell and Michael Littman (53:49.580)
Bad things.
Lex Fridman (53:50.500)
It's clear that things are happening
Lex Fridman (53:51.840)
and characters are being introduced
Lex Fridman (53:52.960)
and we don't yet know anything,
Lex Fridman (53:54.260)
but still I was just struck by how
Lex Fridman (53:57.020)
it's all driven by narrative and story.
Lex Fridman (53:58.580)
And there's all these implied things like programming,
Lex Fridman (54:01.260)
the programming interface is talking to them
Charles Isbell and Michael Littman (54:03.880)
about what's going on in their heads,
Lex Fridman (54:05.640)
which is both, I mean, artistically,
Charles Isbell and Michael Littman (54:08.560)
it's probably useful to film it that way.
Lex Fridman (54:10.120)
But think about how it would work in real life.
Charles Isbell and Michael Littman (54:11.540)
That just seems very great.
Lex Fridman (54:12.480)
But there was, we saw in the second episode,
Charles Isbell and Michael Littman (54:14.280)
there's a screen.
Lex Fridman (54:15.120)
You could see things.
Charles Isbell and Michael Littman (54:15.940)
They were wearing like Kubrick's glasses.
Lex Fridman (54:16.780)
In the world.
Charles Isbell and Michael Littman (54:17.620)
It was quite interesting to just kind of ask this question
Lex Fridman (54:20.360)
so far.
Charles Isbell and Michael Littman (54:21.200)
I mean, I assume it veers off into Never Never Land
Lex Fridman (54:22.940)
at some point.
Lex Fridman (54:23.780)
So we don't know.
Lex Fridman (54:24.940)
We can't answer that question.
Charles Isbell and Michael Littman (54:25.880)
I'm also a fan of a guy named Alex Garland.
Lex Fridman (54:28.680)
He's a director of Ex Machina.
Charles Isbell and Michael Littman (54:30.440)
Mm hmm.
Lex Fridman (54:31.280)
And he is the first,
Charles Isbell and Michael Littman (54:33.980)
I wonder if Kubrick was like this actually,
Lex Fridman (54:36.100)
is he like studies,
Lex Fridman (54:39.260)
what would it take to program an AI systems?
Lex Fridman (54:41.720)
Like he's curious enough to go into that direction.
Charles Isbell and Michael Littman (54:44.720)
On the Westworld side,
Lex Fridman (54:46.420)
I felt there was more emphasis on the narratives
Charles Isbell and Michael Littman (54:49.300)
than like actually asking like computer science questions.
Lex Fridman (54:52.360)
Yeah.
Lex Fridman (54:53.200)
Like, how would you build this?
Lex Fridman (54:54.620)
How would you, and.
Lex Fridman (54:56.940)
How would you debug it?
Lex Fridman (54:57.780)
I still think, to me, that's the key issue.
Charles Isbell and Michael Littman (55:00.960)
They were terrible debuggers.
Lex Fridman (55:02.280)
Yeah.
Charles Isbell and Michael Littman (55:03.120)
Well, they said specifically,
Lex Fridman (55:04.240)
so we make a change and we put it out in the world
Lex Fridman (55:05.880)
and that's bad because something terrible could happen.
Lex Fridman (55:07.800)
Like if you're putting things out in the world
Lex Fridman (55:09.560)
and you're not sure whether something terrible
Lex Fridman (55:11.100)
is going to happen, your process is probably.
Charles Isbell and Michael Littman (55:13.240)
I just feel like there should have been someone
Lex Fridman (55:14.760)
whose sole job it was to walk around and poke his head in
Lex Fridman (55:17.320)
and say, what could possibly go wrong?
Lex Fridman (55:19.160)
Just over and over again.
Charles Isbell and Michael Littman (55:20.660)
I would have loved if there was an,
Lex Fridman (55:22.080)
and I did watch a lot more and I'm not giving anything away.
Charles Isbell and Michael Littman (55:24.800)
I would have loved it if there was like an episode
Lex Fridman (55:27.080)
where like the new intern is like debugging
Charles Isbell and Michael Littman (55:29.960)
a new model or something and like it just keeps failing
Lex Fridman (55:32.720)
and they're like, all right.
Lex Fridman (55:34.160)
And then it's more turns into like a episode
Lex Fridman (55:36.840)
of Silicon Valley or something like that.
Charles Isbell and Michael Littman (55:38.320)
Yes.
Lex Fridman (55:39.320)
Versus like this ominous AI systems
Charles Isbell and Michael Littman (55:41.960)
that are constantly like threatening the fabric
Lex Fridman (55:45.640)
of this world that's been created.
Charles Isbell and Michael Littman (55:47.320)
Yeah.
Lex Fridman (55:48.160)
Yeah, and you know the other,
Charles Isbell and Michael Littman (55:49.360)
this reminds me of something that,
Lex Fridman (55:51.160)
so I agree that that should be very cool,
Charles Isbell and Michael Littman (55:52.600)
at least for the small percentage of people
Lex Fridman (55:54.760)
who care about debugging systems.
Lex Fridman (55:56.680)
But the other thing is.
Lex Fridman (55:57.520)
Right, debugging, the series.
Charles Isbell and Michael Littman (55:59.000)
Yeah, it falls into, think of the sequels,
Lex Fridman (56:01.400)
fear of the debugger.
Charles Isbell and Michael Littman (56:02.240)
Oh my gosh.
Lex Fridman (56:03.400)
Anyway, so.
Charles Isbell and Michael Littman (56:04.240)
It's a nightmare show, it's a horror movie.
Lex Fridman (56:07.400)
I think that's where we lose people, by the way,
Charles Isbell and Michael Littman (56:08.880)
early on is the people who either decide,
Lex Fridman (56:10.660)
either figure out debugging or think debugging is terrible.
Charles Isbell and Michael Littman (56:12.800)
This is where we lose people in computer science.
Lex Fridman (56:14.800)
This is a part of the struggle versus suffering, right?
Charles Isbell and Michael Littman (56:17.000)
You get through it and you kind of get the skills of it,
Lex Fridman (56:19.560)
or you're just like, this is dumb,
Lex Fridman (56:20.880)
and this is a dumb way to do anything.
Lex Fridman (56:22.040)
And I think that's when we lose people.
Charles Isbell and Michael Littman (56:23.480)
But, well, I'll leave it at that.
Lex Fridman (56:26.680)
But I think that there's something really, really neat
Charles Isbell and Michael Littman (56:33.040)
about framing it that way.
Lex Fridman (56:34.160)
But what I don't like about all of these things,
Lex Fridman (56:37.600)
and I love Tex Machina, by the way,
Lex Fridman (56:39.000)
although the ending was very depressing.
Charles Isbell and Michael Littman (56:42.940)
One of the things I have to talk to Alex about,
Lex Fridman (56:46.320)
he says that the thing that nobody noticed he put in
Charles Isbell and Michael Littman (56:49.920)
is at the end, spoiler alert,
Lex Fridman (56:53.880)
the robot turns and looks at the camera and smiles, briefly.
Lex Fridman (57:00.160)
And to him, he thought that his definition
Lex Fridman (57:04.680)
of passing the general version of the Turing test,
Charles Isbell and Michael Littman (57:08.480)
or the consciousness test, is smiling for no one.
Lex Fridman (57:17.960)
It's like the Chinese room kind of experiment.
Charles Isbell and Michael Littman (57:20.520)
It's not always trying to act for others,
Lex Fridman (57:22.760)
but just on your own, being able to have a relationship
Charles Isbell and Michael Littman (57:26.280)
with the actual experience and just take it in.
Lex Fridman (57:29.840)
I don't know, he said nobody noticed the magic of it.
Charles Isbell and Michael Littman (57:32.720)
I have this vague feeling that I remember the smile,
Lex Fridman (57:35.000)
but now you've just put the memory in my head,
Lex Fridman (57:37.080)
so probably not.
Lex Fridman (57:38.120)
But I do think that that's interesting.
Charles Isbell and Michael Littman (57:40.120)
Although, by looking at the camera,
Lex Fridman (57:41.920)
you are smiling for the audience, right?
Charles Isbell and Michael Littman (57:43.600)
You're breaking the fourth wall.
Lex Fridman (57:44.920)
It seems, I mean, well, that's a limitation of the medium.
Lex Fridman (57:48.160)
But I like that idea.
Lex Fridman (57:49.600)
But here's the problem I have with all of those movies,
Charles Isbell and Michael Littman (57:51.480)
all of them, is that, but I know why it's this way,
Lex Fridman (57:54.720)
and I enjoy those movies, and Westworld,
Charles Isbell and Michael Littman (57:57.480)
is it sets up the problem of AI as succeeding
Lex Fridman (58:02.960)
and then having something we cannot control.
Lex Fridman (58:05.400)
But it's not the bad part of AI.
Lex Fridman (58:08.400)
The bad part of AI is the stuff
Lex Fridman (58:10.020)
we're living through now, right?
Lex Fridman (58:11.120)
It's using the data to make decisions that are terrible.
Charles Isbell and Michael Littman (58:13.740)
It's not the intelligence that's gonna go out there
Lex Fridman (58:15.840)
and surpass us and take over the world
Charles Isbell and Michael Littman (58:17.960)
or lock us into a room to starve to death slowly
Lex Fridman (58:21.640)
over multiple days.
Charles Isbell and Michael Littman (58:22.680)
It's instead the tools that we're building
Lex Fridman (58:26.140)
that are allowing us to make the terrible decisions
Lex Fridman (58:30.320)
we would have less efficiently made before, right?
Lex Fridman (58:32.960)
Computers are very good at making us more efficient,
Charles Isbell and Michael Littman (58:35.680)
including being more efficient at doing terrible things.
Lex Fridman (58:38.160)
And that's the part of the AI we have to worry about.
Charles Isbell and Michael Littman (58:40.220)
It's not the true intelligence that we're gonna build
Lex Fridman (58:44.040)
sometime in the future, probably long after we're around.
Lex Fridman (58:48.080)
But I think that whole framing of it
Lex Fridman (58:52.120)
sort of misses the point, even though it is inspiring.
Lex Fridman (58:55.960)
And I was inspired by those ideas, right?
Lex Fridman (58:57.760)
I got into this in part
Charles Isbell and Michael Littman (58:59.040)
because I wanted to build something like that.
Lex Fridman (59:00.800)
Philosophical questions are interesting to me,
Lex Fridman (59:02.800)
but that's not where the terror comes from.
Lex Fridman (59:04.760)
The terror comes from the everyday.
Lex Fridman (59:06.160)
And you can construct situations
Lex Fridman (59:08.320)
in the subtlety of the interaction between AI and the human,
Charles Isbell and Michael Littman (59:11.720)
like with social networks,
Lex Fridman (59:14.120)
all the stuff you're doing
Charles Isbell and Michael Littman (59:15.160)
with interactive artificial intelligence.
Lex Fridman (59:17.800)
But I feel like Cal 9000 came a little bit closer to that
Charles Isbell and Michael Littman (59:22.600)
in 2001 Space Odyssey,
Lex Fridman (59:24.400)
because it felt like a personal assistant.
Charles Isbell and Michael Littman (59:29.000)
It felt like closer to the AI systems we have today.
Lex Fridman (59:31.360)
And the real things we might actually encounter,
Charles Isbell and Michael Littman (59:35.880)
which is over relying in some fundamental way
Lex Fridman (59:40.880)
on our dumb assistants or on social networks,
Charles Isbell and Michael Littman (59:44.860)
like over offloading too much of us
Lex Fridman (59:47.220)
onto things that require internet and power and so on
Lex Fridman (59:55.100)
and thereby becoming powerless as a standalone entity.
Lex Fridman (59:59.580)
And then when that thing starts to misbehave
🔗 相关节目