Michael Kearns: Algorithmic Fairness, Bias, Privacy, and Ethics in Machine Learning
AI 与机器学习技术与编程心理与人性音乐与艺术政治与社会
📋 章节目录
暂无章节信息
🔑 关键词
fairnessdataprivacydonlearningcomputeralgorithmgoingalgorithmicalgorithmsmachinegamesciencegroupmodeltheorydoinggroupsindividualbook
💬 精彩语录
暂无语录
🎙️ 完整对话（1546 条）
Lex Fridman (00:00.000)
The following is a conversation with Michael Kearns.
以下是与迈克尔·卡恩斯的对话。
Lex Fridman (00:03.120)
He's a professor at the University of Pennsylvania
他是宾夕法尼亚大学的教授
Lex Fridman (00:06.240)
and a coauthor of the new book, Ethical Algorithm,
以及新书《道德算法》的合著者，
Lex Fridman (00:09.520)
that is the focus of much of this conversation.
这是本次谈话的重点。
Lex Fridman (00:12.640)
It includes algorithmic fairness, bias, privacy,
它包括算法公平性、偏见、隐私、
Lex Fridman (00:16.640)
and ethics in general.
和一般道德。
Lex Fridman (00:18.080)
But that is just one of many fields
但这只是众多领域之一
Michael Kearns (00:20.000)
that Michael is a world class researcher in,
迈克尔是一位世界级的研究员，
Lex Fridman (00:22.480)
some of which we touch on quickly,
其中一些我们很快就会谈到，
Michael Kearns (00:24.880)
including learning theory
包括学习理论
Lex Fridman (00:26.240)
or the theoretical foundation of machine learning,
或机器学习的理论基础，
Michael Kearns (00:29.120)
game theory, quantitative finance,
博弈论、量化金融、
Lex Fridman (00:31.280)
computational social science, and much more.
计算社会科学等等。
Lex Fridman (00:34.000)
But on a personal note,
但就个人而言，
Lex Fridman (00:35.600)
when I was an undergrad, early on,
当我还是个本科生的时候，很早的时候，
Michael Kearns (00:38.320)
I worked with Michael
我和迈克尔一起工作过
Lex Fridman (00:39.520)
on an algorithmic trading project
算法交易项目
Lex Fridman (00:41.280)
and competition that he led.
以及他领导的竞争。
Lex Fridman (00:43.120)
That's when I first fell in love
那是我第一次恋爱的时候
Michael Kearns (00:44.720)
with algorithmic game theory.
与算法博弈论。
Lex Fridman (00:46.800)
While most of my research life
Michael Kearns (00:48.400)
has been in machine learning
Lex Fridman (00:49.520)
and human robot interaction,
Michael Kearns (00:51.440)
the systematic way that game theory
Lex Fridman (00:53.600)
reveals the beautiful structure
Michael Kearns (00:55.120)
in our competitive and cooperating world of humans
Lex Fridman (00:58.480)
has been a continued inspiration to me.
Lex Fridman (01:01.120)
So for that and other things,
Lex Fridman (01:03.520)
I'm deeply thankful to Michael
Lex Fridman (01:05.600)
and really enjoyed having this conversation
Lex Fridman (01:07.920)
again in person after so many years.
Michael Kearns (01:11.040)
This is the Artificial Intelligence Podcast.
Lex Fridman (01:13.760)
If you enjoy it, subscribe on YouTube,
Michael Kearns (01:16.080)
give it five stars on Apple Podcast,
Lex Fridman (01:18.320)
support on Patreon,
Michael Kearns (01:19.760)
or simply connect with me on Twitter
Lex Fridman (01:21.680)
at Lex Friedman, spelled F R I D M A N.
Michael Kearns (01:25.760)
This episode is supported
Lex Fridman (01:27.280)
by an amazing podcast called Pessimists Archive.
Michael Kearns (01:31.040)
Jason, the host of the show,
Lex Fridman (01:32.720)
reached out to me looking to support this podcast,
Lex Fridman (01:35.360)
and so I listened to it, to check it out.
Lex Fridman (01:38.160)
And by listened, I mean I went through it,
Michael Kearns (01:40.720)
Netflix binge style, at least five episodes in a row.
Lex Fridman (01:44.800)
It's not one of my favorite podcasts,
Lex Fridman (01:46.480)
and I think it should be one of the top podcasts
Lex Fridman (01:48.720)
in the world, frankly.
Michael Kearns (01:50.480)
It's a history show
Lex Fridman (01:51.760)
about why people resist new things.
Michael Kearns (01:54.080)
Each episode looks at a moment in history
Lex Fridman (01:56.400)
when something new was introduced,
Michael Kearns (01:59.040)
something that today we think of as commonplace,
Lex Fridman (02:01.680)
like recorded music, umbrellas, bicycles, cars,
Michael Kearns (02:04.560)
chess, coffee, the elevator,
Lex Fridman (02:06.960)
and the show explores why it freaked everyone out.
Michael Kearns (02:10.160)
The latest episode on mirrors and vanity
Lex Fridman (02:12.800)
still stays with me as I think about vanity
Michael Kearns (02:15.360)
in the modern day of the Twitter world.
Lex Fridman (02:18.640)
That's the fascinating thing about the show,
Michael Kearns (02:20.880)
is that stuff that happened long ago,
Lex Fridman (02:22.640)
especially in terms of our fear of new things,
Michael Kearns (02:25.040)
repeats itself in the modern day,
Lex Fridman (02:26.880)
and so has many lessons for us to think about
Michael Kearns (02:29.280)
in terms of human psychology
Lex Fridman (02:31.040)
and the role of technology in our society.
Michael Kearns (02:34.160)
Anyway, you should subscribe
Lex Fridman (02:35.600)
and listen to Pessimist Archive.
Michael Kearns (02:37.760)
I highly recommend it.
Lex Fridman (02:39.840)
And now, here's my conversation with Michael Kearns.
Michael Kearns (02:44.880)
You mentioned reading Fear and Loathing in Las Vegas
Lex Fridman (02:47.840)
in high school, and having a more,
Michael Kearns (02:50.800)
or a bit more of a literary mind.
Lex Fridman (02:52.560)
So, what books, non technical, non computer science,
Michael Kearns (02:56.880)
would you say had the biggest impact on your life,
Lex Fridman (02:59.360)
either intellectually or emotionally?
Michael Kearns (03:02.400)
You've dug deep into my history, I see.
Lex Fridman (03:04.640)
Went deep.
Michael Kearns (03:05.680)
Yeah, I think, well, my favorite novel is
Lex Fridman (03:08.240)
Infinite Jest by David Foster Wallace,
Michael Kearns (03:10.960)
which actually, coincidentally,
Lex Fridman (03:13.360)
much of it takes place in the halls of buildings
Michael Kearns (03:15.840)
right around us here at MIT.
Lex Fridman (03:18.000)
So that certainly had a big influence on me.
Lex Fridman (03:20.000)
And as you noticed, like, when I was in high school,
Lex Fridman (03:22.320)
I actually even started college as an English major.
Michael Kearns (03:25.440)
So, I was very influenced by sort of that genre of journalism
Lex Fridman (03:29.120)
at the time, and thought I wanted to be a writer,
Lex Fridman (03:30.880)
and then realized that an English major teaches you to read,
Lex Fridman (03:33.840)
but it doesn't teach you how to write,
Lex Fridman (03:35.360)
and then I became interested in math
Lex Fridman (03:36.960)
and computer science instead.
Michael Kearns (03:38.560)
Well, in your new book, Ethical Algorithm,
Lex Fridman (03:41.600)
you kind of sneak up from an algorithmic perspective
Michael Kearns (03:45.920)
on these deep, profound philosophical questions
Lex Fridman (03:48.400)
of fairness, of privacy.
Michael Kearns (03:55.120)
In thinking about these topics,
Lex Fridman (03:56.400)
how often do you return to that literary mind that you had?
Michael Kearns (04:01.440)
Yeah, I'd like to claim there was a deeper connection,
Lex Fridman (04:05.200)
but, you know, I think both Aaron and I
Michael Kearns (04:07.920)
kind of came at these topics first and foremost
Lex Fridman (04:10.320)
from a technical angle.
Michael Kearns (04:11.280)
I mean, you know, I kind of consider myself primarily
Lex Fridman (04:14.720)
and originally a machine learning researcher,
Lex Fridman (04:17.600)
and I think as we just watched, like the rest of the society,
Lex Fridman (04:20.400)
the field technically advance, and then quickly on the heels
Michael Kearns (04:23.440)
of that kind of the buzzkill of all of the antisocial behavior
Lex Fridman (04:27.120)
by algorithms, just kind of realized
Michael Kearns (04:29.360)
there was an opportunity for us to do something about it
Lex Fridman (04:31.840)
from a research perspective.
Michael Kearns (04:34.000)
You know, more to the point in your question,
Lex Fridman (04:36.320)
I mean, I do have an uncle who is literally a moral philosopher,
Lex Fridman (04:41.600)
and so in the early days of my life,
Lex Fridman (04:43.280)
he was a philosopher, and so in the early days
Michael Kearns (04:46.000)
of our technical work on fairness topics,
Lex Fridman (04:48.800)
I would occasionally, you know, run ideas behind him.
Michael Kearns (04:51.280)
So, I mean, I remember an early email I sent to him
Lex Fridman (04:53.440)
in which I said, like, oh, you know,
Michael Kearns (04:55.040)
here's a specific definition of algorithmic fairness
Lex Fridman (04:57.840)
that we think is some sort of variant of Rawlsian fairness.
Lex Fridman (05:02.320)
What do you think?
Lex Fridman (05:03.920)
And I thought I was asking a yes or no question,
Lex Fridman (05:06.880)
and I got back your kind of classical philosopher's
Lex Fridman (05:09.120)
response saying, well, it depends.
Michael Kearns (05:10.800)
Hey, then you might conclude this, and that's when I realized
Lex Fridman (05:14.400)
that there was a real kind of rift between the ways
Michael Kearns (05:19.680)
philosophers and others had thought about things
Lex Fridman (05:21.840)
like fairness, you know, from sort of a humanitarian perspective
Lex Fridman (05:25.680)
and the way that you needed to think about it
Lex Fridman (05:27.840)
as a computer scientist if you were going to kind of
Michael Kearns (05:30.720)
implement actual algorithmic solutions.
Lex Fridman (05:34.080)
But I would say the algorithmic solutions take care
Michael Kearns (05:39.200)
of some of the low hanging fruit.
Lex Fridman (05:41.360)
Sort of the problem is a lot of algorithms,
Michael Kearns (05:44.560)
when they don't consider fairness,
Lex Fridman (05:47.280)
they are just terribly unfair.
Lex Fridman (05:50.240)
And when they don't consider privacy,
Lex Fridman (05:51.680)
they're terribly, they violate privacy.
Michael Kearns (05:55.200)
Sort of the algorithmic approach fixes big problems.
Lex Fridman (05:59.680)
But there's still, when you start pushing into the gray area,
Michael Kearns (06:04.000)
that's when you start getting into this philosophy
Lex Fridman (06:06.240)
of what it means to be fair, starting from Plato,
Lex Fridman (06:09.680)
what is justice kind of questions?
Lex Fridman (06:12.400)
Yeah, I think that's right.
Lex Fridman (06:13.280)
And I mean, I would even not go as far as you want to say
Lex Fridman (06:16.960)
that sort of the algorithmic work in these areas
Michael Kearns (06:19.520)
is solving like the biggest problems.
Lex Fridman (06:22.720)
And, you know, we discuss in the book,
Michael Kearns (06:24.320)
the fact that really we are, there's a sense in which
Lex Fridman (06:27.520)
we're kind of looking where the light is in that,
Michael Kearns (06:30.640)
you know, for example, if police are racist
Lex Fridman (06:34.880)
in who they decide to stop and frisk,
Lex Fridman (06:37.760)
and that goes into the data,
Lex Fridman (06:39.120)
there's sort of no undoing that downstream
Michael Kearns (06:41.520)
by kind of clever algorithmic methods.
Lex Fridman (06:45.760)
And I think, especially in fairness,
Michael Kearns (06:47.520)
I mean, I think less so in privacy,
Lex Fridman (06:49.840)
where we feel like the community kind of really has settled
Michael Kearns (06:52.880)
on the right definition, which is differential privacy.
Lex Fridman (06:56.320)
If you just look at the algorithmic fairness literature
Michael Kearns (06:58.960)
already, you can see it's going to be much more of a mess.
Lex Fridman (07:01.600)
And, you know, you've got these theorems saying,
Michael Kearns (07:03.360)
here are three entirely reasonable,
Lex Fridman (07:06.640)
desirable notions of fairness.
Michael Kearns (07:09.760)
And, you know, here's a proof that you cannot simultaneously
Lex Fridman (07:12.720)
have all three of them.
Lex Fridman (07:14.560)
So I think we know that algorithmic fairness
Lex Fridman (07:17.680)
compared to algorithmic privacy
Michael Kearns (07:19.280)
is going to be kind of a harder problem.
Lex Fridman (07:21.600)
And it will have to revisit, I think,
Michael Kearns (07:23.440)
things that have been thought about by,
Lex Fridman (07:26.080)
you know, many generations of scholars before us.
Lex Fridman (07:29.520)
So it's very early days for fairness, I think.
Lex Fridman (07:32.240)
TK So before we get into the details
Michael Kearns (07:34.560)
of differential privacy, and on the fairness side,
Lex Fridman (07:37.280)
let me linger on the philosophy a bit.
Lex Fridman (07:39.520)
Do you think most people are fundamentally good?
Lex Fridman (07:43.600)
Or do most of us have both the capacity
Lex Fridman (07:46.640)
for good and evil within us?
Lex Fridman (07:48.320)
SB I mean, I'm an optimist.
Michael Kearns (07:50.240)
I tend to think that most people are good
Lex Fridman (07:52.480)
and want to do right.
Lex Fridman (07:55.600)
And that deviations from that are, you know,
Lex Fridman (07:58.480)
kind of usually due to circumstance,
Michael Kearns (08:00.240)
not due to people being bad at heart.
Lex Fridman (08:02.960)
TK With people with power,
Michael Kearns (08:05.520)
are people at the heads of governments,
Lex Fridman (08:08.160)
people at the heads of companies,
Michael Kearns (08:10.320)
people at the heads of, maybe, so financial power markets,
Lex Fridman (08:15.040)
do you think the distribution there is also,
Lex Fridman (08:19.040)
most people are good and have good intent?
Lex Fridman (08:21.040)
SB Yeah, I do.
Michael Kearns (08:22.800)
I mean, my statement wasn't qualified to people
Lex Fridman (08:26.560)
not in positions of power.
Michael Kearns (08:28.560)
I mean, I think what happens in a lot of the, you know,
Lex Fridman (08:30.640)
the cliche about absolute power corrupts absolutely.
Michael Kearns (08:34.080)
I mean, you know, I think even short of that,
Lex Fridman (08:37.680)
you know, having spent a lot of time on Wall Street,
Lex Fridman (08:40.720)
and also in arenas very, very different from Wall Street,
Lex Fridman (08:44.160)
like academia, you know, one of the things
Michael Kearns (08:47.920)
I think I've benefited from by moving between
Lex Fridman (08:50.960)
two very different worlds is you become aware
Michael Kearns (08:53.760)
that, you know, these worlds kind of develop
Lex Fridman (08:57.120)
their own social norms, and they develop
Michael Kearns (08:59.040)
their own rationales for, you know,
Lex Fridman (09:02.080)
behavior, for instance, that might look
Michael Kearns (09:03.920)
unusual to outsiders.
Lex Fridman (09:05.280)
But when you're in that world,
Michael Kearns (09:07.120)
it doesn't feel unusual at all.
Lex Fridman (09:09.600)
And I think this is true of a lot of,
Michael Kearns (09:11.680)
you know, professional cultures, for instance.
Lex Fridman (09:15.440)
And, you know, so then your maybe slippery slope
Michael Kearns (09:18.480)
is too strong of a word.
Lex Fridman (09:19.600)
But, you know, you're in some world
Michael Kearns (09:21.040)
where you're mainly around other people
Lex Fridman (09:23.200)
with the same kind of viewpoints and training
Lex Fridman (09:25.520)
and worldview as you.
Lex Fridman (09:27.360)
And I think that's more of a source of,
Michael Kearns (09:30.640)
of, you know, kind of abuses of power
Lex Fridman (09:34.560)
than sort of, you know, there being good people
Lex Fridman (09:36.720)
and evil people, and that somehow the evil people
Lex Fridman (09:40.640)
are the ones that somehow rise to power.
Michael Kearns (09:43.040)
Oh, that's really interesting.
Lex Fridman (09:44.160)
So it's the, within the social norms
Michael Kearns (09:46.880)
constructed by that particular group of people,
Lex Fridman (09:50.400)
you're all trying to do good.
Lex Fridman (09:52.720)
But because as a group, you might be,
Lex Fridman (09:54.640)
you might drift into something
Michael Kearns (09:56.080)
that for the broader population,
Lex Fridman (09:58.160)
it does not align with the values of society.
Michael Kearns (10:00.480)
That kind of, that's the word.
Lex Fridman (10:01.680)
Yeah, I mean, or not that you drift,
Lex Fridman (10:03.520)
but even the things that don't make sense
Lex Fridman (10:07.440)
to the outside world don't seem unusual to you.
Lex Fridman (10:11.280)
So it's not sort of like a good or a bad thing,
Lex Fridman (10:13.360)
but, you know, like, so for instance,
Lex Fridman (10:14.800)
you know, on, in the world of finance, right?
Lex Fridman (10:18.160)
There's a lot of complicated types of activity
Michael Kearns (10:21.280)
that if you are not immersed in that world,
Lex Fridman (10:22.960)
you cannot see why the purpose of that,
Michael Kearns (10:25.760)
you know, that activity exists at all.
Lex Fridman (10:27.440)
It just seems like, you know, completely useless
Lex Fridman (10:30.640)
and people just like, you know, pushing money around.
Lex Fridman (10:33.440)
And when you're in that world, right,
Michael Kearns (10:34.800)
you're, and you learn more,
Lex Fridman (10:36.640)
your view does become more nuanced, right?
Michael Kearns (10:39.600)
You realize, okay, there is actually a function
Lex Fridman (10:41.680)
to this activity.
Lex Fridman (10:43.840)
And in some cases, you would conclude that actually,
Lex Fridman (10:46.640)
if magically we could eradicate this activity tomorrow,
Michael Kearns (10:50.240)
it would come back because it actually is like
Lex Fridman (10:52.720)
serving some useful purpose.
Michael Kearns (10:54.400)
It's just a useful purpose that's very difficult
Lex Fridman (10:56.880)
for outsiders to see.
Lex Fridman (10:59.200)
And so I think, you know, lots of professional work
Lex Fridman (11:02.720)
environments or cultures, as I might put it,
Michael Kearns (11:06.320)
kind of have these social norms that, you know,
Lex Fridman (11:08.800)
don't make sense to the outside world.
Lex Fridman (11:10.160)
Academia is the same, right?
Lex Fridman (11:11.280)
I mean, lots of people look at academia and say,
Lex Fridman (11:13.600)
you know, what the hell are all of you people doing?
Lex Fridman (11:16.480)
Why are you paid so much in some cases
Michael Kearns (11:18.960)
at taxpayer expenses to do, you know,
Lex Fridman (11:21.840)
to publish papers that nobody reads?
Michael Kearns (11:24.400)
You know, but when you're in that world,
Lex Fridman (11:25.920)
you come to see the value for it.
Michael Kearns (11:27.680)
And, but even though you might not be able to explain it
Lex Fridman (11:30.240)
to, you know, the person in the street.
Michael Kearns (11:33.040)
Right.
Lex Fridman (11:33.360)
And in the case of the financial sector,
Michael Kearns (11:36.000)
tools like credit might not make sense to people.
Lex Fridman (11:39.200)
Like, it's a good example of something that does seem
Michael Kearns (11:41.600)
to pop up and be useful or just the power of markets
Lex Fridman (11:45.120)
and just in general capitalism.
Michael Kearns (11:47.120)
Yeah.
Lex Fridman (11:47.360)
In finance, I think the primary example
Lex Fridman (11:49.360)
I would give is leverage, right?
Lex Fridman (11:51.040)
So being allowed to borrow, to sort of use ten times
Lex Fridman (11:56.320)
as much money as you've actually borrowed, right?
Lex Fridman (11:58.480)
So that's an example of something that before I had
Michael Kearns (12:00.720)
any experience in financial markets,
Lex Fridman (12:02.320)
I might have looked at and said,
Lex Fridman (12:03.440)
well, what is the purpose of that?
Lex Fridman (12:05.280)
That just seems very dangerous and it is dangerous
Lex Fridman (12:08.400)
and it has proven dangerous.
Lex Fridman (12:10.480)
But, you know, if the fact of the matter is that,
Michael Kearns (12:13.280)
you know, sort of on some particular time scale,
Lex Fridman (12:16.560)
you are holding positions that are,
Michael Kearns (12:19.680)
you know, very unlikely to, you know,
Lex Fridman (12:23.040)
lose, you know, your value at risk or variance
Michael Kearns (12:26.800)
is like one or five percent, then it kind of makes sense
Lex Fridman (12:30.320)
that you would be allowed to use a little bit more
Michael Kearns (12:32.160)
than you have because you have, you know,
Lex Fridman (12:35.120)
some confidence that you're not going to lose
Michael Kearns (12:37.760)
it all in a single day.
Lex Fridman (12:39.840)
Now, of course, when that happens,
Michael Kearns (12:42.960)
we've seen what happens, you know, not too long ago.
Lex Fridman (12:45.920)
But, you know, but the idea that it serves
Michael Kearns (12:48.800)
no useful economic purpose under any circumstances
Lex Fridman (12:52.800)
is definitely not true.
Michael Kearns (12:54.800)
We'll return to the other side of the coast,
Lex Fridman (12:57.680)
Silicon Valley, and the problems there as we talk about privacy,
Michael Kearns (13:02.560)
as we talk about fairness.
Lex Fridman (13:05.360)
At the high level, and I'll ask some sort of basic questions
Michael Kearns (13:09.360)
with the hope to get at the fundamental nature of reality.
Lex Fridman (13:12.560)
But from a very high level, what is an ethical algorithm?
Lex Fridman (13:18.160)
So I can say that an algorithm has a running time
Lex Fridman (13:20.960)
of using big O notation n log n.
Michael Kearns (13:24.400)
I can say that a machine learning algorithm
Lex Fridman (13:27.760)
classified cat versus dog with 97 percent accuracy.
Lex Fridman (13:31.440)
Do you think there will one day be a way to measure
Lex Fridman (13:36.320)
sort of in the same compelling way as the big O notation
Lex Fridman (13:39.920)
of this algorithm is 97 percent ethical?
Lex Fridman (13:44.000)
First of all, let me riff for a second on your specific n log n example.
Lex Fridman (13:48.800)
So because early in the book when we're just kind of trying to describe
Lex Fridman (13:51.920)
algorithms period, we say like, okay, you know,
Lex Fridman (13:54.640)
what's an example of an algorithm or an algorithmic problem?
Lex Fridman (13:58.560)
First of all, like it's sorting, right?
Michael Kearns (14:00.160)
You have a bunch of index cards with numbers on them
Lex Fridman (14:02.240)
and you want to sort them.
Lex Fridman (14:03.760)
And we describe, you know, an algorithm that sweeps all the way through,
Lex Fridman (14:07.360)
finds the smallest number, puts it at the front,
Michael Kearns (14:09.680)
then sweeps through again, finds the second smallest number.
Lex Fridman (14:12.640)
So we make the point that this is an algorithm
Lex Fridman (14:14.800)
and it's also a bad algorithm in the sense that, you know,
Lex Fridman (14:17.760)
it's quadratic rather than n log n,
Michael Kearns (14:20.640)
which we know is kind of optimal for sorting.
Lex Fridman (14:23.920)
And we make the point that sort of like, you know,
Lex Fridman (14:26.080)
so even within the confines of a very precisely specified problem,
Lex Fridman (14:31.680)
there, you know, there might be many, many different algorithms
Michael Kearns (14:35.200)
for the same problem with different properties.
Lex Fridman (14:37.520)
Like some might be faster in terms of running time,
Michael Kearns (14:40.400)
some might use less memory, some might have, you know,
Lex Fridman (14:43.520)
better distributed implementations.
Lex Fridman (14:46.240)
And so the point is that already we're used to, you know,
Lex Fridman (14:50.560)
in computer science thinking about trade offs
Michael Kearns (14:53.520)
between different types of quantities and resources
Lex Fridman (14:56.800)
and there being, you know, better and worse algorithms.
Lex Fridman (15:00.960)
And our book is about that part of algorithmic ethics
Michael Kearns (15:08.480)
that we know how to kind of put on that same kind of quantitative footing right now.
Michael Kearns (15:13.520)
So, you know, just to say something that our book is not about,
Lex Fridman (15:17.440)
our book is not about kind of broad, fuzzy notions of fairness.
Michael Kearns (15:22.400)
It's about very specific notions of fairness.
Lex Fridman (15:25.840)
There's more than one of them.
Lex Fridman (15:28.240)
There are tensions between them, right?
Lex Fridman (15:30.880)
But if you pick one of them, you can do something akin to saying
Michael Kearns (15:35.680)
that this algorithm is 97% ethical.
Lex Fridman (15:39.200)
You can say, for instance, the, you know, for this lending model,
Lex Fridman (15:44.080)
the false rejection rate on black people and white people is within 3%, right?
Lex Fridman (15:51.040)
So we might call that a 97% ethical algorithm and a 100% ethical algorithm
Michael Kearns (15:57.040)
would mean that that difference is 0%.
Lex Fridman (15:59.920)
In that case, fairness is specified when two groups, however,
Michael Kearns (16:04.640)
they're defined are given to you.
Lex Fridman (16:06.720)
That's right.
Lex Fridman (16:07.280)
So the, and then you can sort of mathematically start describing the algorithm.
Lex Fridman (16:11.760)
But nevertheless, the part where the two groups are given to you,
Michael Kearns (16:20.080)
I mean, unlike running time, you know, we don't in computer science
Lex Fridman (16:24.480)
talk about how fast an algorithm feels like when it runs.
Michael Kearns (16:29.200)
True.
Lex Fridman (16:29.760)
We measure it and ethical starts getting into feelings.
Michael Kearns (16:33.040)
So, for example, an algorithm runs, you know, if it runs in the background,
Lex Fridman (16:38.160)
it doesn't disturb the performance of my system.
Michael Kearns (16:40.480)
It'll feel nice.
Lex Fridman (16:41.600)
I'll be okay with it.
Lex Fridman (16:42.560)
But if it overloads the system, it'll feel unpleasant.
Lex Fridman (16:45.280)
So in that same way, ethics, there's a feeling of how socially acceptable it is.
Lex Fridman (16:50.320)
How does it represent the moral standards of our society today?
Lex Fridman (16:55.200)
So in that sense, and sorry to linger on that first of high,
Michael Kearns (16:59.040)
low philosophical questions.
Lex Fridman (17:00.640)
Do you have a sense we'll be able to measure how ethical an algorithm is?
Michael Kearns (17:05.920)
First of all, I didn't, certainly didn't mean to give the impression that you can kind of
Michael Kearns (17:09.680)
measure, you know, memory speed trade offs, you know, and that there's a complete mapping from
Michael Kearns (17:16.320)
that onto kind of fairness, for instance, or ethics and accuracy, for example.
Michael Kearns (17:22.880)
In the type of fairness definitions that are largely the objects of study today and starting
Michael Kearns (17:28.960)
to be deployed, you as the user of the definitions, you need to make some hard decisions before you
Lex Fridman (17:35.360)
even get to the point of designing fair algorithms.
Michael Kearns (17:40.240)
One of them, for instance, is deciding who it is that you're worried about protecting,
Michael Kearns (17:45.840)
who you're worried about being harmed by, for instance, some notion of discrimination or
Michael Kearns (17:50.560)
unfairness.
Lex Fridman (17:52.160)
And then you need to also decide what constitutes harm.
Michael Kearns (17:55.520)
So, for instance, in a lending application, maybe you decide that, you know, falsely rejecting
Michael Kearns (18:02.320)
a creditworthy individual, you know, sort of a false negative, is the real harm and that false
Michael Kearns (18:08.960)
positives, i.e. people that are not creditworthy or are not gonna repay your loan, that get a loan,
Lex Fridman (18:14.560)
you might think of them as lucky.
Lex Fridman (18:17.120)
And so that's not a harm, although it's not clear that if you don't have the means to repay a loan,
Lex Fridman (18:22.720)
that being given a loan is not also a harm.
Michael Kearns (18:26.880)
So, you know, the literature is sort of so far quite limited in that you sort of need to say,
Lex Fridman (18:33.600)
who do you want to protect and what would constitute harm to that group?
Lex Fridman (18:37.920)
And when you ask questions like, will algorithms feel ethical?
Michael Kearns (18:42.080)
One way in which they won't, under the definitions that I'm describing, is if, you know, if you are
Michael Kearns (18:47.440)
an individual who is falsely denied a loan, incorrectly denied a loan, all of these definitions
Michael Kearns (18:54.320)
basically say like, well, you know, your compensation is the knowledge that we are also
Michael Kearns (19:00.240)
falsely denying loans to other people, you know, in other groups at the same rate that we're doing
Lex Fridman (19:05.120)
it to you.
Michael Kearns (19:05.680)
And, you know, and so there is actually this interesting even technical tension in the field
Michael Kearns (19:12.800)
right now between these sort of group notions of fairness and notions of fairness that might
Lex Fridman (19:18.400)
actually feel like real fairness to individuals, right?
Michael Kearns (19:22.160)
They might really feel like their particular interests are being protected or thought about
Michael Kearns (19:27.360)
by the algorithm rather than just, you know, the groups that they happen to be members of.
Lex Fridman (19:33.360)
Is there parallels to the big O notation of worst case analysis?
Lex Fridman (19:37.920)
So, is it important to looking at the worst violation of fairness for an individual?
Lex Fridman (19:45.760)
Is it important to minimize that one individual?
Lex Fridman (19:48.080)
So like worst case analysis, is that something you think about or?
Michael Kearns (19:52.320)
I mean, I think we're not even at the point where we can sensibly think about that.
Lex Fridman (19:56.960)
So first of all, you know, we're talking here both about fairness applied at the group level,
Lex Fridman (1:00:05.160)
I definitely think, you know, this is a much narrower statement.
Michael Kearns (1:00:08.560)
I definitely think that kind of algorithmic auditing for different types of unfairness,
Lex Fridman (1:00:12.440)
right?
Lex Fridman (1:00:13.440)
So like in this gerrymandering example where I might want to prevent not just discrimination
Lex Fridman (1:00:18.680)
against very broad categories, but against combinations of broad categories.
Michael Kearns (1:00:23.960)
You know, you quickly get to a point where there's a lot of, a lot of categories.
Michael Kearns (1:00:27.700)
There's a lot of combinations of end features and, you know, you can use algorithmic techniques
Michael Kearns (1:00:33.520)
to sort of try to find the subgroups on which you're discriminating the most and try to
Lex Fridman (1:00:38.000)
fix that.
Michael Kearns (1:00:39.000)
That's actually kind of the form of one of the algorithms we developed for this fairness
Lex Fridman (1:00:42.460)
gerrymandering problem.
Lex Fridman (1:00:44.240)
But I'm, I'm, you know, partly because of our technological, you know, our sort of our
Lex Fridman (1:00:49.440)
scientific ignorance on these topics right now.
Lex Fridman (1:00:53.400)
And also partly just because these topics are so loaded emotionally for people that
Lex Fridman (1:00:58.360)
I just don't see the value.
Michael Kearns (1:01:00.440)
I mean, again, never say never, but I just don't think we're at a moment where it's
Michael Kearns (1:01:03.920)
a great time for computer scientists to be rolling out the idea like, hey, you know,
Michael Kearns (1:01:08.600)
you know, not only have we kind of figured fairness out, but, you know, we think the
Lex Fridman (1:01:12.520)
algorithm should start deciding what's fair or giving input on that decision.
Michael Kearns (1:01:16.880)
I just don't, it's like the cost benefit analysis to the field of kind of going there
Lex Fridman (1:01:22.080)
right now just doesn't seem worth it to me.
Michael Kearns (1:01:24.520)
That said, I should say that I think computer scientists should be more philosophically,
Lex Fridman (1:01:29.200)
like should enrich their thinking about these kinds of things.
Michael Kearns (1:01:32.280)
I think it's been too often used as an excuse for roboticists working on autonomous vehicles,
Michael Kearns (1:01:38.020)
for example, to not think about the human factor or psychology or safety in the same
Michael Kearns (1:01:43.720)
way like computer science design algorithms that have been sort of using it as an excuse.
Lex Fridman (1:01:47.440)
And I think it's time for basically everybody to become a computer scientist.
Michael Kearns (1:01:51.640)
I was about to agree with everything you said except that last point.
Michael Kearns (1:01:54.440)
I think that the other way of looking at it is that I think computer scientists, you know,
Lex Fridman (1:01:59.760)
and many of us are, but we need to weigh it out into the world more, right?
Michael Kearns (1:02:06.120)
I mean, just the influence that computer science and therefore computer scientists have had
Michael Kearns (1:02:12.520)
on society at large just like has exponentially magnified in the last 10 or 20 years or so.
Lex Fridman (1:02:21.520)
And you know, before when we were just tinkering around amongst ourselves and it didn't matter
Michael Kearns (1:02:26.560)
that much, there was no need for sort of computer scientists to be citizens of the world more
Lex Fridman (1:02:32.360)
broadly.
Lex Fridman (1:02:33.440)
And I think those days need to be over very, very fast.
Lex Fridman (1:02:36.760)
And I'm not saying everybody needs to do it, but to me, like the right way of doing it
Michael Kearns (1:02:40.720)
is to not to sort of think that everybody else is going to become a computer scientist.
Lex Fridman (1:02:44.120)
But you know, I think people are becoming more sophisticated about computer science,
Michael Kearns (1:02:49.200)
even lay people.
Michael Kearns (1:02:50.200)
You know, I think one of the reasons we decided to write this book is we thought 10 years
Michael Kearns (1:02:55.520)
ago I wouldn't have tried this just because I just didn't think that sort of people's
Michael Kearns (1:03:00.400)
awareness of algorithms and machine learning, you know, the general population would have
Michael Kearns (1:03:06.240)
been high.
Michael Kearns (1:03:07.240)
I mean, you would have had to first, you know, write one of the many books kind of just explicating
Michael Kearns (1:03:12.060)
that topic to a lay audience first.
Michael Kearns (1:03:14.720)
Now I think we're at the point where like lots of people without any technical training
Michael Kearns (1:03:18.900)
at all know enough about algorithms and machine learning that you can start getting to these
Lex Fridman (1:03:22.800)
nuances of things like ethical algorithms.
Michael Kearns (1:03:26.000)
I think we agree that there needs to be much more mixing, but I think a lot of the onus
Lex Fridman (1:03:31.780)
of that mixing needs to be on the computer science community.
Michael Kearns (1:03:35.360)
Yeah.
Lex Fridman (1:03:36.360)
So just to linger on the disagreement, because I do disagree with you on the point that I
Michael Kearns (1:03:41.920)
think if you're a biologist, if you're a chemist, if you're an MBA business person, all of those
Michael Kearns (1:03:50.780)
things you can, like if you learned a program, and not only program, if you learned to do
Michael Kearns (1:03:57.160)
machine learning, if you learned to do data science, you immediately become much more
Lex Fridman (1:04:02.160)
powerful in the kinds of things you can do.
Lex Fridman (1:04:04.200)
And therefore literature, like library sciences, like, so you were speaking, I think, I think
Lex Fridman (1:04:11.600)
it holds true what you're saying for the next few years.
Lex Fridman (1:04:14.760)
But long term, if you're interested to me, if you're interested in philosophy, you should
Michael Kearns (1:04:21.520)
learn a program, because then you can scrape data and study what people are thinking about
Michael Kearns (1:04:27.700)
on Twitter, and then start making philosophical conclusions about the meaning of life.
Michael Kearns (1:04:33.760)
I just feel like the access to data, the digitization of whatever problem you're trying to solve,
Michael Kearns (1:04:41.440)
will fundamentally change what it means to be a computer scientist.
Michael Kearns (1:04:44.200)
I mean, a computer scientist in 20, 30 years will go back to being Donald Knuth style theoretical
Michael Kearns (1:04:51.200)
computer science, and everybody would be doing basically, exploring the kinds of ideas that
Lex Fridman (1:04:56.560)
you explore in your book.
Michael Kearns (1:04:57.560)
It won't be a computer science major.
Michael Kearns (1:04:58.880)
Yeah, I mean, I don't think I disagree enough, but I think that that trend of more and more
Michael Kearns (1:05:05.000)
people in more and more disciplines adopting ideas from computer science, learning how
Lex Fridman (1:05:11.600)
to code, I think that that trend seems firmly underway.
Michael Kearns (1:05:14.560)
I mean, you know, like an interesting digressive question along these lines is maybe in 50
Michael Kearns (1:05:21.000)
years, there won't be computer science departments anymore, because the field will just sort
Michael Kearns (1:05:27.080)
of be ambient in all of the different disciplines.
Lex Fridman (1:05:30.840)
And people will look back and having a computer science department will look like having an
Michael Kearns (1:05:35.720)
electricity department or something that's like, you know, everybody uses this, it's
Lex Fridman (1:05:39.480)
just out there.
Michael Kearns (1:05:40.480)
I mean, I do think there will always be that kind of Knuth style core to it, but it's not
Michael Kearns (1:05:45.180)
an implausible path that we kind of get to the point where the academic discipline of
Michael Kearns (1:05:50.180)
computer science becomes somewhat marginalized because of its very success in kind of infiltrating
Lex Fridman (1:05:56.160)
all of science and society and the humanities, etcetera.
Lex Fridman (1:06:00.720)
What is differential privacy, or more broadly, algorithmic privacy?
Michael Kearns (1:06:07.720)
Algorithmic privacy more broadly is just the study or the notion of privacy definitions
Michael Kearns (1:06:15.040)
or norms being encoded inside of algorithms.
Lex Fridman (1:06:19.580)
And so, you know, I think we count among this body of work just, you know, the literature
Lex Fridman (1:06:27.520)
and practice of things like data anonymization, which we kind of at the beginning of our discussion
Lex Fridman (1:06:33.980)
of privacy say like, okay, this is sort of a notion of algorithmic privacy.
Michael Kearns (1:06:38.600)
It kind of tells you, you know, something to go do with data, but, you know, our view
Michael Kearns (1:06:44.840)
is that it's, and I think this is now, you know, quite widespread, that it's, you know,
Michael Kearns (1:06:50.120)
despite the fact that those notions of anonymization kind of redacting and coarsening are the most
Michael Kearns (1:06:57.320)
widely adopted technical solutions for data privacy, they are like deeply fundamentally
Michael Kearns (1:07:03.700)
flawed.
Lex Fridman (1:07:05.120)
And so, you know, to your first question, what is differential privacy?
Michael Kearns (1:07:11.240)
Differential privacy seems to be a much, much better notion of privacy that kind of avoids
Michael Kearns (1:07:16.680)
a lot of the weaknesses of anonymization notions while still letting us do useful stuff with
Michael Kearns (1:07:24.520)
data.
Lex Fridman (1:07:25.520)
What is anonymization of data?
Lex Fridman (1:07:27.480)
So by anonymization, I'm, you know, kind of referring to techniques like I have a database.
Lex Fridman (1:07:34.000)
The rows of that database are, let's say, individual people's medical records, okay?
Lex Fridman (1:07:40.240)
And I want to let people use that data.
Michael Kearns (1:07:43.840)
Maybe I want to let researchers access that data to build predictive models for some disease,
Lex Fridman (1:07:49.480)
but I'm worried that that will leak, you know, sensitive information about specific people's
Lex Fridman (1:07:56.200)
medical records.
Lex Fridman (1:07:57.640)
So anonymization broadly refers to the set of techniques where I say like, okay, I'm
Lex Fridman (1:08:01.680)
first going to like, I'm going to delete the column with people's names.
Lex Fridman (1:08:06.160)
I'm going to not put, you know, so that would be like a redaction, right?
Lex Fridman (1:08:09.760)
I'm just redacting that information.
Michael Kearns (1:08:12.040)
I am going to take ages and I'm not going to like say your exact age.
Michael Kearns (1:08:17.040)
I'm going to say whether you're, you know, zero to 10, 10 to 20, 20 to 30, I might put
Michael Kearns (1:08:23.120)
the first three digits of your zip code, but not the last two, et cetera, et cetera.
Lex Fridman (1:08:27.520)
And so the idea is that through some series of operations like this on the data, I anonymize
Michael Kearns (1:08:31.800)
it.
Michael Kearns (1:08:32.800)
You know, another term of art that's used is removing personally identifiable information.
Lex Fridman (1:08:38.880)
And you know, this is basically the most common way of providing data privacy, but that it's
Lex Fridman (1:08:45.600)
in a way that still lets people access the, some variant form of the data.
Lex Fridman (1:08:50.240)
So at a slightly broader picture, as you talk about what does anonymization mean when you
Michael Kearns (1:08:56.080)
have multiple database, like with a Netflix prize, when you can start combining stuff
Michael Kearns (1:09:01.440)
together.
Lex Fridman (1:09:02.440)
So this is exactly the problem with these notions, right?
Michael Kearns (1:09:05.400)
Is that notions of a anonymization, removing personally identifiable information, the kind
Michael Kearns (1:09:10.900)
of fundamental conceptual flaw is that, you know, these definitions kind of pretend as
Michael Kearns (1:09:16.000)
if the data set in question is the only data set that exists in the world or that ever
Lex Fridman (1:09:21.240)
will exist in the future.
Lex Fridman (1:09:23.640)
And of course, things like the Netflix prize and many, many other examples since the Netflix
Michael Kearns (1:09:28.080)
prize, I think that was one of the earliest ones though, you know, you can reidentify
Michael Kearns (1:09:33.320)
people that were, you know, that were anonymized in the data set by taking that anonymized
Michael Kearns (1:09:38.540)
data set and combining it with other allegedly anonymized data sets and maybe publicly available
Michael Kearns (1:09:43.240)
information about you.
Lex Fridman (1:09:44.480)
You know,
Michael Kearns (1:09:45.480)
for people who don't know the Netflix prize was, was being publicly released this data.
Lex Fridman (1:09:50.880)
So the names from those rows were removed, but what was released is the preference or
Michael Kearns (1:09:55.640)
the ratings of what movies you like and you don't like.
Lex Fridman (1:09:58.720)
And from that combined with other things, I think forum posts and so on, you can start
Michael Kearns (1:10:03.360)
to figure out
Michael Kearns (1:10:04.360)
I guess it was specifically the internet movie database where, where lots of Netflix users
Michael Kearns (1:10:10.400)
publicly rate their movie, you know, their movie preferences.
Lex Fridman (1:10:15.280)
And so the anonymized data and Netflix, when it's just this phenomenon, I think that we've
Michael Kearns (1:10:21.840)
all come to realize in the last decade or so is that just knowing a few apparently irrelevant
Lex Fridman (1:10:29.920)
innocuous things about you can often act as a fingerprint.
Michael Kearns (1:10:33.100)
Like if I know, you know, what, what rating you gave to these 10 movies and the date on
Michael Kearns (1:10:39.000)
which you entered these movies, this is almost like a fingerprint for you in the sea of all
Michael Kearns (1:10:43.480)
Netflix users.
Michael Kearns (1:10:44.480)
There were just another paper on this in science or nature of about a month ago that, you know,
Michael Kearns (1:10:49.760)
kind of 18 attributes.
Michael Kearns (1:10:51.240)
I mean, my favorite example of this is, was actually a paper from several years ago now
Michael Kearns (1:10:57.120)
where it was shown that just from your likes on Facebook, just from the time, you know,
Michael Kearns (1:11:03.400)
the things on which you clicked on the thumbs up button on the platform, not using any information,
Michael Kearns (1:11:09.520)
demographic information, nothing about who your friends are, just knowing the content
Michael Kearns (1:11:14.720)
that you had liked was enough to, you know, in the aggregate accurately predict things
Michael Kearns (1:11:20.680)
like sexual orientation, drug and alcohol use, whether you were the child of divorced parents.
Lex Fridman (1:11:27.280)
So we live in this era where, you know, even the apparently irrelevant data that we offer
Michael Kearns (1:11:32.080)
about ourselves on public platforms and forums often unbeknownst to us, more or less acts
Lex Fridman (1:11:38.760)
as signature or, you know, fingerprint.
Lex Fridman (1:11:42.480)
And that if you can kind of, you know, do a join between that kind of data and allegedly
Lex Fridman (1:11:46.980)
anonymized data, you have real trouble.
Lex Fridman (1:11:50.720)
So is there hope for any kind of privacy in a world where a few likes can identify you?
Lex Fridman (1:11:58.380)
So there is differential privacy, right?
Lex Fridman (1:12:00.380)
What is differential privacy?
Michael Kearns (1:12:01.380)
Yeah, so differential privacy basically is a kind of alternate, much stronger notion
Michael Kearns (1:12:06.100)
of privacy than these anonymization ideas.
Michael Kearns (1:12:10.280)
And, you know, it's a technical definition, but like the spirit of it is we compare two
Lex Fridman (1:12:18.760)
alternate worlds, okay?
Lex Fridman (1:12:20.320)
So let's suppose I'm a researcher and I want to do, you know, there's a database of medical
Michael Kearns (1:12:26.120)
records and one of them is yours, and I want to use that database of medical records to
Lex Fridman (1:12:31.600)
build a predictive model for some disease.
Lex Fridman (1:12:33.800)
So based on people's symptoms and test results and the like, I want to, you know, build a
Lex Fridman (1:12:39.440)
probably model predicting the probability that people have disease.
Michael Kearns (1:12:42.180)
So, you know, this is the type of scientific research that we would like to be allowed
Lex Fridman (1:12:46.400)
to continue.
Lex Fridman (1:12:48.060)
And in differential privacy, you ask a very particular counterfactual question.
Lex Fridman (1:12:53.400)
We basically compare two alternatives.
Michael Kearns (1:12:57.480)
One is when I do this, I build this model on the database of medical records, including
Lex Fridman (1:13:04.760)
your medical record.
Lex Fridman (1:13:07.200)
And the other one is where I do the same exercise with the same database with just your medical
Lex Fridman (1:13:15.320)
record removed.
Lex Fridman (1:13:16.320)
So basically, you know, it's two databases, one with N records in it and one with N minus
Lex Fridman (1:13:22.280)
one records in it.
Michael Kearns (1:13:23.840)
The N minus one records are the same, and the only one that's missing in the second
Lex Fridman (1:13:27.960)
case is your medical record.
Lex Fridman (1:13:30.420)
So differential privacy basically says that any harms that might come to you from the
Michael Kearns (1:13:40.580)
analysis in which your data was included are essentially nearly identical to the harms
Michael Kearns (1:13:47.640)
that would have come to you if the same analysis had been done without your medical record
Lex Fridman (1:13:52.720)
included.
Lex Fridman (1:13:53.720)
So in other words, this doesn't say that bad things cannot happen to you as a result of
Lex Fridman (1:13:58.280)
data analysis.
Michael Kearns (1:13:59.760)
It just says that these bad things were going to happen to you already, even if your data
Lex Fridman (1:14:05.080)
wasn't included.
Lex Fridman (1:14:06.080)
And to give a very concrete example, right, you know, like we discussed at some length,
Michael Kearns (1:14:12.360)
the study that, you know, in the 50s that was done that established the link between
Michael Kearns (1:14:17.800)
smoking and lung cancer.
Lex Fridman (1:14:19.960)
And we make the point that, like, well, if your data was used in that analysis and, you
Michael Kearns (1:14:25.200)
know, the world kind of knew that you were a smoker because, you know, there was no stigma
Michael Kearns (1:14:28.980)
associated with smoking before those findings, real harm might have come to you as a result
Michael Kearns (1:14:35.160)
of that study that your data was included in.
Michael Kearns (1:14:37.760)
In particular, your insurer now might have a higher posterior belief that you might have
Michael Kearns (1:14:42.440)
lung cancer and raise your premium.
Lex Fridman (1:14:44.360)
So you've suffered economic damage.
Lex Fridman (1:14:47.820)
But the point is, is that if the same analysis has been done with all the other N minus one
Lex Fridman (1:14:54.960)
medical records and just yours missing, the outcome would have been the same.
Michael Kearns (1:14:58.800)
Or your data wasn't idiosyncratically crucial to establishing the link between smoking and
Michael Kearns (1:15:05.560)
lung cancer because the link between smoking and lung cancer is like a fact about the world
Michael Kearns (1:15:10.440)
that can be discovered with any sufficiently large database of medical records.
Lex Fridman (1:15:14.820)
But that's a very low value of harm.
Michael Kearns (1:15:17.320)
Yeah.
Lex Fridman (1:15:18.320)
So that's showing that very little harm is done.
Michael Kearns (1:15:20.560)
Great.
Lex Fridman (1:15:21.560)
But how what is the mechanism of differential privacy?
Lex Fridman (1:15:24.760)
So that's the kind of beautiful statement of it.
Lex Fridman (1:15:27.600)
It's the mechanism by which privacy is preserved.
Michael Kearns (1:15:30.440)
Yeah.
Lex Fridman (1:15:31.440)
So it's basically by adding noise to computations, right?
Lex Fridman (1:15:34.600)
So the basic idea is that every differentially private algorithm, first of all, or every
Michael Kearns (1:15:40.400)
good differentially private algorithm, every useful one, is a probabilistic algorithm.
Lex Fridman (1:15:45.380)
So it doesn't, on a given input, if you gave the algorithm the same input multiple times,
Lex Fridman (1:15:51.000)
it would give different outputs each time from some distribution.
Lex Fridman (1:15:55.760)
And the way you achieve differential privacy algorithmically is by kind of carefully and
Lex Fridman (1:15:59.820)
tastefully adding noise to a computation in the right places.
Lex Fridman (1:16:05.400)
And to give a very concrete example, if I wanna compute the average of a set of numbers,
Michael Kearns (1:16:11.600)
the non private way of doing that is to take those numbers and average them and release
Michael Kearns (1:16:17.220)
like a numerically precise value for the average.
Lex Fridman (1:16:21.880)
In differential privacy, you wouldn't do that.
Michael Kearns (1:16:24.200)
You would first compute that average to numerical precisions, and then you'd add some noise
Lex Fridman (1:16:29.520)
to it, right?
Michael Kearns (1:16:30.520)
You'd add some kind of zero mean, Gaussian or exponential noise to it so that the actual
Michael Kearns (1:16:37.520)
value you output is not the exact mean, but it'll be close to the mean, but it'll be close...
Michael Kearns (1:16:44.120)
The noise that you add will sort of prove that nobody can kind of reverse engineer any
Lex Fridman (1:16:50.560)
particular value that went into the average.
Lex Fridman (1:16:53.440)
So noise is a savior.
Lex Fridman (1:16:56.200)
How many algorithms can be aided by adding noise?
Michael Kearns (1:17:01.640)
Yeah, so I'm a relatively recent member of the differential privacy community.
Michael Kearns (1:17:07.040)
My co author, Aaron Roth is really one of the founders of the field and has done a great
Michael Kearns (1:17:12.440)
deal of work and I've learned a tremendous amount working with him on it.
Lex Fridman (1:17:15.520)
It's a pretty grown up field already.
Michael Kearns (1:17:17.240)
Yeah, but now it's pretty mature.
Lex Fridman (1:17:18.480)
But I must admit, the first time I saw the definition of differential privacy, my reaction
Michael Kearns (1:17:22.080)
was like, wow, that is a clever definition and it's really making very strong promises.
Lex Fridman (1:17:28.360)
And I first saw the definition in much earlier days and my first reaction was like, well,
Michael Kearns (1:17:34.920)
my worry about this definition would be that it's a great definition of privacy, but that
Lex Fridman (1:17:38.960)
it'll be so restrictive that we won't really be able to use it.
Michael Kearns (1:17:43.180)
We won't be able to compute many things in a differentially private way.
Lex Fridman (1:17:47.200)
So that's one of the great successes of the field, I think, is in showing that the opposite
Michael Kearns (1:17:51.920)
is true and that most things that we know how to compute, absent any privacy considerations,
Lex Fridman (1:18:00.980)
can be computed in a differentially private way.
Lex Fridman (1:18:02.920)
So for example, pretty much all of statistics and machine learning can be done differentially
Lex Fridman (1:18:08.240)
privately.
Lex Fridman (1:18:09.320)
So pick your favorite machine learning algorithm, back propagation and neural networks, cart
Michael Kearns (1:18:15.120)
for decision trees, support vector machines, boosting, you name it, as well as classic
Michael Kearns (1:18:21.060)
hypothesis testing and the like in statistics.
Lex Fridman (1:18:24.920)
None of those algorithms are differentially private in their original form.
Michael Kearns (1:18:29.720)
All of them have modifications that add noise to the computation in different places in
Lex Fridman (1:18:35.700)
different ways that achieve differential privacy.
Lex Fridman (1:18:39.120)
So this really means that to the extent that we've become a scientific community very dependent
Michael Kearns (1:18:47.460)
on the use of machine learning and statistical modeling and data analysis, we really do have
Michael Kearns (1:18:53.400)
a path to provide privacy guarantees to those methods and so we can still enjoy the benefits
Michael Kearns (1:19:02.760)
of the data science era while providing rather robust privacy guarantees to individuals.
Lex Fridman (1:19:10.760)
So perhaps a slightly crazy question, but if we take the ideas of differential privacy
Lex Fridman (1:19:16.160)
and take it to the nature of truth that's being explored currently.
Lex Fridman (1:19:20.680)
So what's your most favorite and least favorite food?
Lex Fridman (1:19:24.880)
Hmm.
Michael Kearns (1:19:25.880)
I'm not a real foodie, so I'm a big fan of spaghetti.
Lex Fridman (1:19:29.880)
Spaghetti?
Michael Kearns (1:19:30.880)
Yeah.
Lex Fridman (1:19:31.880)
What do you really don't like?
Michael Kearns (1:19:35.840)
I really don't like cauliflower.
Lex Fridman (1:19:37.280)
Wow, I love cauliflower.
Michael Kearns (1:19:39.280)
Okay.
Michael Kearns (1:19:40.280)
Is there one way to protect your preference for spaghetti by having an information campaign
Lex Fridman (1:19:46.400)
bloggers and so on of bots saying that you like cauliflower?
Lex Fridman (1:19:51.280)
So like this kind of the same kind of noise ideas, I mean if you think of in our politics
Michael Kearns (1:19:56.640)
today there's this idea of Russia hacking our elections.
Lex Fridman (1:20:01.920)
What's meant there I believe is bots spreading different kinds of information.
Lex Fridman (1:20:07.200)
Is that a kind of privacy or is that too much of a stretch?
Lex Fridman (1:20:10.480)
No it's not a stretch.
Michael Kearns (1:20:12.160)
I've not seen those ideas, you know, that is not a technique that to my knowledge will
Michael Kearns (1:20:19.320)
provide differential privacy, but to give an example like one very specific example
Michael Kearns (1:20:24.400)
about what you're discussing is there was a very interesting project at NYU I think
Michael Kearns (1:20:30.240)
led by Helen Nissenbaum there in which they basically built a browser plugin that tried
Michael Kearns (1:20:38.720)
to essentially obfuscate your Google searches.
Lex Fridman (1:20:41.640)
So to the extent that you're worried that Google is using your searches to build, you
Michael Kearns (1:20:46.440)
know, predictive models about you to decide what ads to show you which they might very
Michael Kearns (1:20:51.480)
reasonably want to do, but if you object to that they built this widget you could plug
Michael Kearns (1:20:56.040)
in and basically whenever you put in a query into Google it would send that query to Google,
Lex Fridman (1:21:01.280)
but in the background all of the time from your browser it would just be sending this
Michael Kearns (1:21:06.200)
torrent of irrelevant queries to the search engine.
Lex Fridman (1:21:11.800)
So you know it's like a weed and chaff thing so you know out of every thousand queries
Michael Kearns (1:21:16.840)
let's say that Google was receiving from your browser one of them was one that you put in
Lex Fridman (1:21:21.560)
but the other 999 were not okay so it's the same kind of idea kind of you know privacy
Michael Kearns (1:21:27.300)
by obfuscation.
Lex Fridman (1:21:29.680)
So I think that's an interesting idea, doesn't give you differential privacy.
Michael Kearns (1:21:34.920)
It's also I was actually talking to somebody at one of the large tech companies recently
Michael Kearns (1:21:39.260)
about the fact that you know just this kind of thing that there are some times when the
Michael Kearns (1:21:45.560)
response to my data needs to be very specific to my data right like I type mountain biking
Michael Kearns (1:21:53.120)
into Google, I want results on mountain biking and I really want Google to know that I typed
Michael Kearns (1:21:58.420)
in mountain biking, I don't want noise added to that.
Lex Fridman (1:22:01.880)
And so I think there's sort of maybe even interesting technical questions around notions
Michael Kearns (1:22:06.180)
of privacy that are appropriate where you know it's not that my data is part of some
Michael Kearns (1:22:10.800)
aggregate like medical records and that we're trying to discover important correlations
Lex Fridman (1:22:15.800)
and facts about the world at large but rather you know there's a service that I really want
Michael Kearns (1:22:20.960)
to you know pay attention to my specific data yet I still want some kind of privacy guarantee
Lex Fridman (1:22:26.120)
and I think these kind of obfuscation ideas are sort of one way of getting at that but
Lex Fridman (1:22:30.200)
maybe there are others as well.
Lex Fridman (1:22:32.160)
So where do you think we'll land in this algorithm driven society in terms of privacy?
Lex Fridman (1:22:36.520)
So sort of China like Kai Fuli describes you know it's collecting a lot of data on its
Michael Kearns (1:22:44.960)
citizens but in the best form it's actually able to provide a lot of sort of protect human
Michael Kearns (1:22:52.360)
rights and provide a lot of amazing services and it's worst forms that can violate those
Michael Kearns (1:22:57.320)
human rights and limit services.
Lex Fridman (1:23:01.080)
So where do you think we'll land because algorithms are powerful when they use data.
Lex Fridman (1:23:08.400)
So as a society do you think we'll give over more data?
Lex Fridman (1:23:12.900)
Is it possible to protect the privacy of that data?
Lex Fridman (1:23:16.400)
So I'm optimistic about the possibility of you know balancing the desire for individual
Michael Kearns (1:23:24.400)
privacy and individual control of privacy with kind of societally and commercially beneficial
Michael Kearns (1:23:32.360)
uses of data not unrelated to differential privacy or suggestions that say like well
Lex Fridman (1:23:37.840)
individuals should have control of their data.
Michael Kearns (1:23:40.560)
They should be able to limit the uses of that data.
Michael Kearns (1:23:43.600)
They should even you know there's you know fledgling discussions going on in research
Michael Kearns (1:23:48.200)
circles about allowing people selective use of their data and being compensated for it.
Lex Fridman (1:23:54.680)
And then you get to sort of very interesting economic questions like pricing right.
Lex Fridman (1:23:59.480)
And one interesting idea is that maybe differential privacy would also you know be a conceptual
Michael Kearns (1:24:05.360)
framework in which you could talk about the relative value of different people's data
Michael Kearns (1:24:09.120)
like you know to demystify this a little bit.
Michael Kearns (1:24:12.080)
If I'm trying to build a predictive model for some rare disease and I'm trying to use
Michael Kearns (1:24:17.320)
machine learning to do it, it's easy to get negative examples because the disease is rare
Lex Fridman (1:24:22.480)
right.
Lex Fridman (1:24:23.740)
But I really want to have lots of people with the disease in my data set okay.
Lex Fridman (1:24:30.880)
And so somehow those people's data with respect to this application is much more valuable
Michael Kearns (1:24:35.380)
to me than just like the background population.
Lex Fridman (1:24:37.840)
And so maybe they should be compensated more for it.
Lex Fridman (1:24:43.160)
And so you know I think these are kind of very, very fledgling conceptual questions
Michael Kearns (1:24:48.800)
that maybe we'll have kind of technical thought on them sometime in the coming years.
Lex Fridman (1:24:54.000)
But I do think we'll you know to kind of get more directly answer your question.
Michael Kearns (1:24:56.760)
I think I'm optimistic at this point from what I've seen that we will land at some you
Michael Kearns (1:25:02.760)
know better compromise than we're at right now where again you know privacy guarantees
Lex Fridman (1:25:08.640)
are few far between and weak and users have very, very little control.
Lex Fridman (1:25:15.400)
And I'm optimistic that we'll land in something that you know provides better privacy overall
Lex Fridman (1:25:20.320)
and more individual control of data and privacy.
Lex Fridman (1:25:22.820)
But you know I think to get there it's again just like fairness it's not going to be enough
Lex Fridman (1:25:27.740)
to propose algorithmic solutions.
Michael Kearns (1:25:29.560)
There's going to have to be a whole kind of regulatory legal process that prods companies
Lex Fridman (1:25:34.880)
and other parties to kind of adopt solutions.
Lex Fridman (1:25:38.880)
And I think you've mentioned the word control a lot and I think giving people control that's
Michael Kearns (1:25:43.040)
something that people don't quite have in a lot of these algorithms and that's a really
Michael Kearns (1:25:48.200)
interesting idea of giving them control.
Michael Kearns (1:25:50.540)
Some of that is actually literally an interface design question sort of just enabling because
Michael Kearns (1:25:57.920)
I think it's good for everybody to give users control.
Michael Kearns (1:26:00.440)
It's almost not a trade off except that you have to hire people that are good at interface
Michael Kearns (1:26:06.160)
design.
Lex Fridman (1:26:07.160)
Yeah.
Michael Kearns (1:26:08.160)
I mean the other thing that has to be said right is that you know it's a cliche but you
Michael Kearns (1:26:13.080)
know we as the users of many systems platforms and apps you know we are the product.
Michael Kearns (1:26:21.720)
We are not the customer.
Lex Fridman (1:26:23.120)
The customer are advertisers and our data is the product.
Michael Kearns (1:26:26.760)
Okay.
Lex Fridman (1:26:27.760)
So it's one thing to kind of suggest more individual control of data and privacy and
Michael Kearns (1:26:32.640)
uses but this you know if this happens in sufficient degree it will upend the entire
Lex Fridman (1:26:40.480)
economic model that has supported the internet to date.
Lex Fridman (1:26:44.520)
And so some other economic model will have to be you know we'll have to replace it.
Lex Fridman (1:26:50.040)
So the idea of markets you mentioned by exposing the economic model to the people they will
Michael Kearns (1:26:56.480)
then become a market.
Lex Fridman (1:26:57.920)
They could be participants in it.
Lex Fridman (1:27:00.280)
And you know this isn't you know this is not a weird idea right because there are markets
Lex Fridman (1:27:04.680)
for data already.
Michael Kearns (1:27:05.720)
It's just that consumers are not participants and there's like you know there's sort of
Michael Kearns (1:27:10.080)
you know publishers and content providers on one side that have inventory and then their
Michael Kearns (1:27:14.780)
advertisers on the others and you know you know Google and Facebook are running you know
Michael Kearns (1:27:19.680)
they're pretty much their entire revenue stream is by running two sided markets between those
Michael Kearns (1:27:25.540)
parties right.
Lex Fridman (1:27:27.380)
And so it's not a crazy idea that there would be like a three sided market or that you know
Michael Kearns (1:27:32.800)
that on one side of the market or the other we would have proxies representing our interest.
Michael Kearns (1:27:37.080)
It's not you know it's not a crazy idea but it would it's not a crazy technical idea but
Michael Kearns (1:27:43.080)
it would have pretty extreme economic consequences.
Michael Kearns (1:27:49.920)
Speaking of markets a lot of fascinating aspects of this world arise not from individual human
Michael Kearns (1:27:55.520)
beings but from the interaction of human beings.
Lex Fridman (1:27:59.880)
You've done a lot of work in game theory.
Lex Fridman (1:28:02.080)
First can you say what is game theory and how does it help us model and study?
Lex Fridman (1:28:07.360)
Yeah game theory of course let us give credit where it's due.
Michael Kearns (1:28:11.080)
You know it comes from the economist first and foremost but as I've mentioned before
Michael Kearns (1:28:16.300)
like you know computer scientists never hesitate to wander into other people's turf and so
Michael Kearns (1:28:22.000)
there is now this 20 year old field called algorithmic game theory.
Lex Fridman (1:28:26.520)
But you know game theory first and foremost is a mathematical framework for reasoning
Michael Kearns (1:28:33.240)
about collective outcomes in systems of interacting individuals.
Michael Kearns (1:28:40.240)
You know so you need at least two people to get started in game theory and many people
Michael Kearns (1:28:46.040)
are probably familiar with Prisoner's Dilemma as kind of a classic example of game theory
Lex Fridman (1:28:50.560)
and a classic example where everybody looking out for their own individual interests leads
Michael Kearns (1:28:57.000)
to a collective outcome that's kind of worse for everybody than what might be possible
Lex Fridman (1:29:02.560)
if they cooperated for example.
Lex Fridman (1:29:05.200)
But cooperation is not an equilibrium in Prisoner's Dilemma.
Lex Fridman (1:29:09.780)
And so my work in the field of algorithmic game theory more generally in these areas
Michael Kearns (1:29:16.120)
kind of looks at settings in which the number of actors is potentially extraordinarily large
Lex Fridman (1:29:24.720)
and their incentives might be quite complicated and kind of hard to model directly but you
Michael Kearns (1:29:31.160)
still want kind of algorithmic ways of kind of predicting what will happen or influencing
Lex Fridman (1:29:36.120)
what will happen in the design of platforms.
Lex Fridman (1:29:39.880)
So what to you is the most beautiful idea that you've encountered in game theory?
Lex Fridman (1:29:47.160)
There's a lot of them.
Michael Kearns (1:29:48.160)
I'm a big fan of the field.
Michael Kearns (1:29:50.760)
I mean you know I mean technical answers to that of course would include Nash's work just
Michael Kearns (1:29:56.400)
establishing that you know there is a competitive equilibrium under very very general circumstances
Michael Kearns (1:30:02.640)
which in many ways kind of put the field on a firm conceptual footing because if you don't
Michael Kearns (1:30:09.840)
have equilibrium it's kind of hard to ever reason about what might happen since you know
Lex Fridman (1:30:14.280)
there's just no stability.
Lex Fridman (1:30:16.200)
So just the idea that stability can emerge when there's multiple.
Lex Fridman (1:30:20.680)
Not that it will necessarily emerge just that it's possible right.
Michael Kearns (1:30:23.840)
Like the existence of equilibrium doesn't mean that sort of natural iterative behavior
Lex Fridman (1:30:28.580)
will necessarily lead to it.
Michael Kearns (1:30:30.640)
In the real world.
Lex Fridman (1:30:31.640)
Yeah.
Michael Kearns (1:30:32.640)
Maybe answering a slightly less personally than you asked the question I think within
Michael Kearns (1:30:35.960)
the field of algorithmic game theory perhaps the single most important kind of technical
Michael Kearns (1:30:43.760)
contribution that's been made is the realization between close connections between machine
Michael Kearns (1:30:49.640)
learning and game theory and in particular between game theory and the branch of machine
Michael Kearns (1:30:53.840)
learning that's known as no regret learning and this sort of provides a very general framework
Michael Kearns (1:31:00.600)
in which a bunch of players interacting in a game or a system each one kind of doing
Michael Kearns (1:31:07.460)
something that's in their self interest will actually kind of reach an equilibrium and
Michael Kearns (1:31:12.440)
actually reach an equilibrium in a you know a pretty you know a rather you know short
Michael Kearns (1:31:18.960)
amount of steps.
Lex Fridman (1:31:21.400)
So you kind of mentioned acting greedily can somehow end up pretty good for everybody.
Michael Kearns (1:31:30.120)
Or pretty bad.
Lex Fridman (1:31:31.320)
Or pretty bad.
Michael Kearns (1:31:32.320)
Yeah.
Lex Fridman (1:31:33.320)
It will end up stable.
Michael Kearns (1:31:34.320)
Yeah.
Lex Fridman (1:31:35.320)
Right.
Lex Fridman (1:31:36.320)
And and you know stability or equilibrium by itself is neither is not necessarily either
Lex Fridman (1:31:41.500)
a good thing or a bad thing.
Lex Fridman (1:31:43.220)
So what's the connection between machine learning and the ideas.
Michael Kearns (1:31:45.840)
Well I think we kind of talked about these ideas already in kind of a non technical way
Michael Kearns (1:31:50.960)
which is maybe the more interesting way of understanding them first which is you know
Michael Kearns (1:31:57.200)
we have many systems platforms and apps these days that work really hard to use our data
Lex Fridman (1:32:04.840)
and the data of everybody else on the platform to selfishly optimize on behalf of each user.
Lex Fridman (1:32:12.120)
OK.
Lex Fridman (1:32:13.120)
So you know let me let me give I think the cleanest example which is just driving apps
Michael Kearns (1:32:17.960)
navigation apps like you know Google Maps and Waze where you know miraculously compared
Michael Kearns (1:32:24.040)
to when I was growing up at least you know the objective would be the same when you wanted
Michael Kearns (1:32:28.680)
to drive from point A to point B spend the least time driving not necessarily minimize
Michael Kearns (1:32:33.440)
the distance but minimize the time.
Lex Fridman (1:32:35.840)
Right.
Lex Fridman (1:32:36.840)
And when I was growing up like the only resources you had to do that were like maps in the car
Michael Kearns (1:32:41.080)
which literally just told you what roads were available and then you might have like half
Michael Kearns (1:32:46.540)
hourly traffic reports just about the major freeways but not about side roads.
Lex Fridman (1:32:51.680)
So you were pretty much on your own.
Lex Fridman (1:32:54.040)
And now we've got these apps you pull it out and you say I want to go from point A to point
Michael Kearns (1:32:57.920)
B and in response kind of to what everybody else is doing if you like what all the other
Michael Kearns (1:33:03.360)
players in this game are doing right now here's the you know the route that minimizes your
Lex Fridman (1:33:09.280)
driving time.
Lex Fridman (1:33:10.280)
So it is really kind of computing a selfish best response for each of us in response to
Lex Fridman (1:33:16.560)
what all of the rest of us are doing at any given moment.
Lex Fridman (1:33:20.240)
And so you know I think it's quite fair to think of these apps as driving or nudging
Lex Fridman (1:33:26.280)
us all towards the competitive or Nash equilibrium of that game.
Michael Kearns (1:33:32.560)
Now you might ask like well that sounds great why is that a bad thing.
Michael Kearns (1:33:36.320)
Well you know it's known both in theory and with some limited studies from actual like
Michael Kearns (1:33:45.400)
traffic data that all of us being in this competitive equilibrium might cause our collective
Michael Kearns (1:33:52.660)
driving time to be higher maybe significantly higher than it would be under other solutions.
Lex Fridman (1:33:59.760)
And then you have to talk about what those other solutions might be and what the algorithms
Michael Kearns (1:34:04.320)
to implement them are which we do discuss in the kind of game theory chapter of the
Michael Kearns (1:34:07.760)
book.
Lex Fridman (1:34:09.880)
But similarly you know on social media platforms or on Amazon you know all these algorithms
Michael Kearns (1:34:17.040)
that are essentially trying to optimize our behalf they're driving us in a colloquial
Michael Kearns (1:34:22.000)
sense towards some kind of competitive equilibrium and you know one of the most important lessons
Michael Kearns (1:34:26.920)
of game theory is that just because we're at equilibrium doesn't mean that there's not
Lex Fridman (1:34:30.360)
a solution in which some or maybe even all of us might be better off.
Lex Fridman (1:34:35.720)
And then the connection to machine learning of course is that in all these platforms I've
Michael Kearns (1:34:39.080)
mentioned the optimization that they're doing on our behalf is driven by machine learning
Michael Kearns (1:34:44.320)
you know like predicting where the traffic will be predicting what products I'm going
Lex Fridman (1:34:48.040)
to like predicting what would make me happy in my newsfeed.
Michael Kearns (1:34:52.220)
Now in terms of the stability and the promise of that I have to ask just out of curiosity
Lex Fridman (1:34:56.720)
how stable are these mechanisms that you game theory is just the economist came up with
Lex Fridman (1:35:02.600)
and we all know that economists don't live in the real world just kidding sort of what's
Lex Fridman (1:35:08.040)
do you think when we look at the fact that we haven't blown ourselves up from the from
Michael Kearns (1:35:15.720)
a game theoretic concept of mutually shared destruction what are the odds that we destroy
Lex Fridman (1:35:21.000)
ourselves with nuclear weapons as one example of a stable game theoretic system?
Michael Kearns (1:35:28.400)
Just to prime your viewers a little bit I mean I think you're referring to the fact
Michael Kearns (1:35:32.080)
that game theory was taken quite seriously back in the 60s as a tool for reasoning about
Michael Kearns (1:35:38.400)
kind of Soviet US nuclear armament disarmament detente things like that.
Michael Kearns (1:35:45.160)
I'll be honest as huge of a fan as I am of game theory and its kind of rich history it
Michael Kearns (1:35:52.320)
still surprises me that you know you had people at the RAND Corporation back in those days
Michael Kearns (1:35:57.700)
kind of drawing up you know two by two tables and one the row player is you know the US
Lex Fridman (1:36:02.800)
and the column player is Russia and that they were taking seriously you know I'm sure if
Michael Kearns (1:36:08.240)
I was there maybe it wouldn't have seemed as naive as it does at the time you know.
Michael Kearns (1:36:12.840)
Seems to have worked which is why it seems naive.
Lex Fridman (1:36:15.440)
Well we're still here.
Michael Kearns (1:36:16.440)
We're still here in that sense.
Michael Kearns (1:36:17.960)
Yeah even though I kind of laugh at those efforts they were more sensible then than
Michael Kearns (1:36:22.600)
they would be now right because there were sort of only two nuclear powers at the time
Lex Fridman (1:36:26.540)
and you didn't have to worry about deterring new entrants and who was developing the capacity
Lex Fridman (1:36:32.480)
and so we have many you know it's definitely a game with more players now and more potential
Lex Fridman (1:36:39.120)
entrants.
Michael Kearns (1:36:40.120)
I'm not in general somebody who advocates using kind of simple mathematical models when
Michael Kearns (1:36:46.200)
the stakes are as high as things like that and the complexities are very political and
Michael Kearns (1:36:51.840)
social but we are still here.
Lex Fridman (1:36:55.760)
So you've worn many hats one of which the one that first caused me to become a big fan
Michael Kearns (1:37:00.600)
of your work many years ago is algorithmic trading.
Lex Fridman (1:37:04.460)
So I have to just ask a question about this because you have so much fascinating work
Michael Kearns (1:37:08.520)
there in the 21st century what role do you think algorithms have in space of trading
Lex Fridman (1:37:15.820)
investment in the financial sector?
Michael Kearns (1:37:19.080)
Yeah it's a good question I mean in the time I've spent on Wall Street and in finance you
Michael Kearns (1:37:27.160)
know I've seen a clear progression and I think it's a progression that kind of models the
Michael Kearns (1:37:31.320)
use of algorithms and automation more generally in society which is you know the things that
Michael Kearns (1:37:38.640)
kind of get taken over by the algos first are sort of the things that computers are
Michael Kearns (1:37:44.240)
obviously better at than people right so you know so first of all there needed to be this
Michael Kearns (1:37:50.320)
era of automation right where just you know financial exchanges became largely electronic
Michael Kearns (1:37:56.200)
which then enabled the possibility of you know trading becoming more algorithmic because
Michael Kearns (1:38:01.720)
once you know that exchanges are electronic an algorithm can submit an order through an
Michael Kearns (1:38:06.720)
API just as well as a human can do at a monitor quickly can read all the data so yeah and
Lex Fridman (1:38:11.800)
so you know I think the places where algorithmic trading have had the greatest inroads and
Michael Kearns (1:38:18.800)
had the first inroads were in kind of execution problems kind of optimized execution problems
Lex Fridman (1:38:24.560)
so what I mean by that is at a large brokerage firm for example one of the lines of business
Michael Kearns (1:38:30.440)
might be on behalf of large institutional clients taking you know what we might consider
Michael Kearns (1:38:36.320)
difficult trade so it's not like a mom and pop investor saying I want to buy a hundred
Michael Kearns (1:38:40.280)
shares of Microsoft it's a large hedge fund saying you know I want to buy a very very
Michael Kearns (1:38:45.940)
large stake in Apple and I want to do it over the span of a day and it's such a large volume
Michael Kearns (1:38:52.760)
that if you're not clever about how you break that trade up not just over time but over
Michael Kearns (1:38:57.260)
perhaps multiple different electronic exchanges that all let you trade Apple on their platform
Michael Kearns (1:39:02.560)
you know you will you will move you'll push prices around in a way that hurts your your
Michael Kearns (1:39:07.760)
execution so you know this is the kind of you know this is an optimization problem this
Michael Kearns (1:39:11.600)
is a control problem right and so machines are better we we know how to design algorithms
Michael Kearns (1:39:19.800)
you know that are better at that kind of thing than a person is going to be able to do because
Michael Kearns (1:39:23.600)
we can take volumes of historical and real time data to kind of optimize the schedule
Michael Kearns (1:39:29.320)
with which we trade and you know similarly high frequency trading you know which is closely
Michael Kearns (1:39:35.080)
related but not the same as optimized execution where you're just trying to spot very very
Michael Kearns (1:39:41.480)
temporary you know mispricings between exchanges or within an asset itself or just predict
Michael Kearns (1:39:48.520)
directional movement of a stock because of the kind of very very low level granular buying
Lex Fridman (1:39:54.800)
and selling data in the in the exchange machines are good at this kind of stuff it's kind of
Michael Kearns (1:40:00.440)
like the mechanics of trading what about the can machines do long terms of prediction yeah
Lex Fridman (1:40:08.080)
so I think we are in an era where you know clearly there have been some very successful
Michael Kearns (1:40:13.280)
you know quant hedge funds that are you know in what we would traditionally call you know
Michael Kearns (1:40:19.760)
still in this the stat arb regime like so you know what's that stat arb referring to
Michael Kearns (1:40:24.520)
statistical arbitrage but but for the purposes of this conversation what it really means
Michael Kearns (1:40:28.920)
is making directional predictions in asset price movement or returns your prediction
Michael Kearns (1:40:35.840)
about that directional movement is good for you know you you have a view that it's valid
Michael Kearns (1:40:42.100)
for some period of time between a few seconds and a few days and that's the amount of time
Michael Kearns (1:40:48.440)
that you're going to kind of get into the position hold it and then hopefully be right
Michael Kearns (1:40:51.920)
about the directional movement and you know buy low and sell high as the cliche goes.
Lex Fridman (1:40:57.360)
So that is a you know kind of a sweet spot I think for quant trading and investing right
Michael Kearns (1:41:04.300)
now and has been for some time when you really get to kind of more Warren Buffett style timescales
Michael Kearns (1:41:11.920)
right like you know my cartoon of Warren Buffett is that you know Warren Buffett sits and thinks
Lex Fridman (1:41:16.800)
what the long term value of Apple really should be and he doesn't even look at what Apple
Michael Kearns (1:41:22.360)
is doing today he just decides you know you know I think that this is what its long term
Michael Kearns (1:41:27.400)
value is and it's far from that right now and so I'm going to buy some Apple or you
Michael Kearns (1:41:31.960)
know short some Apple and I'm going to I'm going to sit on that for 10 or 20 years okay.
Lex Fridman (1:41:37.880)
So when you're at that kind of timescale or even more than just a few days all kinds of
Michael Kearns (1:41:45.600)
other sources of risk and information you know so now you're talking about holding things
Lex Fridman (1:41:51.600)
through recessions and economic cycles, wars can break out.
Lex Fridman (1:41:56.080)
So there you have to understand human nature at a level that.
Michael Kearns (1:41:59.080)
Yeah and you need to just be able to ingest many many more sources of data that are on
Michael Kearns (1:42:03.820)
wildly different timescales right.
Lex Fridman (1:42:06.380)
So if I'm an HFT I'm a high frequency trader like I don't I don't I really my main source
Michael Kearns (1:42:13.120)
of data is just the data from the exchanges themselves about the activity in the exchanges
Michael Kearns (1:42:18.000)
right and maybe I need to pay you know I need to keep an eye on the news right because you
Michael Kearns (1:42:22.880)
know that can cause sudden you know the CEO gets caught in a scandal or you know gets
Michael Kearns (1:42:29.040)
run over by a bus or something that can cause very sudden changes but you know I don't need
Michael Kearns (1:42:33.720)
to understand economic cycles I don't need to understand recessions I don't need to worry
Michael Kearns (1:42:38.480)
about the political situation or war breaking out in this part of the world because you
Michael Kearns (1:42:43.600)
know all I need to know is as long as that's not going to happen in the next 500 milliseconds
Lex Fridman (1:42:49.720)
then you know my model is good.
Michael Kearns (1:42:52.440)
When you get to these longer timescales you really have to worry about that kind of stuff
Lex Fridman (1:42:55.760)
and people in the machine learning community are starting to think about this.
Michael Kearns (1:42:59.280)
We held a we jointly sponsored a workshop at Penn with the Federal Reserve Bank of Philadelphia
Michael Kearns (1:43:06.760)
a little more than a year ago on you know I think the title is something like machine
Michael Kearns (1:43:10.960)
learning for macroeconomic prediction.
Michael Kearns (1:43:14.120)
You know macroeconomic referring specifically to these longer timescales and you know it
Michael Kearns (1:43:19.440)
was an interesting conference but it you know my it left me with greater confidence that
Michael Kearns (1:43:26.800)
we have a long way to go to you know and so I think that people that you know in the grand
Michael Kearns (1:43:32.440)
scheme of things you know if somebody asked me like well whose job on Wall Street is safe
Michael Kearns (1:43:37.480)
from the bots I think people that are at that longer you know timescale and have that appetite
Michael Kearns (1:43:42.840)
for all the risks involved in long term investing and that really need kind of not just algorithms
Michael Kearns (1:43:49.320)
that can optimize from data but they need views on stuff they need views on the political
Michael Kearns (1:43:54.480)
landscape economic cycles and the like and I think you know they're they're they're pretty
Lex Fridman (1:44:01.000)
safe for a while as far as I can tell.
Lex Fridman (1:44:02.680)
So Warren Buffett's job is not seeing you know a robo Warren Buffett anytime soon.
Lex Fridman (1:44:08.320)
Give him comfort.
Michael Kearns (1:44:10.080)
Last question.
Michael Kearns (1:44:11.160)
If you could go back to if there's a day in your life you could relive because it made
Michael Kearns (1:44:18.320)
you truly happy.
Michael Kearns (1:44:21.280)
Maybe you outside family what otherwise you know what what what day would it be.
Lex Fridman (1:44:29.100)
But can you look back you remember just being profoundly transformed in some way or blissful.
Michael Kearns (1:44:40.720)
I'll answer a slightly different question which is like what's a day in my my life or
Michael Kearns (1:44:44.840)
my career that was kind of a watershed moment.
Michael Kearns (1:44:49.040)
I went straight from undergrad to doctoral studies and you know that's not at all atypical
Lex Fridman (1:44:55.760)
and I'm also from an academic family like my my dad was a professor my uncle on his
Lex Fridman (1:45:00.440)
side as a professor both my grandfathers were professors.
Michael Kearns (1:45:03.440)
All kinds of majors to philosophy.
Michael Kearns (1:45:05.640)
Yeah they're kind of all over the map yeah and I was a grad student here just up the
Michael Kearns (1:45:10.820)
river at Harvard and came to study with Les Valiant which was a wonderful experience.
Lex Fridman (1:45:15.840)
But you know I remember my first year of graduate school I was generally pretty unhappy and
Michael Kearns (1:45:21.600)
I was unhappy because you know at Berkeley as an undergraduate you know yeah I studied
Michael Kearns (1:45:25.720)
a lot of math and computer science but it was a huge school first of all and I took
Michael Kearns (1:45:29.960)
a lot of other courses as we've discussed I started as an English major and took history
Michael Kearns (1:45:34.020)
courses and art history classes and had friends you know that did all kinds of different things.
Lex Fridman (1:45:40.200)
And you know Harvard's a much smaller institution than Berkeley and its computer science department
Lex Fridman (1:45:44.840)
especially at that time was was a much smaller place than it is now.
Lex Fridman (1:45:48.720)
And I suddenly just felt very you know like I'd gone from this very big world to this
Michael Kearns (1:45:54.000)
highly specialized world and now all of the classes I was taking were computer science
Michael Kearns (1:45:59.400)
classes and I was only in classes with math and computer science people.
Lex Fridman (1:46:04.600)
And so I was you know I thought often in that first year of grad school about whether I
Michael Kearns (1:46:09.960)
really wanted to stick with it or not and you know I thought like oh I could you know
Michael Kearns (1:46:14.760)
stop with a master's I could go back to the Bay Area and to California and you know this
Michael Kearns (1:46:19.860)
was in one of the early periods where there was you know like you could definitely get
Michael Kearns (1:46:23.660)
a relatively good job paying job at one of the one of the tech companies back you know
Michael Kearns (1:46:28.680)
that were the big tech companies back then.
Lex Fridman (1:46:31.440)
And so I distinctly remember like kind of a late spring day when I was kind of you know
Michael Kearns (1:46:36.880)
sitting in Boston Common and kind of really just kind of chewing over what I wanted to
Michael Kearns (1:46:40.440)
do with my life and I realized like okay and I think this is where my academic background
Michael Kearns (1:46:45.220)
helped me a great deal.
Michael Kearns (1:46:46.220)
I sort of realized you know yeah you're not having a great time right now this feels really
Michael Kearns (1:46:50.420)
narrowing but you know that you're here for research eventually and to do something original
Lex Fridman (1:46:56.320)
and to try to you know carve out a career where you kind of you know choose what you
Michael Kearns (1:47:02.320)
want to think about you know and have a great deal of independence.
Lex Fridman (1:47:06.260)
And so you know at that point I really didn't have any real research experience yet I mean
Michael Kearns (1:47:10.920)
it was trying to think about some problems with very little success but I knew that like
Michael Kearns (1:47:15.840)
I hadn't really tried to do the thing that I knew I'd come to do and so I thought you
Michael Kearns (1:47:23.320)
know I'm going to stick through it for the summer and you know and that was very formative
Michael Kearns (1:47:30.080)
because I went from kind of contemplating quitting to you know a year later it being
Michael Kearns (1:47:37.160)
very clear to me I was going to finish because I still had a ways to go but I kind of started
Michael Kearns (1:47:42.360)
doing research it was going well it was really interesting and it was sort of a complete
Michael Kearns (1:47:46.520)
transformation you know it's just that transition that I think every doctoral student makes
Michael Kearns (1:47:52.400)
at some point which is to sort of go from being like a student of what's been done before
Michael Kearns (1:48:00.040)
to doing you know your own thing and figure out what makes you interested in what your
Michael Kearns (1:48:04.120)
strengths and weaknesses are as a researcher and once you know I kind of made that decision
Michael Kearns (1:48:09.280)
on that particular day at that particular moment in Boston Common you know I'm glad
Lex Fridman (1:48:15.120)
I made that decision.
Lex Fridman (1:48:16.240)
And also just accepting the painful nature of that journey.
Lex Fridman (1:48:19.400)
Yeah exactly exactly.
Michael Kearns (1:48:21.400)
In that moment said I'm gonna I'm gonna stick it out yeah I'm gonna stick around for a while.
Michael Kearns (1:48:26.880)
Well Michael I've looked off do you work for a long time it's really nice to talk to you
Michael Kearns (1:48:30.880)
thank you so much.
Michael Kearns (1:48:31.880)
It's great to get back in touch with you too and see how great you're doing as well.
Michael Kearns (1:48:34.360)
Thanks a lot.
Lex Fridman (1:48:35.360)
Thank you.
Michael Kearns (20:03.280)
which is a relatively weak thing, but it's better than nothing.
Lex Fridman (20:08.000)
And also the more ambitious thing of trying to give some individual promises, but even
Michael Kearns (20:14.960)
that doesn't incorporate, I think something that you're hinting at here is what I might
Lex Fridman (20:18.640)
call subjective fairness, right?
Lex Fridman (20:20.720)
So a lot of the definitions, I mean, all of the definitions in the algorithmic fairness
Lex Fridman (20:25.200)
literature are what I would kind of call received wisdom definitions.
Michael Kearns (20:28.400)
It's sort of, you know, somebody like me sits around and things like, okay, you know, I
Michael Kearns (20:33.440)
think here's a technical definition of fairness that I think people should want or that they
Michael Kearns (20:37.840)
should, you know, think of as some notion of fairness, maybe not the only one, maybe
Lex Fridman (20:41.840)
not the best one, maybe not the last one.
Lex Fridman (20:44.320)
But we really actually don't know from a subjective standpoint, like what people really
Lex Fridman (20:52.480)
think is fair.
Michael Kearns (20:53.360)
You know, we just started doing a little bit of work in our group at actually doing kind
Michael Kearns (21:01.120)
of human subject experiments in which we, you know, ask people about, you know, we ask
Michael Kearns (21:09.120)
them questions about fairness, we survey them, we, you know, we show them pairs of individuals
Michael Kearns (21:15.120)
in, let's say, a criminal recidivism prediction setting, and we ask them, do you think these
Lex Fridman (21:20.320)
two individuals should be treated the same as a matter of fairness?
Lex Fridman (21:24.320)
And to my knowledge, there's not a large literature in which ordinary people are asked
Michael Kearns (21:31.760)
about, you know, they have sort of notions of their subjective fairness elicited from
Lex Fridman (21:37.040)
them.
Michael Kearns (21:38.160)
It's mainly, you know, kind of scholars who think about fairness kind of making up their
Lex Fridman (21:43.840)
own definitions.
Lex Fridman (21:44.400)
And I think this needs to change actually for many social norms, not just for fairness,
Lex Fridman (21:50.320)
right?
Lex Fridman (21:50.560)
So there's a lot of discussion these days in the AI community about interpretable AI
Lex Fridman (21:56.560)
or understandable AI.
Lex Fridman (21:58.560)
And as far as I can tell, everybody agrees that deep learning or at least the outputs
Michael Kearns (22:04.880)
of deep learning are not very understandable, and people might agree that sparse linear
Michael Kearns (22:11.840)
models with integer coefficients are more understandable.
Lex Fridman (22:15.520)
But nobody's really asked people.
Michael Kearns (22:17.440)
You know, there's very little literature on, you know, sort of showing people models
Lex Fridman (22:21.280)
and asking them, do they understand what the model is doing?
Lex Fridman (22:25.280)
And I think that in all these topics, as these fields mature, we need to start doing more
Lex Fridman (22:32.560)
behavioral work.
Michael Kearns (22:34.400)
Yeah, which is one of my deep passions is psychology.
Lex Fridman (22:38.160)
And I always thought computer scientists will be the best future psychologists in a sense
Michael Kearns (22:44.480)
that data is, especially in this modern world, the data is a really powerful way to understand
Lex Fridman (22:51.680)
and study human behavior.
Lex Fridman (22:53.360)
And you've explored that with your game theory side of work as well.
Michael Kearns (22:56.720)
Yeah, I'd like to think that what you say is true about computer scientists and psychology
Michael Kearns (23:02.240)
from my own limited wandering into human subject experiments.
Michael Kearns (23:07.520)
We have a great deal to learn, not just computer science, but AI and machine learning more
Michael Kearns (23:11.600)
specifically, I kind of think of as imperialist research communities in that, you know, kind
Michael Kearns (23:17.040)
of like physicists in an earlier generation, computer scientists kind of don't think of
Michael Kearns (23:22.800)
any scientific topic that's off limits to them.
Michael Kearns (23:25.440)
They will like freely wander into areas that others have been thinking about for decades
Michael Kearns (23:30.880)
or longer.
Michael Kearns (23:31.440)
And, you know, we usually tend to embarrass ourselves in those efforts for some amount
Michael Kearns (23:37.840)
of time.
Lex Fridman (23:38.320)
Like, you know, I think reinforcement learning is a good example, right?
Lex Fridman (23:41.840)
So a lot of the early work in reinforcement learning, I have complete sympathy for the
Michael Kearns (23:48.160)
control theorists that looked at this and said like, okay, you are reinventing stuff
Lex Fridman (23:53.120)
that we've known since like the forties, right?
Michael Kearns (23:55.600)
But, you know, in my view, eventually this sort of, you know, computer scientists have
Michael Kearns (24:01.120)
made significant contributions to that field, even though we kind of embarrassed ourselves
Lex Fridman (24:06.320)
for the first decade.
Lex Fridman (24:07.520)
So I think if computer scientists are gonna start engaging in kind of psychology, human
Michael Kearns (24:12.080)
subjects type of research, we should expect to be embarrassing ourselves for a good 10
Michael Kearns (24:18.080)
years or so, and then hope that it turns out as well as, you know, some other areas that
Lex Fridman (24:23.600)
we've waded into.
Lex Fridman (24:25.600)
So you kind of mentioned this, just to linger on the idea of an ethical algorithm, of idea
Lex Fridman (24:30.400)
of groups, sort of group thinking and individual thinking.
Lex Fridman (24:33.760)
And we're struggling that.
Michael Kearns (24:35.040)
One of the amazing things about algorithms and your book and just this field of study
Michael Kearns (24:39.280)
is it gets us to ask, like forcing machines, converting these ideas into algorithms is
Lex Fridman (24:46.640)
forcing us to ask questions of ourselves as a human civilization.
Lex Fridman (24:50.160)
So there's a lot of people now in public discourse doing sort of group thinking, thinking like
Michael Kearns (24:58.320)
there's particular sets of groups that we don't wanna discriminate against and so on.
Lex Fridman (25:02.000)
And then there is individuals, sort of in the individual life stories, the struggles
Lex Fridman (25:08.560)
they went through and so on.
Michael Kearns (25:10.000)
Now, like in philosophy, it's easier to do group thinking because you don't, it's very
Lex Fridman (25:16.480)
hard to think about individuals.
Michael Kearns (25:17.920)
There's so much variability, but with data, you can start to actually say, you know what
Lex Fridman (25:23.840)
group thinking is too crude.
Michael Kearns (25:26.400)
You're actually doing more discrimination by thinking in terms of groups and individuals.
Lex Fridman (25:30.880)
Can you linger on that kind of idea of group versus individual and ethics?
Lex Fridman (25:36.720)
And is it good to continue thinking in terms of groups in algorithms?
Lex Fridman (25:41.680)
So let me start by answering a very good high level question with a slightly narrow technical
Michael Kearns (25:49.360)
response, which is these group definitions of fairness, like here's a few groups, like
Lex Fridman (25:54.480)
different racial groups, maybe gender groups, maybe age, what have you.
Lex Fridman (25:59.440)
And let's make sure that for none of these groups, do we have a false negative rate,
Lex Fridman (26:06.480)
which is much higher than any other one of these groups.
Michael Kearns (26:09.200)
Okay, so these are kind of classic group aggregate notions of fairness.
Lex Fridman (26:13.760)
And you know, but at the end of the day, an individual you can think of as a combination
Lex Fridman (26:18.000)
of all of their attributes, right?
Michael Kearns (26:19.360)
They're a member of a racial group, they have a gender, they have an age, and many other
Michael Kearns (26:26.800)
demographic properties that are not biological, but that are still very strong determinants
Lex Fridman (26:33.840)
of outcome and personality and the like.
Lex Fridman (26:36.720)
So one, I think, useful spectrum is to sort of think about that array between the group
Lex Fridman (26:43.920)
and the specific individual, and to realize that in some ways, asking for fairness at
Michael Kearns (26:49.600)
the individual level is to sort of ask for group fairness simultaneously for all possible
Lex Fridman (26:56.800)
combinations of groups.
Lex Fridman (26:57.840)
So in particular, you know, if I build a predictive model that meets some definition of fairness,
Michael Kearns (27:06.480)
definition of fairness by race, by gender, by age, by what have you, marginally, to get
Michael Kearns (27:14.160)
it slightly technical, sort of independently, I shouldn't expect that model to not discriminate
Michael Kearns (27:20.960)
against disabled Hispanic women over age 55, making less than $50,000 a year annually,
Michael Kearns (27:27.440)
even though I might have protected each one of those attributes marginally.
Lex Fridman (27:32.480)
So the optimization, actually, that's a fascinating way to put it.
Lex Fridman (27:35.680)
So you're just optimizing, the one way to achieve the optimizing fairness for individuals
Michael Kearns (27:42.160)
is just to add more and more definitions of groups that each individual belongs to.
Michael Kearns (27:46.080)
That's right.
Michael Kearns (27:47.080)
So, you know, at the end of the day, we could think of all of ourselves as groups of size
Michael Kearns (27:50.320)
one because eventually there's some attribute that separates you from me and everybody else
Lex Fridman (27:55.400)
in the world, okay?
Lex Fridman (27:57.020)
And so it is possible to put, you know, these incredibly coarse ways of thinking about fairness
Lex Fridman (28:03.560)
and these very, very individualistic specific ways on a common scale.
Lex Fridman (28:09.960)
And you know, one of the things we've worked on from a research perspective is, you know,
Lex Fridman (28:14.160)
so we sort of know how to, you know, in relative terms, we know how to provide fairness guarantees
Michael Kearns (28:20.520)
at the core system of the scale.
Michael Kearns (28:22.760)
We don't know how to provide kind of sensible, tractable, realistic fairness guarantees at
Michael Kearns (28:28.240)
the individual level, but maybe we could start creeping towards that by dealing with more
Lex Fridman (28:33.120)
refined subgroups.
Michael Kearns (28:35.040)
I mean, we gave a name to this phenomenon where, you know, you protect, you enforce
Michael Kearns (28:41.000)
some definition of fairness for a bunch of marginal attributes or features, but then
Michael Kearns (28:46.580)
you find yourself discriminating against a combination of them.
Michael Kearns (28:49.980)
We call that fairness gerrymandering because like political gerrymandering, you know, you're
Michael Kearns (28:55.480)
giving some guarantee at the aggregate level, but when you kind of look in a more granular
Michael Kearns (29:01.400)
way at what's going on, you realize that you're achieving that aggregate guarantee by sort
Michael Kearns (29:06.440)
of favoring some groups and discriminating against other ones.
Lex Fridman (29:10.880)
And so there are, you know, it's early days, but there are algorithmic approaches that
Michael Kearns (29:15.940)
let you start creeping towards that, you know, individual end of the spectrum.
Michael Kearns (29:22.440)
Does there need to be human input in the form of weighing the value of the importance of
Lex Fridman (29:30.740)
each kind of group?
Lex Fridman (29:33.000)
So for example, is it like, so gender, say crudely speaking, male and female, and then
Michael Kearns (29:42.400)
different races, are we as humans supposed to put value on saying gender is 0.6 and race
Lex Fridman (29:51.980)
is 0.4 in terms of in the big optimization of achieving fairness?
Lex Fridman (29:59.200)
Is that kind of what humans are supposed to do here?
Michael Kearns (30:01.720)
I mean, of course, you know, I don't need to tell you that, of course, technically one
Michael Kearns (30:05.320)
could incorporate such weights if you wanted to into a definition of fairness.
Michael Kearns (30:10.720)
You know, fairness is an interesting topic in that having worked in the book being about
Michael Kearns (30:19.680)
both fairness, privacy, and many other social norms, fairness, of course, is a much, much
Lex Fridman (30:24.820)
more loaded topic.
Lex Fridman (30:27.160)
So privacy, I mean, people want privacy, people don't like violations of privacy, violations
Michael Kearns (30:32.180)
of privacy cause damage, angst, and bad publicity for the companies that are victims of them.
Lex Fridman (30:40.680)
But sort of everybody agrees more data privacy would be better than less data privacy.
Lex Fridman (30:48.020)
And you don't have these, somehow the discussions of fairness don't become politicized along
Michael Kearns (30:53.780)
other dimensions like race and about gender and, you know, whether we, and, you know,
Michael Kearns (31:01.900)
you quickly find yourselves kind of revisiting topics that have been kind of unresolved forever,
Lex Fridman (31:10.760)
like affirmative action, right?
Michael Kearns (31:12.560)
Sort of, you know, like, why are you protecting, and some people will say, why are you protecting
Lex Fridman (31:16.400)
this particular racial group?
Lex Fridman (31:20.320)
And others will say, well, we need to do that as a matter of retribution.
Michael Kearns (31:26.240)
Other people will say, it's a matter of economic opportunity.
Lex Fridman (31:30.040)
And I don't know which of, you know, whether any of these are the right answers, but you
Michael Kearns (31:34.920)
sort of, fairness is sort of special in that as soon as you start talking about it, you
Michael Kearns (31:39.840)
inevitably have to participate in debates about fair to whom, at what expense to who
Michael Kearns (31:46.360)
else.
Michael Kearns (31:47.360)
I mean, even in criminal justice, right, you know, where people talk about fairness in
Michael Kearns (31:56.180)
criminal sentencing or, you know, predicting failures to appear or making parole decisions
Michael Kearns (32:02.840)
or the like, they will, you know, they'll point out that, well, these definitions of
Michael Kearns (32:08.340)
fairness are all about fairness for the criminals.
Lex Fridman (32:13.640)
And what about fairness for the victims, right?
Lex Fridman (32:16.120)
So when I basically say something like, well, the false incarceration rate for black people
Lex Fridman (32:22.840)
and white people needs to be roughly the same, you know, there's no mention of potential
Michael Kearns (32:28.300)
victims of criminals in such a fairness definition.
Lex Fridman (32:33.180)
And that's the realm of public discourse.
Michael Kearns (32:34.960)
I should actually recommend, I just listened to people listening, Intelligence Squares
Lex Fridman (32:41.200)
debates, US edition just had a debate.
Michael Kearns (32:45.080)
They have this structure where you have old Oxford style or whatever they're called, debates,
Michael Kearns (32:50.080)
you know, it's two versus two and they talked about affirmative action and it was incredibly
Michael Kearns (32:55.680)
interesting that there's really good points on every side of this issue, which is fascinating
Lex Fridman (33:03.000)
to listen to.
Michael Kearns (33:04.000)
Yeah, yeah, I agree.
Lex Fridman (33:05.680)
And so it's interesting to be a researcher trying to do, for the most part, technical
Michael Kearns (33:12.400)
algorithmic work, but Aaron and I both quickly learned you cannot do that and then go out
Lex Fridman (33:17.980)
and talk about it and expect people to take it seriously if you're unwilling to engage
Lex Fridman (33:22.640)
in these broader debates that are entirely extra algorithmic, right?
Lex Fridman (33:28.160)
They're not about, you know, algorithms and making algorithms better.
Michael Kearns (33:31.200)
They're sort of, you know, as you said, sort of like, what should society be protecting
Lex Fridman (33:35.160)
in the first place?
Michael Kearns (33:36.160)
When you discuss the fairness, an algorithm that achieves fairness, whether in the constraints
Lex Fridman (33:42.320)
and the objective function, there's an immediate kind of analysis you can perform, which is
Michael Kearns (33:48.520)
saying, if you care about fairness in gender, this is the amount that you have to pay for
Lex Fridman (33:56.520)
it in terms of the performance of the system.
Michael Kearns (33:59.280)
Like do you, is there a role for statements like that in a table, in a paper, or do you
Lex Fridman (34:03.960)
want to really not touch that?
Michael Kearns (34:06.680)
No, no, we want to touch that and we do touch it.
Lex Fridman (34:09.800)
So I mean, just again, to make sure I'm not promising your viewers more than we know how
Michael Kearns (34:16.680)
to provide, but if you pick a definition of fairness, like I'm worried about gender discrimination
Lex Fridman (34:21.760)
and you pick a notion of harm, like false rejection for a loan, for example, and you
Michael Kearns (34:27.100)
give me a model, I can definitely, first of all, go audit that model.
Michael Kearns (34:30.960)
It's easy for me to go, you know, from data to kind of say like, okay, your false rejection
Lex Fridman (34:36.640)
rate on women is this much higher than it is on men, okay?
Lex Fridman (34:41.880)
But once you also put the fairness into your objective function, I mean, I think the table
Lex Fridman (34:47.240)
that you're talking about is what we would call the Pareto curve, right?
Michael Kearns (34:51.640)
You can literally trace out, and we give examples of such plots on real data sets in the book,
Michael Kearns (34:58.740)
you have two axes.
Michael Kearns (34:59.760)
On the X axis is your error, on the Y axis is unfairness by whatever, you know, if it's
Michael Kearns (35:06.360)
like the disparity between false rejection rates between two groups.
Lex Fridman (35:12.240)
And you know, your algorithm now has a knob that basically says, how strongly do I want
Lex Fridman (35:17.080)
to enforce fairness?
Lex Fridman (35:19.400)
And the less unfair, you know, if the two axes are error and unfairness, we'd like to
Michael Kearns (35:24.680)
be at zero, zero.
Lex Fridman (35:26.260)
We'd like zero error and zero unfairness simultaneously.
Michael Kearns (35:31.280)
Anybody who works in machine learning knows that you're generally not going to get to
Lex Fridman (35:34.840)
zero error period without any fairness constraint whatsoever.
Lex Fridman (35:38.840)
So that's not going to happen.
Lex Fridman (35:41.060)
But in general, you know, you'll get this, you'll get some kind of convex curve that
Michael Kearns (35:46.480)
specifies the numerical trade off you face.
Michael Kearns (35:49.960)
You know, if I want to go from 17% error down to 16% error, what will be the increase in
Lex Fridman (35:57.920)
unfairness that I experienced as a result of that?
Lex Fridman (36:02.960)
And so this curve kind of specifies the, you know, kind of undominated models.
Michael Kearns (36:09.520)
Models that are off that curve are, you know, can be strictly improved in one or both dimensions.
Michael Kearns (36:14.480)
You can, you know, either make the error better or the unfairness better or both.
Lex Fridman (36:18.840)
And I think our view is that not only are these objects, these Pareto curves, you know,
Michael Kearns (36:26.000)
with efficient frontiers as you might call them, not only are they valuable scientific
Michael Kearns (36:34.360)
objects, I actually think that they in the near term might need to be the interface between
Lex Fridman (36:41.320)
researchers working in the field and stakeholders in given problems.
Lex Fridman (36:46.180)
So you know, you could really imagine telling a criminal jurisdiction, look, if you're concerned
Lex Fridman (36:55.320)
about racial fairness, but you're also concerned about accuracy.
Michael Kearns (36:58.820)
You want to, you know, you want to release on parole people that are not going to recommit
Lex Fridman (37:05.200)
a violent crime and you don't want to release the ones who are.
Lex Fridman (37:08.600)
So you know, that's accuracy.
Lex Fridman (37:10.600)
But if you also care about those, you know, the mistakes you make not being disproportionately
Michael Kearns (37:15.120)
on one racial group or another, you can show this curve.
Michael Kearns (37:19.160)
I'm hoping that in the near future, it'll be possible to explain these curves to non
Michael Kearns (37:23.980)
technical people that are the ones that have to make the decision, where do we want to
Lex Fridman (37:29.520)
be on this curve?
Lex Fridman (37:30.520)
Like, what are the relative merits or value of having lower error versus lower unfairness?
Lex Fridman (37:38.440)
You know, that's not something computer scientists should be deciding for society, right?
Michael Kearns (37:43.560)
That, you know, the people in the field, so to speak, the policymakers, the regulators,
Lex Fridman (37:49.400)
that's who should be making these decisions.
Lex Fridman (37:51.680)
But I think and hope that they can be made to understand that these trade offs generally
Michael Kearns (37:56.600)
exist and that you need to pick a point and like, and ignoring the trade off, you know,
Lex Fridman (38:03.280)
you're implicitly picking a point anyway, right?
Lex Fridman (38:06.760)
You just don't know it and you're not admitting it.
Michael Kearns (38:09.400)
Just to linger on the point of trade offs, I think that's a really important thing to
Lex Fridman (38:12.740)
sort of think about.
Lex Fridman (38:15.400)
So you think when we start to optimize for fairness, there's almost always in most system
Lex Fridman (38:22.360)
going to be trade offs.
Lex Fridman (38:25.080)
Can you like, what's the trade off between just to clarify, there have been some sort
Lex Fridman (38:30.200)
of technical terms thrown around, but sort of a perfectly fair world.
Lex Fridman (38:39.240)
Why is that?
Lex Fridman (38:40.760)
Why will somebody be upset about that?
Michael Kearns (38:43.760)
The specific trade off I talked about just in order to make things very concrete was
Lex Fridman (38:47.400)
between numerical error and some numerical measure of unfairness.
Lex Fridman (38:53.360)
What is numerical error in the case of...
Michael Kearns (38:56.400)
Just like say predictive error, like, you know, the probability or frequency with which
Michael Kearns (39:01.000)
you release somebody on parole who then goes on to recommit a violent crime or keep incarcerated
Lex Fridman (39:08.480)
somebody who would not have recommitted a violent crime.
Lex Fridman (39:10.920)
So in the case of awarding somebody parole or giving somebody parole or letting them
Lex Fridman (39:17.480)
out on parole, you don't want them to recommit a crime.
Lex Fridman (39:21.480)
So it's your system failed in prediction if they happen to do a crime.
Lex Fridman (39:26.600)
Okay, so that's one axis.
Lex Fridman (39:30.280)
And what's the fairness axis?
Lex Fridman (39:31.800)
So then the fairness axis might be the difference between racial groups in the kind of false
Michael Kearns (39:39.640)
positive predictions, namely people that I kept incarcerated predicting that they would
Lex Fridman (39:47.840)
recommit a violent crime when in fact they wouldn't have.
Michael Kearns (39:51.200)
Right.
Lex Fridman (39:52.200)
And the unfairness of that, just to linger it and allow me to in eloquently to try to
Michael Kearns (40:00.840)
sort of describe why that's unfair, why unfairness is there.
Michael Kearns (40:06.360)
The unfairness you want to get rid of is that in the judge's mind, the bias of having being
Michael Kearns (40:13.280)
brought up to society, the slight racial bias, the racism that exists in the society, you
Lex Fridman (40:18.480)
want to remove that from the system.
Michael Kearns (40:21.760)
Another way that's been debated is sort of equality of opportunity versus equality of
Lex Fridman (40:28.720)
outcome.
Lex Fridman (40:30.440)
And there's a weird dance there that's really difficult to get right.
Lex Fridman (40:35.120)
And we don't, affirmative action is exploring that space.
Michael Kearns (40:40.200)
Right.
Lex Fridman (40:41.200)
And then this also quickly bleeds into questions like, well, maybe if one group really does
Michael Kearns (40:48.840)
recommit crimes at a higher rate, the reason for that is that at some earlier point in
Michael Kearns (40:55.240)
the pipeline or earlier in their lives, they didn't receive the same resources that the
Michael Kearns (41:00.200)
other group did.
Lex Fridman (41:02.560)
And so there's always in kind of fairness discussions, the possibility that the real
Lex Fridman (41:08.480)
injustice came earlier, right?
Michael Kearns (41:11.040)
Earlier in this individual's life, earlier in this group's history, et cetera, et cetera.
Lex Fridman (41:16.360)
And so a lot of the fairness discussion is almost, the goal is for it to be a corrective
Lex Fridman (41:20.840)
mechanism to account for the injustice earlier in life.
Michael Kearns (41:25.440)
By some definitions of fairness or some theories of fairness, yeah.
Michael Kearns (41:29.640)
Others would say like, look, it's not to correct that injustice, it's just to kind of level
Michael Kearns (41:35.120)
the playing field right now and not falsely incarcerate more people of one group than
Lex Fridman (41:40.720)
another group.
Lex Fridman (41:41.720)
But I mean, I think just it might be helpful just to demystify a little bit about the many
Michael Kearns (41:46.960)
ways in which bias or unfairness can come into algorithms, especially in the machine
Lex Fridman (41:54.940)
learning era, right?
Michael Kearns (41:55.940)
I think many of your viewers have probably heard these examples before, but let's say
Lex Fridman (42:00.680)
I'm building a face recognition system, right?
Lex Fridman (42:04.160)
And so I'm kind of gathering lots of images of faces and trying to train the system to
Michael Kearns (42:12.000)
recognize new faces of those individuals from training on a training set of those faces
Lex Fridman (42:17.340)
of individuals.
Lex Fridman (42:19.080)
And it shouldn't surprise anybody or certainly not anybody in the field of machine learning
Michael Kearns (42:24.860)
if my training data set was primarily white males and I'm training the model to maximize
Michael Kearns (42:34.960)
the overall accuracy on my training data set, that the model can reduce its error most by
Michael Kearns (42:44.060)
getting things right on the white males that constitute the majority of the data set, even
Lex Fridman (42:48.800)
if that means that on other groups, they will be less accurate, okay?
Lex Fridman (42:53.640)
Now, there's a bunch of ways you could think about addressing this.
Michael Kearns (42:57.720)
One is to deliberately put into the objective of the algorithm not to optimize the error
Michael Kearns (43:05.760)
at the expense of this discrimination, and then you're kind of back in the land of these
Michael Kearns (43:09.060)
kind of two dimensional numerical trade offs.
Michael Kearns (43:13.140)
A valid counter argument is to say like, well, no, you don't have to, there's no, you know,
Michael Kearns (43:18.660)
the notion of the tension between error and accuracy here is a false one.
Michael Kearns (43:22.840)
You could instead just go out and get much more data on these other groups that are in
Michael Kearns (43:27.760)
the minority and, you know, equalize your data set, or you could train a separate model
Lex Fridman (43:34.580)
on those subgroups and, you know, have multiple models.
Michael Kearns (43:38.800)
The point I think we would, you know, we tried to make in the book is that those things have
Lex Fridman (43:43.120)
cost too, right?
Michael Kearns (43:45.160)
Going out and gathering more data on groups that are relatively rare compared to your
Michael Kearns (43:51.200)
plurality or more majority group that, you know, it may not cost you in the accuracy
Michael Kearns (43:55.520)
of the model, but it's going to cost, you know, it's going to cost the company developing
Michael Kearns (43:59.460)
this model more money to develop that, and it also costs more money to build separate
Michael Kearns (44:04.460)
predictive models and to implement and deploy them.
Lex Fridman (44:07.500)
So even if you can find a way to avoid the tension between error and accuracy in training
Michael Kearns (44:14.100)
a model, you might push the cost somewhere else, like money, like development time, research
Lex Fridman (44:20.720)
time and the like.
Michael Kearns (44:22.920)
There are fundamentally difficult philosophical questions, in fairness, and we live in a very
Lex Fridman (44:30.200)
divisive political climate, outraged culture.
Michael Kearns (44:34.160)
There is alt right folks on 4chan, trolls.
Lex Fridman (44:38.560)
There is social justice warriors on Twitter.
Michael Kearns (44:43.320)
There's very divisive, outraged folks on all sides of every kind of system.
Lex Fridman (44:49.920)
How do you, how do we as engineers build ethical algorithms in such divisive culture?
Lex Fridman (44:57.280)
Do you think they could be disjoint?
Michael Kearns (44:59.540)
The human has to inject your values, and then you can optimize over those values.
Lex Fridman (45:04.700)
But in our times, when you start actually applying these systems, things get a little
Lex Fridman (45:09.560)
bit challenging for the public discourse.
Lex Fridman (45:13.100)
How do you think we can proceed?
Michael Kearns (45:14.920)
Yeah, I mean, for the most part in the book, a point that we try to take some pains to
Michael Kearns (45:21.000)
make is that we don't view ourselves or people like us as being in the position of deciding
Michael Kearns (45:29.560)
for society what the right social norms are, what the right definitions of fairness are.
Michael Kearns (45:34.960)
Our main point is to just show that if society or the relevant stakeholders in a particular
Michael Kearns (45:41.660)
domain can come to agreement on those sorts of things, there's a way of encoding that
Michael Kearns (45:47.160)
into algorithms in many cases, not in all cases.
Michael Kearns (45:50.720)
One other misconception that hopefully we definitely dispel is sometimes people read
Michael Kearns (45:55.640)
the title of the book and I think not unnaturally fear that what we're suggesting is that the
Michael Kearns (46:00.880)
algorithms themselves should decide what those social norms are and develop their own notions
Michael Kearns (46:05.760)
of fairness and privacy or ethics, and we're definitely not suggesting that.
Michael Kearns (46:10.160)
The title of the book is Ethical Algorithm, by the way, and I didn't think of that interpretation
Michael Kearns (46:13.920)
of the title.
Lex Fridman (46:14.920)
That's interesting.
Michael Kearns (46:15.920)
Yeah, yeah.
Michael Kearns (46:16.920)
I mean, especially these days where people are concerned about the robots becoming our
Michael Kearns (46:21.080)
overlords, the idea that the robots would also sort of develop their own social norms
Lex Fridman (46:25.980)
is just one step away from that.
Lex Fridman (46:29.360)
But I do think, obviously, despite disclaimer that people like us shouldn't be making those
Michael Kearns (46:35.240)
decisions for society, we are kind of living in a world where in many ways computer scientists
Michael Kearns (46:40.880)
have made some decisions that have fundamentally changed the nature of our society and democracy
Lex Fridman (46:46.820)
and sort of civil discourse and deliberation in ways that I think most people generally
Lex Fridman (46:53.240)
feel are bad these days, right?
Lex Fridman (46:55.720)
But they had to make, so if we look at people at the heads of companies and so on, they
Lex Fridman (47:01.120)
had to make those decisions, right?
Michael Kearns (47:02.800)
There has to be decisions, so there's two options, either you kind of put your head
Michael Kearns (47:08.440)
in the sand and don't think about these things and just let the algorithm do what it does,
Michael Kearns (47:14.000)
or you make decisions about what you value, you know, of injecting moral values into the
Michael Kearns (47:19.320)
algorithm.
Michael Kearns (47:20.320)
Look, I never mean to be an apologist for the tech industry, but I think it's a little
Michael Kearns (47:26.760)
bit too far to sort of say that explicit decisions were made about these things.
Lex Fridman (47:31.120)
So let's, for instance, take social media platforms, right?
Lex Fridman (47:34.920)
So like many inventions in technology and computer science, a lot of these platforms
Lex Fridman (47:40.160)
that we now use regularly kind of started as curiosities, right?
Michael Kearns (47:45.120)
I remember when things like Facebook came out and its predecessors like Friendster,
Michael Kearns (47:49.240)
which nobody even remembers now, people really wonder, like, why would anybody want to spend
Lex Fridman (47:55.620)
time doing that?
Michael Kearns (47:56.620)
I mean, even the web when it first came out, when it wasn't populated with much content
Lex Fridman (48:01.480)
and it was largely kind of hobbyists building their own kind of ramshackle websites, a lot
Lex Fridman (48:07.100)
of people looked at this and said, well, what is the purpose of this thing?
Lex Fridman (48:09.960)
Why is this interesting?
Lex Fridman (48:11.000)
Who would want to do this?
Lex Fridman (48:12.880)
And so even things like Facebook and Twitter, yes, technical decisions were made by engineers,
Michael Kearns (48:18.120)
by scientists, by executives in the design of those platforms, but, you know, I don't
Michael Kearns (48:23.520)
think 10 years ago anyone anticipated that those platforms, for instance, might kind
Michael Kearns (48:32.240)
of acquire undue, you know, influence on political discourse or on the outcomes of elections.
Lex Fridman (48:42.200)
And I think the scrutiny that these companies are getting now is entirely appropriate, but
Michael Kearns (48:47.600)
I think it's a little too harsh to kind of look at history and sort of say like, oh,
Michael Kearns (48:53.080)
you should have been able to anticipate that this would happen with your platform.
Lex Fridman (48:56.320)
And in this sort of gaming chapter of the book, one of the points we're making is that,
Michael Kearns (49:00.600)
you know, these platforms, right, they don't operate in isolation.
Lex Fridman (49:05.200)
So unlike the other topics we're discussing, like fairness and privacy, like those are
Michael Kearns (49:09.360)
really cases where algorithms can operate on your data and make decisions about you
Lex Fridman (49:13.600)
and you're not even aware of it, okay?
Lex Fridman (49:16.300)
Things like Facebook and Twitter, these are, you know, these are systems, right?
Michael Kearns (49:20.280)
These are social systems and their evolution, even their technical evolution because machine
Michael Kearns (49:25.960)
learning is involved, is driven in no small part by the behavior of the users themselves
Lex Fridman (49:31.680)
and how the users decide to adopt them and how to use them.
Lex Fridman (49:35.680)
And so, you know, I'm kind of like who really knew that, you know, until we saw it happen,
Lex Fridman (49:44.600)
who knew that these things might be able to influence the outcome of elections?
Michael Kearns (49:48.340)
Who knew that, you know, they might polarize political discourse because of the ability
Michael Kearns (49:55.120)
to, you know, decide who you interact with on the platform and also with the platform
Michael Kearns (50:00.840)
naturally using machine learning to optimize for your own interest that they would further
Michael Kearns (50:05.080)
isolate us from each other and, you know, like feed us all basically just the stuff
Michael Kearns (50:10.080)
that we already agreed with.
Lex Fridman (50:12.080)
So I think, you know, we've come to that outcome, I think, largely, but I think it's
Michael Kearns (50:18.120)
something that we all learned together, including the companies as these things happen.
Lex Fridman (50:24.240)
You asked like, well, are there algorithmic remedies to these kinds of things?
Lex Fridman (50:29.940)
And again, these are big problems that are not going to be solved with, you know, somebody
Lex Fridman (50:35.360)
going in and changing a few lines of code somewhere in a social media platform.
Lex Fridman (50:40.040)
But I do think in many ways, there are definitely ways of making things better.
Michael Kearns (50:44.960)
I mean, like an obvious recommendation that we make at some point in the book is like,
Michael Kearns (50:49.360)
look, you know, to the extent that we think that machine learning applied for personalization
Michael Kearns (50:55.280)
purposes in things like newsfeed, you know, or other platforms has led to polarization
Lex Fridman (51:03.480)
and intolerance of opposing viewpoints.
Michael Kearns (51:07.940)
As you know, right, these algorithms have models, right, and they kind of place people
Michael Kearns (51:11.880)
in some kind of metric space, and they place content in that space, and they sort of know
Lex Fridman (51:17.700)
the extent to which I have an affinity for a particular type of content.
Lex Fridman (51:22.180)
And by the same token, they also probably have that same model probably gives you a
Lex Fridman (51:26.400)
good idea of the stuff I'm likely to violently disagree with or be offended by, okay?
Lex Fridman (51:32.760)
So you know, in this case, there really is some knob you could tune that says like, instead
Michael Kearns (51:37.440)
of showing people only what they like and what they want, let's show them some stuff
Michael Kearns (51:43.040)
that we think that they don't like, or that's a little bit further away.
Lex Fridman (51:46.160)
And you could even imagine users being able to control this, you know, just like everybody
Michael Kearns (51:51.680)
gets a slider, and that slider says like, you know, how much stuff do you want to see
Michael Kearns (51:58.240)
that's kind of, you know, you might disagree with, or is at least further from your interest.
Michael Kearns (52:02.960)
It's almost like an exploration button.
Lex Fridman (52:05.720)
So just get your intuition.
Lex Fridman (52:08.360)
Do you think engagement, so like you staying on the platform, you're staying engaged.
Lex Fridman (52:15.160)
Do you think fairness, ideas of fairness won't emerge?
Lex Fridman (52:19.920)
Like how bad is it to just optimize for engagement?
Lex Fridman (52:23.740)
Do you think we'll run into big trouble if we're just optimizing for how much you love
Lex Fridman (52:28.440)
the platform?
Lex Fridman (52:29.440)
Well, I mean, optimizing for engagement kind of got us where we are.
Lex Fridman (52:34.800)
So do you, one, have faith that it's possible to do better?
Lex Fridman (52:39.960)
And two, if it is, how do we do better?
Lex Fridman (52:44.240)
I mean, it's definitely possible to do different, right?
Lex Fridman (52:47.060)
And again, you know, it's not as if I think that doing something different than optimizing
Michael Kearns (52:51.700)
for engagement won't cost these companies in real ways, including revenue and profitability
Lex Fridman (52:57.880)
potentially.
Michael Kearns (52:58.880)
In the short term at least.
Lex Fridman (53:00.600)
Yeah.
Michael Kearns (53:01.600)
In the short term.
Lex Fridman (53:02.600)
Right.
Lex Fridman (53:03.600)
And again, you know, if I worked at these companies, I'm sure that it would have seemed
Lex Fridman (53:08.920)
like the most natural thing in the world also to want to optimize engagement, right?
Lex Fridman (53:12.640)
And that's good for users in some sense.
Michael Kearns (53:14.600)
You want them to be, you know, vested in the platform and enjoying it and finding it useful,
Michael Kearns (53:19.600)
interesting, and or productive.
Lex Fridman (53:21.660)
But you know, my point is, is that the idea that there is, that it's sort of out of their
Michael Kearns (53:27.080)
hands as you said, or that there's nothing to do about it, never say never, but that
Lex Fridman (53:31.560)
strikes me as implausible as a machine learning person, right?
Michael Kearns (53:34.560)
I mean, these companies are driven by machine learning and this optimization of engagement
Lex Fridman (53:39.600)
is essentially driven by machine learning, right?
Michael Kearns (53:42.040)
It's driven by not just machine learning, but you know, very, very large scale A, B
Michael Kearns (53:47.120)
experimentation where you kind of tweak some element of the user interface or tweak some
Michael Kearns (53:53.080)
component of an algorithm or tweak some component or feature of your click through prediction
Lex Fridman (53:59.520)
model.
Lex Fridman (54:01.200)
And my point is, is that anytime you know how to optimize for something, you, you know,
Michael Kearns (54:06.360)
by def, almost by definition, that solution tells you how not to optimize for it or to
Michael Kearns (54:10.600)
do something different.
Lex Fridman (54:13.240)
Engagement can be measured.
Lex Fridman (54:16.200)
So sort of optimizing for sort of minimizing divisiveness or maximizing intellectual growth
Lex Fridman (54:25.320)
over the lifetime of a human being are very difficult to measure.
Michael Kearns (54:30.160)
That's right.
Lex Fridman (54:31.160)
And I'm not claiming that doing something different will immediately make it apparent
Michael Kearns (54:38.240)
that this is a good thing for society and in particular, I mean, I think one way of
Michael Kearns (54:42.320)
thinking about where we are on some of these social media platforms is that, you know,
Lex Fridman (54:47.400)
it kind of feels a bit like we're in a bad equilibrium, right?
Michael Kearns (54:50.880)
That these systems are helping us all kind of optimize something myopically and selfishly
Michael Kearns (54:55.920)
for ourselves and of course, from an individual standpoint at any given moment, like why would
Michael Kearns (55:02.280)
I want to see things in my newsfeed that I found irrelevant, offensive or, you know,
Lex Fridman (55:07.280)
or the like, okay?
Lex Fridman (55:09.240)
But you know, maybe by all of us, you know, having these platforms myopically optimized
Michael Kearns (55:15.320)
in our interests, we have reached a collective outcome as a society that we're unhappy with
Lex Fridman (55:20.920)
in different ways.
Michael Kearns (55:21.920)
Let's say with respect to things like, you know, political discourse and tolerance of
Lex Fridman (55:26.360)
opposing viewpoints.
Lex Fridman (55:28.160)
And if Mark Zuckerberg gave you a call and said, I'm thinking of taking a sabbatical,
Lex Fridman (55:34.840)
could you run Facebook for me for six months?
Lex Fridman (55:37.440)
What would you, how?
Michael Kearns (55:39.240)
I think no thanks would be my first response, but there are many aspects of being the head
Michael Kearns (55:45.720)
of the entire company that are kind of entirely exogenous to many of the things that we're
Lex Fridman (55:51.200)
discussing here.
Michael Kearns (55:52.200)
Yes.
Lex Fridman (55:53.200)
And so I don't really think I would need to be CEO of Facebook to kind of implement the,
Michael Kearns (55:58.680)
you know, more limited set of solutions that I might imagine.
Lex Fridman (56:02.720)
But I think one concrete thing they could do is they could experiment with letting people
Michael Kearns (56:08.940)
who chose to, to see more stuff in their newsfeed that is not entirely kind of chosen to optimize
Lex Fridman (56:17.020)
for their particular interests, beliefs, et cetera.
Lex Fridman (56:22.500)
So the, the kind of thing, so I could speak to YouTube, but I think Facebook probably
Michael Kearns (56:27.240)
does something similar is they're quite effective at automatically finding what sorts of groups
Michael Kearns (56:34.880)
you belong to, not based on race or gender or so on, but based on the kind of stuff you
Lex Fridman (56:40.260)
enjoy watching in the case of YouTube.
Michael Kearns (56:43.120)
Sort of, it's a, it's a difficult thing for Facebook or YouTube to then say, well, you
Lex Fridman (56:50.160)
know what?
Michael Kearns (56:51.160)
We're going to show you something from a very different cluster.
Michael Kearns (56:54.700)
Even though we believe algorithmically, you're unlikely to enjoy that thing sort of that's
Michael Kearns (57:00.020)
a weird jump to make.
Michael Kearns (57:02.340)
There has to be a human, like at the very top of that system that says, well, that will
Michael Kearns (57:07.020)
be longterm healthy for you.
Lex Fridman (57:09.840)
That's more than an algorithmic decision.
Michael Kearns (57:11.880)
Or that same person could say that'll be longterm healthy for the platform or for the platform's
Lex Fridman (57:18.460)
influence on society outside of the platform, right?
Lex Fridman (57:22.560)
And it, you know, it's easy for me to sit here and say these things, but conceptually
Michael Kearns (57:27.020)
I do not think that these are kind of totally or should, they shouldn't be kind of completely
Lex Fridman (57:32.920)
alien ideas, right?
Michael Kearns (57:34.800)
That, you know, you could try things like this and it wouldn't be, you know, we wouldn't
Michael Kearns (57:40.880)
have to invent entirely new science to do it because if we're all already embedded in
Michael Kearns (57:45.820)
some metric space and there's a notion of distance between you and me and every other,
Michael Kearns (57:50.520)
every piece of content, then, you know, we know exactly, you know, the same model that
Michael Kearns (57:56.060)
tells, you know, dictates how to make me really happy also tells how to make me as unhappy
Michael Kearns (58:03.340)
as possible as well.
Lex Fridman (58:04.960)
Right.
Michael Kearns (58:05.960)
The focus in your book and algorithmic fairness research today in general is on machine learning,
Michael Kearns (58:11.000)
like we said, is data, but, and just even the entire AI field right now is captivated
Michael Kearns (58:16.800)
with machine learning, with deep learning.
Lex Fridman (58:19.720)
Do you think ideas in symbolic AI or totally other kinds of approaches are interesting,
Lex Fridman (58:25.480)
useful in the space, have some promising ideas in terms of fairness?
Lex Fridman (58:31.400)
I haven't thought about that question specifically in the context of fairness.
Lex Fridman (58:35.240)
I definitely would agree with that statement in the large, right?
Michael Kearns (58:39.040)
I mean, I am, you know, one of many machine learning researchers who do believe that the
Michael Kearns (58:46.840)
great successes that have been shown in machine learning recently are great successes, but
Lex Fridman (58:51.400)
they're on a pretty narrow set of tasks.
Michael Kearns (58:53.280)
I mean, I don't, I don't think we're kind of notably closer to general artificial intelligence
Lex Fridman (59:00.480)
now than we were when I started my career.
Michael Kearns (59:03.360)
I mean, there's been progress and I do think that we are kind of as a community, maybe
Michael Kearns (59:08.640)
looking a bit where the light is, but the light is shining pretty bright there right
Michael Kearns (59:12.000)
now and we're finding a lot of stuff.
Lex Fridman (59:13.760)
So I don't want to like argue with the progress that's been made in areas like deep learning,
Michael Kearns (59:18.520)
for example.
Michael Kearns (59:19.880)
This touches another sort of related thing that you mentioned and that people might misinterpret
Michael Kearns (59:25.080)
from the title of your book, ethical algorithm.
Lex Fridman (59:27.420)
Is it possible for the algorithm to automate some of those decisions?
Michael Kearns (59:31.800)
Sort of a higher level decisions of what kind of, like what, what should be fair, what should
Lex Fridman (59:37.400)
be fair.
Michael Kearns (59:38.720)
The more you know about a field, the more aware you are of its limitations.
Lex Fridman (59:43.400)
And so I'm a, I'm pretty leery of sort of trying, you know, there's, there's so much
Michael Kearns (59:47.840)
we don't all, we already don't know in fairness, even when we're the ones picking the fairness
Michael Kearns (59:53.760)
definitions and, you know, comparing alternatives and thinking about the tensions between different
Michael Kearns (59:58.960)
definitions that the idea of kind of letting the algorithm start exploring as well.
🔗 相关节目