Po-Shen Loh: Mathematics, Math Olympiad, Combinatorics & Contact Tracing

Lex Fridman (01:16.640)

in those videos.

Po-Shen Loh (01:17.840)

I'm hoping to do a few more short videos

Lex Fridman (01:19.960)

in the coming months that are educational in nature

Po-Shen Loh (01:23.160)

on AI, robotics, math, science, philosophy,

Lex Fridman (01:26.520)

or if all else fails,

Po-Shen Loh (01:28.840)

just fun snippets into my life on music, books, martial arts,

Lex Fridman (01:32.920)

and other random things,

Po-Shen Loh (01:34.720)

if that's of interest to anyone at all.

Lex Fridman (01:38.240)

This is the Lex Friedman Podcast,

Lex Fridman (01:40.200)

and here's my conversation with Po Shenlow.

Lex Fridman (01:43.840)

You know, you mentioned you really enjoy flying

Lex Fridman (01:46.520)

and experiencing different people in different places.

Lex Fridman (01:49.720)

There's something about flying for me,

Po-Shen Loh (01:51.160)

I don't know if you have the same experience,

Lex Fridman (01:53.080)

that every time I get on an airplane,

Po-Shen Loh (01:55.680)

it's incredible to me that human beings

Lex Fridman (01:58.040)

have actually been able to achieve this.

Lex Fridman (02:00.240)

And when I look at like what's happening now

Lex Fridman (02:03.720)

with humans traveling out into space,

Po-Shen Loh (02:06.400)

I see it as all the same thing.

Lex Fridman (02:08.080)

It's incredible that humans are able to get into a box

Lex Fridman (02:11.240)

and fly in the air and safely and land

Lex Fridman (02:16.320)

in the same, it seems like,

Lex Fridman (02:18.040)

and everybody's taking it for granted.

Lex Fridman (02:19.920)

So when I observe them, it's quite fascinating

Po-Shen Loh (02:22.480)

because I see that cleanly mapping to the world

Lex Fridman (02:25.880)

where we're now in rockets and traveling to the moon,

Po-Shen Loh (02:31.520)

traveling to Mars, and at the same kind of way,

Lex Fridman (02:34.360)

I can already see the future

Po-Shen Loh (02:36.480)

where we will all take it for granted.

Lex Fridman (02:40.040)

So I don't know if you have,

Po-Shen Loh (02:42.680)

you personally, when you fly,

Lex Fridman (02:44.320)

have the same kind of magical experience

Lex Fridman (02:46.440)

of like how the heck did humans actually accomplish this?

Lex Fridman (02:49.080)

So I do, especially when there's turbulence,

Po-Shen Loh (02:52.560)

which is like on the way here, there was turbulence

Lex Fridman (02:56.080)

and the plane jiggled, even the flight attendant

Po-Shen Loh (02:58.880)

had to hold onto the side.

Lex Fridman (03:00.200)

And I was just thinking to myself,

Po-Shen Loh (03:01.360)

it's amazing that this happens all the time

Lex Fridman (03:03.200)

and the wings don't fall off,

Po-Shen Loh (03:04.960)

given how many planes are flying.

Lex Fridman (03:06.600)

But then I often think about it and I'm like,

Po-Shen Loh (03:09.160)

a long time ago, I think people didn't trust elevators

Lex Fridman (03:12.160)

in a 40 story building in New York City.

Lex Fridman (03:14.400)

And now we just take it completely for granted

Lex Fridman (03:17.000)

that you can step into this shaft,

Po-Shen Loh (03:18.600)

which is 40 floors up and down, and it will just not fail.

Lex Fridman (03:24.440)

Yeah, again, I'm the same way with elevators,

Lex Fridman (03:26.760)

but also buildings, when I'll stand on the 40th floor

Lex Fridman (03:30.360)

and wonder how the heck are we not falling right now?

Po-Shen Loh (03:35.320)

Like how amazing it is with the high winds,

Lex Fridman (03:39.120)

like structurally, just the earthquakes and the vibrations,

Po-Shen Loh (03:42.200)

I mean, natural vibrations in the ground.

Lex Fridman (03:44.520)

Like how is this, how are all of these,

Po-Shen Loh (03:46.960)

you go to like New York City, all of these buildings standing.

Lex Fridman (03:49.920)

I mean, to me, one of the most beautiful things,

Po-Shen Loh (03:52.120)

actually mathematically too, is bridges.

Lex Fridman (03:54.760)

I used to build bridges in high school from like toothpicks,

Po-Shen Loh (03:57.560)

just like out of the pure joy of like physics,

Lex Fridman (04:02.320)

making some structure really strong.

Po-Shen Loh (04:05.800)

Understanding like from a civil engineering perspective,

Lex Fridman (04:09.600)

what kind of structure will be stronger

Po-Shen Loh (04:12.040)

than another kind of structure, like suspension bridges.

Lex Fridman (04:14.720)

And then you see that at scale,

Po-Shen Loh (04:17.160)

humans being able to span a body of water

Lex Fridman (04:20.400)

with a giant bridge.

Lex Fridman (04:22.240)

And it's, I don't know, it's so humbling.

Lex Fridman (04:26.480)

It makes you realize how dependent we are on each other.

Po-Shen Loh (04:31.920)

Sort of, I talk about love a lot,

Lex Fridman (04:33.360)

but there's a certain element in which we little ants

Po-Shen Loh (04:37.960)

have just a small amount of knowledge

Lex Fridman (04:39.640)

about our particular thing.

Lex Fridman (04:41.880)

And then we're depending on a network of knowledge

Lex Fridman (04:45.360)

that other experts hold.

Lex Fridman (04:47.360)

And then most of our lives,

Lex Fridman (04:49.200)

most of the quality of life we have

Po-Shen Loh (04:51.120)

has to do with the richness of that network of knowledge,

Lex Fridman (04:56.600)

of that collaboration,

Lex Fridman (04:58.280)

and then sort of the ability to build on top of it,

Lex Fridman (05:02.000)

levels of abstractions.

Po-Shen Loh (05:03.200)

You start from like bits in a computer,

Lex Fridman (05:05.760)

then you can have assembly, then you can have C++,

Po-Shen Loh (05:08.800)

or you have an operating system,

Lex Fridman (05:09.920)

then you can have C++ and Python, finally,

Po-Shen Loh (05:12.040)

some machine learning on top.

Lex Fridman (05:13.440)

All of these are abstractions.

Lex Fridman (05:14.960)

And eventually we'll have AI that runs all of us humans.

Lex Fridman (05:17.120)

But anyway, but speaking of abstractions and programming,

Po-Shen Loh (05:21.280)

in high school, you wrote some impressive games

Lex Fridman (05:24.720)

for MS DOS.

Po-Shen Loh (05:26.080)

I got a chance to, in browser somehow,

Lex Fridman (05:28.080)

it's magic, I got a chance to play them.

Po-Shen Loh (05:31.720)

Alien Attack 1, 2, 3, and 4.

Lex Fridman (05:34.560)

What's the hardest part about programming those games?

Lex Fridman (05:37.240)

And maybe can you tell the story about building those games?

Lex Fridman (05:41.080)

Sure.

Po-Shen Loh (05:41.920)

I actually tried to do those in high school

Lex Fridman (05:44.240)

because I was just curious if I could.

Lex Fridman (05:46.480)

That's a good starting point for anything, right?

Lex Fridman (05:49.280)

Yeah, yeah, yeah, it's like, could you?

Lex Fridman (05:50.480)

But the appealing thing was also,

Lex Fridman (05:52.000)

it was a soup to nuts kind of thing.

Lex Fridman (05:54.160)

So something that has always attracted me is,

Lex Fridman (05:56.560)

I like beautiful ideas, I like seeing beautiful ideas,

Lex Fridman (05:59.920)

but I actually also like seeing execution of an idea

Lex Fridman (06:03.760)

all the way from beginning to end in something that works.

Lex Fridman (06:06.360)

So for example, in high school,

Lex Fridman (06:08.000)

I was lucky enough to grow up in the late 90s

Po-Shen Loh (06:10.880)

when even a high school student could hope

Lex Fridman (06:13.720)

to make something sort of comparable

Po-Shen Loh (06:16.200)

to the shareware games that were out there.

Lex Fridman (06:18.640)

I say the word sort of, like still quite far away,

Lex Fridman (06:21.240)

but at least I didn't need to hire a 3D CG artist.

Lex Fridman (06:25.000)

There weren't enough pixels to draw anyway,

Lex Fridman (06:27.200)

even I can draw, right?

Lex Fridman (06:29.360)

Bad art, of course.

Lex Fridman (06:30.280)

But the point is, I wanted to know,

Lex Fridman (06:31.640)

is it possible for me to try to do those things

Po-Shen Loh (06:34.640)

where back in those days,

Lex Fridman (06:36.600)

you didn't even have an easy way

Po-Shen Loh (06:38.440)

to draw letters on the screen in a particular font.

Lex Fridman (06:41.200)

You couldn't just say import a font, it wasn't like Python.

Lex Fridman (06:44.080)

So for example, back then,

Lex Fridman (06:45.440)

if you played those games in the web browser,

Po-Shen Loh (06:47.560)

which is emulating the old school computer,

Lex Fridman (06:51.840)

those, even the letters you see,

Po-Shen Loh (06:53.520)

those are made by individual calls

Lex Fridman (06:55.120)

to draw pixels on the screen.

Lex Fridman (06:56.800)

So you built that from scratch,

Lex Fridman (06:58.240)

almost building a computer graphics library from scratch?

Po-Shen Loh (07:01.000)

Yes, the primitive that I got to use

Lex Fridman (07:03.120)

was some code I copied off of a book in assembly

Po-Shen Loh (07:05.600)

of how to put a pixel on a screen in a particular color.

Lex Fridman (07:08.720)

And the programming language was Pascal?

Po-Shen Loh (07:11.760)

Ah, yeah, the first one was in Pascal,

Lex Fridman (07:14.400)

but then the other ones were in C++ after that.

Lex Fridman (07:18.120)

How's the emulation in the browser work, by the way?

Lex Fridman (07:20.640)

Is that trivial?

Po-Shen Loh (07:21.640)

Because it's pretty cool, you get to play these games

Lex Fridman (07:23.520)

that have a very much 90s feeling to them.

Po-Shen Loh (07:26.480)

Ah, so it's literally making an MSDOS environment,

Lex Fridman (07:29.320)

which is literally running the old.exe file.

Po-Shen Loh (07:32.240)

Wow, in the browser.

Lex Fridman (07:33.520)

This is, that could be more amazing than the airplane.

Lex Fridman (07:37.800)

So it wasn't so much about the video games,

Lex Fridman (07:40.240)

it was more about,

Lex Fridman (07:41.400)

can you build something really cool from scratch?

Lex Fridman (07:44.120)

Yes.

Lex Fridman (07:45.440)

And you did a bunch of programming competitions.

Lex Fridman (07:50.280)

What was your interest, your love for programming?

Lex Fridman (07:54.680)

What did you learn through that experience?

Lex Fridman (07:56.880)

Especially now that as much of your work

Po-Shen Loh (07:59.720)

has taken a long journey through mathematics.

Lex Fridman (08:03.160)

I think I always was amazed

Po-Shen Loh (08:04.920)

by how computers could do things fast.

Lex Fridman (08:08.440)

If I wanted to make it an abstract analysis

Po-Shen Loh (08:11.560)

of why it is that I saw some power in the computer.

Lex Fridman (08:14.640)

Because if the computer can do things

Lex Fridman (08:16.360)

so many times faster than humans,

Lex Fridman (08:18.280)

where the hard part is telling the computer

Lex Fridman (08:20.280)

what to do and how to do it,

Lex Fridman (08:21.840)

if you can master that asking the computer what to do,

Po-Shen Loh (08:25.600)

then you could conceivably achieve more things.

Lex Fridman (08:28.200)

And those contests I was in,

Po-Shen Loh (08:29.640)

those were the opposite in some sense

Lex Fridman (08:31.840)

of making a complete product, like a game is a product.

Po-Shen Loh (08:35.840)

Those contests were effectively write a function

Lex Fridman (08:38.520)

to do something extremely efficiently.

Lex Fridman (08:40.760)

And if you are able to do that,

Lex Fridman (08:42.440)

then you can unlock more of the power of the computer.

Lex Fridman (08:45.800)

But also doing it quickly.

Lex Fridman (08:47.440)

There's a time element from the human perspective

Po-Shen Loh (08:49.640)

to be able to program quickly.

Lex Fridman (08:53.360)

There's something nice.

Lex Fridman (08:54.840)

So there's almost like an athletics component

Lex Fridman (08:57.920)

to where you're almost like an athlete

Po-Shen Loh (09:01.240)

seeking optimal performance as a human being

Lex Fridman (09:03.920)

trying to write these programs.

Lex Fridman (09:05.680)

And at the same time, it's kind of art

Lex Fridman (09:07.440)

because the best way to write a program quickly

Po-Shen Loh (09:11.080)

is to write a simple program.

Lex Fridman (09:13.440)

You just have a damn good solution.

Lex Fridman (09:14.840)

So it's not necessarily you have to type fast.

Lex Fridman (09:17.040)

You have to think through a really clean,

Po-Shen Loh (09:19.520)

beautiful solution.

Lex Fridman (09:22.760)

I mean, what do you think is the use

Lex Fridman (09:26.080)

of those programming competitions?

Lex Fridman (09:27.760)

Do you think they're ultimately something

Po-Shen Loh (09:29.160)

you would recommend for students,

Lex Fridman (09:30.600)

for people interested in programming,

Lex Fridman (09:32.040)

or people interested in building stuff?

Lex Fridman (09:34.000)

Yes, I think so because especially with the work

Po-Shen Loh (09:37.080)

that I've been doing nowadays,

Lex Fridman (09:38.120)

even trying to control COVID,

Po-Shen Loh (09:40.200)

something that was very helpful from day one

Lex Fridman (09:42.560)

was understanding that the kinds of computations

Po-Shen Loh (09:45.560)

we would want to do,

Lex Fridman (09:47.080)

we could conceivably do on like a four core cloud machine

Po-Shen Loh (09:50.440)

on Amazon Web Services out to a population

Lex Fridman (09:53.760)

which might have hundreds of thousands

Po-Shen Loh (09:55.280)

or millions of people.

Lex Fridman (09:56.480)

The reason why that was important

Po-Shen Loh (09:57.840)

to have that back of the envelope calculation

Lex Fridman (10:00.760)

with efficient algorithms

Po-Shen Loh (10:02.720)

is because if we couldn't do that,

Lex Fridman (10:04.960)

then we would bankrupt ourselves

Po-Shen Loh (10:06.280)

before we could get to a big enough scale.

Lex Fridman (10:08.160)

If you think about how you grow anything from small to big,

Po-Shen Loh (10:11.480)

if in order to grow it from small to big,

Lex Fridman (10:13.600)

you also already need 10,000 cloud servers,

Po-Shen Loh (10:16.480)

you'll never get to big.

Lex Fridman (10:19.400)

And also the nice thing about programming competitions

Po-Shen Loh (10:22.320)

is that you actually build a thing that works.

Lex Fridman (10:26.440)

So you finish it, there's a completion thing,

Lex Fridman (10:29.040)

and you realize, I think there's a magic to it,

Lex Fridman (10:32.520)

where you realize that it's not so hard

Po-Shen Loh (10:35.520)

to build something that works.

Lex Fridman (10:37.520)

To have a system that successfully takes in inputs

Lex Fridman (10:40.560)

and produces outputs and solves a difficult problem,

Lex Fridman (10:43.360)

and that directly transfers to building a startup essentially

Po-Shen Loh (10:47.040)

that can help some aspect of this world

Lex Fridman (10:50.000)

as long as it's mostly based on software engineering.

Po-Shen Loh (10:53.840)

Things get really tricky

Lex Fridman (10:54.960)

when you have to manufacture stuff.

Po-Shen Loh (10:58.480)

That's why people like Elon Musk are so impressive

Lex Fridman (11:00.720)

that it's not just software.

Po-Shen Loh (11:02.840)

Tesla Autopilot is not just software.

Lex Fridman (11:05.640)

It's like you have to actually have factories

Po-Shen Loh (11:07.640)

that build cars, and there's like a million components

Lex Fridman (11:11.640)

involved in the machinery required

Po-Shen Loh (11:14.280)

to assemble those cars and so on.

Lex Fridman (11:16.200)

But in software, one person can change the world,

Po-Shen Loh (11:19.000)

which is incredible.

Lex Fridman (11:21.440)

But on the mathematics side,

Po-Shen Loh (11:23.240)

what, if you look back, or maybe today,

Lex Fridman (11:26.760)

what made you fall in love with mathematics?

Po-Shen Loh (11:29.680)

For me, I think I've always been very attracted

Lex Fridman (11:33.320)

to challenge, as I already indicated

Po-Shen Loh (11:35.720)

with writing the program.

Lex Fridman (11:37.320)

I guess if I see something that's hard

Po-Shen Loh (11:40.440)

or supposed to be impossible,

Lex Fridman (11:44.200)

sometimes I say, maybe I want to see if I can pull that off.

Lex Fridman (11:47.120)

And with the mathematics, the math competitions

Lex Fridman (11:49.920)

presented problems that were hard,

Po-Shen Loh (11:53.560)

that I didn't know how to start,

Lex Fridman (11:55.040)

but for which I could conceivably try to learn

Lex Fridman (11:57.640)

how to solve them.

Lex Fridman (11:59.120)

So, I mean, there are other things that are hard

Po-Shen Loh (12:00.480)

called like get something to Mars, get people to Mars.

Lex Fridman (12:03.360)

And I didn't, and I still don't think

Po-Shen Loh (12:05.800)

that I am able to solve that problem.

Lex Fridman (12:08.160)

On the other hand, the math problems struck me

Po-Shen Loh (12:10.000)

as things which are hard

Lex Fridman (12:11.440)

and with significant amount of extra work,

Po-Shen Loh (12:13.560)

I could figure it out.

Lex Fridman (12:14.640)

And maybe they would actually even be useful,

Po-Shen Loh (12:16.560)

like that mathematical skill is the core

Lex Fridman (12:18.920)

of lots of other things.

Po-Shen Loh (12:22.680)

That's really interesting.

Lex Fridman (12:23.720)

Maybe you could speak to that

Po-Shen Loh (12:25.120)

because a lot of people say that math is hard

Lex Fridman (12:29.440)

as a kind of negative statement.

Po-Shen Loh (12:32.880)

It always seemed to me a little bit like

Lex Fridman (12:35.120)

that's kind of a positive statement

Po-Shen Loh (12:37.360)

that all things that are worth having in this world,

Lex Fridman (12:40.360)

they're hard.

Po-Shen Loh (12:41.440)

I mean, everything that people think about

Lex Fridman (12:44.480)

that they would love to do, whether it's sports,

Po-Shen Loh (12:47.800)

whether it's art, music, and all the sciences,

Lex Fridman (12:52.920)

they're going to be hard

Po-Shen Loh (12:54.000)

if you want to do something special.

Lex Fridman (12:56.520)

So is there something you could say to that idea

Lex Fridman (12:59.000)

that math is hard?

Lex Fridman (13:00.160)

Should it be made easy or should it be hard?

Po-Shen Loh (13:04.200)

Ah, so I think maybe I want to dig in a little bit

Lex Fridman (13:07.120)

onto this hard part and say,

Po-Shen Loh (13:09.360)

I think the interesting thing about the math

Lex Fridman (13:12.120)

is that you can see a question

Po-Shen Loh (13:14.800)

that you didn't know how to start doing it before.

Lex Fridman (13:18.240)

And over a course of thinking about it,

Po-Shen Loh (13:20.920)

you can come up with a way to solve it.

Lex Fridman (13:24.400)

And so you can move from a state

Po-Shen Loh (13:25.960)

of not being able to do something

Lex Fridman (13:28.160)

to a state of being able to do something

Po-Shen Loh (13:30.280)

where you help to take yourself through that

Lex Fridman (13:33.280)

instead of somebody else spoon feeding you that technique.

Lex Fridman (13:37.600)

So actually here, I'm already digging into

Lex Fridman (13:39.480)

maybe part of my teaching philosophy also,

Po-Shen Loh (13:42.080)

which is that I actually don't want to ever

Lex Fridman (13:45.400)

just tell somebody, here's how you do something.

Po-Shen Loh (13:48.840)

I actually prefer to say, here's an interesting question.

Lex Fridman (13:52.160)

I know you don't quite know how to do it.

Lex Fridman (13:54.240)

Do you have any ideas?

Lex Fridman (13:55.920)

I'm actually explaining another way

Po-Shen Loh (13:58.920)

that you could try to do teaching.

Lex Fridman (14:00.400)

And I'm contrasting this to a method of watch me do this,

Po-Shen Loh (14:04.160)

now practice it 20 times.

Lex Fridman (14:06.320)

I'm trying to say a lot of people consider math to be hard

Po-Shen Loh (14:09.080)

because maybe they can't remember

Lex Fridman (14:10.800)

all of the methods that were taught.

Lex Fridman (14:12.800)

But for me, I look at the hardness

Lex Fridman (14:15.560)

and I don't think of it as a memory hardness.

Lex Fridman (14:17.600)

I think of it as a, can you invent something hardness?

Lex Fridman (14:21.920)

And I think that if we can teach more people

Lex Fridman (14:24.360)

how to do that art of invention in a pure cognitive way,

Lex Fridman (14:28.640)

not as hard as the actual hardware stuff, right?

Lex Fridman (14:31.240)

But like in terms of the concepts

Lex Fridman (14:32.640)

and the thoughts and the mathematics,

Po-Shen Loh (14:34.000)

teaching people how to invent,

Lex Fridman (14:36.360)

then suddenly actually they might not even find math

Po-Shen Loh (14:38.840)

to be that tiresomeness hard anymore,

Lex Fridman (14:42.680)

but that rewardingness hard of I have the capability

Po-Shen Loh (14:47.480)

of looking at something which I don't know what to do

Lex Fridman (14:49.920)

and coming up with how to do it.

Po-Shen Loh (14:51.400)

I actually think we should be doing that,

Lex Fridman (14:52.840)

giving people that capability.

Lex Fridman (14:55.040)

So hard in the same way that invention is hard,

Lex Fridman (14:58.160)

that is ultimately rewarding.

Lex Fridman (14:59.520)

So maybe you can dig in that a little bit longer,

Lex Fridman (15:03.640)

which is do you see basically the way to teach math

Po-Shen Loh (15:11.200)

is to present a problem and to give a person a chance

Lex Fridman (15:14.840)

to try to invent a solution

Lex Fridman (15:18.480)

with minimal amount of information first?

Lex Fridman (15:21.080)

Is that basically,

Lex Fridman (15:22.840)

how do you build that muscle of invention in a student?

Lex Fridman (15:26.400)

Yes, so the way that,

Po-Shen Loh (15:27.840)

I guess I have two different sort of ways

Lex Fridman (15:30.120)

that I try to teach.

Po-Shen Loh (15:30.960)

Actually, one of them is, in fact, this semester,

Lex Fridman (15:32.960)

because all my classes were remotely delivered,

Po-Shen Loh (15:35.520)

I even threw them all onto my YouTube channel.

Lex Fridman (15:37.200)

So you can see how I teach at Carnegie Mellon,

Lex Fridman (15:39.960)

but I'd often say, hey, everyone, let's try to do this.

Lex Fridman (15:43.280)

Any ideas?

Lex Fridman (15:44.680)

And that actually changes my role as a professor

Lex Fridman (15:47.760)

from a person who shows up for class

Po-Shen Loh (15:50.040)

with a script of what I wanna talk through.

Lex Fridman (15:52.480)

I actually, I don't have a script.

Po-Shen Loh (15:54.000)

The way I show up for classes,

Lex Fridman (15:55.680)

there's something that we want to learn how to do,

Lex Fridman (15:58.080)

and we're gonna do it by improv.

Lex Fridman (16:00.120)

I'm talking about the same method as improv comedy,

Po-Shen Loh (16:02.760)

which is where you tell me some ideas,

Lex Fridman (16:05.240)

and I'll try to yes and them.

Lex Fridman (16:07.040)

You know what I mean?

Lex Fridman (16:09.320)

And then together,

Po-Shen Loh (16:10.400)

we're gonna come up with a proof of this concept

Lex Fridman (16:13.000)

where you were deeply involved in creating the proof.

Po-Shen Loh (16:16.360)

Actually, every time I teach the class,

Lex Fridman (16:18.040)

we do every proof slightly differently

Po-Shen Loh (16:19.800)

because it's based on how the students came up with it.

Lex Fridman (16:23.400)

And that's how I do it when I'm in person.

Po-Shen Loh (16:25.760)

I also have another line of courses that we make

Lex Fridman (16:27.880)

that is delivered online.

Po-Shen Loh (16:29.200)

Those things are where I can't do it live,

Lex Fridman (16:31.640)

but the teaching method became also similar.

Po-Shen Loh (16:34.560)

It was just, here's an interesting question.

Lex Fridman (16:37.160)

I know it's out of reach.

Lex Fridman (16:38.240)

Why don't you think about it?

Lex Fridman (16:39.400)

And then automatic hints.

Po-Shen Loh (16:40.640)

We feed automatically hints through the internet

Lex Fridman (16:44.120)

to go and let the person try to invent.

Lex Fridman (16:47.560)

So that's like a more rigorous prodding of invention.

Lex Fridman (16:52.240)

But you did mention disease and COVID,

Lex Fridman (16:56.000)

and you've been doing some very interesting stuff

Lex Fridman (16:58.160)

from a mathematical, but also software engineering angle

Po-Shen Loh (17:01.920)

of coming up with ideas.

Lex Fridman (17:04.520)

It's back to the, I see a problem.

Po-Shen Loh (17:07.440)

I think I can help.

Lex Fridman (17:09.640)

So you stepped into this world.

Lex Fridman (17:11.120)

Can you tell me about your work there

Lex Fridman (17:13.880)

under the flag of Novid

Lex Fridman (17:16.120)

and both the software and the technical details

Lex Fridman (17:20.560)

of how the thing works?

Po-Shen Loh (17:21.680)

Sure, sure.

Lex Fridman (17:22.520)

So first I want to make sure that I say,

Po-Shen Loh (17:24.520)

this is actually team effort.

Lex Fridman (17:26.000)

I happen to be the one speaking,

Lex Fridman (17:27.480)

but there's no way this would exist

Lex Fridman (17:29.040)

without an incredible team of people

Po-Shen Loh (17:30.560)

who inspire me every day to work on this.

Lex Fridman (17:33.120)

But I'll speak on behalf of them.

Lex Fridman (17:34.600)

So the idea was indeed that we stepped forward

Lex Fridman (17:40.200)

in March of last year, when the world started to become,

Po-Shen Loh (17:43.320)

our part of the world started to become,

Lex Fridman (17:44.800)

our part meaning the United States

Po-Shen Loh (17:46.360)

started to become paralyzed by COVID.

Lex Fridman (17:48.920)

The shutdown started to happen.

Lex Fridman (17:50.760)

And at that time it started as a figment of an idea,

Lex Fridman (17:54.240)

which was network theory,

Po-Shen Loh (17:57.160)

which is the area of math that I work in,

Lex Fridman (17:59.480)

could potentially be combined with smartphones

Lex Fridman (18:02.440)

and some kind of health information anonymized.

Lex Fridman (18:06.040)

Exactly how?

Po-Shen Loh (18:07.080)

We didn't know yet.

Lex Fridman (18:07.960)

We tried to crystallize it.

Lex Fridman (18:09.400)

And many months into this work,

Lex Fridman (18:11.400)

we ended up accidentally discovering a new way

Po-Shen Loh (18:15.200)

to control diseases,

Lex Fridman (18:17.000)

which is now what is the main impetus of all of this work

Po-Shen Loh (18:20.840)

is to take this idea and polish it

Lex Fridman (18:23.440)

and hopefully have it be useful not only now,

Lex Fridman (18:25.960)

but for future pandemics.

Lex Fridman (18:27.400)

The idea is really simple to describe.

Po-Shen Loh (18:29.680)

Actually, my main thing in the world

Lex Fridman (18:31.080)

is I come up with obvious observations.

Po-Shen Loh (18:33.720)

That's that, so I'll explain it now.

Lex Fridman (18:35.080)

Einstein did the same thing

Lex Fridman (18:36.880)

and he wrote a few short papers.

Lex Fridman (18:39.920)

But so the idea is like this.

Po-Shen Loh (18:41.800)

If we describe how usually people control disease

Lex Fridman (18:46.440)

for a lot of history,

Po-Shen Loh (18:47.920)

it was that you'd find out who was sick,

Lex Fridman (18:51.000)

you'd find out who they've been around

Lex Fridman (18:53.280)

and you try to remove all of those people from society

Lex Fridman (18:56.120)

against their will.

Po-Shen Loh (18:57.840)

Now that's the problem.

Lex Fridman (18:58.960)

The against their will part

Po-Shen Loh (19:00.880)

gives you the wrong kind of a feedback loop,

Lex Fridman (19:03.560)

which makes it hard to control the disease

Po-Shen Loh (19:05.520)

because then the people you're trying to control

Lex Fridman (19:07.000)

keep getting other people sick.

Po-Shen Loh (19:08.760)

You can see already how I'm thinking

Lex Fridman (19:10.120)

and talking about this feedback loops.

Po-Shen Loh (19:11.920)

This is actually related to something you said earlier

Lex Fridman (19:14.240)

about even like how skyscrapers stay in the air.

Po-Shen Loh (19:17.120)

The whole point is control theory.

Lex Fridman (19:19.360)

You actually want to, or even how an airplane stays,

Po-Shen Loh (19:22.600)

you need to have control loops

Lex Fridman (19:24.880)

which are feedbacking in the right way.

Lex Fridman (19:27.040)

And what we observed was that the feedback control loop

Lex Fridman (19:29.880)

for controlling disease by asking people

Po-Shen Loh (19:32.160)

to be removed from society against their will

Lex Fridman (19:34.480)

was not working.

Po-Shen Loh (19:35.600)

It was running against human incentives

Lex Fridman (19:37.640)

and you suddenly are trying to control

Po-Shen Loh (19:39.160)

seven billion, eight billion people

Lex Fridman (19:41.440)

in ways that they don't individually want

Po-Shen Loh (19:43.600)

to necessarily do.

Lex Fridman (19:45.160)

So here's the idea.

Lex Fridman (19:47.080)

And this is inspired by the fact

Lex Fridman (19:48.320)

that at the core of our team

Po-Shen Loh (19:49.480)

were user experience designers.

Lex Fridman (19:51.280)

That's actually, in fact, the first thing I knew

Po-Shen Loh (19:53.480)

we needed when we started

Lex Fridman (19:54.560)

was to bring user experience at the core.

Po-Shen Loh (19:57.160)

Okay.

Lex Fridman (19:58.160)

But so the idea was suppose hypothetically

Lex Fridman (1:00:01.200)

and a rigorous deep education,

Lex Fridman (1:00:03.960)

like as opposed to kind of like,

Po-Shen Loh (1:00:06.440)

oh, we wanna make sure we teach

Lex Fridman (1:00:10.320)

to the weakest student in the class,

Po-Shen Loh (1:00:14.240)

which American systems can sometimes do

Lex Fridman (1:00:16.960)

because we don't wanna leave anyone behind.

Po-Shen Loh (1:00:19.360)

The Russian system was anyone can be the strongest student

Lex Fridman (1:00:25.000)

and we're gonna teach you the strongest student

Lex Fridman (1:00:26.840)

and we're going to pretend or force everybody,

Lex Fridman (1:00:30.840)

even the weakest student to be strong.

Lex Fridman (1:00:32.880)

And what that results in, it's obviously,

Lex Fridman (1:00:35.240)

this is what people talk about,

Po-Shen Loh (1:00:36.480)

is a huge amount of pressure.

Lex Fridman (1:00:38.120)

Like it's psychologically very difficult.

Po-Shen Loh (1:00:40.640)

This is why people struggle when they go to MIT,

Lex Fridman (1:00:42.640)

this very competitive environment.

Po-Shen Loh (1:00:44.400)

It can be very psychologically difficult,

Lex Fridman (1:00:46.080)

but at the same time,

Po-Shen Loh (1:00:47.360)

it's bringing out the best out of people.

Lex Fridman (1:00:49.800)

And that mathematics was certainly one of those things.

Lex Fridman (1:00:53.200)

And exactly what you're saying,

Lex Fridman (1:00:54.640)

which kind of clicked with me just now,

Po-Shen Loh (1:00:56.360)

as opposed to kind of a spelling bee in the United States,

Lex Fridman (1:01:00.760)

which I guess you spell, I'm horrible at this,

Lex Fridman (1:01:03.360)

but it's a competition about spelling,

Lex Fridman (1:01:04.960)

which I'm not sure, but you could argue

Po-Shen Loh (1:01:07.160)

it doesn't generalize well to the future skills.

Lex Fridman (1:01:10.000)

Mathematics, especially this kind of mathematics

Po-Shen Loh (1:01:13.320)

is essentially formalized competition of invention,

Lex Fridman (1:01:17.120)

of creating new ideas.

Lex Fridman (1:01:21.760)

And that generalizes really, really well.

Lex Fridman (1:01:23.960)

So that's quite brilliantly put.

Po-Shen Loh (1:01:25.800)

I didn't really think about that.

Lex Fridman (1:01:27.320)

So this is not just about the competition.

Po-Shen Loh (1:01:29.160)

This is about developing minds

Lex Fridman (1:01:31.960)

that will come to do some incredible stuff in the future.

Po-Shen Loh (1:01:37.280)

Yeah, actually, I want to respond

Lex Fridman (1:01:38.640)

to a couple of things there.

Po-Shen Loh (1:01:39.600)

The first one, this one, which is this notion

Lex Fridman (1:01:42.000)

of whether or not that is possible

Po-Shen Loh (1:01:43.800)

in a non authoritarian regime.

Lex Fridman (1:01:46.280)

I think it is.

Lex Fridman (1:01:47.120)

And that's actually why I spent some of my efforts

Lex Fridman (1:01:49.520)

before the COVID thing,

Po-Shen Loh (1:01:51.040)

actually trying to work towards there.

Lex Fridman (1:01:53.240)

The reason is because if you think about it,

Po-Shen Loh (1:01:55.680)

let's say in America,

Lex Fridman (1:01:57.360)

lots of people are pretty serious

Po-Shen Loh (1:01:58.840)

about training very hard for football,

Lex Fridman (1:02:01.160)

or baseball, or basketball.

Po-Shen Loh (1:02:02.520)

Basketball is very, very accessible,

Lex Fridman (1:02:04.040)

but lots of people are doing that.

Lex Fridman (1:02:05.840)

Why?

Lex Fridman (1:02:06.680)

Well, actually, I think that what was going on

Po-Shen Loh (1:02:09.880)

with the authoritarian thing was at least the message

Lex Fridman (1:02:13.520)

that was universally sent was being a good thinker

Lex Fridman (1:02:17.880)

and a creator of ideas is a good thing.

Lex Fridman (1:02:21.800)

Yes, exactly.

Po-Shen Loh (1:02:23.360)

There's no reason why that message can't be sent everywhere.

Lex Fridman (1:02:26.920)

And I think it actually should be.

Lex Fridman (1:02:28.680)

So that's the first thing.

Lex Fridman (1:02:29.720)

The second thing is what you commented about this thing

Po-Shen Loh (1:02:32.600)

about the generalizable skill

Lex Fridman (1:02:35.720)

and what could people do with Olympiads afterwards.

Lex Fridman (1:02:37.960)

So that's actually my interest in the whole thing.

Lex Fridman (1:02:40.360)

I don't just coach students how to do problems.

Po-Shen Loh (1:02:45.400)

In fact, I'm not even the best person for that.

Lex Fridman (1:02:47.160)

I'm not the best at solving these problems.

Po-Shen Loh (1:02:49.480)

There are other people who are much better

Lex Fridman (1:02:50.800)

at making problems and teaching people how to solve problems.

Po-Shen Loh (1:02:53.320)

In fact, when the Mathematical Association of America,

Lex Fridman (1:02:57.160)

which is the group which is in charge

Po-Shen Loh (1:02:58.640)

of the US participation in these Olympiads,

Lex Fridman (1:03:01.280)

when they were deciding whether or not to put me in

Po-Shen Loh (1:03:04.320)

back in 2013 as the head coach,

Lex Fridman (1:03:06.920)

I had a conversation with their executive director

Po-Shen Loh (1:03:09.240)

where I commented that we might do worse

Lex Fridman (1:03:12.680)

because my position was I don't,

Po-Shen Loh (1:03:15.080)

I mean, I actually didn't want to focus on winning.

Lex Fridman (1:03:17.680)

I said, if you're going to let me work

Po-Shen Loh (1:03:19.840)

with 60 very strong minds as picked through this system,

Lex Fridman (1:03:24.400)

because the coach works with these,

Po-Shen Loh (1:03:26.080)

gets to run a camp for these students.

Lex Fridman (1:03:27.680)

I said, I'm actually not going to define my success

Po-Shen Loh (1:03:30.400)

in terms of winning this contest.

Lex Fridman (1:03:33.080)

I said, I wanted to maximize the number of the students

Po-Shen Loh (1:03:36.040)

that I read about in the New York Times in 20 years.

Lex Fridman (1:03:40.240)

And the executive director

Po-Shen Loh (1:03:41.520)

of the Mathematical Association of America

Lex Fridman (1:03:44.000)

was fully in support of this

Po-Shen Loh (1:03:45.840)

because that's also how their philosophy is.

Lex Fridman (1:03:48.000)

So in America, the way we run this

Po-Shen Loh (1:03:49.840)

is we're actually not just training to win,

Lex Fridman (1:03:52.920)

even though the students are very good

Lex Fridman (1:03:54.840)

and they can win anyway.

Lex Fridman (1:03:56.440)

One reason, for example, I went and even did the COVID thing

Po-Shen Loh (1:03:59.120)

involving quite a few of them

Lex Fridman (1:04:01.560)

is so that hopefully some of them get ideas

Po-Shen Loh (1:04:04.280)

because in 20, 30 years, I won't have the energy

Lex Fridman (1:04:06.520)

or the insight to solve problems.

Po-Shen Loh (1:04:08.520)

We'll have another catastrophe.

Lex Fridman (1:04:10.480)

And hopefully some of these people will step up and do it.

Lex Fridman (1:04:13.160)

And ultimately have that longterm impact.

Lex Fridman (1:04:14.960)

I wonder if this is scalable to,

Po-Shen Loh (1:04:17.280)

because that's such a great metric for education,

Lex Fridman (1:04:20.320)

not how to get an A on the test, but how to have,

Lex Fridman (1:04:28.880)

how to be on the cover of New York Times

Lex Fridman (1:04:31.320)

for inventing something new.

Lex Fridman (1:04:33.720)

And do you think that's generalizable to education

Lex Fridman (1:04:37.360)

beyond just this particular Olympia?

Po-Shen Loh (1:04:39.080)

Like, even you saying this feels like a rare statement,

Lex Fridman (1:04:42.760)

almost like a radical statement as a goal for education.

Lex Fridman (1:04:45.920)

So actually the way I teach my classes at Carnegie Mellon,

Lex Fridman (1:04:48.800)

which I will admit right away is not equivalent

Po-Shen Loh (1:04:51.280)

to the average in the world,

Lex Fridman (1:04:52.440)

but it's already not just the top 60 in the country

Po-Shen Loh (1:04:56.280)

as picked by something.

Lex Fridman (1:04:58.120)

Let me just explain.

Po-Shen Loh (1:04:58.960)

I have exams in my class, which are 90% of the grade.

Lex Fridman (1:05:01.480)

So the exams are the whole thing,

Po-Shen Loh (1:05:02.840)

or most of the whole thing.

Lex Fridman (1:05:03.920)

And the way that I let students prepare for the exams

Po-Shen Loh (1:05:06.640)

is I show them all the problems I've ever given

Lex Fridman (1:05:08.880)

on the previous exams.

Lex Fridman (1:05:10.360)

And the exam that they will take is open notes.

Lex Fridman (1:05:12.680)

They can take all the notes they want

Po-Shen Loh (1:05:13.760)

on the previous problems.

Lex Fridman (1:05:14.800)

And the guarantee is that the exam problems this time

Po-Shen Loh (1:05:17.240)

will have no overlap with anything

Lex Fridman (1:05:18.840)

you have seen me give in the past,

Po-Shen Loh (1:05:21.000)

as well as no overlap with anything I taught in the class.

Lex Fridman (1:05:24.400)

So the entire exam is invention.

Po-Shen Loh (1:05:27.360)

Wow.

Lex Fridman (1:05:28.600)

But that's how I go, right?

Po-Shen Loh (1:05:29.760)

My point is I have explained to people when I teach you,

Lex Fridman (1:05:33.240)

I don't want you to have remembered a method I showed you.

Po-Shen Loh (1:05:36.720)

I want you to have learned enough about this area

Lex Fridman (1:05:39.320)

that if you face a new question,

Po-Shen Loh (1:05:40.880)

which I came up with the night before

Lex Fridman (1:05:42.480)

by thinking about like,

Lex Fridman (1:05:43.640)

what could I ask that I have never asked before?

Lex Fridman (1:05:46.200)

Oh, that's cute.

Po-Shen Loh (1:05:47.040)

That's what the answer is.

Lex Fridman (1:05:48.040)

Aha, that's an exam problem.

Po-Shen Loh (1:05:49.160)

That's exactly what I do before the exam.

Lex Fridman (1:05:51.320)

And then that's what I want them to learn.

Lex Fridman (1:05:53.840)

And the first exam, usually people have a rough time

Lex Fridman (1:05:56.080)

because it's like, what kind of crazy class is this?

Po-Shen Loh (1:05:58.400)

The professor doesn't teach you anything for the exam.

Lex Fridman (1:06:01.920)

But then by the second or third,

Lex Fridman (1:06:03.440)

and by the time they finished the class,

Lex Fridman (1:06:05.280)

they have learned how to solve anything in the area.

Lex Fridman (1:06:09.520)

How to invent.

Lex Fridman (1:06:10.360)

How to invent in that area, yeah.

Lex Fridman (1:06:12.240)

Can we walk back to the Mathematical Olympiad?

Lex Fridman (1:06:15.520)

What's the scoring and format like?

Lex Fridman (1:06:18.520)

And also what does it take to win?

Lex Fridman (1:06:20.840)

So the way it works is that each of the six students

Po-Shen Loh (1:06:25.640)

do the problems and there are six problems.

Lex Fridman (1:06:27.800)

All the problems are equally weighted.

Lex Fridman (1:06:29.560)

So each one's worth seven points.

Lex Fridman (1:06:31.480)

That means that your maximum score

Po-Shen Loh (1:06:33.400)

is six problems times seven points,

Lex Fridman (1:06:35.000)

which is the nice number of 42.

Lex Fridman (1:06:37.480)

And now the way that they're scored by the way

Lex Fridman (1:06:40.360)

is there's partial credit.

Lex Fridman (1:06:41.680)

So the question is asking you,

Lex Fridman (1:06:43.200)

explain why this weird fact is true.

Po-Shen Loh (1:06:46.360)

Okay, if you explain why you get seven points.

Lex Fridman (1:06:48.720)

If you make minor mistake, maybe you get six points.

Lex Fridman (1:06:51.320)

But if you don't succeed in explaining why,

Lex Fridman (1:06:53.840)

but you explain some other true fact,

Po-Shen Loh (1:06:57.960)

which is along the way of proving it,

Lex Fridman (1:07:02.200)

then you get partial credit.

Lex Fridman (1:07:03.960)

And actually now this is tricky

Lex Fridman (1:07:05.680)

because how do you score such a thing?

Po-Shen Loh (1:07:07.640)

It's not like the answer was 72

Lex Fridman (1:07:10.800)

and you wrote 71 and it's close, right?

Po-Shen Loh (1:07:13.160)

The answer is 72 and you wrote 36.

Lex Fridman (1:07:15.280)

Oh, but that's pretty close

Po-Shen Loh (1:07:16.200)

because maybe you're just off by it.

Lex Fridman (1:07:18.880)

By the way, they're not numerical anyway,

Lex Fridman (1:07:20.400)

but I'm just giving some numerical analog

Lex Fridman (1:07:22.680)

to the way the scoring might work.

Po-Shen Loh (1:07:24.560)

They're all essays.

Lex Fridman (1:07:25.840)

And that's where I guess I have some role

Po-Shen Loh (1:07:28.240)

as well as some other people

Lex Fridman (1:07:29.320)

who helped me in the US delegation for coaches.

Po-Shen Loh (1:07:32.360)

We actually debate with the country which is organizing it.

Lex Fridman (1:07:37.560)

The country which is organizing the Olympiad

Po-Shen Loh (1:07:39.480)

brings about 50 people to help judge the written solutions.

Lex Fridman (1:07:45.160)

And you schedule these half hour appointments

Po-Shen Loh (1:07:48.360)

where the delegation from one country

Lex Fridman (1:07:50.520)

sits down at a table like this.

Po-Shen Loh (1:07:52.360)

Opposite side is two or three people from the host country.

Lex Fridman (1:07:55.520)

And they're just looking over these exam papers

Po-Shen Loh (1:07:58.000)

saying, well, how many points is this worth

Lex Fridman (1:08:00.640)

based on some rubric that has been designed?

Lex Fridman (1:08:03.120)

And this is a negotiation process

Lex Fridman (1:08:05.600)

where we're not trying to bargain

Lex Fridman (1:08:07.760)

and get the best score we can.

Lex Fridman (1:08:09.320)

In fact, sometimes we go to this table

Lex Fridman (1:08:10.880)

and we will say, we think we want less than what you gave us.

Lex Fridman (1:08:13.960)

This is how our, these are our principles.

Po-Shen Loh (1:08:16.280)

If you give us too much, we say, no, you gave us too much.

Lex Fridman (1:08:18.800)

We do that.

Po-Shen Loh (1:08:19.720)

However, the reason why this is an interesting process

Lex Fridman (1:08:22.280)

is because if you can imagine every country

Po-Shen Loh (1:08:24.160)

which is participating has its own language.

Lex Fridman (1:08:26.760)

And so if you're trying to grade the Mongolian scripts

Lex Fridman (1:08:28.760)

and they're written in Mongolian,

Lex Fridman (1:08:31.040)

if you don't read Mongolian, which most people don't,

Po-Shen Loh (1:08:33.520)

then the coaches are explaining to you,

Lex Fridman (1:08:36.680)

this is what the student has written.

Po-Shen Loh (1:08:38.760)

It's actually quite interesting process.

Lex Fridman (1:08:40.480)

So it's almost like a jury.

Po-Shen Loh (1:08:43.400)

Yes.

Lex Fridman (1:08:44.240)

You have, in the American legal system,

Po-Shen Loh (1:08:47.120)

you have a jury that where they're deliberating,

Lex Fridman (1:08:49.880)

but unlike a jury, there's the members of the jury

Po-Shen Loh (1:08:53.640)

speaking different languages sometimes.

Lex Fridman (1:08:55.800)

Yes. That's fascinating.

Lex Fridman (1:08:57.120)

But I mean, it's hard to know what to do

Lex Fridman (1:09:01.640)

because it's probably really, really competitive.

Lex Fridman (1:09:04.560)

But your sense is that ultimately people,

Lex Fridman (1:09:08.560)

like how do you prevent manipulation here, right?

Po-Shen Loh (1:09:14.240)

Well, we just hope that it's not happening.

Lex Fridman (1:09:17.000)

So we write in English.

Po-Shen Loh (1:09:19.240)

Therefore, everything that the US does,

Lex Fridman (1:09:21.400)

everyone can look at.

Lex Fridman (1:09:22.800)

So it's very hard for me.

Lex Fridman (1:09:24.120)

It's very hard for you to manipulate.

Po-Shen Loh (1:09:25.720)

We don't manipulate.

Lex Fridman (1:09:27.480)

We only hope that other people aren't.

Lex Fridman (1:09:29.360)

But at the same time, as you see, our philosophy was,

Lex Fridman (1:09:32.320)

we want to use this as a way to develop general talent.

Lex Fridman (1:09:35.560)

And although we do this for the six people who go

Lex Fridman (1:09:38.560)

to the International Math Olympiad,

Po-Shen Loh (1:09:40.640)

we really want that everyone at any,

Lex Fridman (1:09:42.880)

touched at any stage of this process

Po-Shen Loh (1:09:45.080)

get some skills that can help to contribute more later.

Lex Fridman (1:09:48.120)

So I don't know if you can say something insightful

Po-Shen Loh (1:09:51.520)

to this question,

Lex Fridman (1:09:52.560)

but what do you think makes a really hard math problem

Po-Shen Loh (1:09:56.200)

on this Olympiad, maybe in the courses you teach

Lex Fridman (1:09:59.760)

or in general?

Lex Fridman (1:10:01.080)

What makes for a hard problem?

Lex Fridman (1:10:03.520)

You've seen, I'm sure, a lot of really difficult problems.

Lex Fridman (1:10:06.280)

What makes a hard problem?

Lex Fridman (1:10:07.880)

So I could quantify it by the number of leaps of insight

Po-Shen Loh (1:10:12.160)

of changes of perspective that are along the way.

Lex Fridman (1:10:14.480)

And here's why.

Po-Shen Loh (1:10:15.800)

This is like a very theoretical computer science

Lex Fridman (1:10:17.560)

way of looking at it, okay?

Po-Shen Loh (1:10:19.080)

It's that each reframing of the problem

Lex Fridman (1:10:22.520)

and using of some tool,

Po-Shen Loh (1:10:23.840)

I actually call that a leap of insight.

Lex Fridman (1:10:25.360)

When you say, oh, wow, now I see,

Po-Shen Loh (1:10:27.400)

I should kind of put these plugs into those sockets

Lex Fridman (1:10:30.480)

like so, and suddenly I get to use that machine.

Po-Shen Loh (1:10:33.600)

Oh, but I'm not done yet.

Lex Fridman (1:10:34.920)

Now I need to do it again.

Po-Shen Loh (1:10:36.160)

Each such step is a large possible,

Lex Fridman (1:10:38.960)

large fan out in the search space.

Po-Shen Loh (1:10:41.120)

The number of these tells you the exponent.

Lex Fridman (1:10:44.040)

The base of the exponent is like how big,

Lex Fridman (1:10:46.600)

how many different possibilities you could try.

Lex Fridman (1:10:49.360)

And that's actually why,

Po-Shen Loh (1:10:51.200)

like if you have a three insight problem,

Lex Fridman (1:10:54.240)

that is not three times as hard as a one insight problem,

Po-Shen Loh (1:10:57.560)

because after you've made the one insight,

Lex Fridman (1:10:59.080)

it's not clear that that was the right track necessarily.

Po-Shen Loh (1:11:03.000)

Well, unless you're very into it.

Lex Fridman (1:11:03.840)

There's still a branching of possibility.

Po-Shen Loh (1:11:06.400)

Yeah.

Lex Fridman (1:11:07.240)

Right.

Po-Shen Loh (1:11:09.800)

You're saying there's problems like on the math Olympia

Lex Fridman (1:11:12.280)

that requires more than one insight?

Po-Shen Loh (1:11:13.680)

Yes.

Lex Fridman (1:11:14.520)

Those are the hard ones.

Lex Fridman (1:11:15.360)

And also I can tell you how you can tell.

Lex Fridman (1:11:17.560)

So this is how I also taught myself math

Po-Shen Loh (1:11:19.760)

when I was in college.

Lex Fridman (1:11:20.720)

So if you are taking a, not taught myself,

Po-Shen Loh (1:11:23.520)

I was taking classes, of course,

Lex Fridman (1:11:24.840)

but I was trying to read the textbook

Lex Fridman (1:11:26.720)

and I found out I was very bad at reading math textbooks.

Lex Fridman (1:11:29.480)

A math textbook has a long page of stuff that is all true,

Po-Shen Loh (1:11:32.760)

which after you read the page,

Lex Fridman (1:11:34.040)

you have no idea what you just read.

Po-Shen Loh (1:11:35.640)

Yeah.

Lex Fridman (1:11:36.480)

This is just a good summary of a math textbook.

Po-Shen Loh (1:11:39.120)

Okay.

Lex Fridman (1:11:39.960)

Yeah, because it's not clear why anything was done that way.

Lex Fridman (1:11:44.120)

And yes, everything is true,

Lex Fridman (1:11:45.400)

but how the heck did anyone think of that?

Lex Fridman (1:11:47.360)

So the way that I taught myself math eventually was,

Lex Fridman (1:11:50.840)

the way I read a math textbook

Po-Shen Loh (1:11:52.960)

is I would look at the theorem statement.

Lex Fridman (1:11:55.640)

I would look at the length of the proof

Lex Fridman (1:11:58.920)

and then I would close the book

Lex Fridman (1:12:00.000)

and attempt to reproof it myself.

Po-Shen Loh (1:12:01.280)

Yeah.

Lex Fridman (1:12:02.120)

That's brilliant.

Po-Shen Loh (1:12:03.520)

The length of the proof is telling you

Lex Fridman (1:12:05.520)

the number of insights,

Po-Shen Loh (1:12:06.800)

because the length of the proof is linear

Lex Fridman (1:12:08.760)

in the number of insights.

Po-Shen Loh (1:12:10.640)

Each insight takes space.

Lex Fridman (1:12:12.200)

Yeah.

Lex Fridman (1:12:13.040)

And if I know that it's a short proof,

Lex Fridman (1:12:14.640)

I know that there's only one insight.

Lex Fridman (1:12:16.080)

So when I'm doing my own way of solving the problem,

Lex Fridman (1:12:19.000)

like finding the proof,

Po-Shen Loh (1:12:20.520)

I quit if I have to do too many plugins.

Lex Fridman (1:12:23.080)

It's equivalent to a math contest.

Po-Shen Loh (1:12:24.880)

In a math contest I look,

Lex Fridman (1:12:25.800)

is it problem one, two, or three?

Po-Shen Loh (1:12:27.200)

That tells me how many insights there are.

Lex Fridman (1:12:28.920)

This is exactly what I did.

Po-Shen Loh (1:12:29.960)

That's brilliant.

Lex Fridman (1:12:30.880)

Linear in the number.

Po-Shen Loh (1:12:32.080)

I don't know.

Lex Fridman (1:12:32.920)

I think it's possible that that's true.

Po-Shen Loh (1:12:36.320)

Approximately, approximately.

Lex Fridman (1:12:37.160)

Approximately, yeah.

Po-Shen Loh (1:12:38.480)

I don't know if somebody out there

Lex Fridman (1:12:41.320)

is gonna try to formally prove this.

Po-Shen Loh (1:12:43.040)

Oh no, I mean, you're right.

Lex Fridman (1:12:44.160)

There are cases where maybe it's not quite linear,

Lex Fridman (1:12:46.040)

but in general.

Lex Fridman (1:12:47.040)

Well, some of it's notation too,

Lex Fridman (1:12:48.360)

and some of it is style and all those kinds of things,

Lex Fridman (1:12:50.920)

but within a textbook.

Po-Shen Loh (1:12:51.880)

Within the same book.

Lex Fridman (1:12:52.720)

Within the same book with the same.

Po-Shen Loh (1:12:54.040)

Within the same book on the same subject.

Lex Fridman (1:12:56.160)

Yeah.

Po-Shen Loh (1:12:57.000)

This is what I was using.

Lex Fridman (1:12:57.840)

That's hilarious.

Po-Shen Loh (1:12:58.760)

Because you know, if it's a two page proof,

Lex Fridman (1:13:00.360)

you just know this is gonna be insane, right?

Po-Shen Loh (1:13:02.760)

That's the scary thing about insights.

Lex Fridman (1:13:06.880)

You look like Andrew Wiles

Po-Shen Loh (1:13:08.200)

working on the Fermat's Last Theorem,

Lex Fridman (1:13:11.480)

is you don't know.

Po-Shen Loh (1:13:13.560)

Something seems like a good idea,

Lex Fridman (1:13:16.200)

and you have that idea,

Lex Fridman (1:13:17.280)

and it feels like this is a leap,

Lex Fridman (1:13:20.160)

like a totally new way to see it,

Lex Fridman (1:13:22.280)

but you have no idea if it's at all useful.

Lex Fridman (1:13:25.760)

Even if you think it's correct,

Po-Shen Loh (1:13:27.000)

you have no idea if this is like going to go down a path

Lex Fridman (1:13:30.120)

that's completely counterproductive

Po-Shen Loh (1:13:32.480)

or not productive at all.

Lex Fridman (1:13:34.520)

That's the crappy thing about invention,

Po-Shen Loh (1:13:38.400)

is like I have, I'm sure you do.

Lex Fridman (1:13:41.080)

I have a lot of really good ideas every single day,

Lex Fridman (1:13:44.640)

but like, and I'll go inside my head along them,

Lex Fridman (1:13:49.600)

along that little trajectory,

Lex Fridman (1:13:52.040)

but it could be just a total waste.

Lex Fridman (1:13:54.640)

And it's, you know what that feels like?

Po-Shen Loh (1:13:57.200)

It just feels like patience is required,

Lex Fridman (1:13:59.480)

not to get excited at any one thing.

Lex Fridman (1:14:01.800)

So I think this is interesting

Lex Fridman (1:14:03.360)

because you raised Andrew Wiles.

Lex Fridman (1:14:04.800)

He spent seven years attacking the same thing, right?

Lex Fridman (1:14:08.400)

And so I think that what attracts

Po-Shen Loh (1:14:10.360)

professional researchers to this

Lex Fridman (1:14:12.320)

is because even though it's very painful

Po-Shen Loh (1:14:14.880)

that you keep fighting with something,

Lex Fridman (1:14:16.680)

when you finally find the right insights

Lex Fridman (1:14:19.480)

and string them together,

Lex Fridman (1:14:20.680)

it feels really good, so.

Po-Shen Loh (1:14:23.080)

Well, there's also like short term,

Lex Fridman (1:14:26.040)

it feels good to, whether it's real or not,

Po-Shen Loh (1:14:31.480)

to pretend like you've solved something

Lex Fridman (1:14:33.560)

in the sense like you have an insight

Lex Fridman (1:14:35.400)

and there's a sense like this might be the insight

Lex Fridman (1:14:37.960)

that solves it.

Lex Fridman (1:14:39.000)

So at least for me, I just enjoy that rush of positivity

Lex Fridman (1:14:44.440)

even though I know statistically speaking

Po-Shen Loh (1:14:46.440)

is probably going to be a dead end.

Lex Fridman (1:14:48.520)

I'm the same way, I'm the same way.

Po-Shen Loh (1:14:49.840)

In fact, that's how I know whether

Lex Fridman (1:14:51.560)

I might want to keep thinking about this general problem.

Po-Shen Loh (1:14:54.320)

It's like, if I still see that I'm getting some insights,

Lex Fridman (1:14:57.280)

I'm not at a dead end yet.

Lex Fridman (1:14:59.000)

But that's also where I learned something

Lex Fridman (1:15:00.680)

from my PhD advisor.

Po-Shen Loh (1:15:01.920)

Actually, he was a real big inspiration on my life.

Lex Fridman (1:15:04.280)

His name is Benny Sudakov.

Po-Shen Loh (1:15:05.760)

In fact, he grew up in the former Soviet Union.

Lex Fridman (1:15:08.120)

He was from Georgia, but he's an incredible person.

Lex Fridman (1:15:12.280)

But one thing I learned was choose the problems to work on

Lex Fridman (1:15:16.680)

that might matter if you succeed.

Po-Shen Loh (1:15:21.040)

Because that's why, for example, we dug into COVID.

Lex Fridman (1:15:23.400)

It was just, well, suppose we succeed

Po-Shen Loh (1:15:25.800)

in finding some interesting insight here.

Lex Fridman (1:15:27.840)

Well, it actually matters.

Po-Shen Loh (1:15:29.120)

That is worth a laugh.

Lex Fridman (1:15:30.960)

Yeah, and I think COVID, the way you're approaching COVID

Po-Shen Loh (1:15:36.480)

has two interesting possibilities.

Lex Fridman (1:15:38.200)

One, it might help with COVID or another pandemic,

Lex Fridman (1:15:41.680)

but two, I mean, just this whole network theory space,

Lex Fridman (1:15:48.360)

you might unlock some deep understanding

Po-Shen Loh (1:15:51.200)

about the interaction with human beings.

Lex Fridman (1:15:53.080)

That might have nothing to do with the pandemic.

Po-Shen Loh (1:15:55.880)

There's a space of possible impacts

Lex Fridman (1:15:58.360)

that may be direct or indirect.

Lex Fridman (1:16:00.440)

And the same thing is with Andrew Wiles's proof.

Lex Fridman (1:16:03.280)

I don't understand, but apparently the pieces of it

Po-Shen Loh (1:16:08.120)

are really impactful for mathematics,

Lex Fridman (1:16:12.040)

even if the main theorem is not.

Lex Fridman (1:16:14.880)

So along the way, the insights you have

Lex Fridman (1:16:18.240)

might be really powerful for unexpected reasons.

Lex Fridman (1:16:22.920)

So I like what you said.

Lex Fridman (1:16:23.760)

This is something that I learned from another friend of mine.

Po-Shen Loh (1:16:26.840)

He's a very famous researcher.

Lex Fridman (1:16:28.080)

All these people are more famous than I am.

Po-Shen Loh (1:16:29.960)

His name is Jacob Fox.

Lex Fridman (1:16:30.960)

He's Jacob Fox at Stanford.

Po-Shen Loh (1:16:32.200)

Also a very big inspiration for me.

Lex Fridman (1:16:33.800)

We were both grad students together at the same time.

Po-Shen Loh (1:16:36.000)

Well, most importantly,

Lex Fridman (1:16:36.840)

you're good at selecting good friends.

Po-Shen Loh (1:16:38.320)

Ah, yeah, well, that's the key.

Lex Fridman (1:16:40.000)

You gotta find good people to learn things from.

Lex Fridman (1:16:42.040)

But his thing was, he often said,

Lex Fridman (1:16:44.400)

if you solve a math problem and have this math proof,

Lex Fridman (1:16:46.840)

math problem for him is like a proof, right?

Lex Fridman (1:16:48.920)

So suppose you came up with this proof.

Po-Shen Loh (1:16:50.440)

He always asks, what have we learned from this

Lex Fridman (1:16:53.440)

that we could potentially use for something else?

Po-Shen Loh (1:16:56.040)

It's not just, did you solve the problem

Lex Fridman (1:16:58.080)

that was supposed to be famous?

Lex Fridman (1:17:00.080)

And is there something new in the course of solving this

Lex Fridman (1:17:03.360)

that you had to invent

Lex Fridman (1:17:04.600)

that we could now use as a tool elsewhere?

Lex Fridman (1:17:06.720)

Yeah, there's this funny effect

Po-Shen Loh (1:17:08.880)

where just looking at different fields

Lex Fridman (1:17:12.680)

where people discover parallels.

Po-Shen Loh (1:17:15.040)

They'll prove something, it'll be a totally new result.

Lex Fridman (1:17:17.520)

And then somebody later realizes

Po-Shen Loh (1:17:19.040)

this was already done 30 years ago

Lex Fridman (1:17:20.680)

in another discipline, in another way.

Lex Fridman (1:17:23.400)

And it's really interesting.

Lex Fridman (1:17:26.440)

Now, we did this offline

Po-Shen Loh (1:17:27.800)

in another illustration he showed to me.

Lex Fridman (1:17:30.240)

It's interesting to see the different perspectives

Po-Shen Loh (1:17:33.920)

on a problem.

Lex Fridman (1:17:35.560)

It kind of points like there's just like

Po-Shen Loh (1:17:38.760)

very few novel ideas that everything else,

Lex Fridman (1:17:42.320)

that most of us are just looking at different perspective

Po-Shen Loh (1:17:45.480)

on the same idea.

Lex Fridman (1:17:47.200)

And it makes you wonder this old silly question

Po-Shen Loh (1:17:51.520)

that I have to ask you is,

Lex Fridman (1:17:53.840)

do you think mathematics is discovered or invented?

Lex Fridman (1:17:58.200)

Do you think we're creating new idea?

Lex Fridman (1:18:02.560)

Are we building a set of knowledge

Lex Fridman (1:18:06.160)

that's distinct from reality?

Lex Fridman (1:18:09.200)

Or are we actually like,

Po-Shen Loh (1:18:12.000)

is math almost like a shovel

Lex Fridman (1:18:13.480)

where we're digging to like this core set of truths

Lex Fridman (1:18:16.880)

that were always there all along?

Lex Fridman (1:18:20.000)

So I personally feel like it's discovered.

Lex Fridman (1:18:22.800)

But that's also because I guess the way that

Lex Fridman (1:18:25.360)

I like to choose what questions to work on

Po-Shen Loh (1:18:27.400)

are questions that maybe we'll get to learn something about

Lex Fridman (1:18:31.120)

why is this hard?

Po-Shen Loh (1:18:32.200)

I mean, I'm often attracted to questions

Lex Fridman (1:18:33.880)

that look simple, but are hard, right?

Lex Fridman (1:18:36.880)

And what could you possibly learn from that?

Lex Fridman (1:18:38.680)

Sort of like probably the attraction

Po-Shen Loh (1:18:40.280)

of Fermat's last theorem, as you mentioned,

Lex Fridman (1:18:42.680)

simple statement, why is it so hard?

Lex Fridman (1:18:44.960)

So I'm more on the discovered side.

Lex Fridman (1:18:47.520)

And I also feel like if we ever ran into

Po-Shen Loh (1:18:49.600)

an intelligent other species in the universe,

Lex Fridman (1:18:54.800)

probably if we compared notes,

Po-Shen Loh (1:18:56.960)

there might be some similarities between both of us

Lex Fridman (1:19:00.120)

realizing that pi is important.

Po-Shen Loh (1:19:02.280)

Because you might say, why, why humans,

Lex Fridman (1:19:04.480)

do humans like circles more than others?

Po-Shen Loh (1:19:06.080)

I think stars also like circles.

Lex Fridman (1:19:08.120)

I think planets like circles.

Po-Shen Loh (1:19:09.880)

They're not perfect circles,

Lex Fridman (1:19:10.920)

but nevertheless, the concept of a circle

Po-Shen Loh (1:19:13.160)

is just point and constant distance.

Lex Fridman (1:19:15.760)

Doesn't get any simpler than that.

Po-Shen Loh (1:19:17.400)

It's possible that like an alien species

Lex Fridman (1:19:19.960)

will have, depending on different cognitive capabilities

Lex Fridman (1:19:23.280)

and different perception systems,

Lex Fridman (1:19:25.000)

will be able to see things

Po-Shen Loh (1:19:28.200)

that are much different than circles.

Lex Fridman (1:19:30.400)

And so if it's discovered,

Po-Shen Loh (1:19:33.640)

it will still be pointing at a lot of same

Lex Fridman (1:19:35.840)

geometrical concepts, mathematical concepts,

Lex Fridman (1:19:38.560)

but it's interesting to think of how many things

Lex Fridman (1:19:43.520)

we would have to still align,

Po-Shen Loh (1:19:45.720)

not just based on notation, but based on understanding,

Lex Fridman (1:19:48.520)

like just like some basic mathematical concepts,

Po-Shen Loh (1:19:53.960)

like how much work is there going to be

Lex Fridman (1:19:56.880)

in trying to find a common language?

Po-Shen Loh (1:19:59.240)

I mean, this is, I think Stephen Wolfram and his son

Lex Fridman (1:20:02.480)

helped with the movie Arrival,

Po-Shen Loh (1:20:04.600)

like the developing an alien language,

Lex Fridman (1:20:07.240)

like how would aliens communicate with humans?

Po-Shen Loh (1:20:10.640)

It's fascinating,

Lex Fridman (1:20:11.800)

because like math seems to be the most promising thing,

Lex Fridman (1:20:14.360)

but even like math,

Lex Fridman (1:20:15.920)

like how do you visualize mathematical ideas?

Po-Shen Loh (1:20:22.000)

It feels like there has to be an interactive component,

Lex Fridman (1:20:24.640)

just like we have a conversation.

Po-Shen Loh (1:20:26.560)

There has to be, this is something we don't,

Lex Fridman (1:20:28.440)

I think, think about often, which is like,

Po-Shen Loh (1:20:31.440)

with somebody who doesn't know anything about math,

Lex Fridman (1:20:33.880)

doesn't know anything about English

Po-Shen Loh (1:20:35.480)

or any other natural language,

Lex Fridman (1:20:37.440)

how would we describe,

Po-Shen Loh (1:20:40.120)

we talked offline about visual proofs.

Lex Fridman (1:20:42.400)

How would we, through visual proofs, have a conversation

Po-Shen Loh (1:20:47.400)

where we say something, here's the concept,

Lex Fridman (1:20:50.160)

the way we see it, does that make sense to you?

Lex Fridman (1:20:53.800)

And like, can you mess with that concept

Lex Fridman (1:20:57.360)

to make it sense for you?

Lex Fridman (1:20:58.760)

And then go back and forth in this kind of way.

Lex Fridman (1:21:01.400)

So purely through mathematics,

Po-Shen Loh (1:21:03.040)

I'm sure it's possible to have those kinds of experiments

Lex Fridman (1:21:04.960)

with like tribes on earth that don't,

Po-Shen Loh (1:21:07.040)

there's no common language.

Lex Fridman (1:21:08.400)

Through math, like draw a circle

Lex Fridman (1:21:10.840)

and see what they do with it, right?

Lex Fridman (1:21:13.200)

Do some of these visual proofs,

Po-Shen Loh (1:21:15.640)

like the summation of the odds and adds up to the squares.

Lex Fridman (1:21:19.720)

Yes, I wonder how difficult that is

Po-Shen Loh (1:21:21.960)

before one or the other species murders themselves.

Lex Fridman (1:21:24.760)

That's a good question.

Po-Shen Loh (1:21:27.720)

I hope that the curiosity for knowledge

Lex Fridman (1:21:29.760)

will overpower the greedy,

Po-Shen Loh (1:21:31.480)

this is back to our game theory thing,

Lex Fridman (1:21:33.520)

that the curiosity of like discovering math together

Po-Shen Loh (1:21:37.280)

will overpower the desire for resources

Lex Fridman (1:21:40.040)

and ultimately like willing to commit violence

Po-Shen Loh (1:21:44.440)

in order to gain those resources.

Lex Fridman (1:21:46.440)

I think as we progress,

Po-Shen Loh (1:21:47.920)

become more and more intelligent as a species,

Lex Fridman (1:21:50.080)

I'm hoping we would value more and more of the knowledge

Po-Shen Loh (1:21:53.320)

because we'll come up with clever ways

Lex Fridman (1:21:54.880)

to gain more resources so we won't be so resource starved.

Po-Shen Loh (1:21:58.160)

I don't know.

Lex Fridman (1:21:59.000)

That's a hopeful message for when we finally meet aliens.

Po-Shen Loh (1:22:01.240)

Yeah, yeah.

Lex Fridman (1:22:02.600)

The cool thing about the Math Olympiad,

Po-Shen Loh (1:22:07.120)

I don't know if you know work from Francois Chollet

Lex Fridman (1:22:11.400)

from Google, he came up with this kind of IQ test slash,

Po-Shen Loh (1:22:16.400)

it kind of has similar aspects to it

Lex Fridman (1:22:18.880)

that also the Math Olympiad does for AI.

Lex Fridman (1:22:24.320)

So he came up with these tests

Lex Fridman (1:22:25.920)

where they're very simple for humans,

Lex Fridman (1:22:29.040)

but very difficult for AI to illustrate exactly

Lex Fridman (1:22:32.280)

why we're just not good at seeing a totally new problem.

Po-Shen Loh (1:22:38.960)

Sorry, AI systems are not good at looking at a new problem

Lex Fridman (1:22:43.960)

that requires you to detect

Po-Shen Loh (1:22:46.680)

that there's a symmetry of some kind,

Lex Fridman (1:22:48.560)

or there's a pattern that hasn't seen before.

Po-Shen Loh (1:22:53.880)

The pattern is like obvious to us humans,

Lex Fridman (1:22:56.680)

but it's not so obvious to find that kind of,

Po-Shen Loh (1:22:59.480)

you're inventing a pattern that's there

Lex Fridman (1:23:03.640)

in order to then find a solution.

Po-Shen Loh (1:23:09.720)

I don't know if you can comment on that.

Lex Fridman (1:23:12.720)

If you can comment on, but from an AI perspective

Lex Fridman (1:23:16.280)

and from a math problem perspective,

Lex Fridman (1:23:19.440)

what do you think is intelligence?

Lex Fridman (1:23:22.120)

What do you think is the thing

Lex Fridman (1:23:23.640)

that allows us to solve that problem?

Lex Fridman (1:23:25.760)

And how hard is it to build a machine to do that?

Lex Fridman (1:23:29.560)

Asking for a friend.

Po-Shen Loh (1:23:30.560)

Yeah.

Lex Fridman (1:23:31.400)

So I guess, you see,

Po-Shen Loh (1:23:33.000)

because if I just think of the raw search space, it's huge.

Lex Fridman (1:23:35.920)

That's why you can't do it.

Lex Fridman (1:23:37.000)

And if I think about what makes somebody

Lex Fridman (1:23:38.640)

good at doing these things, they have this heuristic sense.

Po-Shen Loh (1:23:42.320)

It's almost like a good chess player of saying,

Lex Fridman (1:23:44.320)

let's not keep analyzing down this way

Po-Shen Loh (1:23:45.960)

because there's some heuristic reason

Lex Fridman (1:23:47.600)

why that's a bad way to go.

Lex Fridman (1:23:49.240)

Where did they get that heuristic from?

Lex Fridman (1:23:50.520)

Now, that's a good question.

Po-Shen Loh (1:23:51.760)

I don't know.

Lex Fridman (1:23:53.000)

Because that, if you asked them to explain to you,

Po-Shen Loh (1:23:56.680)

they could probably say something in words

Lex Fridman (1:23:58.200)

that sounds like it makes sense,

Lex Fridman (1:23:59.640)

but I'm guessing that's only a part

Lex Fridman (1:24:01.240)

of what's really going on in their brain

Po-Shen Loh (1:24:03.000)

of evaluating that position.

Lex Fridman (1:24:04.800)

You know what I mean?

Po-Shen Loh (1:24:05.640)

If you ask Gary Kasparov, what is good,

Lex Fridman (1:24:06.800)

or why is this position good, he will say something,

Lex Fridman (1:24:10.240)

but probably not approximating everything

Lex Fridman (1:24:12.520)

that's going on inside.

Lex Fridman (1:24:14.000)

So there's basically a function being computed,

Lex Fridman (1:24:16.920)

but it's hard to articulate what that function is.

Po-Shen Loh (1:24:19.200)

Now, the question is, could a computer get as good

Lex Fridman (1:24:21.760)

at computing these kinds of heuristic functions?

Po-Shen Loh (1:24:24.200)

Maybe.

Lex Fridman (1:24:25.040)

I'm not enough of an expert to understand,

Lex Fridman (1:24:27.840)

but one bit of me has always been a little bit curious

Lex Fridman (1:24:30.320)

of whether or not the human brain has a particular tendency

Po-Shen Loh (1:24:34.200)

due to its wiring to come up with certain kinds of things,

Lex Fridman (1:24:37.480)

which is just natural due to the way

Po-Shen Loh (1:24:39.520)

that the topology of the neurons and whatever is there,

Lex Fridman (1:24:43.520)

for which if you tried to just build from scratch

Po-Shen Loh (1:24:46.120)

a computer to do it,

Lex Fridman (1:24:47.280)

would it naturally have different tendencies?

Po-Shen Loh (1:24:49.640)

I don't know.

Lex Fridman (1:24:50.480)

This is just me being completely ignorant

Lex Fridman (1:24:52.280)

and just saying a few ideas.

Lex Fridman (1:24:53.640)

Well, this is a good thing that mathematics shows

Po-Shen Loh (1:24:56.160)

is we don't have to be,

Lex Fridman (1:24:57.840)

so math and physics or mathematical physics

Po-Shen Loh (1:25:00.920)

operates in a world that's different

Lex Fridman (1:25:02.440)

than our descendants of eight brains operate in.

Lex Fridman (1:25:07.440)

So it allows us to have multiple, many, many dimensions.

Lex Fridman (1:25:11.880)

It allows us to work on weird surfaces.

Po-Shen Loh (1:25:15.440)

I would like topology as a discipline is just weird to me.

Lex Fridman (1:25:19.920)

It's really complicated,

Lex Fridman (1:25:21.120)

but it allows us to work in that space,

Lex Fridman (1:25:23.560)

the differential geometry and all those kinds of things

Po-Shen Loh (1:25:25.840)

where it's totally outside of our natural day to day

Lex Fridman (1:25:30.560)

four dimensional experience,

Po-Shen Loh (1:25:33.240)

3D dimensional with time experience.

Lex Fridman (1:25:35.320)

So math gives me hope that we can discover

Po-Shen Loh (1:25:44.200)

the processes of intelligence outside the limited nature

Lex Fridman (1:25:49.720)

of our own human experiences.

Lex Fridman (1:25:51.960)

But you said that you're not an expert.

Lex Fridman (1:25:55.920)

It's kind of funny.

Po-Shen Loh (1:25:57.200)

I find that we know so little about intelligence

Lex Fridman (1:26:02.200)

that I honestly think like almost children are more expert

Po-Shen Loh (1:26:08.760)

at creating artificial intelligence systems than adults.

Lex Fridman (1:26:14.400)

I feel like we know so little,

Po-Shen Loh (1:26:15.760)

we really need to think outside the box.

Lex Fridman (1:26:18.200)

And those little,

Po-Shen Loh (1:26:19.840)

I found people should check out

Lex Fridman (1:26:22.120)

Francois Chollet's little exams,

Lex Fridman (1:26:24.640)

but even just solving math problems,

Lex Fridman (1:26:27.960)

I don't know if you've ever done this for yourself,

Lex Fridman (1:26:30.560)

but when you solve a math problem,

Lex Fridman (1:26:33.400)

you kind of then trace back and try to figure out

Lex Fridman (1:26:38.160)

where did that idea come from?

Lex Fridman (1:26:40.440)

Like what was I visualizing in my head?

Lex Fridman (1:26:45.240)

How did I start visualizing it that way?

Lex Fridman (1:26:48.640)

Why did I start rotating that cube in my head in that way?

Lex Fridman (1:26:52.280)

Like what is that?

Lex Fridman (1:26:53.120)

If I were to try to build a program that does that,

Lex Fridman (1:26:55.480)

where did that come from?

Lex Fridman (1:26:56.880)

So this is interesting.

Lex Fridman (1:26:58.960)

So I try to do this to teach middle school students

Lex Fridman (1:27:02.680)

how to learn how to create and think and invent.

Lex Fridman (1:27:05.560)

And the way I do it

Lex Fridman (1:27:06.840)

is there are these math competition problems

Lex Fridman (1:27:08.920)

and I'm working in collaboration

Lex Fridman (1:27:10.560)

with the people who run those.

Lex Fridman (1:27:12.000)

And I will turn on my YouTube live

Lex Fridman (1:27:14.040)

and for the first time,

Po-Shen Loh (1:27:15.080)

look at those questions and live solve them.

Lex Fridman (1:27:18.480)

The reason I do this is to let the middle school students

Lex Fridman (1:27:21.160)

and the high school students and the adults

Lex Fridman (1:27:22.400)

whoever wants to watch,

Po-Shen Loh (1:27:23.440)

just see what exactly goes on through someone's head

Lex Fridman (1:27:27.360)

as they go and attempt to invent what they need to do

Po-Shen Loh (1:27:30.400)

to solve the question.

Lex Fridman (1:27:32.080)

So I've actually thought about that.

Po-Shen Loh (1:27:34.520)

I think that, first of all, as a teacher,

Lex Fridman (1:27:37.600)

I think about that because whenever I want to explain

Po-Shen Loh (1:27:39.920)

to a student how to do something,

Lex Fridman (1:27:42.000)

I want to explain how it made sense,

Lex Fridman (1:27:44.080)

why it's intuitive to do the following things

Lex Fridman (1:27:46.160)

and why the wrong things are wrong.

Po-Shen Loh (1:27:48.720)

Not just why this one short fast way,

Lex Fridman (1:27:51.840)

well, why this is the right way, if that makes sense.

Lex Fridman (1:27:54.600)

So my point is I'm actually always thinking about that.

Lex Fridman (1:27:57.200)

Like how would you think about these things?

Lex Fridman (1:27:58.880)

And then I eventually decided the easiest way

Lex Fridman (1:28:00.560)

to expose this would just be to go live on YouTube

Lex Fridman (1:28:04.440)

and just say, I've never seen any of these questions before.

Lex Fridman (1:28:06.560)

Here we go.

Po-Shen Loh (1:28:07.760)

Don't you get, that's anxiety inducing for me.

Lex Fridman (1:28:12.680)

Don't you get trapped in a kind of like little dead ends

Lex Fridman (1:28:17.240)

of confusion, even on middle school problems?

Lex Fridman (1:28:20.320)

Yes, that's what the comments are for.

Po-Shen Loh (1:28:21.960)

The live comments come in and students say, try this.

Lex Fridman (1:28:24.600)

Oh wow.

Po-Shen Loh (1:28:25.600)

It's actually pretty good.

Lex Fridman (1:28:26.640)

And I'll never get stuck.

Po-Shen Loh (1:28:27.760)

I mean, I'm willing to go on camera and say,

Lex Fridman (1:28:30.400)

guess what, Potion Dough can't do this.

Po-Shen Loh (1:28:32.160)

That's fine.

Lex Fridman (1:28:33.120)

But then what ends up happening is you will then see

Lex Fridman (1:28:35.760)

how maybe somebody saying something and I look at the chat

Lex Fridman (1:28:38.080)

and I say, aha, that actually looks useful.

Po-Shen Loh (1:28:40.600)

Now that also shows how not all ideas,

Lex Fridman (1:28:44.000)

not all suggestions are the same power, if that makes sense.

Po-Shen Loh (1:28:46.880)

Because if I actually do get stuck,

Lex Fridman (1:28:48.120)

I'll go fishing through the chat, if you've got any ideas.

Po-Shen Loh (1:28:52.000)

I don't know if you can speak to this,

Lex Fridman (1:28:53.280)

but is there a moment for the middle school students,

Po-Shen Loh (1:28:57.320)

maybe high school as well,

Lex Fridman (1:28:59.640)

where there's like a turning point for them

Po-Shen Loh (1:29:04.360)

where they maybe fall in love with mathematics

Lex Fridman (1:29:07.800)

or they get it?

Po-Shen Loh (1:29:09.720)

Is there something to be said about like discovering

Lex Fridman (1:29:13.360)

that moment and trying to grab them to get them

Po-Shen Loh (1:29:17.640)

to understand that mathematics is something,

Lex Fridman (1:29:20.200)

no matter what they wanna do in life

Lex Fridman (1:29:21.520)

could be part of their life?

Lex Fridman (1:29:23.040)

Yes.

Po-Shen Loh (1:29:23.880)

I actually do think that the middle school

Lex Fridman (1:29:25.440)

is exactly the right time

Po-Shen Loh (1:29:26.800)

because that's the place

Lex Fridman (1:29:27.880)

where your mathematical understanding

Po-Shen Loh (1:29:30.400)

gets just sophisticated enough

Lex Fridman (1:29:32.440)

that you can start doing interesting things.

Po-Shen Loh (1:29:34.640)

Because if you're early on and counting,

Lex Fridman (1:29:37.600)

I'm honestly not very good at teaching you new insights.

Po-Shen Loh (1:29:40.720)

My wife is pretty good at that.

Lex Fridman (1:29:41.880)

But somehow once you get to this part

Po-Shen Loh (1:29:44.120)

where you know what a fraction is

Lex Fridman (1:29:45.680)

and when you know how to add and how to multiply

Lex Fridman (1:29:49.560)

and what the area of a triangle is,

Lex Fridman (1:29:51.200)

at that point to me, the whole world opens up

Lex Fridman (1:29:54.120)

and you can start observing

Lex Fridman (1:29:55.240)

there are really nifty coincidences,

Po-Shen Loh (1:29:57.360)

the things that made the Greek mathematicians

Lex Fridman (1:29:59.840)

and the ancient mathematicians excited.

Po-Shen Loh (1:30:02.120)

Actually back then it was exciting

Lex Fridman (1:30:03.880)

to discover the Pythagorean theorem.

Po-Shen Loh (1:30:05.680)

It wasn't just homework.

Lex Fridman (1:30:07.840)

So is there,

Po-Shen Loh (1:30:10.280)

which discipline do you think

Lex Fridman (1:30:11.760)

has the most exciting coincidences?

Lex Fridman (1:30:14.080)

So is it geometry?

Lex Fridman (1:30:16.360)

Is it algebra?

Lex Fridman (1:30:17.920)

Or is it calculus?

Lex Fridman (1:30:21.560)

Well, you see, you're asking me

Lex Fridman (1:30:22.680)

and I'm the guy who gets the most excited

Lex Fridman (1:30:24.520)

when the combinatorics shows up in the geometry.

Po-Shen Loh (1:30:27.320)

Is it, okay.

Lex Fridman (1:30:30.040)

So it's the combinatorics in the geometry.

Lex Fridman (1:30:33.080)

So first of all, the nice thing about geometry,

Lex Fridman (1:30:35.240)

this is the same nice thing about computer vision

Po-Shen Loh (1:30:37.920)

is it's visual.

Lex Fridman (1:30:40.040)

So geometry, you can draw circles and triangles and stuff.

Lex Fridman (1:30:42.720)

So it naturally presents itself

Lex Fridman (1:30:46.720)

to the visual proof, right?

Lex Fridman (1:30:49.280)

But also the nice thing about geometry,

Lex Fridman (1:30:51.160)

I think for me is the earliest class,

Po-Shen Loh (1:30:56.360)

the earliest discipline where there's,

Lex Fridman (1:30:59.680)

that's most amenable to the exploration,

Po-Shen Loh (1:31:02.600)

the invention through proofs.

Lex Fridman (1:31:04.960)

The idea of proofs I think is most easily shown in geometry

Po-Shen Loh (1:31:09.640)

because it's so visual, I guess.

Lex Fridman (1:31:12.280)

So that to me is like,

Po-Shen Loh (1:31:14.400)

if I were to think about

Lex Fridman (1:31:15.440)

when I first fell in love with math, it would be geometry.

Lex Fridman (1:31:18.360)

And sadly enough, that's not used.

Lex Fridman (1:31:21.280)

Geometry only has a little,

Po-Shen Loh (1:31:23.240)

appears briefly in the journey of a student.

Lex Fridman (1:31:29.000)

And it kind of disappears.

Lex Fridman (1:31:30.600)

And not until much later,

Lex Fridman (1:31:32.320)

which there may be like differential geometry,

Po-Shen Loh (1:31:36.560)

I don't know where else it shows up.

Lex Fridman (1:31:37.720)

For me in computer science,

Po-Shen Loh (1:31:38.840)

like you could start to think about

Lex Fridman (1:31:40.960)

like computational geometry or even graph theory

Po-Shen Loh (1:31:44.520)

as a kind of geometry.

Lex Fridman (1:31:45.600)

You could start to think about it visually,

Po-Shen Loh (1:31:47.200)

although it's pretty tricky.

Lex Fridman (1:31:49.320)

But yeah, it was always,

Po-Shen Loh (1:31:51.080)

that was the most beautiful one.

Lex Fridman (1:31:53.000)

Everything else, I guess calculus can be kind of visual too.

Po-Shen Loh (1:31:56.200)

That can be pretty beautiful.

Lex Fridman (1:31:57.800)

But is there something you try to look for in the student

Lex Fridman (1:32:05.360)

to see like, how can I inspire them at this moment?

Lex Fridman (1:32:09.920)

Or is this like individual student to student?

Lex Fridman (1:32:11.960)

Is there something you could say there?

Lex Fridman (1:32:13.880)

So first of all,

Po-Shen Loh (1:32:14.720)

I really think that every student

Lex Fridman (1:32:16.440)

can pick up all of this skill.

Po-Shen Loh (1:32:17.880)

I really do think so.

Lex Fridman (1:32:18.720)

I don't think it's something only for a few.

Lex Fridman (1:32:20.520)

And so if I'm looking for a student,

Lex Fridman (1:32:23.680)

actually oftentimes what I'm,

Po-Shen Loh (1:32:25.080)

if I'm looking at a particular student,

Lex Fridman (1:32:26.800)

the question is,

Lex Fridman (1:32:28.400)

how can we help you feel like

Lex Fridman (1:32:30.560)

you have the power to invent also?

Po-Shen Loh (1:32:32.880)

Because I think a lot of people

Lex Fridman (1:32:34.200)

are used to thinking about math

Po-Shen Loh (1:32:35.600)

as something where the teacher will show you what to do

Lex Fridman (1:32:37.960)

and then you will do it.

Po-Shen Loh (1:32:39.240)

Yes.

Lex Fridman (1:32:40.080)

So I think that the key is to show that they have some,

Po-Shen Loh (1:32:42.160)

let them see that they have some power to invent.

Lex Fridman (1:32:44.280)

And at that point,

Po-Shen Loh (1:32:45.120)

it's often starting by trying to give a question

Lex Fridman (1:32:47.640)

that they don't know how to do.

Po-Shen Loh (1:32:48.720)

You want to find these questions

Lex Fridman (1:32:49.880)

that they don't know how to do,

Po-Shen Loh (1:32:51.160)

that they can think about,

Lex Fridman (1:32:53.000)

and then they can solve.

Lex Fridman (1:32:54.480)

And then suddenly they say,

Lex Fridman (1:32:55.600)

my gosh, I've had a situation,

Po-Shen Loh (1:32:57.800)

I've had an experience where I didn't know what to do.

Lex Fridman (1:33:00.440)

And after a while, I did.

Po-Shen Loh (1:33:03.320)

Is there advice you can give on how to learn math

Lex Fridman (1:33:08.760)

for people, whether it's a middle school,

Po-Shen Loh (1:33:10.720)

whether it's somebody as an adult

Lex Fridman (1:33:14.520)

kind of gave up on math maybe early on?

Po-Shen Loh (1:33:19.360)

I actually think that these math competition problems,

Lex Fridman (1:33:22.080)

middle school and high school are really good.

Po-Shen Loh (1:33:23.800)

They're actually very hard.

Lex Fridman (1:33:25.080)

So if you haven't had this kind of experience before

Lex Fridman (1:33:29.160)

and you grab a middle school math competition problem

Lex Fridman (1:33:32.960)

from the state level,

Po-Shen Loh (1:33:33.920)

which is used to decide who represents the state

Lex Fridman (1:33:36.320)

in the country, in the United States, for example,

Po-Shen Loh (1:33:39.160)

those are pretty tricky.

Lex Fridman (1:33:40.680)

And even if you are a professional,

Po-Shen Loh (1:33:43.560)

maybe not doing mathematical things

Lex Fridman (1:33:45.480)

and you're not a middle school student, you'll struggle.

Lex Fridman (1:33:48.280)

So I find that these things really do teach you things

Lex Fridman (1:33:51.080)

by trying to work on these questions.

Po-Shen Loh (1:33:53.080)

Is there a Googleable term that you could use

Lex Fridman (1:33:56.760)

for the organization, for the state competitions?

Po-Shen Loh (1:33:59.440)

Ah, yeah.

Lex Fridman (1:34:00.280)

So there are a number of different ones

Po-Shen Loh (1:34:02.280)

that are quite popular.

Lex Fridman (1:34:03.600)

One of them is called Math Counts, M A T H C O U N T S.

Lex Fridman (1:34:07.760)

And that's a big tournament,

Lex Fridman (1:34:08.880)

which actually has a state level.

Po-Shen Loh (1:34:10.480)

There's also a mathleague.org,

Lex Fridman (1:34:12.680)

mathleague, L E A G U E dot org,

Po-Shen Loh (1:34:15.360)

also has this kind of tiered tournament structure.

Lex Fridman (1:34:18.680)

There's also the American math competitions, AMC 8.

Po-Shen Loh (1:34:22.600)

AMC also has AMC 10, that's for 10th grade and below

Lex Fridman (1:34:25.920)

and AMC 12.

Po-Shen Loh (1:34:27.240)

These are all run by the Mathematical Association

Lex Fridman (1:34:29.240)

of America.

Lex Fridman (1:34:30.280)

And these are all ways to find old questions.

Lex Fridman (1:34:32.880)

What about the daily challenges that you run?

Lex Fridman (1:34:34.840)

What are those about?

Lex Fridman (1:34:36.000)

We do that too.

Lex Fridman (1:34:36.840)

But I mean, the difference was ours isn't,

Lex Fridman (1:34:39.000)

that one's not free.

Lex Fridman (1:34:39.920)

So I should actually probably be careful.

Lex Fridman (1:34:42.080)

The things that I've just mentioned are also not free.

Po-Shen Loh (1:34:44.280)

Not all of those things I mentioned just now

Lex Fridman (1:34:45.720)

are free either.

Po-Shen Loh (1:34:46.600)

Well, people can figure out what is free and what's not,

Lex Fridman (1:34:48.760)

but this is really nice to know what's out there.

Lex Fridman (1:34:51.040)

But can you speak a little bit to the daily challenges?

Lex Fridman (1:34:53.720)

Sure, sure.

Lex Fridman (1:34:54.560)

So that's actually what we did when,

Lex Fridman (1:34:56.800)

I guess I was thinking about,

Lex Fridman (1:34:58.480)

how would I try to develop that skill in people

Lex Fridman (1:35:02.040)

if we had the power to architect the entire system ourselves?

Lex Fridman (1:35:05.360)

So that's called the daily challenge with Po Shan Luo.

Lex Fridman (1:35:07.440)

It's not free because that's actually how I pay

Po-Shen Loh (1:35:09.720)

for everything else I do.

Lex Fridman (1:35:11.240)

So that was the idea.

Lex Fridman (1:35:12.840)

But the concept was, aha, now let's invent from scratch.

Lex Fridman (1:35:16.480)

So if we're gonna go from scratch

Lex Fridman (1:35:17.960)

and we're gonna use technology,

Lex Fridman (1:35:19.720)

what if we made every single lesson

Po-Shen Loh (1:35:23.120)

something where first I say,

Lex Fridman (1:35:24.680)

hey, here's an interesting question.

Po-Shen Loh (1:35:25.840)

Recorded, of course, not live.

Lex Fridman (1:35:27.080)

But it's like, I say,

Po-Shen Loh (1:35:27.920)

hey, here's an interesting question.

Lex Fridman (1:35:28.760)

Why don't we think about this?

Lex Fridman (1:35:29.880)

But I know you don't know how to do it.

Lex Fridman (1:35:32.000)

So now you think,

Lex Fridman (1:35:32.960)

and a minute later a hint pops on the screen.

Lex Fridman (1:35:35.520)

But you still think.

Lex Fridman (1:35:36.360)

And a minute later a big hint pops on the screen.

Lex Fridman (1:35:38.560)

You still think.

Lex Fridman (1:35:39.400)

And then finally, after the three minutes,

Lex Fridman (1:35:41.280)

hopefully you got some ideas you tried to answer.

Lex Fridman (1:35:43.520)

And then suddenly there's like this

Lex Fridman (1:35:45.440)

pretty extended explanation of,

Po-Shen Loh (1:35:47.720)

oh yeah, so here's like multiple different ways

Lex Fridman (1:35:50.280)

that you can do the question.

Lex Fridman (1:35:51.600)

And by accident, you also just learned this other concept.

Lex Fridman (1:35:54.640)

That's what we did.

Lex Fridman (1:35:55.480)

So yeah.

Lex Fridman (1:35:56.320)

Is this targeted towards middle school students,

Lex Fridman (1:35:58.160)

high school students?

Lex Fridman (1:35:59.480)

It's targeted towards middle school students

Po-Shen Loh (1:36:01.360)

with competitions.

Lex Fridman (1:36:02.400)

But there's a lot of high school students

Po-Shen Loh (1:36:04.040)

who didn't do competitions in middle school

Lex Fridman (1:36:06.400)

where they would also learn how to think.

Po-Shen Loh (1:36:07.840)

If you can see the whole concept was,

Lex Fridman (1:36:09.800)

can we teach people how to think?

Lex Fridman (1:36:11.720)

How would you do that?

Lex Fridman (1:36:12.760)

You need to give people the chance to,

Po-Shen Loh (1:36:14.440)

on their own, invent without that kid in the front row

Lex Fridman (1:36:17.960)

answering every question in two seconds.

Lex Fridman (1:36:20.320)

And people can find it, I think, what daily.

Lex Fridman (1:36:24.040)

It's daily.potionlo.com.

Lex Fridman (1:36:26.080)

But if you go to find my website,

Lex Fridman (1:36:27.840)

you'll be able to find it.

Po-Shen Loh (1:36:29.200)

Beautiful.

Lex Fridman (1:36:30.600)

Can we zoom out a little bit in the,

Lex Fridman (1:36:32.880)

so day to day, week to week, month to month,

Lex Fridman (1:36:36.240)

year to year, what does the lifelong educational process

Lex Fridman (1:36:41.480)

look like, do you think?

Lex Fridman (1:36:43.880)

For yourself, but for me,

Lex Fridman (1:36:46.600)

what would you recommend in the world of mathematics

Lex Fridman (1:36:50.280)

or sort of as opposed to studying for a test,

Lex Fridman (1:36:52.600)

but just like lifelong expanding of knowledge

Lex Fridman (1:36:59.240)

in that skill for invention?

Po-Shen Loh (1:37:02.120)

I think I often articulate this as,

Lex Fridman (1:37:05.040)

can you always try to do more than you could do in the past?

Po-Shen Loh (1:37:10.040)

Yeah.

Lex Fridman (1:37:11.440)

But that comes in many ways.

Lex Fridman (1:37:13.640)

And I will say it's great

Lex Fridman (1:37:15.280)

if one wants to build that with mathematics,

Lex Fridman (1:37:17.680)

but it's also great to use that philosophy

Lex Fridman (1:37:20.000)

with all other things.

Po-Shen Loh (1:37:21.080)

In fact, if I just think of myself, I just think,

Lex Fridman (1:37:23.920)

what do I know now that I didn't know a year ago

Lex Fridman (1:37:26.480)

or a month ago or a week ago?

Lex Fridman (1:37:28.520)

And not just know,

Lex Fridman (1:37:29.400)

but what do I have the capability of doing?

Lex Fridman (1:37:32.040)

And if you just have that attitude, it brings more.

Po-Shen Loh (1:37:35.040)

See, the thing is, there's also a habit,

Lex Fridman (1:37:38.280)

like it is a skill, like I've been using Anki,

Po-Shen Loh (1:37:43.040)

it's an app for helps you memorize things.

Lex Fridman (1:37:46.640)

And I've actually, a few months ago,

Po-Shen Loh (1:37:50.720)

started doing this daily of setting aside time

Lex Fridman (1:37:54.960)

to think about an idea that's outside of my work.

Po-Shen Loh (1:37:59.960)

Like, let's say, it's all over the place, by the way,

Lex Fridman (1:38:04.680)

but let's say politics, like gun control.

Lex Fridman (1:38:08.320)

Is it good to have a lot of guns or not in society?

Lex Fridman (1:38:11.800)

And just, I've set aside time every day,

Po-Shen Loh (1:38:15.000)

I do at least 10 minutes, but I try to do 30,

Lex Fridman (1:38:17.640)

where I think about a problem.

Lex Fridman (1:38:19.000)

And I kind of outline it for myself from scratch,

Lex Fridman (1:38:20.920)

from not looking anything up,

Po-Shen Loh (1:38:22.200)

just thinking about it, using common sense.

Lex Fridman (1:38:24.960)

And I think the practice of that is really important.

Po-Shen Loh (1:38:29.120)

It's the daily routine of it, it's the discipline of it.

Lex Fridman (1:38:32.600)

It's not just that I figured something out

Po-Shen Loh (1:38:35.880)

from thinking about gun control,

Lex Fridman (1:38:38.800)

it's more that that muscle is built too,

Po-Shen Loh (1:38:43.080)

it's that thinking muscle.

Lex Fridman (1:38:44.200)

So I'm kind of interested in, you know, math has,

Po-Shen Loh (1:38:49.600)

because especially because I've gotten specialized

Lex Fridman (1:38:52.080)

into machine learning,

Lex Fridman (1:38:53.040)

and because I love programming so much,

Lex Fridman (1:38:55.360)

I've lost touch with math a little bit

Po-Shen Loh (1:38:59.760)

to where I feel quite sad about it,

Lex Fridman (1:39:02.400)

and I want to fix that.

Po-Shen Loh (1:39:05.040)

Even just not math, like pure knowledge math,

Lex Fridman (1:39:07.520)

but math, like these middle school problems,

Lex Fridman (1:39:10.200)

the challenges, right?

Lex Fridman (1:39:13.360)

Is that something you see a person

Po-Shen Loh (1:39:14.680)

be able to do every single day,

Lex Fridman (1:39:16.160)

kind of just practice every single day for years?

Lex Fridman (1:39:19.600)

So I can give an answer to that,

Lex Fridman (1:39:21.520)

that gives a practical way you could do it,

Po-Shen Loh (1:39:23.160)

assuming you have kids.

Lex Fridman (1:39:24.480)

So, no, you can do it yourself.

Po-Shen Loh (1:39:26.920)

Step one, get kids.

Lex Fridman (1:39:28.000)

No, no, I'm just saying this

Po-Shen Loh (1:39:29.560)

because I'm just thinking out loud right now,

Lex Fridman (1:39:31.720)

what could I do to suggest?

Po-Shen Loh (1:39:33.720)

Because what I have noticed is that, for example,

Lex Fridman (1:39:35.960)

if you do have kids who are in elementary school

Po-Shen Loh (1:39:37.760)

or middle school, if you yourself go and look

Lex Fridman (1:39:41.080)

at those middle school math problems

Po-Shen Loh (1:39:43.080)

to think about interesting ways

Lex Fridman (1:39:44.480)

that you can teach your elementary school

Po-Shen Loh (1:39:46.160)

or middle school kid, it works.

Lex Fridman (1:39:48.120)

That's what my wife did.

Po-Shen Loh (1:39:48.960)

She never did any of those contests before,

Lex Fridman (1:39:51.040)

but now she knows quite a lot about them.

Lex Fridman (1:39:53.000)

And I didn't teach her anything.

Lex Fridman (1:39:53.920)

I don't do that.

Po-Shen Loh (1:39:55.160)

She just was messing around with them

Lex Fridman (1:39:57.320)

and taught herself all of that stuff.

Lex Fridman (1:39:59.600)

And that had the automatic daily.

Lex Fridman (1:40:01.400)

I'm always thinking, how do you make it practical, right?

Lex Fridman (1:40:03.840)

And the way to make it practical

Lex Fridman (1:40:04.880)

is if the timer on the automatically daily

Po-Shen Loh (1:40:07.680)

is that you are going to automatically daily

Lex Fridman (1:40:09.520)

do something with your own kid.

Po-Shen Loh (1:40:11.240)

Now it feeds back.

Lex Fridman (1:40:13.360)

And that includes the whole lesson

Po-Shen Loh (1:40:14.760)

that if you wanna learn something, you should teach it.

Lex Fridman (1:40:16.960)

Oh, I strongly believe that.

Po-Shen Loh (1:40:19.160)

I strongly believe that.

Lex Fridman (1:40:21.760)

So I currently don't have kids.

Lex Fridman (1:40:23.440)

So that's, maybe I should just get kids

Lex Fridman (1:40:25.600)

to help me with the math thing.

Lex Fridman (1:40:27.040)

But outside of that,

Lex Fridman (1:40:29.320)

I do want to do great math into daily practice.

Lex Fridman (1:40:32.040)

So I'll definitely check out the daily challenges

Lex Fridman (1:40:35.920)

and see, because what is it?

Po-Shen Loh (1:40:39.200)

Grant Sanderson, we talked about offline,

Lex Fridman (1:40:41.080)

the three blue and brown.

Po-Shen Loh (1:40:42.720)

He speaks to this as well,

Lex Fridman (1:40:44.920)

that his videos aren't necessarily,

Po-Shen Loh (1:40:48.200)

they don't speak to the thing that I'm referring to,

Lex Fridman (1:40:50.520)

which is the daily practice.

Po-Shen Loh (1:40:52.640)

They're more almost tools of inspiration.

Lex Fridman (1:40:56.320)

They kind of show you the beauty

Po-Shen Loh (1:40:58.520)

of a particular problem in mathematics,

Lex Fridman (1:41:03.640)

but they're not a daily ritual.

Lex Fridman (1:41:05.760)

And I'm in search of that daily ritual mathematics.

Lex Fridman (1:41:09.560)

It's not trivial to find,

Lex Fridman (1:41:14.600)

but I hope to find that

Lex Fridman (1:41:16.880)

because I think math gives you a perspective on the world

Po-Shen Loh (1:41:20.960)

that enriches everything else.

Lex Fridman (1:41:23.960)

So I like what you said about the daily also,

Po-Shen Loh (1:41:25.800)

because that's also one reason

Lex Fridman (1:41:27.400)

why I put my Carnegie Mellon class online.

Po-Shen Loh (1:41:29.960)

It's not every day.

Lex Fridman (1:41:30.800)

It's every other day.

Po-Shen Loh (1:41:31.960)

Semester is almost over.

Lex Fridman (1:41:33.280)

But the idea was, I guess my philosophy was,

Po-Shen Loh (1:41:35.840)

if I'm already doing the class,

Lex Fridman (1:41:37.320)

let's just put it there, right?

Lex Fridman (1:41:38.880)

But I do know that there are people

Lex Fridman (1:41:40.880)

who have been following it,

Po-Shen Loh (1:41:42.280)

who are not in my class at all,

Lex Fridman (1:41:43.880)

who have just been following it because,

Po-Shen Loh (1:41:45.920)

yes, it's combinatorics.

Lex Fridman (1:41:47.560)

And the value of that is you could,

Po-Shen Loh (1:41:49.840)

you don't really need to know calculus to follow it,

Lex Fridman (1:41:51.800)

if that makes sense.

Lex Fridman (1:41:52.720)

So it's actually something that people could follow.

Lex Fridman (1:41:54.600)

So again, and that one's free.

Lex Fridman (1:41:56.000)

So that one's just there on YouTube.

Lex Fridman (1:41:58.680)

Well, speaking of combinatorics,

Lex Fridman (1:42:01.440)

what is it, what do you find interesting,

Lex Fridman (1:42:03.800)

what do you find beautiful about combinatorics?

Lex Fridman (1:42:07.240)

So combinatorics to me is the study of things

Lex Fridman (1:42:11.520)

where they might be more finite and more discreet.

Lex Fridman (1:42:17.240)

What I mean is like, if I look at a network,

Lex Fridman (1:42:18.960)

actually a lot of times the combinatorics

Po-Shen Loh (1:42:20.680)

will boil down to something,

Lex Fridman (1:42:21.880)

and the combinatorics I think about

Po-Shen Loh (1:42:23.240)

might be something related to graphs or networks.

Lex Fridman (1:42:25.960)

And they're very discreet because if you have a node,

Po-Shen Loh (1:42:28.840)

it's not that you have 0.7 of a node

Lex Fridman (1:42:32.160)

and 0.3 of a node over there.

Po-Shen Loh (1:42:33.520)

It's that you've got one node,

Lex Fridman (1:42:34.720)

and then you jump one step to go to the next node.

Lex Fridman (1:42:37.440)

So that notion is different from say, calculus,

Lex Fridman (1:42:39.880)

which is very continuous,

Po-Shen Loh (1:42:42.080)

where you go and say, I have this speed,

Lex Fridman (1:42:44.360)

which is changing over time.

Lex Fridman (1:42:46.200)

And now what's the distance I've traveled?

Lex Fridman (1:42:47.800)

That's the notion of an integral,

Po-Shen Loh (1:42:49.040)

where you have to think of subdividing time

Lex Fridman (1:42:50.920)

into very, very small pieces.

Lex Fridman (1:42:52.640)

So the kinds of things that you do

Lex Fridman (1:42:54.400)

when you reason about these finite discreet structures

Po-Shen Loh (1:42:59.360)

often might be iterative, algorithmic, inductive.

Lex Fridman (1:43:03.280)

These are ideas where I go from one step to the next step

Lex Fridman (1:43:06.320)

and so on and make progress.

Lex Fridman (1:43:08.160)

I guess I actually personally like all kinds of math.

Po-Shen Loh (1:43:11.280)

My area of research just ended up in here

Lex Fridman (1:43:13.560)

because I met a really interesting PhD advisor,

Po-Shen Loh (1:43:17.160)

potential, that's honestly the reason

Lex Fridman (1:43:19.000)

I went into that direction.

Po-Shen Loh (1:43:20.320)

I met a really interesting guy.

Lex Fridman (1:43:22.040)

He seemed like he did good stuff, interesting stuff,

Lex Fridman (1:43:24.880)

and he looked like he cared about students.

Lex Fridman (1:43:26.680)

And I said, let me just go and learn whatever you do,

Po-Shen Loh (1:43:29.240)

even though my prior practice and preparation

Lex Fridman (1:43:32.240)

before my PhD was not combinatorics,

Lex Fridman (1:43:34.280)

but analysis, the continuous stuff.

Lex Fridman (1:43:36.920)

So the annoying thing about combinatorics

Lex Fridman (1:43:40.600)

and discreet stuff is it's often really difficult to solve

Lex Fridman (1:43:45.600)

from a sort of running time complexity perspective.

Po-Shen Loh (1:43:53.840)

Could you speak to the idea of complexity analysis

Lex Fridman (1:43:59.640)

of problems, do you find it useful, do you find it interesting?

Lex Fridman (1:44:04.480)

Do you find that lens of studying the difficulty

Lex Fridman (1:44:08.240)

of how difficult the computer science problem is

Lex Fridman (1:44:12.600)

a useful lens onto the world?

Lex Fridman (1:44:15.200)

Oh, very much so.

Po-Shen Loh (1:44:16.200)

Because if you want to make something practical

Lex Fridman (1:44:20.400)

which has large numbers of people using it,

Po-Shen Loh (1:44:22.760)

the computational complexity to me is almost question one.

Lex Fridman (1:44:27.240)

And that's, again, that's at the origin

Po-Shen Loh (1:44:29.080)

of when we started doing this stuff with disease control.

Lex Fridman (1:44:31.760)

From the very beginning, the deep questions

Po-Shen Loh (1:44:33.480)

that were running through my mind were,

Lex Fridman (1:44:35.400)

would we be able to support a large population

Lex Fridman (1:44:38.480)

with only one server?

Lex Fridman (1:44:41.160)

And if the answer is no, we can't start

Po-Shen Loh (1:44:43.680)

because I don't have enough money.

Lex Fridman (1:44:48.200)

Yeah, and there the question is very much

Po-Shen Loh (1:44:51.080)

linear time versus anything slower than linear time.

Lex Fridman (1:44:58.360)

As a very specific thing, you have a bunch

Po-Shen Loh (1:45:00.280)

of really interesting papers.

Lex Fridman (1:45:01.360)

If I could ask, maybe we could pull out some cool insights

Po-Shen Loh (1:45:04.240)

at the high level.

Lex Fridman (1:45:06.160)

Can you describe the data structure of a voting tree

Lex Fridman (1:45:08.920)

and what are some interesting results on it?

Lex Fridman (1:45:11.240)

You have a paper that I noticed on it.

Po-Shen Loh (1:45:13.720)

Yeah, so this is an example of, I guess,

Lex Fridman (1:45:17.400)

how in math we might say here's an interesting

Po-Shen Loh (1:45:20.960)

kind of a question that we just can't seem

Lex Fridman (1:45:24.400)

to understand enough about.

Po-Shen Loh (1:45:25.720)

Maybe there's something else going on here.

Lex Fridman (1:45:27.560)

And the way to describe this is you could imagine

Po-Shen Loh (1:45:30.920)

trying to hold elections where if you have

Lex Fridman (1:45:33.800)

only two candidates, that's kind of easy.

Po-Shen Loh (1:45:35.920)

You just run them against each other

Lex Fridman (1:45:37.200)

and see who gets more votes.

Lex Fridman (1:45:38.600)

But as you know, once you have more candidates,

Lex Fridman (1:45:40.680)

it's very difficult to decide who wins the election.

Lex Fridman (1:45:43.120)

And there's an entire voting theory around this.

Lex Fridman (1:45:46.280)

So a theoretical question became,

Lex Fridman (1:45:49.400)

what if you made like a system of runoffs,

Lex Fridman (1:45:53.560)

like a system of head to head contests,

Po-Shen Loh (1:45:57.160)

which is structured like a tree,

Lex Fridman (1:45:58.600)

almost looking like a circuit.

Po-Shen Loh (1:46:00.200)

I'm using that way of thinking because it's sort of like

Lex Fridman (1:46:03.160)

in electrical engineering or computer science,

Po-Shen Loh (1:46:05.680)

you might imagine having a bunch of leads

Lex Fridman (1:46:08.120)

that carry signal, which are going through AND gates

Lex Fridman (1:46:10.520)

and OR gates and whatnot.

Lex Fridman (1:46:11.640)

And you've managed to compute beautiful things.

Po-Shen Loh (1:46:13.640)

This is just from a purely abstract point of view.

Lex Fridman (1:46:16.280)

What if the inputs are candidates?

Lex Fridman (1:46:18.600)

And for every two candidates, it is known

Lex Fridman (1:46:20.840)

which of the candidates is more popular than the other.

Po-Shen Loh (1:46:23.480)

Now can you build some kind of a circuit board

Lex Fridman (1:46:25.880)

which says, first candidate number four

Po-Shen Loh (1:46:28.000)

will play against five and see who wins and so on.

Lex Fridman (1:46:31.600)

Okay, so now what would be a nice outcome, right?

Po-Shen Loh (1:46:34.600)

This is a general question of,

Lex Fridman (1:46:35.840)

could I make a big circuit board to feed an election into?

Po-Shen Loh (1:46:39.120)

Like maybe one nice outcome would be whoever wins

Lex Fridman (1:46:41.320)

at least is preferred over a lot of people.

Po-Shen Loh (1:46:44.680)

Yes.

Lex Fridman (1:46:45.520)

So for example, if you ran in 1,024 candidates,

Po-Shen Loh (1:46:48.480)

ideally we would like a guarantee that says

Lex Fridman (1:46:51.080)

that the winner beats a lot of people.

Po-Shen Loh (1:46:54.080)

Actually in any system where there are 1,024 candidates,

Lex Fridman (1:46:58.520)

there's always a candidate who beats

Po-Shen Loh (1:47:00.320)

at least 512 of the others.

Lex Fridman (1:47:02.840)

This is a mathematical fact

Po-Shen Loh (1:47:04.360)

that there's actually always a person who beats

Lex Fridman (1:47:06.200)

at least half of the other people.

Po-Shen Loh (1:47:09.680)

I'm trying to make sense of that mathematical fact.

Lex Fridman (1:47:13.400)

Is this supposed to be obvious?

Po-Shen Loh (1:47:15.000)

No, but I can explain it.

Lex Fridman (1:47:17.000)

No, I can't.

Po-Shen Loh (1:47:17.840)

The way it works is that, think of it this way.

Lex Fridman (1:47:21.400)

Every time I think, imagine I have all these candidates

Lex Fridman (1:47:24.280)

and everyone is competing,

Lex Fridman (1:47:26.040)

everyone is like compared with everyone else at some point.

Po-Shen Loh (1:47:29.200)

Well, think of it this way.

Lex Fridman (1:47:30.600)

Whenever there's a comparison, somebody gets a point.

Po-Shen Loh (1:47:34.200)

That's the one who is better than the other one.

Lex Fridman (1:47:37.000)

My claim is there's somebody whose score

Po-Shen Loh (1:47:39.440)

is at least half of how many other people there are.

Lex Fridman (1:47:42.880)

Yeah, I'm just trying to,

Po-Shen Loh (1:47:44.840)

like my intuition is very close to that being true,

Lex Fridman (1:47:47.600)

but it's beautiful.

Po-Shen Loh (1:47:48.680)

I didn't at first, that's not an obvious fact.

Lex Fridman (1:47:52.240)

No, it's not.

Lex Fridman (1:47:53.080)

And it feels like a beautiful fact.

Lex Fridman (1:47:55.760)

Well, let me explain it this way.

Po-Shen Loh (1:47:57.120)

Imagine that for every match,

Lex Fridman (1:48:00.520)

you didn't give one point, but you gave two points.

Po-Shen Loh (1:48:03.840)

You gave one point to each person.

Lex Fridman (1:48:05.920)

Now that's not what we're really doing.

Po-Shen Loh (1:48:07.040)

We really want to give one point to the winner of the match,

Lex Fridman (1:48:10.840)

but instead we'll just give two.

Po-Shen Loh (1:48:12.240)

If you gave two points to everyone on every matchup,

Lex Fridman (1:48:15.880)

actually everyone has the same number of points.

Lex Fridman (1:48:18.400)

And the number of points they get

Lex Fridman (1:48:19.680)

is how many other people there are.

Lex Fridman (1:48:22.520)

Does that sort of make sense?

Lex Fridman (1:48:23.560)

I'm just like saying.

Po-Shen Loh (1:48:24.400)

No, no, everything you're saying makes perfect sense.

Lex Fridman (1:48:26.640)

So the point is if for every comparison between two people,

Po-Shen Loh (1:48:30.720)

which I'm doing for every two people,

Lex Fridman (1:48:32.480)

I gave one point to each person,

Po-Shen Loh (1:48:34.400)

your score, everyone's score is the same.

Lex Fridman (1:48:36.640)

It's how many other people there are.

Po-Shen Loh (1:48:38.560)

Now we only make one change.

Lex Fridman (1:48:40.200)

For each matchup, you give one point only to the winner.

Lex Fridman (1:48:44.320)

So we're awarding half the points.

Lex Fridman (1:48:47.040)

So now the deal is if in the original situation,

Po-Shen Loh (1:48:50.240)

everyone's score was equal,

Lex Fridman (1:48:52.120)

which is how many other people there are.

Po-Shen Loh (1:48:54.840)

Now there's only half the number of points to go around.

Lex Fridman (1:48:58.480)

So what ends up happening is that

Po-Shen Loh (1:49:01.000)

there's always going to be,

Lex Fridman (1:49:02.360)

like the average number of points per person

Po-Shen Loh (1:49:04.680)

is going to be half of how many other people there are.

Lex Fridman (1:49:07.120)

And somebody is gonna be above average.

Po-Shen Loh (1:49:08.560)

Somebody is going to be above that.

Lex Fridman (1:49:09.880)

At least average.

Po-Shen Loh (1:49:10.720)

Yeah, this is this notion of expected value,

Lex Fridman (1:49:13.200)

that if I have a random variable,

Po-Shen Loh (1:49:14.520)

which has an expected value,

Lex Fridman (1:49:16.200)

there's going to be some possibility

Po-Shen Loh (1:49:17.800)

in the probability space

Lex Fridman (1:49:19.280)

where you're at least as big as the expected value.

Po-Shen Loh (1:49:21.560)

Yeah, when you describe it like that, it's obvious.

Lex Fridman (1:49:23.680)

But when you're first saying in this little circuit

Po-Shen Loh (1:49:26.640)

that there's going to be one candidate better than half,

Lex Fridman (1:49:32.040)

that's not obvious.

Po-Shen Loh (1:49:33.360)

Yeah, it's not obvious.

Lex Fridman (1:49:34.200)

It's funny.

Po-Shen Loh (1:49:35.040)

It's not obvious.

Lex Fridman (1:49:35.880)

Math, this is nice.

Po-Shen Loh (1:49:37.160)

Okay, so you have this,

Lex Fridman (1:49:38.600)

but ultimately you're trying to with a voting tree,

Po-Shen Loh (1:49:42.800)

I don't know if you're trying this,

Lex Fridman (1:49:43.880)

but to have a circuit that's like, that's small.

Po-Shen Loh (1:49:48.360)

Well, you'd like it to be small.

Lex Fridman (1:49:49.200)

That achieves the same kind of,

Po-Shen Loh (1:49:53.960)

I mean, the smaller it is,

Lex Fridman (1:49:56.440)

if we look at practically speaking,

Po-Shen Loh (1:49:59.080)

the lower the cost of running the election,

Lex Fridman (1:50:01.560)

of running through, of computing the circuit.

Po-Shen Loh (1:50:03.800)

That is true.

Lex Fridman (1:50:04.640)

But actually at this point,

Po-Shen Loh (1:50:05.800)

the reason the question was interesting

Lex Fridman (1:50:08.600)

is because there was no good guarantee

Po-Shen Loh (1:50:12.760)

that the winner of that circuit

Lex Fridman (1:50:15.440)

would have like have beaten a lot of people.

Po-Shen Loh (1:50:18.440)

Let me give an example.

Lex Fridman (1:50:19.680)

The best known circuit,

Po-Shen Loh (1:50:20.800)

when we started thinking about this,

Lex Fridman (1:50:22.400)

was the circuit called candidate one

Po-Shen Loh (1:50:24.760)

plays against candidate two,

Lex Fridman (1:50:26.400)

candidate three plays against four,

Lex Fridman (1:50:28.640)

and then the winners play against each other.

Lex Fridman (1:50:30.480)

And then by the way, five plays against six,

Po-Shen Loh (1:50:32.560)

seven against eight, the winners play against each other.

Lex Fridman (1:50:34.640)

You understand, it's like a giant binary tree.

Po-Shen Loh (1:50:36.600)

Yeah, it's a binary, like a balanced binary tree.

Lex Fridman (1:50:39.400)

It's a balanced binary tree.

Po-Shen Loh (1:50:40.720)

One, two, three, four, up to 1,024,

Lex Fridman (1:50:42.680)

everyone going up to find the winner.

Lex Fridman (1:50:44.520)

Well, you know what?

Lex Fridman (1:50:45.480)

There's a system in the world

Po-Shen Loh (1:50:47.400)

where it could just be

Lex Fridman (1:50:49.720)

that there's a candidate called number one,

Po-Shen Loh (1:50:52.280)

that just beats like 10 other people,

Lex Fridman (1:50:56.200)

just the 10 that they need to be on their way up

Lex Fridman (1:50:59.760)

and they lose to everyone else.

Lex Fridman (1:51:02.240)

But somehow they would get all the way up.

Po-Shen Loh (1:51:04.720)

My point is it is possible to outsmart that circuit

Lex Fridman (1:51:11.040)

in one weird way of the world,

Po-Shen Loh (1:51:13.720)

which makes that circuit a bad one

Lex Fridman (1:51:15.320)

because you want to say,

Po-Shen Loh (1:51:16.160)

I will use this circuit for all elections.

Lex Fridman (1:51:18.920)

And you might have a system of inputs that go in there

Po-Shen Loh (1:51:22.480)

where the winner only beat 10 other people,

Lex Fridman (1:51:24.800)

which is the people they had to beat on the way up.

Lex Fridman (1:51:26.720)

So you want to have a circuit where there's as many,

Lex Fridman (1:51:29.720)

like the final result is as strong as possible.

Po-Shen Loh (1:51:33.160)

Yes.

Lex Fridman (1:51:34.200)

And so what ideas do you have for that?

Lex Fridman (1:51:37.440)

So we actually only managed to improve it

Lex Fridman (1:51:40.480)

to square root of N.

Lex Fridman (1:51:41.800)

So if N is number of vertices,

Lex Fridman (1:51:43.680)

N over two would be the ideal.

Po-Shen Loh (1:51:46.080)

We got it to square root of N.

Lex Fridman (1:51:48.320)

Versus log base two.

Po-Shen Loh (1:51:50.120)

Yeah, exactly.

Lex Fridman (1:51:51.080)

Yeah.

Po-Shen Loh (1:51:52.400)

Which is...

Lex Fridman (1:51:53.440)

Well, that is halfway.

Po-Shen Loh (1:51:54.280)

It could be a lot.

Lex Fridman (1:51:55.920)

Yeah.

Po-Shen Loh (1:51:56.760)

Could be a big improvement.

Lex Fridman (1:51:57.760)

So that's a, okay, cool.

Po-Shen Loh (1:51:59.000)

Is there something you can say with words

Lex Fridman (1:52:01.600)

about what kind of circuit, what that looks like?

Po-Shen Loh (1:52:04.600)

I can give an idea of one of the tools inside,

Lex Fridman (1:52:08.040)

but the actual execution ends up being more complicated.

Lex Fridman (1:52:10.560)

But one of the widgets inside this

Lex Fridman (1:52:12.800)

is building a system where you have like a candidate

Po-Shen Loh (1:52:16.960)

who plays, like one part of the whole huge, huge tree

Lex Fridman (1:52:20.720)

is that that same candidate, let's call them seven.

Po-Shen Loh (1:52:23.240)

Seven plays against somebody,

Lex Fridman (1:52:25.520)

let's make up some numbers.

Po-Shen Loh (1:52:26.680)

Let's call the others like letters.

Lex Fridman (1:52:27.920)

So seven plays against A.

Po-Shen Loh (1:52:30.720)

Seven's also gonna play against B separately.

Lex Fridman (1:52:33.680)

And the winners of each of those will play each other.

Po-Shen Loh (1:52:36.560)

By the way, seven's also gonna play C.

Lex Fridman (1:52:38.440)

Seven's gonna play D.

Lex Fridman (1:52:39.680)

And the winners are gonna play each other.

Lex Fridman (1:52:41.040)

And the winners are gonna play each other.

Po-Shen Loh (1:52:42.520)

We call this seven against all.

Lex Fridman (1:52:44.960)

Well, seven against like everyone from a bunch of.

Po-Shen Loh (1:52:47.720)

Got it.

Lex Fridman (1:52:48.560)

So there's some nice overlap between the matchups

Po-Shen Loh (1:52:50.840)

that somehow has a nice feature to it.

Lex Fridman (1:52:53.000)

Yes, and I can tell you the nice feature

Po-Shen Loh (1:52:54.200)

because if at the base of this giant tree,

Lex Fridman (1:52:56.480)

at the base of this giant circuit,

Po-Shen Loh (1:52:57.880)

like this is a widget.

Lex Fridman (1:52:58.720)

We build the things out of widgets.

Lex Fridman (1:52:59.920)

So I'm just describing one widget.

Lex Fridman (1:53:01.320)

But in the base of this widget,

Po-Shen Loh (1:53:03.400)

you have lots of things which are seven against someone,

Lex Fridman (1:53:05.560)

seven against someone, seven against someone.

Po-Shen Loh (1:53:07.360)

In fact, every matchup at the bottom

Lex Fridman (1:53:09.800)

is seven against someone.

Lex Fridman (1:53:11.520)

What that means is

Lex Fridman (1:53:12.840)

if seven actually beat everyone they were matched up against,

Po-Shen Loh (1:53:16.840)

well, seven would rise to the top.

Lex Fridman (1:53:18.640)

So one possibility is if you see a seven

Po-Shen Loh (1:53:21.240)

emerge from the top,

Lex Fridman (1:53:22.360)

you know that seven actually beat everyone

Po-Shen Loh (1:53:24.280)

they were against.

Lex Fridman (1:53:25.600)

On the other hand, if anyone else is on top,

Po-Shen Loh (1:53:28.360)

let's call it F.

Lex Fridman (1:53:29.600)

If F is on top, how did F get there?

Po-Shen Loh (1:53:31.920)

Well, F beat seven on the way at the beginning.

Lex Fridman (1:53:34.640)

So the point is the outcome of this circuit

Po-Shen Loh (1:53:37.000)

has a certain property.

Lex Fridman (1:53:38.400)

If you see a seven,

Po-Shen Loh (1:53:39.640)

you know that the seven actually beat a person

Lex Fridman (1:53:41.680)

but the seven actually beat a bazillion people.

Po-Shen Loh (1:53:43.960)

If you see anyone else,

Lex Fridman (1:53:45.240)

at least you know they beat seven.

Po-Shen Loh (1:53:47.040)

Yeah, then you can prove that it has a nice property.

Lex Fridman (1:53:49.880)

That's really interesting.

Po-Shen Loh (1:53:50.880)

Is there something you can say,

Lex Fridman (1:53:54.120)

perhaps going completely outside

Po-Shen Loh (1:53:55.760)

of what we're talking about,

Lex Fridman (1:53:56.720)

is how we may

Po-Shen Loh (1:54:00.720)

have mathematical ideas

Lex Fridman (1:54:03.040)

of improving the electoral process?

Po-Shen Loh (1:54:06.440)

That one, no.

Lex Fridman (1:54:07.320)

No, I can't give you that one.

Po-Shen Loh (1:54:09.120)

I mean, is there, like, do you ever see it as,

Lex Fridman (1:54:14.640)

do you see as there being a lot of opportunities

Lex Fridman (1:54:17.400)

for improving how we vote?

Lex Fridman (1:54:20.480)

Like from your, I don't know if you saw parallels,

Po-Shen Loh (1:54:23.720)

but, you know, it seems like if,

Lex Fridman (1:54:26.560)

this actually kind of maps to your sort of COVID work,

Lex Fridman (1:54:29.680)

which is there's a network effect, right?

Lex Fridman (1:54:32.240)

It seems like we should be able to apply similar kind

Po-Shen Loh (1:54:34.920)

of effects of how we decide other things in our lives.

Lex Fridman (1:54:39.400)

And one of the big decisions we'll make

Po-Shen Loh (1:54:42.160)

is who represents us in government.

Lex Fridman (1:54:44.480)

Do you ever think about like mathematically

Lex Fridman (1:54:46.200)

about those kinds of systems?

Lex Fridman (1:54:48.160)

I think a little bit about those,

Po-Shen Loh (1:54:49.520)

because where I went to college,

Lex Fridman (1:54:51.480)

the way we voted for student government

Lex Fridman (1:54:53.200)

was based on this, is it called ranked choice?

Lex Fridman (1:54:56.120)

Where you eliminate the bottom

Lex Fridman (1:54:58.600)

and there was runoff elections.

Lex Fridman (1:55:00.840)

So that was the first time I ever saw that.

Lex Fridman (1:55:02.840)

And I thought that made sense.

Lex Fridman (1:55:04.480)

The only problem is it doesn't seem so easy

Po-Shen Loh (1:55:06.920)

to get something that makes sense adopted

Lex Fridman (1:55:08.520)

as the new voting system.

Po-Shen Loh (1:55:09.760)

That's a whole nother, that's not a math solution.

Lex Fridman (1:55:12.760)

That's a, well, it's math in the sense that it's game theory.

Lex Fridman (1:55:16.040)

So you have to come up with incentive,

Lex Fridman (1:55:17.280)

it's mechanism design.

Po-Shen Loh (1:55:18.320)

You have to figure out how to trick us

Lex Fridman (1:55:21.040)

despite our basic human nature

Po-Shen Loh (1:55:24.280)

to adopt solutions that are better.

Lex Fridman (1:55:27.600)

That's a whole nother conversation, I think.

Lex Fridman (1:55:30.520)

Can you just, cause it sounded really cool,

Lex Fridman (1:55:33.240)

talk a little bit about stochastic coalescence

Lex Fridman (1:55:36.120)

and you have a paper on showing that,

Lex Fridman (1:55:39.400)

so you could describe what it is,

Lex Fridman (1:55:40.760)

but I guess it's a super linear, super logarithmic time

Lex Fridman (1:55:44.760)

and you came up with some kind of trick

Po-Shen Loh (1:55:46.360)

that make it faster.

Lex Fridman (1:55:47.960)

Can you just talk about it a little bit?

Po-Shen Loh (1:55:49.280)

Yeah, so this was something which came up

Lex Fridman (1:55:51.680)

when I was at Microsoft Research for a summer.

Lex Fridman (1:55:54.160)

And I'm putting that context because that shows

Lex Fridman (1:55:56.680)

that it has some practical motivation at some point.

Po-Shen Loh (1:56:01.040)

Actually, I think it's still.

Lex Fridman (1:56:01.920)

It doesn't need to.

Po-Shen Loh (1:56:02.840)

It doesn't need to.

Lex Fridman (1:56:03.680)

It can be beautiful and it's all right.

Po-Shen Loh (1:56:05.160)

Yeah, so the easiest way to describe this is

Lex Fridman (1:56:07.360)

suppose you got like a big crowd of people

Lex Fridman (1:56:09.920)

and everybody knows how many hours of sleep

Lex Fridman (1:56:12.120)

they got last night.

Lex Fridman (1:56:13.240)

And you wanna know how many total hours of sleep

Lex Fridman (1:56:15.240)

were gotten by this big crowd of people.

Po-Shen Loh (1:56:17.560)

At the beginning, you might say,

Lex Fridman (1:56:18.760)

that sounds like a linear time algorithm

Lex Fridman (1:56:20.680)

of saying, hey, how many hours you got?

Lex Fridman (1:56:22.680)

How many you got?

Lex Fridman (1:56:23.520)

How many you got?

Lex Fridman (1:56:24.360)

Add, add, add.

Lex Fridman (1:56:25.560)

But there's a way to do this

Lex Fridman (1:56:26.760)

if you remember that there are people

Lex Fridman (1:56:28.520)

and they presumably know how to add.

Lex Fridman (1:56:30.360)

You could make a distributed algorithm

Po-Shen Loh (1:56:32.160)

to make this happen.

Lex Fridman (1:56:33.400)

For example, while we're thinking of these trees,

Po-Shen Loh (1:56:35.840)

imagine you had 1,024 people.

Lex Fridman (1:56:38.560)

If you could just say, hey, person number one

Lex Fridman (1:56:40.720)

and person number two, you will add your hours of sleep.

Lex Fridman (1:56:44.920)

Person number two will go away

Lex Fridman (1:56:46.040)

and person number one is gonna remember the sum.

Lex Fridman (1:56:48.360)

Person three and four add up

Lex Fridman (1:56:50.920)

and person three takes charge of remembering it.

Lex Fridman (1:56:53.760)

Person four goes away.

Po-Shen Loh (1:56:54.840)

Now this like person one knows the sum of these two.

Lex Fridman (1:56:56.960)

Person three knows the sum of those two.

Po-Shen Loh (1:56:58.160)

They talk.

Lex Fridman (1:56:59.000)

You see what I mean?

Po-Shen Loh (1:56:59.840)

You're going up this tree,

Lex Fridman (1:57:02.200)

same tree that we talked about earlier.

Po-Shen Loh (1:57:03.600)

Built up a tree from the bottom up.

Lex Fridman (1:57:05.920)

Yeah, build up a tree from the bottom up.

Lex Fridman (1:57:07.560)

And the beautiful thing is

Lex Fridman (1:57:09.160)

since everyone's doing stuff in parallel,

Po-Shen Loh (1:57:11.360)

the amount of time it takes to get the total sum

Lex Fridman (1:57:14.560)

is actually just the number of layers in the tree,

Po-Shen Loh (1:57:17.360)

which is 10.

Lex Fridman (1:57:18.960)

So now that's logarithmic time

Po-Shen Loh (1:57:20.360)

to add up the number of hours that people slept today.

Lex Fridman (1:57:23.720)

Sounds fantastic.

Po-Shen Loh (1:57:25.240)

There's only one problem.

Lex Fridman (1:57:26.400)

How do you decide who's person number one

Lex Fridman (1:57:27.960)

and person number two?

Lex Fridman (1:57:29.720)

Yes.

Lex Fridman (1:57:30.560)

So if, for example, you just went out into the downtown

Lex Fridman (1:57:32.680)

and said, hey, get these thousand people, go.

Po-Shen Loh (1:57:34.760)

Well, if you're gonna go and say,

Lex Fridman (1:57:35.960)

and by the way, you're one and you're two and you're three,

Po-Shen Loh (1:57:37.720)

that's linear time.

Lex Fridman (1:57:38.760)

Yes.

Po-Shen Loh (1:57:39.640)

That's cheating.

Lex Fridman (1:57:40.480)

So now the question is how to do this

Po-Shen Loh (1:57:41.800)

in a distributed way.

Lex Fridman (1:57:43.000)

And there were some people who proposed

Po-Shen Loh (1:57:44.840)

a very elegant algorithm and they wanted to analyze it.

Lex Fridman (1:57:48.880)

So I came in onto the analyze side,

Lex Fridman (1:57:50.680)

but the elegant algorithm was like this.

Lex Fridman (1:57:52.800)

It was like, well, we don't actually know

Lex Fridman (1:57:55.760)

what this big tree is.

Lex Fridman (1:57:57.640)

There isn't any big tree.

Lex Fridman (1:57:58.880)

So what's gonna happen is first,

Lex Fridman (1:58:01.040)

everyone is going to decide right now.

Po-Shen Loh (1:58:04.280)

Oh, one important thing.

Lex Fridman (1:58:05.680)

Everyone is going to,

Po-Shen Loh (1:58:07.000)

at the very beginning of the whole game,

Lex Fridman (1:58:09.960)

they will have delegated responsibility to themselves

Po-Shen Loh (1:58:13.400)

as the one who knows the sum so far.

Lex Fridman (1:58:16.400)

So the point is there's gonna be,

Po-Shen Loh (1:58:18.920)

people are all gonna have like a pointer which says,

Lex Fridman (1:58:22.120)

you are the one who knows my,

Po-Shen Loh (1:58:24.440)

you've taken care of my ticket, my number.

Lex Fridman (1:58:26.680)

Yeah.

Po-Shen Loh (1:58:27.520)

You're the representative for this particular piece

Lex Fridman (1:58:30.560)

of knowledge.

Lex Fridman (1:58:31.400)

And at the very beginning, you're your own representative.

Lex Fridman (1:58:33.680)

The thing has to start simple, right?

Lex Fridman (1:58:35.120)

So at the beginning, you're your own representative.

Lex Fridman (1:58:36.440)

You're pointing to yourself, got it.

Po-Shen Loh (1:58:38.040)

Yup, yup.

Lex Fridman (1:58:38.880)

And now the way this works is that at every time step,

Po-Shen Loh (1:58:41.560)

someone blares a ding dong on the town clock or whatever.

Lex Fridman (1:58:45.840)

And each person flips a coin themselves to decide,

Po-Shen Loh (1:58:48.680)

am I going to hunt for somebody to give my number to

Lex Fridman (1:58:53.600)

and let them represent me?

Lex Fridman (1:58:55.120)

Or am I going to sit here and wait for someone to come?

Lex Fridman (1:58:58.920)

Okay.

Po-Shen Loh (1:58:59.960)

Okay.

Lex Fridman (1:59:00.800)

Well, they flipped their coin.

Po-Shen Loh (1:59:02.600)

Some of the people start asking other people saying,

Lex Fridman (1:59:04.840)

hey, I would like you to be my representative.

Po-Shen Loh (1:59:08.560)

Here is my number.

Lex Fridman (1:59:10.200)

But the problem is that there's limited bandwidth

Po-Shen Loh (1:59:12.040)

of the people who are getting asked.

Lex Fridman (1:59:13.240)

It's like, you can't get,

Po-Shen Loh (1:59:14.440)

you can't go out to prom with five people.

Lex Fridman (1:59:16.680)

But this is not what we're doing.

Lex Fridman (1:59:17.840)

We're adding numbers, okay?

Lex Fridman (1:59:19.160)

But you can only add one number.

Lex Fridman (1:59:20.720)

So the person who has suddenly gotten asked

Lex Fridman (1:59:22.600)

by all these people,

Po-Shen Loh (1:59:23.920)

well, they'll have to decide who they're going

Lex Fridman (1:59:25.720)

to take it from.

Lex Fridman (1:59:27.280)

And they randomly just choose one.

Lex Fridman (1:59:29.440)

When they randomly choose one,

Po-Shen Loh (1:59:30.720)

all the others are rejected

Lex Fridman (1:59:31.880)

and they don't get to delegate anything in that round.

Lex Fridman (1:59:34.920)

But now if this person has absorbed this one who said,

Lex Fridman (1:59:38.360)

okay, here, you take charge of my number.

Po-Shen Loh (1:59:40.680)

This person now updates their pointer.

Lex Fridman (1:59:42.600)

You're in charge.

Lex Fridman (1:59:44.840)

And this person adds the two numbers.

Lex Fridman (1:59:47.680)

That was the first round.

Po-Shen Loh (1:59:50.000)

In the next round, when they do the coin flipping,

Lex Fridman (1:59:52.840)

this person doesn't flip anymore

Po-Shen Loh (1:59:54.280)

because they're just delegating.

Lex Fridman (1:59:56.120)

It's that anyone who has the pointers themselves,

Po-Shen Loh (1:59:59.000)

that's like a person who is in charge

Lex Fridman (20:03.600)

there was a pandemic.

Lex Fridman (20:05.320)

What would you want?

Lex Fridman (20:08.000)

You would want a way to be able to live your life

Po-Shen Loh (20:10.320)

as much as possible and avoid getting sick.

Lex Fridman (20:13.680)

Can we make an app to help you avoid getting sick?

Po-Shen Loh (20:17.400)

Notice how I've just articulated the problem.

Lex Fridman (20:19.680)

It is not, can we make an app

Lex Fridman (20:21.840)

so that after you are around somebody who's sick

Lex Fridman (20:24.720)

you can be removed from society.

Po-Shen Loh (20:27.080)

It's can we make an app so that you can avoid getting sick.

Lex Fridman (20:30.520)

That would run a positive feed.

Po-Shen Loh (20:33.520)

I don't know if I want to call it positive or negative

Lex Fridman (20:35.320)

but it would run a good feedback loop.

Po-Shen Loh (20:36.840)

Okay.

Lex Fridman (20:37.680)

So then how would you do this?

Po-Shen Loh (20:38.840)

The only problem is that you don't know who's sick

Lex Fridman (20:41.480)

because especially with this disease

Po-Shen Loh (20:44.120)

if I see somebody who looks perfectly healthy

Lex Fridman (20:46.960)

the disease spreads two days before you have any symptoms.

Lex Fridman (20:50.000)

And so it's actually not possible.

Lex Fridman (20:52.360)

That's where the network theory comes in.

Po-Shen Loh (20:54.760)

You caught it from someone.

Lex Fridman (20:56.560)

What if we changed the paradigm

Lex Fridman (20:59.560)

and we said, whenever there's a sickness

Lex Fridman (21:02.440)

tell everybody how many physical relationships

Po-Shen Loh (21:06.200)

separate them from the sickness.

Lex Fridman (21:08.000)

That is the trivial idea we added.

Po-Shen Loh (21:09.840)

The trivial idea was the distance between you and a disease

Lex Fridman (21:13.360)

is not measured in feet or seconds.

Po-Shen Loh (21:16.600)

It's measured in terms of how many

Lex Fridman (21:19.040)

close physical relationships separate you

Po-Shen Loh (21:22.040)

like these six degrees of separation like LinkedIn.

Lex Fridman (21:25.400)

Simple idea.

Lex Fridman (21:26.480)

What if we told everyone that?

Lex Fridman (21:28.080)

It turns out that actually unlocks

Po-Shen Loh (21:30.200)

some interesting behavioral feedback loops

Lex Fridman (21:33.040)

which for example, let me now jump to a non COVID example

Po-Shen Loh (21:37.040)

to show why this maybe could be useful.

Lex Fridman (21:39.240)

Actually we think it could be quite useful.

Po-Shen Loh (21:40.920)

Imagine there was Ebola or some hemorrhagic fever.

Lex Fridman (21:44.080)

Imagine it spread through contact through the air.

Po-Shen Loh (21:46.400)

In fact, pretend, pretend.

Lex Fridman (21:50.520)

That's a disastrous disease.

Po-Shen Loh (21:52.800)

It has high fatality rate.

Lex Fridman (21:54.720)

And as you die, you're bleeding out of every orifice.

Po-Shen Loh (22:00.640)

Okay.

Lex Fridman (22:01.480)

So.

Po-Shen Loh (22:02.320)

Yeah, not pleasant.

Lex Fridman (22:03.160)

Not pleasant.

Lex Fridman (22:04.000)

So the question is, suppose that such a disease broke

Lex Fridman (22:07.680)

who would want to install an app that would tell them

Lex Fridman (22:10.440)

how many relationships away from them

Lex Fridman (22:12.640)

this disease had struck?

Po-Shen Loh (22:14.240)

Like a lot of people.

Lex Fridman (22:15.600)

A lot of people.

Po-Shen Loh (22:16.440)

In fact, almost, I don't want to say almost everyone.

Lex Fridman (22:20.160)

That's a very strong statement

Lex Fridman (22:21.000)

but a very large number of people.

Lex Fridman (22:22.840)

That's fascinating framing.

Po-Shen Loh (22:24.200)

Like the more deadly and transmissible the disease

Lex Fridman (22:28.040)

the stronger the incentive to install it in a positive sense

Po-Shen Loh (22:32.080)

the, in the good feedback loop sense.

Lex Fridman (22:36.000)

That's a really good example.

Po-Shen Loh (22:37.200)

It's a really good way to frame it.

Lex Fridman (22:38.520)

Cause with COVID, it was not as deadly

Po-Shen Loh (22:42.240)

as potential pandemics could have been

Lex Fridman (22:45.440)

viruses could have been.

Lex Fridman (22:46.280)

So it's sometimes muddled with how we think about it

Lex Fridman (22:49.160)

but yeah, this is a really good framing.

Po-Shen Loh (22:51.040)

If the virus was a lot more deadly

Lex Fridman (22:53.720)

you want to create a system that has a set of incentives

Po-Shen Loh (22:56.280)

that it quickly spreads to the population

Lex Fridman (22:59.400)

where everybody is using it

Lex Fridman (23:00.960)

and it's contributing in a positive way to the system.

Lex Fridman (23:04.200)

Exactly.

Lex Fridman (23:05.040)

And actually that point you just made

Lex Fridman (23:06.080)

I don't take credit for that observation.

Po-Shen Loh (23:07.760)

There was another person I talked to

Lex Fridman (23:09.000)

who pointed out that it's very interesting

Po-Shen Loh (23:11.360)

that this feedback loop is even more effective

Lex Fridman (23:14.600)

when the disease is worse.

Lex Fridman (23:16.960)

And that's actually not a bad characteristic to have

Lex Fridman (23:19.720)

in your feedback loop

Po-Shen Loh (23:20.640)

if you're trying to help civilization keep running.

Lex Fridman (23:24.320)

Yeah, it's a really, it's in this dynamic

Po-Shen Loh (23:27.320)

like people figure out, they dynamically figure out

Lex Fridman (23:30.440)

how bad the disease is.

Po-Shen Loh (23:31.920)

The more it spreads and the deadlier it is

Lex Fridman (23:35.080)

as the people observe it

Po-Shen Loh (23:37.200)

as long as the spread of information

Lex Fridman (23:39.640)

like semantic information, natural language information

Po-Shen Loh (23:43.440)

is closely aligned with the reality of the disease

Lex Fridman (23:46.040)

which is a whole nother conversation, right?

Po-Shen Loh (23:48.120)

We, that's, we might, maybe we'll chat about that

Lex Fridman (23:51.000)

how we sort of make sure there's not misinformation

Po-Shen Loh (23:53.800)

while there's accurate information

Lex Fridman (23:54.960)

but that aside, okay, so this is a really nice property.

Po-Shen Loh (23:58.840)

Right, and just going on on that

Lex Fridman (24:00.880)

actually just talking more about what that could do

Lex Fridman (24:02.840)

and why we're so excited about it.

Lex Fridman (24:04.440)

It's that not only would people want to install it

Lex Fridman (24:07.320)

but what would they do if you start to see

Lex Fridman (24:10.800)

that this disease is getting closer and closer?

Po-Shen Loh (24:13.200)

We surveyed informally people

Lex Fridman (24:15.080)

but they said, as we saw it getting closer, we would hide.

Po-Shen Loh (24:18.600)

We would try to not have contacts.

Lex Fridman (24:21.800)

But now you notice what this has just achieved.

Po-Shen Loh (24:24.080)

The whole goal on this whole exercise was

Lex Fridman (24:27.400)

you got the people who might be sick

Lex Fridman (24:29.720)

and you got everyone else, set A and set B.

Lex Fridman (24:32.000)

Set A is the people who might be sick,

Po-Shen Loh (24:33.360)

set B is everyone else.

Lex Fridman (24:34.840)

And for the entirety of the past

Po-Shen Loh (24:37.680)

contact tracing approaches, you tried to get set A

Lex Fridman (24:41.680)

to do things that might not be to their liking or their will

Po-Shen Loh (24:44.920)

because that's removing them from society.

Lex Fridman (24:47.680)

We found out that there's two ways

Po-Shen Loh (24:49.040)

to separate set A from set B.

Lex Fridman (24:51.120)

You can also let the people at set B

Po-Shen Loh (24:53.400)

at the fringe of set A

Lex Fridman (24:55.720)

attempt to remove themselves from this interface.

Po-Shen Loh (24:58.920)

It's the symmetry of A and B separation.

Lex Fridman (25:01.560)

Everyone was looking at A, we look at B

Lex Fridman (25:04.520)

and suddenly B is in their incentive to do so.

Lex Fridman (25:07.960)

Beautiful.

Lex Fridman (25:08.960)

So there's a virus that jumps from human to human.

Lex Fridman (25:11.960)

So there's a network sometimes called graph

Po-Shen Loh (25:16.240)

of the spread of a virus.

Lex Fridman (25:18.400)

It hops from person to person to person to person.

Lex Fridman (25:21.640)

And each one of us individuals are sitting

Lex Fridman (25:25.760)

or plopped into that network.

Po-Shen Loh (25:28.200)

We have close friends and relations and so on.

Lex Fridman (25:31.680)

It's kind of fascinating

Po-Shen Loh (25:32.520)

to actually think about this network

Lex Fridman (25:33.880)

and we can maybe talk about the shapes

Po-Shen Loh (25:35.520)

of this kind of network.

Lex Fridman (25:37.680)

Because I was trying to think exactly this,

Po-Shen Loh (25:39.920)

like how many people do I,

Lex Fridman (25:41.400)

well, I'm kind of an introvert, not kind of,

Po-Shen Loh (25:43.480)

I'm very much an introvert.

Lex Fridman (25:45.160)

But so can I be explicit about the kind of people

Lex Fridman (25:48.200)

I meet in regular life?

Lex Fridman (25:50.360)

Say when it was completely opened up, there's no pandemic.

Po-Shen Loh (25:54.840)

There is a kind of network and there's maybe

Lex Fridman (25:59.440)

in the graph theoretic sense, there's some weights

Po-Shen Loh (26:02.280)

or something about how close that relationship is

Lex Fridman (26:06.920)

in terms of the frequency of visits,

Po-Shen Loh (26:08.800)

the duration of visits and all of those kinds of things.

Lex Fridman (26:11.560)

So you're saying we might want to be,

Po-Shen Loh (26:14.840)

to create on top of that network,

Lex Fridman (26:18.040)

a spread of information to let you know

Po-Shen Loh (26:22.240)

as the virus travels through this network,

Lex Fridman (26:24.400)

how close is it getting to you?

Lex Fridman (26:26.240)

And the number of hops away it is on that network

Lex Fridman (26:29.240)

is really powerful information

Po-Shen Loh (26:31.320)

that creates a positive feedback loop

Lex Fridman (26:33.960)

where you can act essentially anonymously

Lex Fridman (26:39.080)

and on your own.

Lex Fridman (26:41.200)

Like nobody's telling you what to do,

Po-Shen Loh (26:43.960)

which is really important, is decentralized

Lex Fridman (26:46.920)

and not whatever the opposite of authoritarian is.

Lex Fridman (26:52.200)

But you get to sort of the American way.

Lex Fridman (26:54.640)

You get to choose to do it yourself.

Po-Shen Loh (26:56.320)

You have the freedom to do it yourself

Lex Fridman (26:58.440)

and you're incentivized to do it.

Lex Fridman (27:00.160)

And you're most likely going to do it

Lex Fridman (27:01.920)

to protect yourself against you getting the disease

Po-Shen Loh (27:08.400)

as the closer it gets to you

Lex Fridman (27:10.000)

based on the information that you have.

Lex Fridman (27:12.000)

But can you maybe elaborate, first of all, brilliant.

Lex Fridman (27:17.880)

Whenever I saw the thing you're working on,

Lex Fridman (27:20.320)

so forget for COVID, this is of course,

Lex Fridman (27:23.360)

really relevant for COVID, but it's also probably relevant

Po-Shen Loh (27:26.800)

for future diseases as well.

Lex Fridman (27:28.160)

So that was the thing I'm nervous about.

Po-Shen Loh (27:30.400)

I was like, if this whole,

Lex Fridman (27:31.840)

if our society shut down because of COVID,

Po-Shen Loh (27:34.720)

like what the heck is gonna happen

Lex Fridman (27:38.040)

when there's a much deadlier disease?

Po-Shen Loh (27:40.480)

Like this, this is disappointing.

Lex Fridman (27:41.920)

The whole time, 2020, the whole time

Po-Shen Loh (27:44.760)

I'm just sitting like this,

Lex Fridman (27:45.960)

like is the incompetence of everybody

Po-Shen Loh (27:49.840)

except the people developing vaccines.

Lex Fridman (27:53.000)

The biologists are the only ones

Po-Shen Loh (27:54.400)

that got their stuff together.

Lex Fridman (27:56.040)

But in terms of institutions and all that kind of stuff,

Po-Shen Loh (27:58.560)

it's just been terrible.

Lex Fridman (28:00.720)

But this is exactly the power of information

Lex Fridman (28:04.080)

and the power of information

Lex Fridman (28:05.960)

that doesn't limit personal freedom.

Lex Fridman (28:08.080)

So your idea is brilliant.

Lex Fridman (28:09.400)

Okay, mathematically, can you maybe elaborate

Lex Fridman (28:12.680)

what are we talking about?

Lex Fridman (28:14.080)

Like how do you actually make that work?

Lex Fridman (28:16.480)

What's involved?

Lex Fridman (28:17.680)

Sure, first I'm gonna reply to something you said

Po-Shen Loh (28:19.720)

about the freedom inside this,

Lex Fridman (28:22.400)

because actually that was the idea.

Lex Fridman (28:24.480)

The idea is this is game theory, right?

Lex Fridman (28:27.320)

And effectively what we did is analogous

Po-Shen Loh (28:29.600)

to free market economy, as opposed to central planning.

Lex Fridman (28:34.840)

If you just line up the set of incentives correctly

Lex Fridman (28:38.120)

so that people have in their purely selfish behavior

Lex Fridman (28:43.240)

are contributing to the optimization of the global function,

Po-Shen Loh (28:47.120)

that's it.

Lex Fridman (28:47.960)

And the point of what we do, I guess in mathematics

Po-Shen Loh (28:50.400)

is we try to explore the search space

Lex Fridman (28:52.800)

to go and find out as many possibilities as there are.

Lex Fridman (28:54.960)

And in this case, it's an applied search space.

Lex Fridman (28:58.080)

That's why the inputs from design,

Po-Shen Loh (29:00.120)

user experience design and actual people are important.

Lex Fridman (29:02.720)

But you asked about, I guess, the mathematical

Po-Shen Loh (29:05.760)

or the technical things underpinning it.

Lex Fridman (29:07.640)

So I think the first thing I'll say is

Po-Shen Loh (29:09.800)

we wanted to make this thing

Lex Fridman (29:12.600)

not require your personal information.

Lex Fridman (29:14.760)

And so in order to do that,

Lex Fridman (29:16.360)

what gave me the confidence to, I guess,

Po-Shen Loh (29:18.800)

lead our team to run at the beginning

Lex Fridman (29:20.560)

is we saw that this could be done without using GPS information.

Lex Fridman (29:24.960)

So technically what's going on is if two smartphones,

Lex Fridman (29:28.160)

it's a smartphone app.

Po-Shen Loh (29:29.040)

If two smartphones have this thing installed,

Lex Fridman (29:31.480)

they just communicate with each other by Bluetooth

Po-Shen Loh (29:35.440)

to go and find out how far,

Lex Fridman (29:38.200)

they can detect nearby things by Bluetooth.

Lex Fridman (29:40.080)

And then they can find out that these two phones

Lex Fridman (29:42.040)

were approximately such and such distance apart.

Lex Fridman (29:44.880)

And that kind of relative proximity information

Lex Fridman (29:47.680)

is enough to construct this big network.

Po-Shen Loh (29:50.640)

Okay, so the physical network is constructed

Lex Fridman (29:53.160)

based on proximity that's through Bluetooth

Lex Fridman (29:56.040)

and you don't have to specify your exact location,

Lex Fridman (29:59.320)

it's the proximity.

Po-Shen Loh (2:00:01.600)

of some number of informations,

Lex Fridman (2:00:03.320)

they flip the coin to decide,

Lex Fridman (2:00:04.680)

should I find other people who are agents?

Lex Fridman (2:00:08.240)

Or should I wait for people to ask me?

Po-Shen Loh (2:00:10.040)

Yes.

Lex Fridman (2:00:10.880)

Brilliant.

Po-Shen Loh (2:00:11.720)

This is somebody else's idea.

Lex Fridman (2:00:12.840)

And then now the idea is, okay,

Po-Shen Loh (2:00:14.320)

if you just keep doing this process,

Lex Fridman (2:00:15.720)

what ends up happening?

Po-Shen Loh (2:00:16.920)

Oh yeah, and also by the way,

Lex Fridman (2:00:18.680)

if you decide that you want to go reach out

Po-Shen Loh (2:00:20.320)

to other people, here's the catch.

Lex Fridman (2:00:23.120)

When you're one of these agents saying,

Po-Shen Loh (2:00:24.680)

okay, I'm going to go look for someone.

Lex Fridman (2:00:27.400)

You have no idea who in this crowd is an agent

Po-Shen Loh (2:00:30.760)

or somebody who delegated it to someone else.

Lex Fridman (2:00:33.400)

You just pick a random person.

Po-Shen Loh (2:00:35.640)

When you pick the random person,

Lex Fridman (2:00:37.080)

if it lands on someone and the person says,

Po-Shen Loh (2:00:38.880)

oh, I actually delegated it to someone,

Lex Fridman (2:00:41.840)

then you follow the point.

Po-Shen Loh (2:00:43.240)

You walk up the delegation chain.

Lex Fridman (2:00:45.280)

Walk up the delegation chain.

Lex Fridman (2:00:46.120)

And you can do like path compression in the algorithm

Lex Fridman (2:00:49.080)

to make it so you don't consistently

Po-Shen Loh (2:00:50.560)

do lots of walking up.

Lex Fridman (2:00:52.040)

But the bottom line is that what ends up happening

Po-Shen Loh (2:00:54.680)

is that you end up reaching out.

Lex Fridman (2:00:57.240)

Whenever you're one of the ones reaching out,

Po-Shen Loh (2:00:59.080)

you can think of it as each agent is responsible

Lex Fridman (2:01:01.840)

for some number of people.

Po-Shen Loh (2:01:03.320)

It's almost like they're the leader of a bunch.

Lex Fridman (2:01:05.480)

As the process is evolving, you have these lumps.

Po-Shen Loh (2:01:09.920)

Each lump has an agent.

Lex Fridman (2:01:11.680)

And when the agent reaches out,

Po-Shen Loh (2:01:13.360)

they reach out to another lump

Lex Fridman (2:01:15.680)

where the probability of them hitting that lump

Po-Shen Loh (2:01:18.160)

is proportional to the size of the lump.

Lex Fridman (2:01:21.880)

That is the one funny thing about this process.

Po-Shen Loh (2:01:25.720)

This is not that they can reach out

Lex Fridman (2:01:27.360)

to a uniformly random lump

Po-Shen Loh (2:01:29.280)

where every lump has the same chance

Lex Fridman (2:01:30.920)

of getting reached out to.

Po-Shen Loh (2:01:32.440)

The bigger the lump is,

Lex Fridman (2:01:34.520)

the more likely it is that you end up reaching that lump.

Lex Fridman (2:01:38.760)

Which is a problem?

Lex Fridman (2:01:40.160)

Let me explain why that's a problem.

Po-Shen Loh (2:01:41.600)

Because you see, you're hoping

Lex Fridman (2:01:43.280)

that this has a small number of steps,

Lex Fridman (2:01:45.400)

but here's a bad situation that could happen.

Lex Fridman (2:01:47.680)

Imagine if you had like,

Po-Shen Loh (2:01:50.000)

there are n people that you're adding up.

Lex Fridman (2:01:52.040)

Imagine that you have exactly square root of n lumps left,

Po-Shen Loh (2:01:57.040)

of which almost all of them are just one person

Lex Fridman (2:02:01.440)

who's still their own boss, their own manager.

Po-Shen Loh (2:02:04.280)

Except one giant one.

Lex Fridman (2:02:06.080)

Now what's gonna happen?

Po-Shen Loh (2:02:06.920)

It's gonna be a huge bottleneck

Lex Fridman (2:02:08.360)

because every round the giant one

Po-Shen Loh (2:02:09.760)

can only absorb one of the others.

Lex Fridman (2:02:11.880)

And now you suddenly have time

Po-Shen Loh (2:02:13.560)

which is about square root of n.

Lex Fridman (2:02:15.680)

The square root of n is chosen

Po-Shen Loh (2:02:16.840)

because that is one where the lumps are such

Lex Fridman (2:02:20.280)

that you really are limited by this large one

Po-Shen Loh (2:02:23.560)

slowly sucking up the rest of them.

Lex Fridman (2:02:26.000)

So the heart of the question became,

Po-Shen Loh (2:02:28.160)

well, but is that just so unusual

Lex Fridman (2:02:30.040)

that it doesn't usually happen?

Po-Shen Loh (2:02:32.760)

Because remember you start with everyone

Lex Fridman (2:02:34.680)

just being independent.

Po-Shen Loh (2:02:36.080)

It's like a lot of lumps of size one.

Lex Fridman (2:02:37.640)

How naturally do the big lumps emerge?

Po-Shen Loh (2:02:39.640)

Yes.

Lex Fridman (2:02:40.480)

And so what that heart of the proof was,

Po-Shen Loh (2:02:42.120)

was showing that that was a joint work with Eyal Lubezki.

Lex Fridman (2:02:45.120)

That one was showing that actually in that thing

Po-Shen Loh (2:02:49.040)

the lumps do kind of get out of whack.

Lex Fridman (2:02:50.920)

And so it's not the purely logarithmic number of steps.

Lex Fridman (2:02:54.720)

But if you make one very slight change,

Lex Fridman (2:02:56.720)

which is if you are one of the agents

Lex Fridman (2:03:00.400)

and you have just been propositioned,

Lex Fridman (2:03:02.400)

possibly relayed along by a couple of different people.

Po-Shen Loh (2:03:05.320)

If you just say, don't take a random one,

Lex Fridman (2:03:07.600)

but accept the smallest lump.

Po-Shen Loh (2:03:12.600)

That actually does enough to even the whole economy.

Lex Fridman (2:03:14.400)

Distributes the lump size.

Po-Shen Loh (2:03:16.320)

I mean, yeah, it's fascinating how

Lex Fridman (2:03:17.800)

with the distributed algorithms,

Po-Shen Loh (2:03:19.000)

a little adjustment can make all the difference

Lex Fridman (2:03:21.520)

in the world.

Po-Shen Loh (2:03:22.440)

Yeah.

Lex Fridman (2:03:23.280)

Actually, by the way, this does,

Po-Shen Loh (2:03:25.320)

back to our voting conversation,

Lex Fridman (2:03:26.960)

this makes me think of like,

Po-Shen Loh (2:03:29.440)

these networking systems are so fascinating to study.

Lex Fridman (2:03:32.240)

They immediately spring to mind ideas

Po-Shen Loh (2:03:35.120)

of how to have representation.

Lex Fridman (2:03:37.680)

Like maybe as opposed to me voting for a president,

Po-Shen Loh (2:03:42.240)

I want to vote for like,

Lex Fridman (2:03:45.600)

for you, Paul, to represent me,

Po-Shen Loh (2:03:48.400)

maybe on a particular issue.

Lex Fridman (2:03:50.600)

And then you will delegate that further.

Lex Fridman (2:03:52.560)

And then we naturally construct those kinds of networks

Lex Fridman (2:03:55.120)

because that feels like I can have a good conversation

Po-Shen Loh (2:03:58.880)

with you and figure out that you know what you're doing

Lex Fridman (2:04:00.760)

and I can delegate it to you.

Lex Fridman (2:04:01.800)

And in that way, construct a representative government,

Lex Fridman (2:04:05.560)

a representative decision maker.

Po-Shen Loh (2:04:08.440)

That feels really nice as opposed to like us,

Lex Fridman (2:04:12.480)

like a tree of height one or something,

Po-Shen Loh (2:04:14.600)

where it's like everybody's just,

Lex Fridman (2:04:18.400)

it feels like there's a lot of room for layers

Po-Shen Loh (2:04:20.440)

of representation to form organically from the bottom up.

Lex Fridman (2:04:23.840)

I wonder if there are systems like that.

Po-Shen Loh (2:04:25.400)

This is the cool thing about the internet

Lex Fridman (2:04:27.080)

and the digital space where we're so well connected,

Po-Shen Loh (2:04:29.560)

just like with the Novid app to distribute information

Lex Fridman (2:04:34.080)

about the spread of the disease.

Po-Shen Loh (2:04:37.000)

We can in the same way, in a distributed sense,

Lex Fridman (2:04:39.280)

form anything like any kind of knowledge bases

Po-Shen Loh (2:04:44.880)

that are formed in a decentralized way

Lex Fridman (2:04:48.760)

and in a hierarchical way,

Po-Shen Loh (2:04:51.480)

as opposed to sort of old way

Lex Fridman (2:04:54.320)

where there is no mechanism for large scale,

Po-Shen Loh (2:04:56.840)

fast distributed transactional information.

Lex Fridman (2:05:01.760)

This is really interesting.

Po-Shen Loh (2:05:02.600)

This is where almost like network graph theory,

Lex Fridman (2:05:06.800)

becomes practical.

Po-Shen Loh (2:05:09.280)

Most of that exciting work was done in the 20th century,

Lex Fridman (2:05:11.840)

but most of the application will be in the 21st,

Po-Shen Loh (2:05:14.080)

which is cool to think about.

Lex Fridman (2:05:15.960)

Let me ask the most ridiculous question.

Lex Fridman (2:05:17.720)

You think P equals NP?

Lex Fridman (2:05:19.880)

Wow.

Po-Shen Loh (2:05:21.240)

I don't know.

Lex Fridman (2:05:22.360)

I mean, I would say,

Po-Shen Loh (2:05:26.640)

I know there are enough people who have very strong interest

Lex Fridman (2:05:29.520)

in trying to show that it is.

Po-Shen Loh (2:05:32.680)

I'm talking about government agencies.

Lex Fridman (2:05:34.760)

For security purposes.

Po-Shen Loh (2:05:38.520)

For security purposes.

Lex Fridman (2:05:39.360)

And most computer scientists,

Po-Shen Loh (2:05:40.560)

we should say believe that P equals NP.

Lex Fridman (2:05:43.800)

My question almost like,

Po-Shen Loh (2:05:45.280)

this is back to our aliens discussion.

Lex Fridman (2:05:47.000)

You want to think outside the box,

Po-Shen Loh (2:05:48.440)

the low probability event,

Lex Fridman (2:05:51.800)

what is the world,

Lex Fridman (2:05:54.040)

what kind of discoveries would lead us to prove

Lex Fridman (2:05:58.640)

that P does not equal to NP?

Po-Shen Loh (2:06:01.560)

Like there could be giant misunderstandings

Lex Fridman (2:06:05.480)

or gaps in our knowledge about computer science,

Po-Shen Loh (2:06:08.080)

about theoretical computer science, about computation,

Lex Fridman (2:06:11.040)

which allow us to think like flatten all problems.

Po-Shen Loh (2:06:14.800)

Yeah, so I don't know the answer to this question.

Lex Fridman (2:06:17.080)

I think it's very interesting, but I actually,

Po-Shen Loh (2:06:19.440)

I know, let's put it this way.

Lex Fridman (2:06:21.280)

By being at Carnegie Mellon

Lex Fridman (2:06:22.520)

and being around the theoretical computer scientists,

Lex Fridman (2:06:24.880)

I know enough about what I don't know to say.

Po-Shen Loh (2:06:27.920)

To be humble.

Lex Fridman (2:06:28.760)

I'm the wrong person to answer this question.

Po-Shen Loh (2:06:32.320)

It's a great one.

Lex Fridman (2:06:33.240)

Well, Scott Aaronson, who's now here at UT Austin,

Po-Shen Loh (2:06:35.920)

he used to be at MIT,

Lex Fridman (2:06:37.800)

puts the probability of P not equals to NP at 3%.

Po-Shen Loh (2:06:45.920)

I always love it when you ask,

Lex Fridman (2:06:48.040)

it's very rare in science and academics

Po-Shen Loh (2:06:51.040)

because most folks are humble

Lex Fridman (2:06:54.680)

in the face of the mystery,

Po-Shen Loh (2:06:56.800)

the uncertainty of everything around us.

Lex Fridman (2:06:59.400)

To have both the humor and the guts to say like,

Lex Fridman (2:07:03.360)

what are the chance that there's aliens in our galaxy,

Lex Fridman (2:07:07.160)

intelligent alien civilizations?

Po-Shen Loh (2:07:09.440)

As opposed to saying, I don't know, it could be zero.

Lex Fridman (2:07:12.280)

It could be, depending on the fact, you're saying it's 2.5%.

Po-Shen Loh (2:07:15.920)

There's something very pleasant about just having,

Lex Fridman (2:07:20.280)

it's the number thing.

Po-Shen Loh (2:07:24.400)

It's powered to the number.

Lex Fridman (2:07:25.480)

It's just like 42.

Lex Fridman (2:07:26.440)

It's like, why 42?

Lex Fridman (2:07:27.280)

I don't know, but it's a powerful number.

Lex Fridman (2:07:29.640)

And then everything,

Lex Fridman (2:07:30.640)

this is the power of human psychology

Po-Shen Loh (2:07:32.600)

is once you have the number 42,

Lex Fridman (2:07:36.560)

it's not that the number has meaning,

Lex Fridman (2:07:39.240)

but because it's placed in a book with humor around it,

Lex Fridman (2:07:43.960)

it has the meme effect of actually creating reality.

Po-Shen Loh (2:07:49.400)

I mean, you could say that 42 has a strong contribution

Lex Fridman (2:07:53.520)

of helping us colonize Mars

Po-Shen Loh (2:07:55.920)

because it created,

Lex Fridman (2:07:57.720)

it gave the whatever existential crisis to many of us,

Po-Shen Loh (2:08:00.480)

including Elon Musk when he was young,

Lex Fridman (2:08:03.160)

reading a book like that.

Lex Fridman (2:08:04.280)

And then now 42 is now part of his humor

Lex Fridman (2:08:07.040)

that he doesn't shut up about,

Po-Shen Loh (2:08:08.840)

it's constantly joking about.

Lex Fridman (2:08:09.680)

And that humor is spreading through our minds

Lex Fridman (2:08:12.200)

and somehow this like silly number just had an effect.

Lex Fridman (2:08:15.160)

In that same way, after Scott told me like the 3% chance,

Po-Shen Loh (2:08:19.360)

it's stuck in my head.

Lex Fridman (2:08:20.680)

And I think it's been having a ripple effect

Po-Shen Loh (2:08:22.720)

in everybody else.

Lex Fridman (2:08:23.720)

The believing that P is not equal to NP,

Po-Shen Loh (2:08:29.120)

Scott almost as a joke saying it's 3%

Lex Fridman (2:08:32.200)

is actually motivating a large number of researchers

Po-Shen Loh (2:08:34.920)

to work on it.

Lex Fridman (2:08:35.760)

3% is high.

Po-Shen Loh (2:08:37.080)

It's very high.

Lex Fridman (2:08:37.920)

Because for the potential impact that that would have.

Lex Fridman (2:08:39.920)

But then 3% is not that high because it's only,

Lex Fridman (2:08:44.280)

you know, like we're not very good.

Po-Shen Loh (2:08:46.480)

I feel like humans are only able to really think about

Lex Fridman (2:08:48.840)

like 1%, 50%.

Lex Fridman (2:08:51.280)

And we kind of, I think a lot of people around 3%

Lex Fridman (2:08:55.400)

up to 50% like in our minds.

Po-Shen Loh (2:08:58.800)

Like 3% at this point.

Lex Fridman (2:09:00.640)

It could happen.

Po-Shen Loh (2:09:01.480)

It could happen.

Lex Fridman (2:09:02.480)

And it could happen and it's like, yeah.

Po-Shen Loh (2:09:04.760)

Like half the time it'll probably happen.

Lex Fridman (2:09:07.040)

So we're not very good at that.

Po-Shen Loh (2:09:08.200)

That's the other thing with the pandemic

Lex Fridman (2:09:10.240)

is we're not the exponential growth

Po-Shen Loh (2:09:13.560)

that we also talked about offline

Lex Fridman (2:09:15.800)

is something that we can't quite intuit.

Lex Fridman (2:09:20.200)

And that's something we probably should

Lex Fridman (2:09:22.680)

if we're to predict the future, to anticipate the future

Lex Fridman (2:09:25.240)

and to understand how to create technologies

Lex Fridman (2:09:27.920)

that let us sort of control the future.

Po-Shen Loh (2:09:32.320)

Can I ask you for some recommendations

Lex Fridman (2:09:35.040)

maybe for books or movies in your life?

Po-Shen Loh (2:09:39.120)

Long ago when you were baby Po or today

Lex Fridman (2:09:45.320)

that you found insightful or you learned a lot from

Lex Fridman (2:09:50.160)

what you would recommend to others.

Lex Fridman (2:09:52.080)

Yeah.

Lex Fridman (2:09:52.920)

So I think I don't necessarily have an exact name

Lex Fridman (2:09:55.760)

of these old things, but I was generally inspired

Po-Shen Loh (2:09:58.600)

by stories, true or fictional of campaigns.

Lex Fridman (2:10:05.920)

For example, like the Lord of the rings, that's a campaign.

Lex Fridman (2:10:09.080)

But the thing that always inspired me was

Lex Fridman (2:10:12.480)

it could be possible for somebody who's crazy enough

Po-Shen Loh (2:10:16.480)

to go up against adversity after adversity,

Lex Fridman (2:10:18.680)

and it succeeds.

Po-Shen Loh (2:10:20.240)

I mean, those are false, those are fictitious.

Lex Fridman (2:10:23.040)

But I also spent a lot of time, I guess, reading about,

Po-Shen Loh (2:10:25.480)

I don't know, I was interested somehow

Lex Fridman (2:10:26.600)

in like World War II history for whatever reason.

Po-Shen Loh (2:10:29.520)

That's a campaign which is much more brutal.

Lex Fridman (2:10:31.440)

But nevertheless, the idea of difficulty, strategy,

Po-Shen Loh (2:10:37.560)

fighting even when things,

Lex Fridman (2:10:39.480)

in that case it was really fighting,

Lex Fridman (2:10:40.800)

but just pushing on even when things are difficult.

Lex Fridman (2:10:43.600)

I guess these are the kinds of general stories

Po-Shen Loh (2:10:46.840)

that made me, I guess, want to work on things

Lex Fridman (2:10:50.880)

that would be hard and where it could be a campaign.

Po-Shen Loh (2:10:54.920)

It could be that you work on something for a year,

Lex Fridman (2:10:57.520)

multiple years, because that was the point, I guess.

Po-Shen Loh (2:11:02.200)

Yeah, it starts with a single person.

Lex Fridman (2:11:04.160)

That's the interesting thing.

Po-Shen Loh (2:11:05.680)

I've obviously been, don't shut up about it recently

Lex Fridman (2:11:08.880)

about World War II, especially on the Hitler side

Lex Fridman (2:11:11.400)

and the Stalin side.

Lex Fridman (2:11:12.640)

Some of that has really affected my own family.

Po-Shen Loh (2:11:15.600)

The roots of my family very much.

Lex Fridman (2:11:18.760)

But it's interesting to think that it was just an idea

Lex Fridman (2:11:24.840)

and one person decided to do stuff

Lex Fridman (2:11:26.920)

and it just builds and builds and builds.

Lex Fridman (2:11:29.520)

And you can truly have an impact on the world,

Lex Fridman (2:11:31.840)

both horrendous and exceptionally positive and inspiring.

Lex Fridman (2:11:36.840)

So yeah, it's like it's a agency of us individuals.

Lex Fridman (2:11:47.840)

Sometimes we think we're just reacting to the world,

Lex Fridman (2:11:49.920)

but we have the full power to actually change the world.

Lex Fridman (2:11:53.080)

Is there advice you can give to young folks?

Po-Shen Loh (2:11:56.040)

We talked, we gave a bunch of advice

Lex Fridman (2:11:58.000)

on middle school, high school mathematics.

Po-Shen Loh (2:12:00.600)

Is there more general advice you would give

Lex Fridman (2:12:02.480)

about how to succeed in life,

Lex Fridman (2:12:04.800)

how to learn for high school students,

Lex Fridman (2:12:07.400)

for college students, career or life in general?

Lex Fridman (2:12:10.600)

So I think the first one would be

Lex Fridman (2:12:12.280)

to make sure that you're learning to invent

Lex Fridman (2:12:14.680)

and to make sure you're not just learning how to mimic.

Lex Fridman (2:12:19.080)

Because a lot of times you learn how to do X

Po-Shen Loh (2:12:21.880)

by watching somebody do X and then repeating X

Lex Fridman (2:12:24.280)

many times with different inputs.

Po-Shen Loh (2:12:26.360)

I've just been very generic in explaining this.

Lex Fridman (2:12:28.520)

But I guess this is just my own attitude towards the world.

Po-Shen Loh (2:12:31.840)

I didn't like ever following anyone's directions exactly.

Lex Fridman (2:12:34.920)

Even if you told me this is the way to do your homework

Po-Shen Loh (2:12:37.680)

is to write in pencil, I would say,

Lex Fridman (2:12:39.400)

but I think pen is nice, let's try, right?

Lex Fridman (2:12:42.880)

So I've been that kind of a funny person.

Lex Fridman (2:12:45.640)

But I do encourage that if you can learn how to invent

Po-Shen Loh (2:12:50.720)

as your core skill, then you can do a lot.

Lex Fridman (2:12:52.760)

But then the second piece that comes with that

Po-Shen Loh (2:12:54.320)

is something I learned from my PhD advisor,

Lex Fridman (2:12:57.400)

which was, well, make sure that what you're working on

Po-Shen Loh (2:13:01.040)

is big enough.

Lex Fridman (2:13:02.280)

And so in that sense, I usually advise to people

Po-Shen Loh (2:13:04.760)

once they have learned how to invent,

Lex Fridman (2:13:07.600)

ideally don't just try to settle for something comfortable,

Po-Shen Loh (2:13:11.520)

try to see if you can aim for something which is hard,

Lex Fridman (2:13:15.000)

which might involve a campaign, which might be important,

Po-Shen Loh (2:13:18.000)

which might make a difference.

Lex Fridman (2:13:20.000)

And it's more of, I guess, rather than worrying

Lex Fridman (2:13:23.920)

what if you didn't achieve that,

Lex Fridman (2:13:27.040)

there's also the regret of what if I didn't try?

Po-Shen Loh (2:13:30.320)

See, that's how I operate.

Lex Fridman (2:13:31.600)

I don't operate based on did I succeed or fail?

Po-Shen Loh (2:13:33.720)

It was hard anyway.

Lex Fridman (2:13:34.640)

If I did this novid thing and the whole thing failed,

Lex Fridman (2:13:36.640)

would I feel terrible?

Lex Fridman (2:13:37.640)

No, it's a very hard problem.

Lex Fridman (2:13:39.480)

But would I have had the regret of not jumping in?

Lex Fridman (2:13:42.640)

Yes.

Lex Fridman (2:13:44.080)

So it's that different mentality of don't worry

Lex Fridman (2:13:46.000)

about the failing part as much of the,

Po-Shen Loh (2:13:48.680)

make sure you give yourself the shot

Lex Fridman (2:13:50.760)

at those potentially unbounded opportunities.

Po-Shen Loh (2:13:55.160)

You almost make it sound like there's a meaning to it all.

Lex Fridman (2:13:58.320)

Let me ask the big ridiculous question.

Lex Fridman (2:13:59.960)

What do you think is the meaning of life?

Lex Fridman (2:14:01.840)

Or maybe the easier version of that

Lex Fridman (2:14:04.200)

is what brings your life joy?

Lex Fridman (2:14:06.120)

So I'll just answer that one personally.

Po-Shen Loh (2:14:07.880)

For me, I'm a little bit weird.

Lex Fridman (2:14:10.080)

I sort of, I guess you can tell by now.

Po-Shen Loh (2:14:13.520)

See the pen and pencil discussion from earlier, yes.

Lex Fridman (2:14:15.960)

Yeah, yeah.

Po-Shen Loh (2:14:16.920)

So, I mean, my thing is, I guess I personally

Lex Fridman (2:14:20.360)

just wanted to maximize a certain score,

Po-Shen Loh (2:14:24.360)

which was for how many person years

Lex Fridman (2:14:28.160)

after I'm no longer here anymore,

Lex Fridman (2:14:31.000)

did what I do mattered?

Lex Fridman (2:14:33.400)

Yeah.

Lex Fridman (2:14:34.240)

And it didn't matter if it's necessarily attributed to me.

Lex Fridman (2:14:36.480)

It's just like, did it matter?

Lex Fridman (2:14:38.600)

And so that's what I wanted.

Lex Fridman (2:14:41.800)

I guess that is very inspired by how scientists work.

Lex Fridman (2:14:45.620)

It's like, why do we keep talking about Newton?

Lex Fridman (2:14:47.760)

It's because Newton discovered some interesting things.

Lex Fridman (2:14:50.840)

And so Newton's score is pretty high.

Lex Fridman (2:14:53.880)

It's going to be infinity, right?

Po-Shen Loh (2:14:56.240)

Well, let's hope it's infinity, but pretty high.

Lex Fridman (2:14:59.040)

Yes, yes.

Lex Fridman (2:15:00.120)

So you're going for, so person years,

Lex Fridman (2:15:03.780)

you're going for like triple digits.

Po-Shen Loh (2:15:05.480)

You're going for, so like Newton is like four digits,

Lex Fridman (2:15:08.940)

probably like a thousand years or personal lifetimes.

Lex Fridman (2:15:13.400)

How do you like to think, well, what are we?

Lex Fridman (2:15:15.340)

Sorry, I meant people times years.

Po-Shen Loh (2:15:17.480)

People times.

Lex Fridman (2:15:18.320)

So then it's like, actually his is huge.

Po-Shen Loh (2:15:19.980)

His is like going to be billions or trillions, trillions.

Lex Fridman (2:15:23.240)

But I guess for me, I actually changed the metric

Po-Shen Loh (2:15:27.680)

after a while.

Lex Fridman (2:15:28.520)

And the reason is because you may have seen,

Po-Shen Loh (2:15:30.080)

I found some simple way to solve quadratic equations

Lex Fridman (2:15:33.640)

that is easier than every textbook.

Lex Fridman (2:15:35.780)

So my score might already be not bad,

Lex Fridman (2:15:39.040)

which is why I decided then let's change it

Po-Shen Loh (2:15:40.760)

into the number of hours in the lifetimes as well.

Lex Fridman (2:15:44.640)

So the way I was doing it before is that

Po-Shen Loh (2:15:48.480)

if a person was sort of remembering or using

Lex Fridman (2:15:53.440)

or appreciating what I had done

Po-Shen Loh (2:15:56.040)

for like 10 years of their life.

Lex Fridman (2:16:00.280)

Oh, I see.

Po-Shen Loh (2:16:01.120)

That would count as 10.

Lex Fridman (2:16:01.940)

I see.

Lex Fridman (2:16:02.780)

So if there was one person who for 10 years remembered

Lex Fridman (2:16:05.060)

or appreciated something I did,

Po-Shen Loh (2:16:06.200)

that counts as a score of 10 and we add up overall people.

Lex Fridman (2:16:09.400)

And then, and that was with the hypothesis

Po-Shen Loh (2:16:13.480)

that the score would be very finite in the sense

Lex Fridman (2:16:16.680)

that if I didn't come up with anything

Po-Shen Loh (2:16:19.080)

that might potentially help a lot of generations

Lex Fridman (2:16:21.400)

in a forever way, then your score will be finite

Po-Shen Loh (2:16:23.840)

because at some point it's not,

Lex Fridman (2:16:25.960)

people don't remember that you made like nice bottles

Lex Fridman (2:16:28.760)

or something, right?

Lex Fridman (2:16:30.160)

But then after the quadratic equation thing,

Po-Shen Loh (2:16:33.040)

it was that there's some chance

Lex Fridman (2:16:34.980)

that that actually might make it into textbooks.

Lex Fridman (2:16:37.720)

And if it makes it in textbooks,

Lex Fridman (2:16:39.080)

the chance that there'll be an easier way discovered

Po-Shen Loh (2:16:40.960)

is actually quite small.

Lex Fridman (2:16:42.680)

So in that case, then the score might get bigger.

Po-Shen Loh (2:16:46.220)

I was just saying the score might actually already

Lex Fridman (2:16:48.040)

have been achieved in a non trivial way.

Po-Shen Loh (2:16:51.240)

I see.

Lex Fridman (2:16:52.080)

Because it's fun to think about,

Po-Shen Loh (2:16:53.720)

cause it could be different.

Lex Fridman (2:16:54.620)

You can achieve a high score by a small number of people

Po-Shen Loh (2:16:58.980)

using it for most of their lifetime

Lex Fridman (2:17:01.240)

and then generations and generations.

Po-Shen Loh (2:17:03.400)

Or you can have, if we do dissipate,

Lex Fridman (2:17:05.960)

if we do split colonize, become multi planetary species,

Po-Shen Loh (2:17:10.120)

you could have that little,

Lex Fridman (2:17:12.280)

a clever way to solve differential equations,

Po-Shen Loh (2:17:17.280)

spread through like trillions of people

Lex Fridman (2:17:19.640)

as they spread throughout the galaxy.

Lex Fridman (2:17:21.600)

And they would only use it each one,

Lex Fridman (2:17:24.800)

a few hours in their lifetime,

Lex Fridman (2:17:26.760)

but their kids will use it,

Lex Fridman (2:17:28.280)

the kids of kids will use it, it will spread

Lex Fridman (2:17:30.080)

and you'll have that impact in that kind of way.

Lex Fridman (2:17:33.240)

Yes, so that's why I renormalized it

Po-Shen Loh (2:17:34.920)

because I was like, well, that's kind of dumb

Lex Fridman (2:17:36.440)

because what's the importance of that?

Po-Shen Loh (2:17:37.760)

That'll save people 15 minutes.

Lex Fridman (2:17:39.880)

But, so what I meant is I didn't want to count that

Po-Shen Loh (2:17:42.380)

as the main score.

Lex Fridman (2:17:46.360)

Well, I'm gonna have to try to come up

Po-Shen Loh (2:17:47.880)

with some kind of device that everyone would want to use,

Lex Fridman (2:17:50.360)

maybe to make coffee,

Po-Shen Loh (2:17:51.520)

cause coffee seems to be the prevalent

Lex Fridman (2:17:54.720)

performance enhancing chemical that everyone uses.

Lex Fridman (2:17:57.160)

So I'll have to think about those kinds of metrics.

Lex Fridman (2:17:59.880)

Yeah, but you see that's just giving an idea

Po-Shen Loh (2:18:02.560)

of I guess what I found meaningful in general,

Lex Fridman (2:18:05.080)

like whether or not it's like,

Po-Shen Loh (2:18:06.200)

whether or not that quadratic thing is important or not.

Lex Fridman (2:18:08.340)

The general idea was I wanted to do things

Po-Shen Loh (2:18:10.740)

that would outlast me.

Lex Fridman (2:18:12.080)

And that was what inspired me

Lex Fridman (2:18:13.400)

and that's just how I choose what problems to work on.

Lex Fridman (2:18:15.680)

And that's a kind of immortality is ideas

Po-Shen Loh (2:18:18.520)

that you've invented living on long after you

Lex Fridman (2:18:23.060)

in the minds of others.

Lex Fridman (2:18:24.920)

And humans are ultimately not,

Lex Fridman (2:18:27.160)

are like meat vehicles that carry ideas for brief

Po-Shen Loh (2:18:32.400)

for just a few years may not be the important thing.

Lex Fridman (2:18:34.880)

It might be the ideas that we carry with us

Lex Fridman (2:18:37.280)

and invent new ones.

Lex Fridman (2:18:38.680)

Like we get a bunch of baby ideas in our head.

Po-Shen Loh (2:18:41.640)

We borrow them from others

Lex Fridman (2:18:43.120)

and then maybe we invent a new one

Lex Fridman (2:18:45.060)

and that new one might have a life of its own.

Lex Fridman (2:18:47.920)

And it's fun to think about that idea

Po-Shen Loh (2:18:50.880)

of living for many centuries to come

Lex Fridman (2:18:53.440)

unless we destroy ourselves.

Lex Fridman (2:18:54.960)

But maybe AI will borrow it

Lex Fridman (2:18:56.960)

and we'll remember Po as like that one human

Po-Shen Loh (2:19:00.160)

that helped us out before we of course killed him

Lex Fridman (2:19:04.680)

and the rest of human civilization.

Po-Shen Loh (2:19:06.760)

On that note, Po, this is a huge honor.

Lex Fridman (2:19:09.840)

You're one of the great educators

Po-Shen Loh (2:19:12.880)

I've ever gotten a chance to interact with.

Lex Fridman (2:19:15.260)

So it's truly an honor that you would talk with me today.

Po-Shen Loh (2:19:18.520)

It means especially a lot that you would travel a lot

Lex Fridman (2:19:21.160)

to Austin to talk to me.

Po-Shen Loh (2:19:22.480)

It really means a lot.

Lex Fridman (2:19:23.480)

So thank you so much.

Po-Shen Loh (2:19:25.160)

Keep on inspiring.

Lex Fridman (2:19:26.440)

And I'm one of your many, many students.

Po-Shen Loh (2:19:30.320)

Thank you so much for talking today.

Lex Fridman (2:19:32.140)

Thank you, thank you.

Po-Shen Loh (2:19:32.980)

It's actually a real honor for me to talk to you

Lex Fridman (2:19:34.640)

and to get this chance to have this really

Po-Shen Loh (2:19:37.120)

intellectual conversation through all of these topics.

Lex Fridman (2:19:39.800)

Thanks, Po.

Po-Shen Loh (2:19:41.440)

Thanks for listening to this conversation with Po Chenlo

Lex Fridman (2:19:44.260)

and thank you to Jordan Harmer, the show,

Po-Shen Loh (2:19:47.040)

Onnit, BetterHelp, AidSleep and Element.

Lex Fridman (2:19:51.400)

Check them out in the description to support this podcast.

Lex Fridman (2:19:54.480)

And now let me leave you with some words from Isaac Newton.

Lex Fridman (2:19:58.260)

I can calculate the motion of heavenly bodies

Lex Fridman (2:20:01.280)

but not the madness of people.

Lex Fridman (2:20:03.700)

Thank you for listening and hope to see you next time.

Po-Shen Loh (30:01.200)

I'm not using the Pythagorean theorem basically.

Lex Fridman (30:03.320)

I mean, if I just knew the GPS coordinates,

Po-Shen Loh (30:05.520)

we could use the Pythagorean theorem too.

Lex Fridman (30:07.320)

Sorry, that's just how I call it.

Po-Shen Loh (30:08.520)

Distance formula, whatever you want to call it.

Lex Fridman (30:10.520)

Yeah, so we're not doing

Po-Shen Loh (30:14.800)

the old Pythagorean based violation of privacy.

Lex Fridman (30:18.280)

Okay.

Lex Fridman (30:21.240)

But so is that enough to form,

Lex Fridman (30:27.280)

to give you enough information about physical connection

Lex Fridman (30:31.160)

to another human being?

Lex Fridman (30:32.880)

Is there a time element there?

Po-Shen Loh (30:35.240)

Is there, so, okay.

Lex Fridman (30:37.400)

That sounds like a really strong, like low hanging fruit.

Po-Shen Loh (30:41.560)

Like if you have that,

Lex Fridman (30:42.480)

you could probably go really, really far.

Po-Shen Loh (30:44.920)

My natural question is,

Lex Fridman (30:46.880)

is there extra information you can add on top of that?

Lex Fridman (30:49.760)

Like the duration of the physical proximity?

Lex Fridman (30:53.520)

So first of all, we actually do estimate the duration,

Lex Fridman (30:56.640)

but the way we estimate the duration

Lex Fridman (30:58.360)

is like how a movie is filmed,

Po-Shen Loh (31:00.440)

in the sense that every so often, every few minutes,

Lex Fridman (31:03.200)

we check what's nearby.

Po-Shen Loh (31:04.640)

It's like how a movie is filmed.

Lex Fridman (31:06.240)

You take lots of snapshots.

Lex Fridman (31:07.960)

So there's no way in a battery efficient way

Lex Fridman (31:11.280)

to really keep track of that proximity.

Po-Shen Loh (31:14.680)

However, fortunately, we're using probability.

Lex Fridman (31:17.120)

The fact is the paradigm that we're using

Po-Shen Loh (31:20.120)

is it's not super important

Lex Fridman (31:22.000)

if you run into that person only for 10 minutes

Po-Shen Loh (31:24.080)

at the grocery store.

Lex Fridman (31:25.680)

If that's a stranger that you run into 10 minutes

Po-Shen Loh (31:27.640)

in this grocery store,

Lex Fridman (31:28.920)

that's not gonna be relevant for our paradigm

Po-Shen Loh (31:30.880)

because our paradigm is not telling you

Lex Fridman (31:33.080)

who were you around before

Lex Fridman (31:35.000)

and might therefore have gotten infected by already.

Lex Fridman (31:38.600)

Ours is about predicting the future.

Po-Shen Loh (31:40.160)

We change from, I mean, the standard paradigm was

Lex Fridman (31:42.680)

what already happened, quick damage control.

Po-Shen Loh (31:45.200)

Ours is predict the future.

Lex Fridman (31:46.560)

If you run into that person once in the grocery store today

Lex Fridman (31:49.280)

and never see them again,

Lex Fridman (31:50.360)

it's irrelevant for predicting the future.

Lex Fridman (31:52.520)

And therefore, for ours, what really matters

Lex Fridman (31:54.800)

is the many hours around the other person,

Po-Shen Loh (31:57.960)

at which point, if you're scanning every five

Lex Fridman (31:59.600)

to eight minutes.

Po-Shen Loh (32:00.640)

That's going to come out in the problem,

Lex Fridman (32:02.040)

like statistically speaking,

Po-Shen Loh (32:03.240)

it's going to come out as a strong relationship

Lex Fridman (32:05.400)

and a person in the grocery store is going to wash out

Po-Shen Loh (32:08.760)

is not an important physical relationship.

Lex Fridman (32:11.360)

I mean, this is brilliant.

Lex Fridman (32:14.400)

How difficult is it to make work?

Lex Fridman (32:15.880)

So you said, one, there's a mathematical component

Po-Shen Loh (32:19.000)

that we just kind of talked about,

Lex Fridman (32:21.280)

and then there's the user experience component.

Lex Fridman (32:24.000)

So how difficult does it to go,

Lex Fridman (32:26.080)

just like you built the video game, Alien Attack,

Lex Fridman (32:29.920)

from zero to completion, what's involved?

Lex Fridman (32:33.800)

How difficult is it?

Lex Fridman (32:34.920)

So I'm going to answer that question

Lex Fridman (32:36.720)

in terms of building the product,

Lex Fridman (32:39.360)

but then I'm also going to acknowledge

Lex Fridman (32:40.880)

that just having an app doesn't make it useful

Po-Shen Loh (32:44.760)

because that's actually maybe the easy part.

Lex Fridman (32:48.560)

If you know what I mean,

Po-Shen Loh (32:49.400)

there's like all of this stuff

Lex Fridman (32:50.320)

about rollout adoption and awareness,

Lex Fridman (32:52.040)

but let's focus on the app part first.

Lex Fridman (32:53.760)

So that's again, why I said the team is incredible.

Lex Fridman (32:56.320)

So we have a bunch of people who,

Lex Fridman (32:59.800)

let's just say that the technology that we use to make it

Po-Shen Loh (33:02.720)

is not the standard way you make an app.

Lex Fridman (33:04.960)

If you think about a standard iOS app or Android app,

Po-Shen Loh (33:08.680)

those are a user interface that contacts a web server

Lex Fridman (33:12.120)

and sends some information back and forth.

Po-Shen Loh (33:14.080)

We're doing some stuff that has to hook

Lex Fridman (33:16.000)

into the operating system of saying,

Po-Shen Loh (33:17.880)

let's go use Bluetooth for something

Lex Fridman (33:19.280)

it wasn't really meant for, right?

Lex Fridman (33:21.600)

So there's that part.

Lex Fridman (33:22.960)

By the way, what is the app called?

Po-Shen Loh (33:24.520)

Oh, it's called Novid, COVID with an N.

Lex Fridman (33:28.760)

Very nice.

Lex Fridman (33:29.600)

So you have to hook into Bluetooth.

Lex Fridman (33:31.360)

You're saying you have to do that beyond the permissions

Po-Shen Loh (33:36.240)

that are like at the very surface level

Lex Fridman (33:39.560)

provided on the phone?

Po-Shen Loh (33:40.680)

Well, I don't want to call them permissions.

Lex Fridman (33:42.440)

I just want to say,

Po-Shen Loh (33:43.280)

that's not what you usually do with Bluetooth.

Lex Fridman (33:45.440)

Gotcha.

Po-Shen Loh (33:46.280)

Usually with Bluetooth, you say,

Lex Fridman (33:47.680)

do I have headphones nearby?

Po-Shen Loh (33:49.080)

Yes.

Lex Fridman (33:49.920)

Okay, I'm done.

Lex Fridman (33:50.760)

You don't go and say, do I have headphones nearby?

Lex Fridman (33:53.160)

Or do I have another phone nearby, which is doing something?

Lex Fridman (33:55.520)

And then keep asking that same question.

Lex Fridman (33:56.760)

Keep asking the question.

Lex Fridman (33:58.320)

Right?

Lex Fridman (33:59.160)

So it's actually not easy.

Lex Fridman (34:00.240)

And I mean, there were some parts of it,

Lex Fridman (34:02.160)

which actually a lot of people had tried unsuccessfully.

Po-Shen Loh (34:05.840)

Actually, it's known that, for example,

Lex Fridman (34:07.960)

the UK was trying to do something similar.

Lex Fridman (34:11.600)

And the problem they ran into was,

Lex Fridman (34:13.640)

when you program things on iOS,

Po-Shen Loh (34:16.360)

iOS is very good at making it hard

Lex Fridman (34:19.320)

to do things in the background.

Lex Fridman (34:21.760)

And so there was quite a lot of effort required

Lex Fridman (34:23.800)

to go and make this thing work.

Lex Fridman (34:25.440)

So the whole point, this thing would run in the background

Lex Fridman (34:28.160)

and iOS, I mean, most Android probably as well, right?

Lex Fridman (34:33.720)

But yeah, iOS certainly makes it difficult

Lex Fridman (34:35.480)

for something to run in the background,

Lex Fridman (34:36.880)

especially when it's eating up your battery, right?

Lex Fridman (34:40.400)

Well, we wanted to make sure we didn't eat up the battery.

Lex Fridman (34:42.240)

So that one we can,

Lex Fridman (34:43.480)

we actually are very proud of the fact

Po-Shen Loh (34:44.920)

that ours uses very little battery.

Lex Fridman (34:47.760)

Actually, even if compared to Apple's own system, so.

Po-Shen Loh (34:51.560)

Beautiful.

Lex Fridman (34:52.400)

So what else is required to make this thing work?

Po-Shen Loh (34:54.480)

Right, so the key was that you had to do

Lex Fridman (34:56.840)

a significant amount of work on the actual

Po-Shen Loh (34:58.800)

mobile app development,

Lex Fridman (35:00.160)

which fortunately the team that we brought

Po-Shen Loh (35:02.120)

was this kind of general thinkers

Lex Fridman (35:04.000)

where we would dig in deep into the operating system

Po-Shen Loh (35:07.120)

documentation and the API libraries.

Lex Fridman (35:09.400)

So we got that working.

Lex Fridman (35:10.600)

But there's another angle, which is,

Lex Fridman (35:12.280)

you also need the servers to be able to compute fast enough,

Po-Shen Loh (35:15.000)

which is tying back to this old school

Lex Fridman (35:17.360)

computer programming competitions and math Olympiads.

Po-Shen Loh (35:20.040)

In fact, our team that was working on the algorithm

Lex Fridman (35:23.000)

and backend side included several people

Po-Shen Loh (35:25.440)

who had been in these competitions from before,

Lex Fridman (35:29.320)

which I happen to know because I do coach the team

Po-Shen Loh (35:32.360)

for the math.

Lex Fridman (35:33.360)

And so we were able to bring people in to build servers,

Po-Shen Loh (35:37.240)

a server infrastructure in C++ actually,

Lex Fridman (35:40.120)

so that we could support significant numbers of people

Po-Shen Loh (35:43.080)

without needing tons of servers.

Lex Fridman (35:45.360)

Is there some distributed algorithms working here

Po-Shen Loh (35:48.000)

or you basically have to keep in the same place

Lex Fridman (35:51.560)

the entire graph as it builds?

Po-Shen Loh (35:53.720)

Cause especially the more and more people use it,

Lex Fridman (35:56.280)

the bigger, the bigger the graph gets.

Lex Fridman (35:58.240)

I mean, this is very difficult scaling problem, right?

Lex Fridman (36:02.280)

Ah, so that's actually why this computer algorithm

Po-Shen Loh (36:05.600)

competition stuff was handy.

Lex Fridman (36:07.120)

It's because there are only about seven to eight

Po-Shen Loh (36:11.200)

giga people in the world.

Lex Fridman (36:12.920)

Yeah.

Po-Shen Loh (36:14.000)

That's not that many.

Lex Fridman (36:15.160)

So if you can make your algorithms linear time

Po-Shen Loh (36:17.520)

or almost linear time, a computer operates in gigahertz.

Lex Fridman (36:22.320)

I only need to do one run, one recalculation every hour

Po-Shen Loh (36:26.000)

in terms of telling people how far away these dangers are.

Lex Fridman (36:29.080)

Yes.

Lex Fridman (36:29.920)

So I suddenly have 3,600 seconds

Lex Fridman (36:33.520)

and my CPU cores are running in gigahertz.

Lex Fridman (36:36.520)

And at most they're eight giga people.

Lex Fridman (36:39.520)

Well, you skipping over the fact that there's N squared

Po-Shen Loh (36:44.000)

potential connections between people.

Lex Fridman (36:46.840)

So how do you get around the fact that, you know,

Po-Shen Loh (36:51.400)

that we, you know, the potential set of relationship

Lex Fridman (36:54.440)

any one of us could have is a billion.

Lex Fridman (36:56.160)

So it's a billion times squared.

Lex Fridman (36:59.200)

That's the potential amount of data you have to be storing

Lex Fridman (37:02.720)

and computing over and constantly updating.

Lex Fridman (37:05.280)

So the way we dealt with that is we actually expect

Po-Shen Loh (37:08.120)

that the typical network is very sparse.

Lex Fridman (37:10.840)

The technical term sparse would mean that the average degree

Po-Shen Loh (37:14.840)

or the average number of connections that a person has

Lex Fridman (37:17.640)

is going to be at most like a hundred strong connections

Po-Shen Loh (37:21.160)

that you care about.

Lex Fridman (37:22.520)

If you think of it almost in terms of the heavy hitters,

Po-Shen Loh (37:25.520)

actually in most people's lives,

Lex Fridman (37:28.200)

a hundred, if we just kept track

Po-Shen Loh (37:29.880)

of their top hundred interactions,

Lex Fridman (37:32.440)

that's probably most of the signal.

Po-Shen Loh (37:35.480)

Yeah, yeah.

Lex Fridman (37:37.680)

I'm saddened to think that I might not be even

Po-Shen Loh (37:40.400)

in a double digits, but.

Lex Fridman (37:42.040)

Oh, I was intentionally giving a crazy number

Po-Shen Loh (37:44.720)

to account for college students.

Lex Fridman (37:46.160)

You call, oh, those are the,

Po-Shen Loh (37:48.960)

who you call on the heavy hitters,

Lex Fridman (37:50.000)

the people who are like the social butterflies.

Po-Shen Loh (37:52.040)

Yeah, I need to,

Lex Fridman (37:54.840)

I'd love to know that information about myself,

Po-Shen Loh (37:56.920)

by the way, that, do you expose the graph,

Lex Fridman (38:01.960)

like how many, like about yourself,

Lex Fridman (38:04.480)

how many connections you have?

Lex Fridman (38:06.080)

We do expose to each person

Lex Fridman (38:07.920)

how many direct connections they have.

Lex Fridman (38:09.600)

That's great.

Lex Fridman (38:10.440)

But for privacy purposes,

Lex Fridman (38:11.600)

we don't tell anybody who their connections,

Po-Shen Loh (38:14.120)

like how their connections are interconnected.

Lex Fridman (38:16.000)

Yes, gotcha.

Lex Fridman (38:16.880)

But at the same time, we do expose also to everyone

Lex Fridman (38:19.400)

an interesting chart that says,

Po-Shen Loh (38:21.000)

here's how many people you have

Lex Fridman (38:22.840)

that you're connected to directly.

Po-Shen Loh (38:24.400)

Here's how many at distance two,

Lex Fridman (38:26.720)

meaning via people.

Lex Fridman (38:27.880)

And then here's how many at distance three.

Lex Fridman (38:29.600)

And the reason we do that,

Po-Shen Loh (38:30.920)

is that actually ends up being a dynamic

Lex Fridman (38:32.880)

that also boosts adoption.

Po-Shen Loh (38:34.440)

It drives another feedback loop.

Lex Fridman (38:36.360)

The reason is because we saw, actually,

Po-Shen Loh (38:38.120)

when we deployed this in some universities,

Lex Fridman (38:40.520)

that when people see on their app

Po-Shen Loh (38:42.760)

that they are indirectly connected to hundreds

Lex Fridman (38:46.000)

or thousands of other people,

Po-Shen Loh (38:47.600)

they get excited and they tell other people,

Lex Fridman (38:49.120)

hey, let's download this app.

Lex Fridman (38:50.760)

But you know, we also saw in those examples,

Lex Fridman (38:52.960)

especially looking at the screenshots people gave,

Po-Shen Loh (38:55.480)

that is hit as soon as the typical person

Lex Fridman (38:58.600)

has two or three other direct connections on the system.

Po-Shen Loh (39:02.600)

Because that means that our app

Lex Fridman (39:04.440)

has reached a virality or not of two to three.

Po-Shen Loh (39:07.680)

The key is we were making a viral app to fight a virus

Lex Fridman (39:10.880)

spreading on the same network that the virus spreads on.

Lex Fridman (39:14.880)

So you're trying to out virus the virus.

Lex Fridman (39:17.120)

That's right.

Po-Shen Loh (39:17.960)

That's exactly right.

Lex Fridman (39:20.280)

Okay, great.

Lex Fridman (39:21.400)

What have you learned from this whole experience

Lex Fridman (39:23.640)

in terms of, let's say for COVID,

Lex Fridman (39:26.520)

but for future pandemics as well,

Lex Fridman (39:29.840)

is it possible to use the power information here

Po-Shen Loh (39:33.760)

of networked information as a virus spreads and travels

Lex Fridman (39:38.520)

in order to basically keep the society open?

Po-Shen Loh (39:41.400)

Is it possible for people to protect themselves

Lex Fridman (39:44.840)

with this information?

Po-Shen Loh (39:46.200)

Or do you still have to have most,

Lex Fridman (39:48.920)

like in this overarching policy

Lex Fridman (39:50.680)

of everybody should stay at home, that kind of thing?

Lex Fridman (39:53.680)

We are trying to answer that question right now.

Lex Fridman (39:55.400)

So the answer is we don't know yet,

Lex Fridman (39:57.600)

but that's actually why we're very happy

Po-Shen Loh (39:59.320)

that now the idea has started to become more widely known.

Lex Fridman (40:02.720)

And we're already starting to collaborate

Po-Shen Loh (40:04.640)

with epidemiologists.

Lex Fridman (40:06.520)

Again, I'm just a mathematician, right?

Lex Fridman (40:08.880)

And a mathematician should not be the person

Lex Fridman (40:11.040)

who is telling everybody, this will definitely work.

Lex Fridman (40:13.720)

But because of the potential power of this approach,

Lex Fridman (40:17.680)

especially the potential power

Po-Shen Loh (40:19.160)

of this being an end game for COVID,

Lex Fridman (40:22.880)

we have gotten the interest of real researchers.

Lex Fridman (40:26.240)

And we're now working together

Lex Fridman (40:27.720)

to try to actually understand the answer to that question.

Po-Shen Loh (40:30.240)

Because you see, there's a theory.

Lex Fridman (40:31.600)

So what I can share is the mathematics of,

Po-Shen Loh (40:34.160)

here's why there's some hope that this would work.

Lex Fridman (40:36.680)

And that's because I'm talking about end game now.

Po-Shen Loh (40:39.440)

End game means you have very few cases.

Lex Fridman (40:41.560)

But everywhere, we're always thinking,

Lex Fridman (40:43.600)

once there's few cases, then does that mean we now open up?

Lex Fridman (40:46.480)

Once you open up in the past, then the cases go up again

Po-Shen Loh (40:49.560)

until you have to lock down again.

Lex Fridman (40:51.440)

And now when we talk about the dynamic process that makes,

Po-Shen Loh (40:54.200)

it's guaranteeing you always have cases

Lex Fridman (40:55.880)

until you have the great vaccines,

Po-Shen Loh (40:57.160)

which is, we both got vaccinated, this is good.

Lex Fridman (41:00.680)

But at the same time, why I'm thinking

Po-Shen Loh (41:02.520)

this is still important is because we know

Lex Fridman (41:04.360)

that many vaccine makers have said

Po-Shen Loh (41:06.560)

they're preparing for the next dose next year.

Lex Fridman (41:09.760)

And if we have a perpetual thing

Po-Shen Loh (41:11.720)

where you just always need a new vaccine every year,

Lex Fridman (41:14.480)

it could actually be beneficial to make sure

Po-Shen Loh (41:16.280)

we have as many other techniques as possible

Lex Fridman (41:18.800)

for parts of the world that can't afford,

Po-Shen Loh (41:20.840)

for example, that kind of distribution.

Lex Fridman (41:23.160)

Yeah, so actually, no matter how deadly the virus is,

Po-Shen Loh (41:26.360)

no matter how many things,

Lex Fridman (41:27.720)

whether you have a vaccine or not,

Po-Shen Loh (41:29.680)

it's still useful to be having this information.

Lex Fridman (41:31.920)

Yes.

Po-Shen Loh (41:32.760)

Because to stay home or not, depending on how risk,

Lex Fridman (41:35.880)

I'm a big fan, just like you said, of having the freedom

Lex Fridman (41:39.160)

for you to decide how risk averse you wanna be, right?

Lex Fridman (41:43.240)

Depending on your own conditions,

Lex Fridman (41:44.400)

but also on the state of like what you,

Lex Fridman (41:47.240)

just how dangerously you like to live.

Lex Fridman (41:50.080)

So I think that actually makes a lot of sense.

Lex Fridman (41:51.960)

And I also think that since we're,

Po-Shen Loh (41:54.920)

when you think of disease spreading,

Lex Fridman (41:56.960)

it spreads in aggregate in the sense that

Po-Shen Loh (42:00.520)

if there are some people who maybe are more risk tolerant

Lex Fridman (42:04.680)

because of other things in their life,

Po-Shen Loh (42:06.480)

well, there might also be other people

Lex Fridman (42:08.040)

who are less risk tolerance.

Lex Fridman (42:09.800)

And then those people decide to isolate.

Lex Fridman (42:12.720)

But what matters is in the aggregate

Po-Shen Loh (42:14.520)

that this R naught of the infection spreading

Lex Fridman (42:17.680)

drops below one.

Lex Fridman (42:19.000)

And so the key is if you can empower people

Lex Fridman (42:21.200)

with that power to make that decision,

Po-Shen Loh (42:23.440)

you might actually still be able to drive

Lex Fridman (42:25.080)

that R naught down below one.

Po-Shen Loh (42:27.600)

Yeah, and also, this is me talking,

Lex Fridman (42:31.480)

people get a little bit nervous, I think,

Po-Shen Loh (42:33.720)

with information somehow mapping to privacy violation.

Lex Fridman (42:38.280)

But first of all, in the approach you're describing,

Po-Shen Loh (42:42.240)

that's respecting anonymity.

Lex Fridman (42:46.240)

But I would love to have information

Po-Shen Loh (42:49.240)

from the very beginning, from March and April of last year,

Lex Fridman (42:54.040)

almost like a map of like where it's risky

Lex Fridman (42:59.080)

and where it's not to go.

Lex Fridman (43:01.360)

And not map based on sort of the exact location of people,

Lex Fridman (43:05.200)

but where people usually hang out kind of thing.

Lex Fridman (43:07.560)

Just, and maybe not necessarily about actual location,

Lex Fridman (43:13.080)

but just maybe activities,

Lex Fridman (43:15.240)

like just to have information about what is good to do

Lex Fridman (43:19.680)

and not, in terms of like safety,

Lex Fridman (43:23.080)

is it okay to run outside and not,

Po-Shen Loh (43:25.440)

is it okay to go to a restaurant and not,

Lex Fridman (43:27.720)

I just feel like we're operating in the blind.

Lex Fridman (43:29.640)

And then what you had is a very imperfect signal,

Lex Fridman (43:33.880)

which is like basically politicians desperately trying

Po-Shen Loh (43:37.160)

to make statements about what is safe and not.

Lex Fridman (43:40.040)

They don't know what the heck they're doing.

Po-Shen Loh (43:41.600)

They have a bunch of smart scientists telling them stuff.

Lex Fridman (43:44.120)

And the scientists themselves also, very important,

Po-Shen Loh (43:47.720)

don't always know what they're doing.

Lex Fridman (43:49.600)

Epidemiology is not, is as much an art as a science.

Po-Shen Loh (43:54.600)

You're desperately trying to predict the future,

Lex Fridman (43:56.400)

which nobody can do.

Lex Fridman (43:57.960)

And then you're trying to speak with some level of authority.

Lex Fridman (44:01.200)

I mean, if I were to criticize scientists,

Po-Shen Loh (44:02.920)

they spoke with too much authority.

Lex Fridman (44:04.440)

It's okay to say, I'm not sure.

Lex Fridman (44:06.480)

But then they think like, if I say, I'm not sure,

Lex Fridman (44:10.640)

then there's going to be a distrust.

Lex Fridman (44:12.400)

What they realize is when you're wrong and you say,

Lex Fridman (44:14.480)

I'm sure, it's going to lead to more distrust.

Lex Fridman (44:16.840)

So there's this imperfect, like just chaotic,

Lex Fridman (44:19.760)

messy system of people trying to figure out

Po-Shen Loh (44:23.560)

with very little information.

Lex Fridman (44:25.320)

And what you're proposing is just a huge amount

Po-Shen Loh (44:27.880)

of information, and information is power.

Lex Fridman (44:31.080)

Is there challenges with adoption that you see

Lex Fridman (44:34.320)

in the future here?

Lex Fridman (44:36.240)

So there's, maybe we could speak to,

Po-Shen Loh (44:38.640)

there's approaches, I guess, from Google.

Lex Fridman (44:40.520)

There's different people that have tried

Po-Shen Loh (44:42.600)

similar kind of ideas.

Lex Fridman (44:44.800)

Not, you have quite a novel idea, actually.

Lex Fridman (44:49.600)

But speaking, the umbrella idea of contact tracing,

Lex Fridman (44:53.600)

is there something you can comment about

Lex Fridman (44:58.800)

why their approaches haven't been fully adopted?

Lex Fridman (45:02.040)

Is there challenges there?

Po-Shen Loh (45:03.200)

Is there reasons why Novid might be a better idea

Lex Fridman (45:06.400)

moving forward, in general, just about adoption?

Po-Shen Loh (45:09.240)

Yeah, so first of all, I want to say,

Lex Fridman (45:10.680)

I always have respect for the methods that other people use.

Lex Fridman (45:13.280)

And so it's good to see the other people I've been trying.

Lex Fridman (45:16.160)

But what we have noticed is that the difference

Po-Shen Loh (45:19.040)

between our value proposition to the user

Lex Fridman (45:22.360)

and the value proposition to the user delivered

Po-Shen Loh (45:24.320)

by everything that was made before is that,

Lex Fridman (45:27.800)

unfortunately, the action of installing

Po-Shen Loh (45:30.480)

a standard contact tracing app will then tell you

Lex Fridman (45:34.480)

after you have already been exposed to the disease

Lex Fridman (45:37.760)

so that you can protect other people from you.

Lex Fridman (45:40.640)

And what that does to your own direct probability

Po-Shen Loh (45:43.640)

of getting sick, if you think about it,

Lex Fridman (45:45.720)

suppose you were making the decision,

Lex Fridman (45:47.120)

should I or should I not install one of those apps?

Lex Fridman (45:50.040)

What does that do to your own probability of getting sick?

Po-Shen Loh (45:55.000)

It's close to zero.

Lex Fridman (45:56.320)

This is the sad thing you're speaking to, not sad.

Po-Shen Loh (46:00.680)

I suppose it's the way the world is.

Lex Fridman (46:03.280)

The only incentive there is to just help other people,

Po-Shen Loh (46:06.080)

I suppose, but a much stronger incentive

Lex Fridman (46:09.640)

is anything that allows you to help yourself.

Po-Shen Loh (46:13.160)

Yes, so what I'm saying is that,

Lex Fridman (46:15.520)

let's just say free market capitalism

Po-Shen Loh (46:17.160)

was not based on altruism, I think it's based on,

Lex Fridman (46:21.680)

if you make a system of incentives

Lex Fridman (46:23.280)

so that everybody trying to maximize their own situation

Lex Fridman (46:26.960)

somehow contributes to the whole,

Po-Shen Loh (46:28.760)

that's a game theoretic solution to a very hard problem.

Lex Fridman (46:31.800)

And so this is actually basically mechanism design,

Po-Shen Loh (46:34.200)

that we've basically come up with a different mechanism,

Lex Fridman (46:36.640)

different set of incentives,

Po-Shen Loh (46:38.280)

which incentivizes the adoption,

Lex Fridman (46:40.960)

because actually whenever we've been rolling it out,

Po-Shen Loh (46:43.120)

usually the first question we ask people,

Lex Fridman (46:45.240)

like say in a university is,

Lex Fridman (46:46.800)

do you know what Novid does?

Lex Fridman (46:48.160)

And most of them have read about the other apps

Lex Fridman (46:50.640)

and they say, Oh, Novid will tell you

Lex Fridman (46:51.920)

after you've been around someone so you can quarantine.

Lex Fridman (46:54.120)

And we have to explain to them,

Lex Fridman (46:55.400)

actually, Novid never wants to ask you to quarantine.

Po-Shen Loh (46:58.440)

That's not the principle.

Lex Fridman (46:59.280)

Our principle isn't based on that at all.

Po-Shen Loh (47:01.240)

We just want to let you know if something is coming close

Lex Fridman (47:04.560)

so that you can protect yourself.

Po-Shen Loh (47:07.360)

If you want.

Lex Fridman (47:08.200)

If you want, if you want, if you want.

Lex Fridman (47:09.320)

And then the quarantine is like, yes,

Lex Fridman (47:11.320)

in that case, if you're quarantining,

Po-Shen Loh (47:13.440)

it's because you're shutting the door from the inside,

Lex Fridman (47:16.240)

if that makes sense.

Po-Shen Loh (47:17.080)

Yes, exactly.

Lex Fridman (47:18.120)

Exactly.

Po-Shen Loh (47:18.960)

I mean, this is brilliant.

Lex Fridman (47:20.080)

So what do you think the future looks like

Lex Fridman (47:23.360)

for future pandemics?

Lex Fridman (47:24.560)

What's your plan with Novid?

Lex Fridman (47:26.680)

What's your plan with these set of ideas?

Lex Fridman (47:28.760)

I am actually still an academic and a researcher.

Lex Fridman (47:31.160)

So the biggest work I'm working on right now

Lex Fridman (47:33.360)

is to try to build as many collaborations

Po-Shen Loh (47:35.560)

with other public health researchers at other universities

Lex Fridman (47:39.120)

to actually work on pilot deployments together

Po-Shen Loh (47:42.160)

in various places.

Lex Fridman (47:43.040)

That's the goal.

Po-Shen Loh (47:44.080)

That's actually ongoing work right now.

Lex Fridman (47:45.960)

And so, for example, if anyone's watching this

Lex Fridman (47:47.800)

and you happen to be a public health researcher

Lex Fridman (47:49.760)

and you want to be involved in something like this,

Po-Shen Loh (47:52.360)

I'm just gonna say, I'm still incentive thinking.

Lex Fridman (47:55.240)

There's something in it for the researchers too.

Po-Shen Loh (47:57.320)

This could open up an entire new way

Lex Fridman (47:59.360)

of controlling disease.

Po-Shen Loh (48:00.360)

That's my hope.

Lex Fridman (48:01.920)

I mean, it might actually be true.

Lex Fridman (48:03.560)

And people who are involved in figuring out

Lex Fridman (48:06.360)

how to make this work,

Po-Shen Loh (48:08.040)

well, it could actually be good for their careers too.

Lex Fridman (48:09.960)

I always have to think like,

Po-Shen Loh (48:11.080)

if a researcher was getting involved,

Lex Fridman (48:12.680)

what are they getting out of it?

Po-Shen Loh (48:14.240)

Oh, so you mean like from a research perspective,

Lex Fridman (48:16.600)

you can like publications and sets of ideas

Po-Shen Loh (48:20.200)

about how to, from a sort of network theory perspective,

Lex Fridman (48:27.000)

understand how we control the spread of a pandemic.

Po-Shen Loh (48:30.080)

Yes, and what I'm doing right now

Lex Fridman (48:31.680)

is this is basically interdisciplinary research

Po-Shen Loh (48:33.720)

where maybe our side is bringing the technology

Lex Fridman (48:35.960)

and the network theory,

Lex Fridman (48:37.040)

and the missing parts are epidemiology

Lex Fridman (48:39.200)

and public health expertise.

Lex Fridman (48:40.960)

And if the two things start to join,

Lex Fridman (48:42.840)

also because everywhere that you deploy,

Po-Shen Loh (48:45.320)

let's just say that the world is different

Lex Fridman (48:46.800)

in the Philippines as it is in the United States.

Lex Fridman (48:49.400)

And just the natures of the locality

Lex Fridman (48:52.040)

would mean that someone like me

Po-Shen Loh (48:53.560)

should not be trying to figure out how to do that.

Lex Fridman (48:55.280)

But if we can work with the researchers

Po-Shen Loh (48:56.840)

who are based there,

Lex Fridman (48:57.840)

now suddenly we might come up with a solution

Po-Shen Loh (48:59.720)

that will help scale in parts of the world

Lex Fridman (49:01.960)

where they aren't all getting the Moderna and Pfizer vaccines

Po-Shen Loh (49:04.680)

which cost like $20 a pop in the US.

Lex Fridman (49:07.400)

So if they want to participate,

Lex Fridman (49:09.240)

who do they reach out to?

Lex Fridman (49:10.720)

Oh, that would just be us.

Po-Shen Loh (49:11.640)

I mean, the novid.org website has...

Lex Fridman (49:13.600)

Novid.org.

Po-Shen Loh (49:14.440)

It has a feedback reach out form.

Lex Fridman (49:16.880)

And actually we are, I mean, again,

Po-Shen Loh (49:18.840)

this is the DNA of being a researcher.

Lex Fridman (49:21.040)

I am actually very excited by the idea

Po-Shen Loh (49:23.320)

that this could contribute knowledge

Lex Fridman (49:25.560)

that will outlast all of our generations,

Po-Shen Loh (49:28.240)

like all of our lifetimes.

Lex Fridman (49:29.960)

There you go.

Po-Shen Loh (49:30.800)

Reach out to novid.org.

Lex Fridman (49:34.440)

What about individual people?

Lex Fridman (49:36.040)

Should they install the app and try it out?

Lex Fridman (49:37.680)

Or is this really geographically restricted?

Po-Shen Loh (49:40.080)

Oh, yeah, I didn't come on here to tell everyone

Lex Fridman (49:42.040)

to install the app.

Po-Shen Loh (49:42.880)

I did not come to tell everyone to install the app

Lex Fridman (49:44.760)

because it works best

Po-Shen Loh (49:46.440)

if your local health authority is working with us.

Lex Fridman (49:49.400)

Gotcha.

Po-Shen Loh (49:50.240)

There's a reason.

Lex Fridman (49:51.080)

It's because, this is back to the game theory.

Po-Shen Loh (49:54.720)

If anyone could just say, I'm positive,

Lex Fridman (49:58.320)

the high school senior prank would be to say that

Po-Shen Loh (50:01.840)

we have a massive outbreak on finals week.

Lex Fridman (50:03.960)

Let's not have final exams.

Lex Fridman (50:05.240)

So the way that our system works,

Lex Fridman (50:06.640)

it actually borrows some ideas, not borrows,

Po-Shen Loh (50:08.760)

we came up with them independently.

Lex Fridman (50:10.280)

But this idea is similar to what Google and Apple do,

Po-Shen Loh (50:13.240)

which is that if the local health authority

Lex Fridman (50:14.880)

is working with this, they can,

Po-Shen Loh (50:16.800)

for everyone who's positive,

Lex Fridman (50:17.960)

give them a passcode that expires in a short time.

Lex Fridman (50:20.680)

So for ours, if you're on the app and saying, I'm positive,

Lex Fridman (50:23.600)

you can either just say that,

Lex Fridman (50:25.120)

and that's called unverified,

Lex Fridman (50:26.800)

or you can enter in one of these codes

Po-Shen Loh (50:28.360)

that you got from the local health authority.

Lex Fridman (50:30.280)

So basically, for anyone who's watching this,

Po-Shen Loh (50:32.440)

it's not that you should just go and download it

Lex Fridman (50:34.000)

unless you want to go and look at it.

Po-Shen Loh (50:35.280)

That's cool.

Lex Fridman (50:36.120)

But if you, on the other hand,

Po-Shen Loh (50:37.480)

if you happen to know anyone at the local health authority,

Lex Fridman (50:39.880)

which is trying to figure out how to handle COVID,

Po-Shen Loh (50:42.640)

well then, I mean, we'd be very happy

Lex Fridman (50:44.280)

to also work with you.

Po-Shen Loh (50:46.440)

Gotcha.

Lex Fridman (50:47.280)

So the verified there is really important

Po-Shen Loh (50:49.120)

because you're maintaining anonymity.

Lex Fridman (50:51.320)

And because of that,

Po-Shen Loh (50:52.160)

you have to have some source of verification

Lex Fridman (50:54.600)

in order to make sure that it's not possible to manipulate

Po-Shen Loh (50:59.080)

because it's ultimately about trust and information.

Lex Fridman (51:02.040)

So it could be, verification is really important there.

Lex Fridman (51:06.120)

So basically, individual people should

Lex Fridman (51:09.480)

ask their local health authorities

Po-Shen Loh (51:11.280)

to sign up to contact you.

Lex Fridman (51:15.040)

I hope this spreads.

Po-Shen Loh (51:16.320)

I hope this spreads for future pandemics

Lex Fridman (51:18.680)

because I'm really, it's the amount,

Po-Shen Loh (51:21.440)

the millions of people who are hurt by this,

Lex Fridman (51:25.600)

I think our response to the virus,

Po-Shen Loh (51:28.720)

economically speaking,

Lex Fridman (51:30.040)

the number of people who lost their dream,

Po-Shen Loh (51:32.520)

lost their jobs, but also lost their dream.

Lex Fridman (51:35.120)

Entrepreneurs, jobs often give meaning.

Po-Shen Loh (51:38.400)

There's people who financially and psychologically

Lex Fridman (51:41.080)

are suffering because of our,

Po-Shen Loh (51:43.760)

I'll say, incompetent response to the virus

Lex Fridman (51:47.440)

across the world, but certainly the United States,

Po-Shen Loh (51:49.760)

that should be the beacon of entrepreneurial hope

Lex Fridman (51:53.960)

for the world.

Lex Fridman (51:54.800)

So I hope that we'll be able to respond

Lex Fridman (52:00.600)

to these kinds of events much better in the future.

Lex Fridman (52:02.760)

And this is exactly the right kind of idea.

Lex Fridman (52:05.040)

And now is the time to do the investment.

Po-Shen Loh (52:08.280)

Let's step back to the beauty of mathematics.

Lex Fridman (52:13.040)

Maybe ask the big, silly question first,

Lex Fridman (52:16.040)

which is, what do you find beautiful about mathematics?

Lex Fridman (52:20.840)

I think that being able to look at a complicated problem,

Po-Shen Loh (52:26.880)

which looks unsolvable,

Lex Fridman (52:28.560)

and then to be able to change the perspective

Po-Shen Loh (52:30.800)

to come from a different angle

Lex Fridman (52:32.480)

and suddenly see that there's a nice solution.

Po-Shen Loh (52:36.160)

I don't mean that every problem in math

Lex Fridman (52:37.920)

is supposed to be this way,

Lex Fridman (52:39.160)

but I think that these reframings

Lex Fridman (52:40.920)

and changing of perspectives

Po-Shen Loh (52:42.160)

that cause difficult things to get simplified

Lex Fridman (52:44.640)

and crystallized and factored in certain ways is beautiful.

Po-Shen Loh (52:48.600)

Actually, that's related to what we were just talking about

Lex Fridman (52:50.920)

with even this fighting pandemics.

Po-Shen Loh (52:52.640)

The crystal idea was just quantify proximity

Lex Fridman (52:57.840)

by the number of relationships in the physical network,

Lex Fridman (53:01.680)

instead of just by the feet and meters, right?

Lex Fridman (53:04.960)

It's just that if you change that perspective,

Po-Shen Loh (53:07.360)

now all of these things follow.

Lex Fridman (53:09.240)

And so mathematics to me is beautiful

Po-Shen Loh (53:12.280)

in the pure sense just for that.

Lex Fridman (53:15.000)

Yeah, it's quite interesting to see a human civilization

Po-Shen Loh (53:17.560)

as a network, as a graph,

Lex Fridman (53:20.040)

and our relationships as kind of edges in that graph.

Lex Fridman (53:25.240)

And to then do, outside of just pandemic,

Lex Fridman (53:29.360)

do interesting inferences based on that.

Po-Shen Loh (53:33.760)

This is true for like Twitter, social networks and so on,

Lex Fridman (53:36.920)

how we expand the kind of things we talk about,

Po-Shen Loh (53:40.080)

think about sort of politically,

Lex Fridman (53:42.200)

if you have this little bubble, quote unquote,

Po-Shen Loh (53:44.680)

of ideas that you play with,

Lex Fridman (53:46.880)

it's nice from a recommender system perspective,

Lex Fridman (53:50.200)

how do you jump out of those bubbles?

Lex Fridman (53:52.000)

It's really fascinating.

Po-Shen Loh (53:53.600)

YouTube was working on that, Twitter's working on that,

Lex Fridman (53:57.520)

but not always so successfully,

Lex Fridman (53:59.680)

but there's a lot of interesting work

Lex Fridman (54:02.680)

from a mathematical and a psychological,

Po-Shen Loh (54:05.160)

sociological perspective there within those graphs.

Lex Fridman (54:09.520)

But if we look at the cleanest formulation of that,

Po-Shen Loh (54:13.360)

of looking at a problem from a different perspective,

Lex Fridman (54:16.280)

you're also involved

Po-Shen Loh (54:17.360)

with the International Mathematics Olympiad,

Lex Fridman (54:20.200)

which takes small, clean problems that are really hard,

Lex Fridman (54:27.640)

but once you look at them differently, can become easy.

Lex Fridman (54:31.120)

But that little jump of innovation is the entire trick.

Lex Fridman (54:36.320)

So maybe at the high level,

Lex Fridman (54:38.560)

can you say what is the International Mathematical Olympiad?

Po-Shen Loh (54:41.480)

Sure, so this is the competition

Lex Fridman (54:44.600)

for people who aren't yet in college, math competition,

Po-Shen Loh (54:47.840)

which is the most prestigious one in the entire world.

Lex Fridman (54:50.720)

It's the Olympics of mathematics,

Lex Fridman (54:52.720)

but only for people who aren't yet in college.

Lex Fridman (54:55.040)

Now, the kinds of questions that they ask you to do

Po-Shen Loh (54:58.000)

are not computational.

Lex Fridman (54:59.600)

Usually you're not supposed to find that the answer is 42.

Lex Fridman (55:02.400)

Right?

Lex Fridman (55:03.760)

Instead, you're supposed to explain why something is true.

Lex Fridman (55:07.280)

And the problem is that at the beginning,

Lex Fridman (55:09.840)

when you look at each of the questions,

Po-Shen Loh (55:11.560)

first of all, you have four and a half hours

Lex Fridman (55:13.560)

to solve three questions, and this is one day,

Lex Fridman (55:16.080)

and then you have a second day,

Lex Fridman (55:16.920)

which is four and a half hours, three questions.

Lex Fridman (55:19.360)

But when you look at the questions,

Lex Fridman (55:20.600)

they're all asking you,

Po-Shen Loh (55:21.440)

explain why the following thing is true,

Lex Fridman (55:23.240)

which you've never seen before.

Lex Fridman (55:25.160)

And by the way, even though there are six questions,

Lex Fridman (55:27.360)

if you solve any one of them, you're a genius

Lex Fridman (55:29.120)

and you get an honorable mention.

Lex Fridman (55:30.320)

So this is hard to solve.

Lex Fridman (55:32.560)

So what about, is it one person, is it a team?

Lex Fridman (55:35.280)

Ah, so each country can send six people

Lex Fridman (55:38.720)

and the score of the country is actually unofficial.

Lex Fridman (55:42.320)

There's not an official country versus country system,

Po-Shen Loh (55:45.360)

although everyone just adds up the point scores

Lex Fridman (55:47.480)

of the six people and they say,

Lex Fridman (55:48.760)

well, now which country stacked up where?

Lex Fridman (55:51.440)

Yeah, so maybe as a side comment,

Po-Shen Loh (55:53.400)

I should say that there's a bunch of countries,

Lex Fridman (55:56.560)

including the former Soviet Union and Russia,

Po-Shen Loh (55:59.480)

where I grew up, where this is one of the

Lex Fridman (56:04.320)

most important competitions that the country participates in.

Po-Shen Loh (56:08.360)

It was a source of pride for a lot of the country.

Lex Fridman (56:11.880)

You look at the Olympic sports,

Po-Shen Loh (56:14.400)

like wrestling, weightlifting,

Lex Fridman (56:17.080)

there's certain sports and hockey

Po-Shen Loh (56:20.280)

that Russia and the Soviet Union truly took pride in.

Lex Fridman (56:24.960)

And actually the Mathematical Olympiad,

Po-Shen Loh (56:28.600)

it was one of them for many years.

Lex Fridman (56:30.880)

It's still one of them.

Lex Fridman (56:32.560)

And that's kind of fascinating.

Lex Fridman (56:33.720)

We don't think about it this way in the United States.

Po-Shen Loh (56:36.680)

Maybe you can correct me if I'm wrong,

Lex Fridman (56:38.200)

but it's not nearly as popular in the United States

Po-Shen Loh (56:42.360)

in terms of its integration into the culture,

Lex Fridman (56:45.560)

into just basic conversation, into the pride.

Po-Shen Loh (56:49.080)

Like, if you won an Olympic gold medal

Lex Fridman (56:52.360)

or if you win the Super Bowl, you can walk around proud.

Po-Shen Loh (56:56.080)

I think that was the case

Lex Fridman (56:57.040)

with the Mathematical Olympiad in Russia.

Po-Shen Loh (56:59.200)

Not as much the case in the United States, I think.

Lex Fridman (57:03.040)

So I just wanna give that a little aside

Po-Shen Loh (57:04.880)

because beating anybody from Russia,

Lex Fridman (57:07.520)

from the Eastern Republic or from China

Po-Shen Loh (57:09.400)

is very, very difficult.

Lex Fridman (57:11.840)

Like, if I remember correctly,

Po-Shen Loh (57:14.880)

there's people, this was a multiyear training process.

Lex Fridman (57:18.880)

They train hard.

Lex Fridman (57:20.680)

And this is everything that they're focused on.

Lex Fridman (57:25.040)

My dad was a participant in this.

Lex Fridman (57:29.320)

And it's, I mean, it's as serious as Olympic sports.

Lex Fridman (57:33.120)

You think about like gymnastics,

Po-Shen Loh (57:34.520)

like young athletes participating in gymnastics.

Lex Fridman (57:36.600)

This is as serious as that, if not more serious.

Lex Fridman (57:38.920)

So I just wanna give that a little bit of context

Lex Fridman (57:41.360)

because we're talking about serious high level math,

Po-Shen Loh (57:44.640)

athletics almost here.

Lex Fridman (57:46.360)

Yeah, and actually I also think that it made sense

Po-Shen Loh (57:49.800)

from the Soviet Union's perspective

Lex Fridman (57:51.400)

because if you look at what these people do eventually,

Po-Shen Loh (57:55.480)

even though, let's look at the USSR's

Lex Fridman (57:58.800)

International Math Olympiad record.

Po-Shen Loh (58:00.600)

Even though they, I say, even though they won

Lex Fridman (58:03.080)

a lot of awards at the high school thing,

Po-Shen Loh (58:05.280)

many of them went on to do incredible things

Lex Fridman (58:07.800)

in research mathematics or research other things.

Lex Fridman (58:10.600)

And that's showing the generalization,

Lex Fridman (58:13.000)

generalizability of what they were working on.

Po-Shen Loh (58:15.960)

Because ultimately we're just playing with ideas

Lex Fridman (58:20.160)

of how to prove things.

Lex Fridman (58:22.360)

And if you get pretty good at inventing creative ways

Lex Fridman (58:26.040)

to turn problems apart, split them apart,

Po-Shen Loh (58:29.040)

observe neat ways to turn messy things into simple crystals.

Lex Fridman (58:34.080)

Well, if you're gonna try to solve any real problem

Po-Shen Loh (58:36.200)

in the real world, that could be a really handy tool too.

Lex Fridman (58:39.240)

So I don't think it was a bad investment.

Po-Shen Loh (58:41.160)

I think it clearly worked well for Soviet Union.

Lex Fridman (58:44.920)

Yeah, so this is interesting.

Po-Shen Loh (58:47.120)

People sometimes ask me, you know,

Lex Fridman (58:48.600)

you go up and under communism, you know,

Lex Fridman (58:52.080)

was there anything good about communism?

Lex Fridman (58:55.560)

And it's difficult for me to talk about it

Po-Shen Loh (58:58.120)

because it's not, communism is one of those things

Lex Fridman (59:00.800)

that's looked down on like without,

Po-Shen Loh (59:02.920)

in absolutist terms currently.

Lex Fridman (59:05.360)

But you could still, in my perspective,

Po-Shen Loh (59:07.360)

talk about the actual, forget communism

Lex Fridman (59:09.520)

or whatever the actual term is,

Lex Fridman (59:11.760)

but you know, certain ways that the society functioned

Lex Fridman (59:16.760)

that we can learn lessons from.

Lex Fridman (59:18.120)

And one of the things in the Soviet Union

Lex Fridman (59:20.280)

that was highly prized is knowledge,

Po-Shen Loh (59:25.160)

not even knowledge, it's wisdom

Lex Fridman (59:27.280)

and the skill of invention, of innovation at a young age.

Lex Fridman (59:34.120)

So we're not talking about a selection process

Lex Fridman (59:37.240)

where you pick the best students in the school

Po-Shen Loh (59:40.440)

to do the mathematics or to read literature.

Lex Fridman (59:44.120)

It's like, everybody did it.

Po-Shen Loh (59:47.000)

Everybody, it was almost treated

Lex Fridman (59:49.920)

as if anyone could be the next Einstein,

Po-Shen Loh (59:53.000)

anybody could be the next, I don't know,

Lex Fridman (59:55.560)

Hemingway, James Joyce.

Lex Fridman (59:56.920)

And so you're forcing an education on the populace