Vladimir Vapnik: Predicates, Invariants, and the Essence of Intelligence
音乐与艺术心理与人性AI 与机器学习技术与编程生物与进化
📋 章节目录
暂无章节信息
🔑 关键词
functionpredicatedonfunctionspredicatessymmetryrecognitionconvergenceideasdigitintelligencedataimagessmallhumantalkadmissiblemusiclookingunderstanding
💬 精彩语录
暂无语录
🎙️ 完整对话（1766 条）
Lex Fridman (00:00.000)
The following is a conversation with Vladimir Vapnik, part two, the second
Lex Fridman (00:05.280)
time we spoke on the podcast.
Lex Fridman (00:07.280)
He's the coinventor of support vector machines, support vector clustering, VC
Lex Fridman (00:11.440)
theory, and many foundational ideas and statistical learning.
Lex Fridman (00:14.960)
He was born in the Soviet Union, worked at the Institute of Control Sciences
Vladimir Vapnik (00:19.320)
in Moscow, then in the US, worked at AT&T, NEC labs, Facebook AI research,
Lex Fridman (00:26.080)
and now is a professor at Columbia University.
Vladimir Vapnik (00:28.640)
His work has been cited over 200,000 times.
Lex Fridman (00:32.320)
The first time we spoke on the podcast was just over a year
Vladimir Vapnik (00:35.040)
ago, one of the early episodes.
Lex Fridman (00:38.240)
This time we spoke after a lecture he gave titled complete statistical theory
Vladimir Vapnik (00:42.760)
of learning as part of the MIT series of lectures on deep learning
Lex Fridman (00:46.720)
and AI that I organized.
Vladimir Vapnik (00:49.520)
I'll release the video of the lecture in the next few days.
Lex Fridman (00:53.040)
This podcast and lecture are independent from each other, so you don't need
Vladimir Vapnik (00:56.840)
one to understand the other.
Lex Fridman (00:59.000)
The lecture is quite technical and math heavy, so if you do watch both, I
Vladimir Vapnik (01:04.040)
recommend listening to this podcast first, since the podcast is
Lex Fridman (01:07.320)
probably a bit more accessible.
Vladimir Vapnik (01:10.800)
This is the artificial intelligence podcast.
Lex Fridman (01:13.560)
If you enjoy it, subscribe on YouTube, give it five stars on Apple podcasts,
Vladimir Vapnik (01:17.520)
support it on Patreon, or simply connect with me on Twitter
Lex Fridman (01:20.720)
at Lex Friedman spelled F R I D M A N.
Vladimir Vapnik (01:23.680)
As usual, I'll do one or two minutes of ads now and never any ads in
Lex Fridman (01:27.800)
the middle that can break the flow of the conversation.
Vladimir Vapnik (01:30.440)
I hope that works for you and doesn't hurt the listening experience.
Vladimir Vapnik (01:35.080)
This show is presented by Cash App, the number one finance app in the app store.
Vladimir Vapnik (01:39.480)
When you get it, use code LexPodcast.
Lex Fridman (01:42.760)
Cash App lets you send money to friends, buy Bitcoin, and invest in the
Vladimir Vapnik (01:46.440)
stock market with as little as $1.
Lex Fridman (01:48.680)
Broker services are provided by Cash App Investing, a subsidiary of Square
Lex Fridman (01:52.720)
and member SIPC, since Cash App allows you to send and receive money
Vladimir Vapnik (01:57.520)
digitally, peer to peer, and security in all digital transactions is very important.
Vladimir Vapnik (02:02.400)
Let me mention that PCI data security standard, PCI DSS level one,
Lex Fridman (02:07.760)
that Cash App is compliant with.
Vladimir Vapnik (02:10.920)
I'm a big fan of standards for safety and security and PCI DSS is a good
Lex Fridman (02:16.480)
example of that, where a bunch of competitors got together and agreed
Vladimir Vapnik (02:20.480)
that there needs to be a global standard around the security of transactions.
Lex Fridman (02:24.480)
Now we just need to do the same for autonomous vehicles
Lex Fridman (02:27.480)
and AI systems in general.
Lex Fridman (02:30.000)
So again, if you get Cash App from the app store or Google Play and use the code
Vladimir Vapnik (02:34.200)
LexPodcast, you get $10 and Cash App will also donate $10 to FIRST, one of my
Lex Fridman (02:40.240)
favorite organizations that is helping to advance robotics and STEM education
Vladimir Vapnik (02:45.040)
for young people around the world.
Lex Fridman (02:46.800)
And now here's my conversation with Vladimir Vapnik.
Vladimir Vapnik (02:52.440)
You and I talked about Alan Turing yesterday a little bit and that he, as the
Lex Fridman (02:58.040)
father of artificial intelligence, may have instilled in our field, an ethic
Vladimir Vapnik (03:02.080)
of engineering and not science, seeking more to build intelligence
Lex Fridman (03:06.560)
rather than to understand it.
Lex Fridman (03:09.160)
What do you think is the difference between these two paths of engineering
Lex Fridman (03:13.760)
intelligence and the science of intelligence?
Vladimir Vapnik (03:18.120)
It's a completely different story.
Lex Fridman (03:20.520)
Engineering is a mutation of human activity.
Vladimir Vapnik (03:25.320)
You have to make a device which behaves as humans behave, have all the functions
Lex Fridman (03:34.640)
of humans.
Vladimir Vapnik (03:36.120)
It doesn't matter how you do it, but to understand what is intelligence,
Lex Fridman (03:41.360)
but to understand what is intelligence about, it's quite a different problem.
Lex Fridman (03:48.920)
So I think, I believe that it's somehow related to the predicate we talked
Lex Fridman (03:55.160)
yesterday about, because look at the Vladimir Propp's idea.
Vladimir Vapnik (04:04.760)
He just found 31 here, predicates, he called it units, which can explain
Lex Fridman (04:17.600)
human behavior, at least in Russian tales.
Vladimir Vapnik (04:20.760)
You look at Russian tales and derive from that.
Lex Fridman (04:24.840)
And then people realize that it's more wide than in Russian tales.
Vladimir Vapnik (04:29.480)
It is in TV, in movie serials and so on and so on.
Lex Fridman (04:33.720)
So you're talking about Vladimir Propp, who in 1928 published a book,
Vladimir Vapnik (04:39.960)
Morphology of the Folktale, describing 31 predicates that have this kind of
Lex Fridman (04:46.400)
sequential structure that a lot of the stories, narratives follow in Russian
Vladimir Vapnik (04:53.320)
folklore and in other contexts.
Lex Fridman (04:54.960)
We'll talk about it.
Vladimir Vapnik (04:56.040)
I'd like to talk about predicates in a focused way, but let me, if you allow
Lex Fridman (05:00.400)
me to stay zoomed out on our friend, Alan Turing, and, you know, he inspired
Vladimir Vapnik (05:06.600)
a generation with the imitation game.
Lex Fridman (05:10.080)
Yes.
Lex Fridman (05:11.560)
Do you think if we can linger on that a little bit longer, do you think we can
Lex Fridman (05:17.480)
learn, do you think learning to imitate intelligence can get us closer to the
Lex Fridman (05:22.960)
science, to understanding intelligence?
Lex Fridman (05:24.920)
So why do you think imitation is so far from understanding?
Vladimir Vapnik (05:32.200)
I think that it is different between you have different goals.
Lex Fridman (05:37.000)
So your goal is to create something, something useful.
Vladimir Vapnik (05:43.080)
Yeah.
Lex Fridman (05:43.560)
And that is great.
Lex Fridman (05:45.400)
And you can see how much things was done and I believe that it will be done even
Lex Fridman (05:51.240)
more, it's self driving cars and also the business, it is great.
Lex Fridman (05:57.920)
And it was inspired by Turing's vision.
Lex Fridman (06:02.640)
But understanding is very difficult.
Vladimir Vapnik (06:05.000)
It's more or less philosophical category.
Lex Fridman (06:07.840)
What means understand the world?
Vladimir Vapnik (06:10.800)
I believe in scheme which starts from Plato, that there exists world of ideas.
Vladimir Vapnik (06:18.040)
I believe that intelligence, it is world of ideas, but it is world of pure ideas.
Lex Fridman (06:24.840)
And when you combine them with reality things, it creates, as in my case,
Lex Fridman (06:34.400)
invariants, which is very specific.
Lex Fridman (06:37.520)
And that's, I believe, the combination of ideas in way to constructing invariants.
Lex Fridman (06:47.320)
Constructing invariant is intelligence.
Lex Fridman (06:49.760)
But first of all, predicate, if you know, predicate and hopefully
Lex Fridman (06:56.080)
then not too much predicate exists.
Vladimir Vapnik (07:00.760)
For example, 31 predicate for human behavior, it is not a lot.
Lex Fridman (07:06.040)
Vladimir Propp used 31, you can even call them predicate, 31
Vladimir Vapnik (07:12.720)
predicates to describe stories, narratives.
Lex Fridman (07:17.640)
Do you think human behavior, how much of human behavior, how much of our
Vladimir Vapnik (07:22.560)
world, our universe, all the things that matter in our existence can be
Lex Fridman (07:28.000)
summarized in predicates of the kind that Propp was working with?
Vladimir Vapnik (07:32.600)
I think that we have a lot of form of behavior, but I think that
Lex Fridman (07:38.760)
predicate is much less because even in this example, which I gave you
Vladimir Vapnik (07:43.840)
yesterday, you saw that predicate can be, one predicate can construct many
Lex Fridman (07:55.000)
different invariants depending on your data.
Vladimir Vapnik (07:59.360)
They're applying to different data and they give different invariants.
Lex Fridman (08:04.200)
So, but pure ideas, maybe not so much.
Vladimir Vapnik (08:08.600)
Not so many.
Lex Fridman (08:09.880)
I don't know about that, but my guess, I hope that's why challenge
Vladimir Vapnik (08:15.000)
about digit recognition, how much you need.
Lex Fridman (08:19.600)
I think we'll talk about computer vision and 2D images a little bit
Vladimir Vapnik (08:23.560)
in your challenge.
Lex Fridman (08:24.800)
That's exactly about intelligence.
Vladimir Vapnik (08:26.720)
That's exactly, that's exactly about, no, that hopes to be exactly about
Lex Fridman (08:33.880)
the spirit of intelligence in the simplest possible way.
Vladimir Vapnik (08:37.160)
Yeah, absolutely you should start the simplest way, otherwise you
Lex Fridman (08:40.760)
will not be able to do it.
Vladimir Vapnik (08:42.320)
Well, there's an open question whether starting at the MNIST digit
Lex Fridman (08:46.680)
recognition is a step towards intelligence or it's an entirely different thing.
Vladimir Vapnik (08:52.320)
I think that to beat records using say 100, 200 times less examples,
Lex Fridman (08:59.360)
you need intelligence.
Vladimir Vapnik (09:00.360)
You need intelligence.
Lex Fridman (09:01.200)
So let's, because you use this term and it would be nice, I'd like to
Vladimir Vapnik (09:05.800)
ask simple, maybe even dumb questions.
Lex Fridman (09:09.640)
Let's start with a predicate.
Lex Fridman (09:12.520)
In terms of terms and how you think about it, what is a predicate?
Lex Fridman (09:17.160)
I don't know.
Vladimir Vapnik (09:18.520)
I have a feeling formally they exist, but I believe that predicate for
Lex Fridman (09:26.440)
2D images, one of them is symmetry.
Vladimir Vapnik (09:31.960)
Hold on a second.
Lex Fridman (09:32.560)
Sorry.
Vladimir Vapnik (09:32.960)
Sorry, sorry to interrupt and pull you back.
Lex Fridman (09:36.440)
At the simplest level, we're not even, we're not being profound currently.
Vladimir Vapnik (09:40.680)
A predicate is a statement of something that is true.
Lex Fridman (09:44.880)
Yes.
Lex Fridman (09:46.600)
Do you think of predicates as somehow probabilistic in nature or is this binary?
Lex Fridman (09:54.640)
This is truly constraints of logical statements about the world.
Vladimir Vapnik (09:59.840)
In my definition, the simplest predicate is function.
Vladimir Vapnik (10:03.800)
Function, and you can use this function to make inner product that is predicate.
Lex Fridman (10:10.480)
What's the input and what's the output of the function?
Lex Fridman (10:13.600)
Input is X, something which is input in reality.
Vladimir Vapnik (10:18.240)
Say if you consider digit recognition, it pixel space input, but it is
Vladimir Vapnik (10:25.440)
function which in pixel space, but it can be any function from pixel space and you
Vladimir Vapnik (10:36.240)
choose, and I believe that there are several functions which is important for
Lex Fridman (10:43.160)
understanding of images.
Vladimir Vapnik (10:46.400)
One of them is symmetry.
Lex Fridman (10:48.240)
It's not so simple construction as I described with the derivative, with all
Vladimir Vapnik (10:53.720)
this stuff, but another, I believe, I don't know how many, is how well
Lex Fridman (10:59.600)
structurized is picture.
Lex Fridman (11:03.240)
Structurized?
Lex Fridman (11:04.280)
Yeah.
Lex Fridman (11:04.840)
What do you mean by structurized?
Lex Fridman (11:06.960)
It is formal definition.
Vladimir Vapnik (11:09.040)
Say something heavy on the left corner, not so heavy in the middle and so on.
Lex Fridman (11:17.040)
You describe in general concept of what you assume.
Vladimir Vapnik (11:21.840)
Concepts, some kind of universal concepts.
Lex Fridman (11:25.200)
Yeah, but I don't know how to formalize this.
Lex Fridman (11:29.160)
Do you?
Lex Fridman (11:29.840)
So this is the thing.
Vladimir Vapnik (11:31.560)
There's a million ways we can talk about this.
Lex Fridman (11:33.600)
I'll keep bringing it up, but we humans have such concepts when we look at
Vladimir Vapnik (11:40.000)
digits, but it's hard to put them, just like you're saying now, it's
Lex Fridman (11:44.000)
hard to put them into words.
Vladimir Vapnik (11:45.480)
You know, that is example, when critics in music, trying to describe music,
Lex Fridman (11:55.440)
they use predicate and not too many predicate, but in different combination,
Lex Fridman (12:02.600)
but they have some special words for describing music and the same
Lex Fridman (12:10.440)
should be for images, but maybe there are critics who understand essence
Vladimir Vapnik (12:16.920)
of what this image is about.
Lex Fridman (12:20.960)
Do you think there exists critics who can summarize the essence of
Lex Fridman (12:26.960)
images, human beings?
Lex Fridman (12:29.120)
I hope so, yes, but that...
Vladimir Vapnik (12:32.440)
Explicitly state them on paper.
Lex Fridman (12:34.520)
The fundamental question I'm asking is, do you think there exists a small
Lex Fridman (12:41.840)
set of predicates that will summarize images?
Lex Fridman (12:45.040)
It feels to our mind, like it does, that the concept of what makes a two
Lex Fridman (12:50.840)
and a three and a four...
Lex Fridman (12:53.000)
No, no, no, it's not on this level.
Vladimir Vapnik (12:58.040)
It should not describe two, three, four.
Lex Fridman (13:01.240)
It describes some construction, which allow you to create invariance.
Lex Fridman (13:08.040)
And invariance, sorry to stick on this, but terminology.
Lex Fridman (13:12.360)
Invariance, it is property of your image.
Vladimir Vapnik (13:21.040)
Say, I can say, looking on my image, it is more or less symmetric.
Lex Fridman (13:27.760)
Looking on my image, it is more or less symmetric, and I can give you value
Vladimir Vapnik (13:33.360)
of symmetry, say, level of symmetry, using this function which I gave
Lex Fridman (13:40.560)
yesterday. And you can describe that your image has these characteristics
Vladimir Vapnik (13:51.560)
exactly in the way how musical critics describe music.
Lex Fridman (13:56.640)
So, but this is invariant applied to specific data, to specific music,
Vladimir Vapnik (14:05.400)
to something.
Lex Fridman (14:07.640)
I strongly believe in this plot ideas that there exists world of predicate
Lex Fridman (14:14.960)
and world of reality, and predicate and reality is somehow connected,
Lex Fridman (14:20.160)
and you have to know that.
Vladimir Vapnik (14:22.400)
Let's talk about Plato a little bit.
Lex Fridman (14:23.960)
So you draw a line from Plato, to Hegel, to Wigner, to today.
Lex Fridman (14:30.120)
So Plato has forms, the theory of forms.
Lex Fridman (14:35.440)
So there's a world of ideas and a world of things, as you talk about,
Lex Fridman (14:39.400)
and there's a connection.
Lex Fridman (14:40.400)
And presumably the world of ideas is very small, and the world of things
Vladimir Vapnik (14:45.720)
is arbitrarily big, but they're all what Plato calls them like, it's a shadow.
Lex Fridman (14:52.520)
The real world is a shadow from the world of forms.
Vladimir Vapnik (14:55.040)
Yeah, you have projection of a world of ideas.
Lex Fridman (14:58.840)
Yeah, very poetic.
Vladimir Vapnik (15:00.640)
In reality, you can realize this projection using these invariants
Vladimir Vapnik (15:07.040)
because it is projection for own specific examples, which create specific features
Vladimir Vapnik (15:13.600)
of specific objects.
Lex Fridman (15:14.840)
So the essence of intelligence is while only being able to observe
Vladimir Vapnik (15:22.920)
the world of things, try to come up with a world of ideas.
Lex Fridman (15:26.720)
Exactly.
Vladimir Vapnik (15:27.720)
Like in this music story, intelligent musical critics knows all these words
Lex Fridman (15:33.040)
and have a feeling about what they mean.
Vladimir Vapnik (15:34.840)
I feel like that's a contradiction, intelligent music critics.
Lex Fridman (15:38.800)
But I think music is to be enjoyed in all its forms.
Vladimir Vapnik (15:47.280)
The notion of critic, like a food critic.
Lex Fridman (15:49.840)
No, I don't want touch emotion.
Vladimir Vapnik (15:51.800)
That's an interesting question.
Lex Fridman (15:53.440)
Does emotion...
Vladimir Vapnik (15:54.640)
There's certain elements of the human psychology, of the human experience,
Lex Fridman (15:59.240)
which seem to almost contradict intelligence and reason.
Vladimir Vapnik (16:04.720)
Like emotion, like fear, like love, all of those things,
Lex Fridman (16:11.160)
are those not connected in any way to the space of ideas?
Vladimir Vapnik (16:16.520)
That I don't know.
Lex Fridman (16:18.560)
I just want to be concentrate on very simple story, on digit recognition.
Lex Fridman (16:27.720)
So you don't think you have to love and fear death in order to recognize digits?
Lex Fridman (16:31.800)
I don't know.
Vladimir Vapnik (16:33.600)
Because it's so complicated.
Lex Fridman (16:36.560)
It involves a lot of stuff which I never considered.
Lex Fridman (16:41.200)
But I know about digit recognition.
Lex Fridman (16:44.720)
And I know that for digit recognition,
Vladimir Vapnik (16:50.360)
to get records from small number of observations, you need predicate.
Lex Fridman (16:59.040)
But not special predicate for this problem.
Lex Fridman (17:03.240)
But universal predicate, which understand world of images.
Lex Fridman (17:08.480)
Of visual information.
Vladimir Vapnik (17:09.920)
Visual, yes.
Lex Fridman (17:11.120)
But on the first step, they understand, say, world of handwritten digits,
Vladimir Vapnik (17:18.440)
or characters, or something simple.
Lex Fridman (17:21.400)
So like you said, symmetry is an interesting one.
Vladimir Vapnik (17:23.800)
No, that's what I think one of the predicate is related to symmetry.
Lex Fridman (17:28.720)
The level of symmetry.
Vladimir Vapnik (17:30.720)
Okay, degree of symmetry.
Lex Fridman (17:32.120)
So you think symmetry at the bottom is a universal notion,
Lex Fridman (17:37.200)
and there's degrees of a single kind of symmetry,
Lex Fridman (17:41.480)
or is there many kinds of symmetries?
Vladimir Vapnik (17:44.160)
Many kinds of symmetries.
Lex Fridman (17:46.000)
There is a symmetry, antisymmetry, say, letter S.
Lex Fridman (17:52.360)
So it has vertical antisymmetry.
Lex Fridman (17:58.400)
And it could be diagonal symmetry, vertical symmetry.
Lex Fridman (18:02.640)
So when you cut vertically the letter S...
Lex Fridman (18:07.760)
Yeah, then the upper part and lower part in different directions.
Vladimir Vapnik (18:16.600)
Inverted, along the Y axis.
Lex Fridman (18:18.920)
But that's just like one example of symmetry, right?
Vladimir Vapnik (18:21.240)
Isn't there like...
Lex Fridman (18:21.960)
Right, but there is a degree of symmetry.
Vladimir Vapnik (18:26.320)
If you play all this iterative stuff to do tangent distance,
Lex Fridman (18:35.040)
whatever I describe, you can have a degree of symmetry.
Lex Fridman (18:40.480)
And that is what describing reason of image.
Lex Fridman (18:45.920)
It is the same as you will describe this image.
Vladimir Vapnik (18:53.200)
Think about digit S, it has antisymmetry.
Lex Fridman (18:57.920)
Digit three is symmetric.
Vladimir Vapnik (19:00.880)
More or less, look for symmetry.
Lex Fridman (19:04.480)
Do you think such concepts like symmetry,
Lex Fridman (19:07.840)
predicates like symmetry, is it a hierarchical set of concepts?
Lex Fridman (19:14.360)
Or are these independent, distinct predicates
Vladimir Vapnik (19:20.080)
that we want to discover as some set of...
Lex Fridman (19:23.600)
No, there is an idea of symmetry.
Lex Fridman (19:25.960)
And you can, this idea of symmetry, make very general.
Lex Fridman (19:34.880)
Like degree of symmetry.
Vladimir Vapnik (19:37.120)
If degree of symmetry can be zero, no symmetry at all.
Lex Fridman (19:40.680)
Or degree of symmetry, say, more or less symmetrical.
Lex Fridman (19:46.960)
But you have one of these descriptions.
Lex Fridman (19:50.480)
And symmetry can be different.
Vladimir Vapnik (19:52.480)
As I told, horizontal, vertical, diagonal,
Lex Fridman (19:56.320)
and antisymmetry is also concept of symmetry.
Vladimir Vapnik (1:00:03.320)
Yeah, but you should make a projection on reality.
Lex Fridman (1:00:07.520)
But understanding is, it is abstract ideas.
Vladimir Vapnik (1:00:11.820)
You have in your mind several abstract ideas
Lex Fridman (1:00:15.880)
which you can apply to reality.
Lex Fridman (1:00:17.760)
And reality in this case,
Lex Fridman (1:00:19.160)
so if you look at machine learning as data.
Vladimir Vapnik (1:00:21.400)
This example, data.
Lex Fridman (1:00:22.720)
Data.
Vladimir Vapnik (1:00:24.080)
Okay, let me put this on you
Lex Fridman (1:00:26.280)
because I'm an emotional creature.
Vladimir Vapnik (1:00:28.320)
I'm not a mathematical creature like you.
Lex Fridman (1:00:30.800)
I find compelling the idea,
Vladimir Vapnik (1:00:33.400)
forget the space, the sea of functions.
Lex Fridman (1:00:36.680)
There's also a sea of data in the world.
Lex Fridman (1:00:39.520)
And I find compelling that there might be,
Lex Fridman (1:00:42.320)
like you said, teacher,
Vladimir Vapnik (1:00:44.640)
small examples of data that are most useful
Lex Fridman (1:00:49.240)
for discovering good,
Vladimir Vapnik (1:00:53.000)
whether it's predicates or good functions,
Lex Fridman (1:00:55.560)
that the selection of data may be a powerful journey,
Vladimir Vapnik (1:01:00.320)
a useful, you know, coming up with a mechanism
Lex Fridman (1:01:03.760)
for selecting good data might be useful too.
Lex Fridman (1:01:07.480)
Do you find this idea of finding the right data set
Lex Fridman (1:01:12.440)
interesting at all?
Lex Fridman (1:01:14.000)
Or do you kind of take the data set as a given?
Lex Fridman (1:01:17.760)
I think that it is, you know, my theme is very simple.
Vladimir Vapnik (1:01:22.680)
You have huge set of functions.
Lex Fridman (1:01:25.900)
If you will apply, and you have not too many data,
Vladimir Vapnik (1:01:31.500)
if you pick up function which describes this data,
Lex Fridman (1:01:37.560)
you will do not very well.
Vladimir Vapnik (1:01:41.200)
You will.
Lex Fridman (1:01:42.040)
Like randomly pick up.
Vladimir Vapnik (1:01:42.860)
Yeah, you will overfit.
Lex Fridman (1:01:43.700)
Yeah, it will be overfitting.
Lex Fridman (1:01:46.380)
So you should decrease set of function
Lex Fridman (1:01:50.160)
from which you're picking up one.
Lex Fridman (1:01:53.640)
So you should go somehow to admissible set of function.
Lex Fridman (1:01:59.560)
And this, what about weak conversions?
Vladimir Vapnik (1:02:03.800)
So, but from another point of view,
Lex Fridman (1:02:08.040)
to make admissible set of function,
Vladimir Vapnik (1:02:13.200)
you need just a DG, just function
Lex Fridman (1:02:15.320)
which you will take in inner product,
Vladimir Vapnik (1:02:19.400)
which you will measure property of your function.
Lex Fridman (1:02:27.440)
And that is how it works.
Vladimir Vapnik (1:02:31.200)
No, I get it, I get it, I understand it,
Lex Fridman (1:02:32.720)
but do you, the reality is.
Lex Fridman (1:02:34.960)
But let's think about examples.
Lex Fridman (1:02:40.040)
You have huge set of function,
Lex Fridman (1:02:41.880)
and you have several examples.
Lex Fridman (1:02:44.640)
If you just trying to keep, take function
Vladimir Vapnik (1:02:50.360)
which satisfies these examples, you still will overfit.
Lex Fridman (1:02:56.620)
You need decrease, you need admissible set of function.
Vladimir Vapnik (1:02:59.320)
Absolutely, but what, say you have more data than functions.
Lex Fridman (1:03:06.120)
So sort of consider the, I mean,
Vladimir Vapnik (1:03:08.280)
maybe not more data than functions,
Lex Fridman (1:03:09.760)
because that's impossible.
Lex Fridman (1:03:12.040)
But what, I was trying to be poetic for a second.
Lex Fridman (1:03:15.120)
I mean, you have a huge amount of data,
Vladimir Vapnik (1:03:17.200)
a huge amount of examples.
Lex Fridman (1:03:19.840)
But amount of function can be even bigger.
Vladimir Vapnik (1:03:22.440)
It can get bigger, I understand.
Lex Fridman (1:03:24.320)
Everything is.
Vladimir Vapnik (1:03:25.520)
There's always a bigger boat.
Lex Fridman (1:03:27.560)
Full Hilbert space.
Vladimir Vapnik (1:03:29.200)
I got you, but okay.
Lex Fridman (1:03:31.800)
But you don't find the world of data
Vladimir Vapnik (1:03:35.800)
to be an interesting optimization space.
Lex Fridman (1:03:38.720)
Like the optimization should be in the space of functions.
Vladimir Vapnik (1:03:45.040)
Creating admissible set of functions.
Lex Fridman (1:03:47.080)
Admissible set of functions.
Vladimir Vapnik (1:03:48.120)
No, you know, even from the classical business theory,
Lex Fridman (1:03:54.480)
from structure risk minimization,
Vladimir Vapnik (1:03:56.400)
you should organize function in the way
Lex Fridman (1:04:02.240)
that they will be useful for you.
Vladimir Vapnik (1:04:06.560)
Right.
Lex Fridman (1:04:07.560)
And that is admissible set.
Vladimir Vapnik (1:04:10.280)
The way you're thinking about useful
Lex Fridman (1:04:13.560)
is you're given a small set of examples.
Vladimir Vapnik (1:04:17.000)
Useful small, small set of function
Lex Fridman (1:04:19.040)
which contain function I'm looking for.
Vladimir Vapnik (1:04:21.800)
Yeah, but looking for based on
Lex Fridman (1:04:25.320)
the empirical set of small examples.
Vladimir Vapnik (1:04:27.640)
Yeah, but that is another story.
Lex Fridman (1:04:29.640)
I don't touch it.
Vladimir Vapnik (1:04:31.160)
Because I believe that this small examples
Lex Fridman (1:04:35.720)
is not too small.
Vladimir Vapnik (1:04:37.400)
Say 60 per class.
Lex Fridman (1:04:39.200)
Law of large numbers works.
Vladimir Vapnik (1:04:41.360)
I don't need uniform law.
Lex Fridman (1:04:43.400)
The story is that in statistics there are two law.
Vladimir Vapnik (1:04:46.740)
Law of large numbers and uniform law of large numbers.
Lex Fridman (1:04:51.120)
So I want to be in situation where I use
Vladimir Vapnik (1:04:54.760)
law of large numbers but not uniform law of large numbers.
Lex Fridman (1:04:58.280)
Right, so 60 is law of large, it's large enough.
Vladimir Vapnik (1:05:01.440)
I hope, no, it still need some evaluations,
Lex Fridman (1:05:05.640)
some bonds.
Lex Fridman (1:05:07.880)
But the idea is the following that
Lex Fridman (1:05:11.560)
if you trust that
Vladimir Vapnik (1:05:15.580)
say this average gives you something close to expectations
Lex Fridman (1:05:21.080)
so you can talk about that, about this predicate.
Lex Fridman (1:05:26.240)
And that is basis of human intelligence.
Lex Fridman (1:05:30.720)
Good predicates is the,
Vladimir Vapnik (1:05:32.280)
the discovery of good predicates is the basis of human intelligence.
Lex Fridman (1:05:34.880)
It is discoverer of your understanding world.
Vladimir Vapnik (1:05:39.880)
Of your methodology of understanding world.
Lex Fridman (1:05:45.280)
Because you have several function
Vladimir Vapnik (1:05:47.240)
which you will apply to reality.
Lex Fridman (1:05:51.200)
Can you say that again?
Lex Fridman (1:05:52.480)
So you're...
Lex Fridman (1:05:54.440)
You have several functions predicate.
Lex Fridman (1:05:58.680)
But they're abstract.
Lex Fridman (1:06:00.240)
Yes.
Vladimir Vapnik (1:06:01.080)
Then you will apply them to reality, to your data.
Lex Fridman (1:06:04.360)
And you will create in this way predicate.
Vladimir Vapnik (1:06:07.400)
Which is useful for your task.
Lex Fridman (1:06:11.420)
But predicate are not related specifically to your task.
Vladimir Vapnik (1:06:16.840)
To this your task.
Lex Fridman (1:06:17.840)
It is abstract functions.
Vladimir Vapnik (1:06:20.080)
Which being applying, applied to...
Lex Fridman (1:06:23.240)
Many tasks that you might be interested in.
Vladimir Vapnik (1:06:25.280)
It might be many tasks, I don't know.
Lex Fridman (1:06:27.640)
Or...
Vladimir Vapnik (1:06:28.640)
Different tasks.
Lex Fridman (1:06:29.960)
Well they should be many tasks, right?
Vladimir Vapnik (1:06:31.640)
I believe like, like in prop case.
Lex Fridman (1:06:35.680)
It was for fairytales, but it's happened everywhere.
Vladimir Vapnik (1:06:40.080)
Okay, so we talked about images a little bit.
Lex Fridman (1:06:42.160)
But, can we talk about Noam Chomsky for a second?
Vladimir Vapnik (1:06:49.800)
No, I believe I...
Lex Fridman (1:06:52.280)
I don't know him very well.
Vladimir Vapnik (1:06:54.240)
Personally, well...
Lex Fridman (1:06:55.680)
Not personally, I don't know.
Vladimir Vapnik (1:06:57.040)
His ideas.
Lex Fridman (1:06:57.880)
His ideas.
Vladimir Vapnik (1:06:58.720)
Well let me just say,
Lex Fridman (1:06:59.840)
do you think language, human language,
Lex Fridman (1:07:02.360)
is essential to expressing ideas?
Lex Fridman (1:07:05.760)
As Noam Chomsky believes.
Lex Fridman (1:07:08.320)
So like, language is at the core
Lex Fridman (1:07:10.080)
of our formation of predicates.
Vladimir Vapnik (1:07:13.800)
The human language.
Lex Fridman (1:07:14.960)
For me, language and all the story of language
Vladimir Vapnik (1:07:18.560)
is very complicated.
Lex Fridman (1:07:20.720)
I don't understand this.
Lex Fridman (1:07:22.920)
And I am not...
Lex Fridman (1:07:24.080)
I thought about...
Vladimir Vapnik (1:07:25.680)
Nobody does.
Lex Fridman (1:07:26.520)
I am not ready to work on that.
Vladimir Vapnik (1:07:28.260)
Because it's so huge.
Lex Fridman (1:07:30.720)
It is not for me, and I believe not for our century.
Vladimir Vapnik (1:07:35.880)
The 21st century.
Lex Fridman (1:07:37.280)
Not for 21st century.
Vladimir Vapnik (1:07:39.440)
You should learn something, a lot of stuff,
Lex Fridman (1:07:42.160)
from simple task like digit recognition.
Lex Fridman (1:07:45.040)
So you think, okay, you think digital recognition,
Lex Fridman (1:07:49.200)
2D image, how would you more abstractly define
Lex Fridman (1:07:55.120)
digit recognition?
Lex Fridman (1:07:56.440)
It's 2D image, symbol recognition, essentially.
Vladimir Vapnik (1:08:03.760)
I mean, I'm trying to get a sense,
Lex Fridman (1:08:08.080)
sort of thinking about it now,
Vladimir Vapnik (1:08:09.680)
having worked with MNIST forever,
Lex Fridman (1:08:12.880)
how small of a subset is this
Vladimir Vapnik (1:08:16.040)
of the general vision recognition problem
Lex Fridman (1:08:18.560)
and the general intelligence problem?
Vladimir Vapnik (1:08:21.580)
Is it...
Lex Fridman (1:08:24.360)
Yeah.
Lex Fridman (1:08:25.200)
Is it a giant subset?
Lex Fridman (1:08:26.360)
Is it not?
Lex Fridman (1:08:27.840)
And how far away is language?
Lex Fridman (1:08:30.200)
You know, let me refer to Einstein.
Vladimir Vapnik (1:08:34.600)
Take the simplest problem, as simple as possible,
Lex Fridman (1:08:38.280)
but not simpler.
Lex Fridman (1:08:39.800)
And this is challenge, this simple problem.
Lex Fridman (1:08:44.280)
But it's simple by idea, but not simple to get it.
Vladimir Vapnik (1:08:50.360)
When you will do this, you will find some predicate,
Lex Fridman (1:08:55.360)
which helps it a bit.
Vladimir Vapnik (1:08:57.160)
Well, yeah, I mean, with Einstein, you can,
Lex Fridman (1:09:01.320)
you look at general relativity,
Lex Fridman (1:09:04.120)
but that doesn't help you with quantum mechanics.
Lex Fridman (1:09:07.280)
That's another story.
Vladimir Vapnik (1:09:08.760)
You don't have any universal instrument.
Lex Fridman (1:09:11.840)
Yes, so I'm trying to wonder which space we're in,
Vladimir Vapnik (1:09:16.520)
whether handwritten recognition is like general relativity,
Lex Fridman (1:09:21.120)
and then language is like quantum mechanics.
Lex Fridman (1:09:23.160)
So you're still gonna have to do a lot of mess
Lex Fridman (1:09:27.000)
to universalize it.
Lex Fridman (1:09:28.720)
But I'm trying to see,
Lex Fridman (1:09:35.120)
so what's your intuition why handwritten recognition
Lex Fridman (1:09:39.160)
is easier than language?
Lex Fridman (1:09:42.020)
Just, I think a lot of people would agree with that,
Lex Fridman (1:09:45.320)
but if you could elucidate sort of the intuition of why.
Lex Fridman (1:09:50.200)
I don't know, no, I don't think in this direction.
Vladimir Vapnik (1:09:56.460)
I just think in directions that this is problem,
Lex Fridman (1:10:00.880)
which if we will solve it well,
Vladimir Vapnik (1:10:07.760)
we will create some abstract understanding of images.
Lex Fridman (1:10:18.040)
Maybe not all images.
Vladimir Vapnik (1:10:19.680)
I would like to talk to guys who doing in real images
Lex Fridman (1:10:24.000)
in Columbia University.
Lex Fridman (1:10:26.280)
What kind of images, unreal?
Lex Fridman (1:10:28.400)
Real images.
Vladimir Vapnik (1:10:29.240)
Real images.
Lex Fridman (1:10:30.060)
Yeah, what they're ready, is there a predicate,
Lex Fridman (1:10:33.400)
what can be predicate?
Lex Fridman (1:10:35.160)
I still symmetry will play role in real life images,
Vladimir Vapnik (1:10:40.960)
in any real life images, 2D images.
Lex Fridman (1:10:43.920)
Let's talk about 2D images.
Vladimir Vapnik (1:10:46.320)
Because that's what we know.
Lex Fridman (1:10:52.520)
A neural network was created for 2D images.
Lex Fridman (1:10:55.880)
So the people I know in vision science, for example,
Lex Fridman (1:10:58.680)
the people who study human vision,
Vladimir Vapnik (1:11:01.000)
that they usually go to the world of symbols
Lex Fridman (1:11:04.520)
and like handwritten recognition,
Lex Fridman (1:11:06.360)
but not really, it's other kinds of symbols
Lex Fridman (1:11:08.480)
to study our visual perception system.
Vladimir Vapnik (1:11:11.560)
As far as I know, not much predicate type of thinking
Lex Fridman (1:11:15.160)
is understood about our vision system.
Vladimir Vapnik (1:11:17.640)
They did not think in this direction.
Lex Fridman (1:11:19.400)
They don't, yeah, but how do you even begin
Lex Fridman (1:11:21.720)
to think in that direction?
Lex Fridman (1:11:23.480)
That's a, I would like to discuss with them.
Vladimir Vapnik (1:11:26.920)
Yeah.
Lex Fridman (1:11:27.760)
Because if we will be able to show that it is what working,
Lex Fridman (1:11:35.600)
and theoretical scheme, it's not so bad.
Lex Fridman (1:11:40.360)
So the unfortunate, so if we compare to language,
Vladimir Vapnik (1:11:43.360)
language is like letters, finite set of letters,
Lex Fridman (1:11:46.520)
and a finite set of ways you can put together those letters.
Lex Fridman (1:11:50.480)
So it feels more amenable to kind of analysis.
Lex Fridman (1:11:53.720)
With natural images, there is so many pixels.
Vladimir Vapnik (1:11:58.680)
No, no, no, letter, language is much, much more complicated.
Lex Fridman (1:12:03.680)
It's involved a lot of different stuff.
Vladimir Vapnik (1:12:08.040)
It's not just understanding of very simple class of tasks.
Lex Fridman (1:12:15.280)
I would like to see list of task with language involved.
Vladimir Vapnik (1:12:19.960)
Yes, so there's a lot of nice benchmarks now
Lex Fridman (1:12:23.200)
in natural language processing from the very trivial,
Vladimir Vapnik (1:12:27.400)
like understanding the elements of a sentence,
Lex Fridman (1:12:30.200)
to question answering, to much more complicated
Vladimir Vapnik (1:12:33.040)
where you talk about open domain dialogue.
Lex Fridman (1:12:36.120)
The natural question is, with handwritten recognition,
Vladimir Vapnik (1:12:39.240)
is really the first step of understanding
Lex Fridman (1:12:42.960)
visual information.
Vladimir Vapnik (1:12:44.600)
Right.
Lex Fridman (1:12:46.440)
But even our records show that we go in the wrong direction
Vladimir Vapnik (1:12:54.160)
because we need 60,000 digits.
Lex Fridman (1:12:56.600)
So even this first step, so forget about talking
Vladimir Vapnik (1:12:59.680)
about the full journey, this first step
Lex Fridman (1:13:01.880)
should be taking in the right direction.
Vladimir Vapnik (1:13:03.280)
No, no, wrong direction because 60,000 is unacceptable.
Lex Fridman (1:13:07.160)
No, I'm saying it should be taken in the right direction
Vladimir Vapnik (1:13:11.000)
because 60,000 is not acceptable.
Lex Fridman (1:13:13.640)
If you can talk, it's great, we have half percent of error.
Lex Fridman (1:13:18.440)
And hopefully the step from doing hand recognition
Lex Fridman (1:13:22.720)
using very few examples, the step towards what babies do
Vladimir Vapnik (1:13:26.760)
when they crawl and understand their physical environment.
Lex Fridman (1:13:30.160)
I know you don't know about babies.
Vladimir Vapnik (1:13:31.720)
If you will do from very small examples,
Lex Fridman (1:13:36.040)
you will find principles which are different
Vladimir Vapnik (1:13:40.520)
from what we're using now.
Lex Fridman (1:13:44.440)
And so it's more or less clear.
Vladimir Vapnik (1:13:48.320)
That means that you will use weak convergence,
Lex Fridman (1:13:52.240)
not just strong convergence.
Lex Fridman (1:13:54.440)
Do you think these principles
Lex Fridman (1:13:58.440)
will naturally be human interpretable?
Vladimir Vapnik (1:14:01.640)
Oh, yeah.
Lex Fridman (1:14:02.560)
So like when we'll be able to explain them
Lex Fridman (1:14:04.480)
and have a nice presentation to show
Lex Fridman (1:14:06.240)
what those principles are, or are they very,
Lex Fridman (1:14:10.760)
going to be very kind of abstract kinds of functions?
Lex Fridman (1:14:14.440)
For example, I talked yesterday about symmetry.
Vladimir Vapnik (1:14:17.640)
Yes.
Lex Fridman (1:14:18.680)
And I gave very simple examples.
Vladimir Vapnik (1:14:20.440)
The same will be like that.
Lex Fridman (1:14:22.000)
You gave like a predicate of a basic for?
Vladimir Vapnik (1:14:24.680)
For symmetries.
Lex Fridman (1:14:25.760)
Yes, for different symmetries and you have for?
Vladimir Vapnik (1:14:29.520)
Degree of symmetries, that is important.
Lex Fridman (1:14:31.840)
Not just symmetry.
Vladimir Vapnik (1:14:33.680)
Existence doesn't exist, degree of symmetry.
Lex Fridman (1:14:38.360)
Yeah, for handwritten recognition.
Vladimir Vapnik (1:14:41.320)
No, it's not for handwritten, it's for any images.
Lex Fridman (1:14:45.160)
But I would like apply to handwritten.
Vladimir Vapnik (1:14:47.720)
Right, in theory it's more general, okay, okay.
Lex Fridman (1:14:55.280)
So a lot of the things we've been talking about
Vladimir Vapnik (1:14:58.160)
falls, we've been talking about philosophy a little bit,
Lex Fridman (1:15:01.800)
but also about mathematics and statistics.
Vladimir Vapnik (1:15:05.480)
A lot of it falls into this idea,
Lex Fridman (1:15:08.040)
a universal idea of statistical theory of learning.
Lex Fridman (1:15:11.760)
What is the most beautiful and sort of powerful
Lex Fridman (1:15:16.760)
or essential idea you've come across,
Vladimir Vapnik (1:15:19.080)
even just for yourself personally in the world
Lex Fridman (1:15:22.040)
of statistics or statistic theory of learning?
Vladimir Vapnik (1:15:25.440)
Probably uniform convergence, which we did
Lex Fridman (1:15:29.480)
with Alexei Chilvonenkis.
Lex Fridman (1:15:33.000)
Can you describe universal convergence?
Lex Fridman (1:15:36.080)
You have law of large numbers.
Lex Fridman (1:15:40.080)
So for any function, expectation of function,
Lex Fridman (1:15:44.480)
average of function converged to expectation.
Lex Fridman (1:15:48.120)
But if you have set of functions,
Lex Fridman (1:15:50.520)
for any function it is true.
Lex Fridman (1:15:52.340)
But it should converge simultaneously
Lex Fridman (1:15:55.580)
for all set of functions.
Lex Fridman (1:15:59.020)
And for learning, you need uniform convergence.
Lex Fridman (1:16:06.700)
Just convergence is not enough.
Vladimir Vapnik (1:16:11.220)
Because when you pick up one which gives minimum,
Lex Fridman (1:16:16.660)
you can pick up one function which does not converge
Lex Fridman (1:16:21.660)
and it will give you the best answer for this function.
Lex Fridman (1:16:31.460)
So you need uniform convergence to guarantee learning.
Lex Fridman (1:16:34.900)
So learning does not rely on trivial law of large numbers,
Lex Fridman (1:16:40.220)
it relies on universal law.
Lex Fridman (1:16:42.940)
But idea of convergence exists in statistics for a long time.
Lex Fridman (1:16:51.940)
But it is interesting that as I think about myself,
Lex Fridman (1:17:02.140)
how stupid I was 50 years, I did not see weak convergence.
Lex Fridman (1:17:08.160)
I work on strong convergence.
Lex Fridman (1:17:10.940)
But now I think that most powerful is weak convergence.
Lex Fridman (1:17:15.260)
Because it makes admissible set of functions.
Lex Fridman (1:17:18.860)
And even in all proverbs,
Lex Fridman (1:17:22.720)
when people try to understand recognition about dog law,
Vladimir Vapnik (1:17:28.300)
looks like a dog and so on, they use weak convergence.
Lex Fridman (1:17:32.400)
People in language, they understand this.
Lex Fridman (1:17:34.600)
But when we're trying to create artificial intelligence,
Lex Fridman (1:17:42.260)
we want event in different way.
Vladimir Vapnik (1:17:46.220)
We just consider strong convergence arguments.
Lex Fridman (1:17:50.540)
So reducing the set of admissible functions,
Vladimir Vapnik (1:17:52.740)
you think there should be effort put into understanding
Lex Fridman (1:17:58.780)
the properties of weak convergence?
Vladimir Vapnik (1:18:01.260)
You know, in classical mathematics, in Gilbert space,
Lex Fridman (1:18:07.260)
there are only two ways,
Vladimir Vapnik (1:18:08.820)
two form of convergence, strong and weak.
Lex Fridman (1:18:14.180)
Now we can use both.
Vladimir Vapnik (1:18:16.900)
That means that we did everything.
Lex Fridman (1:18:21.180)
And it so happened that when we use Hilbert space,
Vladimir Vapnik (1:18:26.180)
which is very rich space, space of continuous functions,
Lex Fridman (1:18:34.780)
which has integral and square.
Lex Fridman (1:18:38.020)
So we can apply weak and strong convergence for learning
Lex Fridman (1:18:42.420)
and have closed form solution.
Lex Fridman (1:18:45.140)
So for computationally simple.
Lex Fridman (1:18:47.660)
For me, it is sign that it is right way.
Vladimir Vapnik (1:18:51.080)
Because you don't need any heuristic here,
Lex Fridman (1:18:55.740)
just do whatever you want.
Lex Fridman (1:18:59.620)
But now the only what left is this concept
Lex Fridman (1:19:03.380)
of what is predicate, but it is not statistics.
Vladimir Vapnik (1:19:08.020)
By the way, I like the fact that you think that heuristics
Lex Fridman (1:19:11.660)
are a mess that should be removed from the system.
Lex Fridman (1:19:14.900)
So closed form solution is the ultimate goal.
Lex Fridman (1:19:18.460)
No, it so happened that when you're using right instrument,
Vladimir Vapnik (1:19:23.980)
you have closed form solution.
Lex Fridman (1:19:28.500)
Do you think intelligence, human level intelligence,
Vladimir Vapnik (1:19:32.780)
when we create it,
Lex Fridman (1:19:37.660)
will have something like a closed form solution?
Vladimir Vapnik (1:19:42.360)
You know, now I'm looking on bounds,
Lex Fridman (1:19:46.380)
which I gave bounds for convergence.
Lex Fridman (1:19:51.220)
And when I'm looking for bounds,
Lex Fridman (1:19:53.900)
I'm thinking what is the most appropriate kernel
Vladimir Vapnik (1:19:59.620)
for this bound would be.
Lex Fridman (1:20:02.500)
So we know that in say,
Vladimir Vapnik (1:20:05.960)
all our businesses, we use radial basis function.
Lex Fridman (1:20:11.460)
But looking on the bound,
Vladimir Vapnik (1:20:13.220)
I think that I start to understand that maybe
Lex Fridman (1:20:17.140)
we need to make corrections to radial basis function
Vladimir Vapnik (1:20:21.140)
to be closer to work better for this bounds.
Lex Fridman (1:20:28.440)
So I'm again trying to understand what type of kernel
Vladimir Vapnik (1:20:33.940)
have best approximation,
Lex Fridman (1:20:37.580)
best fit to this bound.
Vladimir Vapnik (1:20:43.420)
Sure, so there's a lot of interesting work
Lex Fridman (1:20:45.580)
that could be done in discovering better functions
Vladimir Vapnik (1:20:47.780)
than radial basis functions for bounds you find.
Lex Fridman (1:20:53.160)
It still comes from,
Vladimir Vapnik (1:20:55.860)
you're looking to mass and trying to understand what.
Lex Fridman (1:21:00.220)
From your own mind, looking at the, I don't know.
Vladimir Vapnik (1:21:03.540)
Then I'm trying to understand what will be good for that.
Lex Fridman (1:21:11.260)
Yeah, but to me, there's still a beauty.
Vladimir Vapnik (1:21:14.020)
Again, maybe I'm a descendant of Alan Turing to heuristics.
Lex Fridman (1:21:17.980)
To me, ultimately, intelligence will be a mess of heuristics.
Lex Fridman (1:21:23.620)
And that's the engineering answer, I guess.
Lex Fridman (1:21:26.300)
Absolutely.
Vladimir Vapnik (1:21:27.460)
When you're doing say, self driving cars,
Lex Fridman (1:21:31.060)
the great guy who will do this.
Vladimir Vapnik (1:21:35.020)
It doesn't matter what theory behind that.
Lex Fridman (1:21:40.640)
Who has a better feeling how to apply it.
Lex Fridman (1:21:43.800)
But by the way, it is the same story about predicates.
Lex Fridman (1:21:50.400)
Because you cannot create rule for,
Vladimir Vapnik (1:21:53.880)
situation is much more than you have rule for that.
Lex Fridman (1:21:56.660)
But maybe you can have more abstract rule
Vladimir Vapnik (1:22:04.780)
than it will be less literal.
Lex Fridman (1:22:08.780)
It is the same story about ideas
Lex Fridman (1:22:10.820)
and ideas applied to specific cases.
Lex Fridman (1:22:16.500)
But still you should reach.
Vladimir Vapnik (1:22:17.340)
You cannot avoid this.
Lex Fridman (1:22:18.900)
Yes, of course.
Lex Fridman (1:22:19.740)
But you should still reach for the ideas
Lex Fridman (1:22:21.620)
to understand the science.
Vladimir Vapnik (1:22:22.940)
Okay, let me kind of ask, do you think neural networks
Lex Fridman (1:22:27.980)
or functions can be made to reason?
Lex Fridman (1:22:34.100)
So what do you think, we've been talking about intelligence,
Lex Fridman (1:22:37.100)
but this idea of reasoning,
Vladimir Vapnik (1:22:39.620)
there's an element of sequentially disassembling,
Lex Fridman (1:22:44.500)
interpreting the images.
Lex Fridman (1:22:48.380)
So when you think of handwritten recognition, we kind of think
Lex Fridman (1:22:54.100)
that there'll be a single, there's an input and output.
Vladimir Vapnik (1:22:56.940)
There's not a recurrence.
Lex Fridman (1:23:01.060)
What do you think about sort of the idea of recurrence,
Vladimir Vapnik (1:23:04.440)
of going back to memory and thinking through this
Lex Fridman (1:23:06.860)
sort of sequentially mangling the different representations
Lex Fridman (1:23:11.860)
over and over until you arrive at a conclusion?
Lex Fridman (1:23:20.100)
Or is ultimately all that can be wrapped up into a function?
Vladimir Vapnik (1:23:23.460)
No, you're suggesting that let us use this type of algorithm.
Lex Fridman (1:23:29.860)
When I started thinking, I first of all,
Vladimir Vapnik (1:23:33.300)
starting to understand what I want.
Lex Fridman (1:23:36.580)
Can I write down what I want?
Lex Fridman (1:23:39.560)
And then I'm trying to formalize.
Lex Fridman (1:23:45.020)
And when I do that, I think I have to solve this problem.
Lex Fridman (1:23:52.120)
And till now I did not see a situation where you need recurrence.
Lex Fridman (1:24:04.280)
But do you observe human beings?
Vladimir Vapnik (1:24:07.840)
Yeah.
Lex Fridman (1:24:08.680)
You try to, it's the imitation question, right?
Vladimir Vapnik (1:24:12.400)
It seems that human beings reason
Lex Fridman (1:24:14.880)
this kind of sequentially sort of,
Vladimir Vapnik (1:24:20.680)
does that inspire in you a thought that we need to add that
Lex Fridman (1:24:24.120)
into our intelligence systems?
Vladimir Vapnik (1:24:30.760)
You're saying, okay, I mean, you've kind of answered saying
Lex Fridman (1:24:34.440)
until now I haven't seen a need for it.
Lex Fridman (1:24:37.040)
And so because of that, you don't see a reason
Lex Fridman (1:24:40.080)
to think about it.
Vladimir Vapnik (1:24:41.740)
You know, most of things I don't understand.
Lex Fridman (1:24:45.880)
In reasoning in human, it is for me too complicated.
Vladimir Vapnik (1:24:52.740)
For me, the most difficult part is to ask questions,
Lex Fridman (1:25:01.160)
to good questions, how it works,
Lex Fridman (1:25:03.900)
how people asking questions, I don't know this.
Lex Fridman (1:25:11.720)
You said that machine learning is not only
Vladimir Vapnik (1:25:13.640)
about technical things, speaking of questions,
Lex Fridman (1:25:16.480)
but it's also about philosophy.
Lex Fridman (1:25:19.720)
So what role does philosophy play in machine learning?
Lex Fridman (1:25:23.480)
We talked about Plato, but generally thinking
Vladimir Vapnik (1:25:28.240)
in this philosophical way, does it have,
Lex Fridman (1:25:32.480)
how does philosophy and math fit together in your mind?
Vladimir Vapnik (1:25:36.640)
First ideas and then their implementation.
Lex Fridman (1:25:39.520)
It's like predicate, like say admissible set of functions.
Vladimir Vapnik (1:25:48.940)
It comes together, everything.
Lex Fridman (1:25:51.500)
Because the first iteration of theory was done 50 years ago.
Vladimir Vapnik (1:25:58.360)
I told that, this is theory.
Lex Fridman (1:26:00.380)
So everything's there, if you have data you can,
Lex Fridman (1:26:04.080)
and your set of function has not big capacity.
Lex Fridman (1:26:13.600)
So low VC dimension, you can do that.
Vladimir Vapnik (1:26:15.760)
You can make structural risk minimization, control capacity.
Lex Fridman (1:26:21.140)
But you was not able to make admissible set of function good.
Vladimir Vapnik (1:26:26.140)
Now when suddenly realize that we did not use
Lex Fridman (1:26:33.680)
another idea of convergence, which we can,
Vladimir Vapnik (1:26:39.480)
everything comes together.
Lex Fridman (1:26:41.480)
But those are mathematical notions.
Vladimir Vapnik (1:26:43.320)
Philosophy plays a role of simply saying
Lex Fridman (1:26:48.000)
that we should be swimming in the space of ideas.
Vladimir Vapnik (1:26:52.080)
Let's talk what is philosophy.
Lex Fridman (1:26:54.320)
Philosophy means understanding of life.
Lex Fridman (1:26:58.080)
So understanding of life, say people like Plata,
Lex Fridman (1:27:03.480)
they understand on very high abstract level of life.
Vladimir Vapnik (1:27:07.640)
So, and whatever I doing,
Lex Fridman (1:27:12.040)
just implementation of my understanding of life.
Lex Fridman (1:27:16.740)
But every new step, it is very difficult.
Lex Fridman (1:27:21.400)
For example, to find this idea
Vladimir Vapnik (1:27:28.880)
that we need big convergence was not simple for me.
Lex Fridman (1:27:40.600)
So that required thinking about life a little bit.
Vladimir Vapnik (1:27:44.260)
Hard to trace, but there was some thought process.
Lex Fridman (1:27:48.840)
I'm working, I'm thinking about the same problem
Vladimir Vapnik (1:27:52.960)
for 50 years or more, and again, and again, and again.
Lex Fridman (1:28:00.020)
I'm trying to be honest and that is very important.
Vladimir Vapnik (1:28:02.680)
Not to be very enthusiastic, but concentrate
Lex Fridman (1:28:06.320)
on whatever we was not able to achieve, for example.
Lex Fridman (1:28:12.040)
And understand why.
Lex Fridman (1:28:13.360)
And now I understand that because I believe in math,
Vladimir Vapnik (1:28:18.920)
I believe that in Wigner's idea.
Lex Fridman (1:28:23.740)
But now when I see that there are only two way
Vladimir Vapnik (1:28:28.720)
of convergence and we're using both,
Lex Fridman (1:28:32.960)
that means that we must do as well as people doing.
Lex Fridman (1:28:37.960)
But now, exactly in philosophy
Lex Fridman (1:28:42.880)
and what we know about predicate,
Lex Fridman (1:28:45.760)
how we understand life, can we describe as a predicate.
Lex Fridman (1:28:51.400)
I thought about that and that is more or less obvious
Vladimir Vapnik (1:28:57.840)
level of symmetry.
Lex Fridman (1:29:00.760)
But next, I have a feeling,
Vladimir Vapnik (1:29:05.100)
it's something about structures.
Lex Fridman (1:29:09.540)
But I don't know how to formulate,
Lex Fridman (1:29:11.820)
how to measure measure of structure and all this stuff.
Lex Fridman (1:29:16.180)
And the guy who will solve this challenge problem,
Vladimir Vapnik (1:29:22.220)
then when we were looking how he did it,
Lex Fridman (1:29:27.060)
probably just only symmetry is not enough.
Lex Fridman (1:29:30.340)
But something like symmetry will be there.
Lex Fridman (1:29:33.980)
Structure will be there.
Vladimir Vapnik (1:29:34.820)
Oh yeah, absolutely.
Lex Fridman (1:29:35.640)
Symmetry will be there and level of symmetry will be there.
Lex Fridman (1:29:40.760)
And level of symmetry, antisymmetry, diagonal, vertical.
Lex Fridman (1:29:44.740)
And I even don't know how you can use
Vladimir Vapnik (1:29:48.780)
in different direction idea of symmetry, it's very general.
Lex Fridman (1:29:52.300)
But it will be there.
Vladimir Vapnik (1:29:54.940)
I think that people very sensitive to idea of symmetry.
Lex Fridman (1:29:58.600)
But there are several ideas like symmetry.
Vladimir Vapnik (1:30:04.900)
As I would like to learn.
Lex Fridman (1:30:07.020)
But you cannot learn just thinking about that.
Vladimir Vapnik (1:30:11.820)
You should do challenging problems
Lex Fridman (1:30:14.100)
and then analyze them, why it was able to solve them.
Lex Fridman (1:30:20.240)
And then you will see.
Lex Fridman (1:30:22.740)
Very simple things, it's not easy to find.
Lex Fridman (1:30:25.420)
But even with talking about this every time.
Lex Fridman (1:30:32.900)
I was surprised, I tried to understand.
Vladimir Vapnik (1:30:36.340)
These people describe in language
Lex Fridman (1:30:40.120)
strong convergence mechanism for learning.
Vladimir Vapnik (1:30:44.460)
I did not see, I don't know.
Lex Fridman (1:30:46.660)
But weak convergence, this dark story
Lex Fridman (1:30:50.100)
and story like that when you will explain to kid,
Lex Fridman (1:30:54.700)
you will use weak convergence argument.
Vladimir Vapnik (1:30:57.620)
It looks like it does like it does that.
Lex Fridman (1:31:00.900)
But when you try to formalize, you're just ignoring this.
Lex Fridman (1:31:05.820)
Why, why 50 years from start of machine learning?
Lex Fridman (1:31:10.140)
And that's the role of philosophy, thinking about life.
Vladimir Vapnik (1:31:12.420)
I think that maybe, I don't know.
Lex Fridman (1:31:18.300)
Maybe this is theory also, we should blame for that
Vladimir Vapnik (1:31:22.780)
because empirical risk minimization and all this stuff.
Lex Fridman (1:31:27.100)
And if you read now textbooks,
Vladimir Vapnik (1:31:30.660)
they just about bound about empirical risk minimization.
Lex Fridman (1:31:34.420)
They don't looking for another problem like admissible set.
Lex Fridman (1:31:41.820)
But on the topic of life, perhaps we,
Lex Fridman (1:31:47.340)
you could talk in Russian for a little bit.
Lex Fridman (1:31:50.020)
What's your favorite memory from childhood?
Lex Fridman (1:31:53.180)
What's your favorite memory from childhood?
Vladimir Vapnik (1:31:56.740)
Oh, music.
Lex Fridman (1:31:59.500)
How about, can you try to answer in Russian?
Lex Fridman (1:32:02.700)
Music?
Lex Fridman (1:32:04.980)
It was very cool when...
Lex Fridman (1:32:08.100)
What kind of music?
Lex Fridman (1:32:09.980)
Classic music.
Lex Fridman (1:32:11.860)
What's your favorite?
Lex Fridman (1:32:13.340)
Well, different composers.
Vladimir Vapnik (1:32:15.900)
At first, it was Vivaldi, I was surprised that it was possible.
Lex Fridman (1:32:23.500)
And then when I understood Bach, I was absolutely shocked.
Vladimir Vapnik (1:32:29.020)
By the way, from him I think that there is a predicate,
Lex Fridman (1:32:35.180)
like a structure.
Lex Fridman (1:32:36.740)
In Bach?
Lex Fridman (1:32:37.580)
Well, of course.
Vladimir Vapnik (1:32:38.420)
Because you can just feel the structure.
Lex Fridman (1:32:42.700)
And I don't think that different elements of life
Vladimir Vapnik (1:32:49.020)
are very much divided, in the sense of predicates.
Lex Fridman (1:32:53.020)
Everywhere structure, in painting structure,
Vladimir Vapnik (1:32:56.900)
in human relations structure.
Lex Fridman (1:32:59.820)
Here's how to find these high level predicates, it's...
Vladimir Vapnik (1:33:05.540)
In Bach and in life, everything is connected.
Lex Fridman (1:33:08.460)
Now that we're talking about Bach,
Vladimir Vapnik (1:33:14.100)
let's switch back to English,
Lex Fridman (1:33:15.700)
because I like Beethoven and Chopin, so...
Vladimir Vapnik (1:33:18.580)
Well, Chopin, it's another amusing story.
Lex Fridman (1:33:21.300)
But Bach, if we talk about predicates,
Vladimir Vapnik (1:33:23.940)
Bach probably has the most sort of
Lex Fridman (1:33:29.300)
well defined predicates that underlie it.
Vladimir Vapnik (1:33:31.860)
It is very interesting to read what critics
Lex Fridman (1:33:36.860)
are writing about Bach, which words they're using.
Vladimir Vapnik (1:33:40.460)
They're trying to describe predicates.
Lex Fridman (1:33:43.500)
And then Chopin, it is very different vocabulary,
Vladimir Vapnik (1:33:52.100)
very different predicates.
Lex Fridman (1:33:55.140)
And I think that if you will make collection of that,
Lex Fridman (1:34:02.700)
so maybe from this you can describe predicate
Lex Fridman (1:34:05.860)
for digit recognition as well.
Vladimir Vapnik (1:34:08.780)
From Bach and Chopin.
Lex Fridman (1:34:10.460)
No, no, no, not from Bach and Chopin.
Vladimir Vapnik (1:34:12.540)
From the critic interpretation of the music, yeah.
Lex Fridman (1:34:15.260)
When they're trying to explain you music, what they use.
Vladimir Vapnik (1:34:22.300)
As they use, they describe high level ideas
Lex Fridman (1:34:25.260)
of platos ideas, what behind this music.
Vladimir Vapnik (1:34:28.900)
That's brilliant.
Lex Fridman (1:34:29.740)
So art is not self explanatory in some sense.
Lex Fridman (1:34:34.740)
So you have to try to convert it into ideas.
Lex Fridman (1:34:39.060)
It is ill post problems.
Vladimir Vapnik (1:34:40.940)
When you go from ideas to the representation,
Lex Fridman (1:34:46.060)
it is easy way.
Lex Fridman (1:34:47.580)
But when you're trying to go Bach, it is ill post problems.
Lex Fridman (1:34:51.420)
But nevertheless, I believe that when you're looking
Vladimir Vapnik (1:34:55.660)
from that, even from art, you will be able to find
Lex Fridman (1:35:00.340)
predicates for digit recognition.
Vladimir Vapnik (1:35:02.100)
That's such a fascinating and powerful notion.
Lex Fridman (1:35:08.500)
Do you ponder your own mortality?
Lex Fridman (1:35:11.660)
Do you think about it?
Lex Fridman (1:35:12.540)
Do you fear it?
Lex Fridman (1:35:13.660)
Do you draw insight from it?
Lex Fridman (1:35:16.820)
About mortality, no, yeah.
Lex Fridman (1:35:21.540)
Are you afraid of death?
Lex Fridman (1:35:25.860)
Not too much, not too much.
Vladimir Vapnik (1:35:29.660)
It is pity that I will not be able to do something
Lex Fridman (1:35:33.700)
which I think I have a feeling to do that.
Vladimir Vapnik (1:35:39.460)
For example, I will be very happy to work with guys
Lex Fridman (1:35:48.020)
theoretician from music to write this collection
Vladimir Vapnik (1:35:52.060)
of description, how they describe music,
Lex Fridman (1:35:55.060)
how they use that predicate, and from art as well.
Vladimir Vapnik (1:36:00.140)
Then take what is in common and try to understand
Lex Fridman (1:36:04.580)
predicate which is absolute for everything.
Lex Fridman (1:36:08.660)
And then use that for visual recognition
Lex Fridman (1:36:10.460)
and see if there is a connection.
Vladimir Vapnik (1:36:12.180)
Yeah, exactly.
Lex Fridman (1:36:13.540)
Ah, there's still time.
Vladimir Vapnik (1:36:14.660)
We got time.
Lex Fridman (1:36:16.980)
Ha ha ha ha.
Vladimir Vapnik (1:36:18.660)
Yeah.
Lex Fridman (1:36:19.500)
We got time.
Vladimir Vapnik (1:36:20.340)
It take years and years and years.
Lex Fridman (1:36:24.100)
Yes, yeah, it's a long way.
Vladimir Vapnik (1:36:26.460)
Well, see, you've got the patient mathematicians mind.
Lex Fridman (1:36:30.900)
I think it could be done very quickly and very beautifully.
Vladimir Vapnik (1:36:34.060)
I think it's a really elegant idea.
Lex Fridman (1:36:35.820)
Yeah, but also.
Vladimir Vapnik (1:36:36.940)
Some of many.
Lex Fridman (1:36:37.780)
Yeah, you know, the most time,
Vladimir Vapnik (1:36:40.580)
it is not to make this collection to understand
Lex Fridman (1:36:45.280)
what is the common to think about that once again
Lex Fridman (1:36:48.700)
and again and again.
Lex Fridman (1:36:49.540)
Again and again and again, but I think sometimes,
Vladimir Vapnik (1:36:52.660)
especially just when you say this idea now,
Lex Fridman (1:36:55.700)
even just putting together the collection
Lex Fridman (1:36:58.780)
and looking at the different sets of data,
Lex Fridman (1:37:03.300)
language, trying to interpret music,
Vladimir Vapnik (1:37:05.520)
criticize music, and images,
Lex Fridman (1:37:08.740)
I think there'll be sparks of ideas that'll come.
Vladimir Vapnik (1:37:10.940)
Of course, again and again, you'll come up with better ideas,
Lex Fridman (1:37:13.420)
but even just that notion is a beautiful notion.
Vladimir Vapnik (1:37:16.940)
I even have some example.
Lex Fridman (1:37:19.340)
Yes, so I have friend
Vladimir Vapnik (1:37:25.200)
who was specialist in Russian poetry.
Lex Fridman (1:37:30.960)
She is professor of Russian poetry.
Vladimir Vapnik (1:37:35.320)
He did not write poems,
Lex Fridman (1:37:39.400)
but she know a lot of stuff.
Vladimir Vapnik (1:37:43.340)
She make book, several books,
Lex Fridman (1:37:48.080)
and one of them is a collection of Russian poetry.
Vladimir Vapnik (1:37:54.680)
She have images of Russian poetry.
Lex Fridman (1:37:57.140)
She collect all images of Russian poetry.
Lex Fridman (1:38:00.720)
And I ask her to do following.
Lex Fridman (1:38:05.420)
You have NIPS, digit recognition,
Lex Fridman (1:38:09.720)
and we get 100 digits,
Lex Fridman (1:38:13.520)
or maybe less than 100.
Vladimir Vapnik (1:38:15.280)
I don't remember, maybe 50 digits.
Lex Fridman (1:38:18.920)
And try from poetical point of view,
Vladimir Vapnik (1:38:21.680)
describe every image which she see,
Lex Fridman (1:38:25.260)
using only words of images of Russian poetry.
Lex Fridman (1:38:31.320)
And she did it.
Lex Fridman (1:38:34.280)
And then we tried to,
Vladimir Vapnik (1:38:41.140)
I call it learning using privileged information.
Lex Fridman (1:38:43.600)
I call it privileged information.
Vladimir Vapnik (1:38:45.920)
You have on two languages.
Lex Fridman (1:38:48.040)
One language is just image of digit,
Lex Fridman (1:38:53.140)
and another language, poetic description of this image.
Lex Fridman (1:38:57.760)
And this is privileged information.
Lex Fridman (1:39:02.360)
And there is an algorithm when you're working
Lex Fridman (1:39:04.520)
using privileged information, you're doing better.
Vladimir Vapnik (1:39:08.320)
Much better, so.
Lex Fridman (1:39:10.400)
So there's something there.
Vladimir Vapnik (1:39:11.560)
Something there.
Lex Fridman (1:39:12.880)
And there is a, in NEC,
Vladimir Vapnik (1:39:16.980)
she unfortunately died.
Lex Fridman (1:39:20.880)
The collection of digits
Vladimir Vapnik (1:39:24.840)
in poetic descriptions of these digits.
Lex Fridman (1:39:29.160)
Yeah.
Lex Fridman (1:39:30.000)
So there's something there in that poetic description.
Lex Fridman (1:39:32.920)
But I think that there is a abstract ideas
Vladimir Vapnik (1:39:38.320)
on the plot of level of ideas.
Lex Fridman (1:39:40.680)
Yeah, that they're there.
Vladimir Vapnik (1:39:42.000)
That could be discovered.
Lex Fridman (1:39:43.120)
And music seems to be a good entry point.
Lex Fridman (1:39:45.000)
But as soon as we start with this challenge problem.
Lex Fridman (1:39:50.340)
The challenge problem.
Vladimir Vapnik (1:39:51.180)
Listen.
Lex Fridman (1:39:52.020)
It immediately connected to all this stuff.
Vladimir Vapnik (1:39:55.400)
Especially with your talk and this podcast,
Lex Fridman (1:39:58.060)
and I'll do whatever I can to advertise it.
Vladimir Vapnik (1:40:00.120)
It's such a clean, beautiful Einstein like formulation
Lex Fridman (1:40:03.280)
of the challenge before us.
Vladimir Vapnik (1:40:05.240)
Right.
Lex Fridman (1:40:06.060)
Let me ask another absurd question.
Vladimir Vapnik (1:40:09.520)
We talked about mortality.
Lex Fridman (1:40:12.800)
We talked about philosophy of life.
Lex Fridman (1:40:14.640)
What do you think is the meaning of life?
Lex Fridman (1:40:17.560)
What's the predicate for mysterious existence here on earth?
Vladimir Vapnik (1:40:29.620)
I don't know.
Lex Fridman (1:40:33.620)
It's very interesting how we have,
Vladimir Vapnik (1:40:37.640)
in Russia, I don't know if you know the guy Strugatsky.
Lex Fridman (1:40:43.100)
They are writing fiction.
Vladimir Vapnik (1:40:46.320)
They're thinking about human, what's going on.
Lex Fridman (1:40:51.680)
And they have idea that there are developing
Vladimir Vapnik (1:41:00.560)
two type of people, common people and very smart people.
Lex Fridman (1:41:05.120)
They just started.
Lex Fridman (1:41:06.080)
And these two branches of people will go
Lex Fridman (1:41:10.420)
in different direction very soon.
Lex Fridman (1:41:13.540)
So that's what they're thinking about that.
Lex Fridman (1:41:18.980)
So the purpose of life is to create two paths.
Vladimir Vapnik (1:41:23.800)
Two paths.
Lex Fridman (1:41:24.640)
Of human societies.
Vladimir Vapnik (1:41:25.940)
Yes.
Lex Fridman (1:41:27.020)
Simple people and more complicated people.
Lex Fridman (1:41:29.980)
Which do you like best?
Lex Fridman (1:41:31.540)
The simple people or the complicated ones?
Vladimir Vapnik (1:41:34.500)
I don't know that it is just his fantasy,
Lex Fridman (1:41:38.260)
but you know, every week we have guy
Vladimir Vapnik (1:41:41.700)
who is just a writer and also a theorist of literature.
Lex Fridman (1:41:51.820)
And he explain how he understand literature
Lex Fridman (1:41:56.600)
and human relationship.
Lex Fridman (1:41:58.800)
How he see life.
Lex Fridman (1:42:00.340)
And I understood that I'm just small kids
Lex Fridman (1:42:06.920)
comparing to him.
Vladimir Vapnik (1:42:09.500)
He's very smart guy in understanding life.
Lex Fridman (1:42:13.880)
He knows this predicate.
Vladimir Vapnik (1:42:15.640)
He knows big blocks of life.
Lex Fridman (1:42:19.760)
I am used every time when I listen to him.
Lex Fridman (1:42:24.800)
And he just talking about literature.
Lex Fridman (1:42:27.400)
And I think that I was surprised.
Lex Fridman (1:42:33.200)
So the managers in big companies,
Lex Fridman (1:42:41.460)
most of them are guys who study English language
Lex Fridman (1:42:48.760)
and English literature.
Lex Fridman (1:42:51.120)
So why?
Vladimir Vapnik (1:42:52.520)
Because they understand life.
Lex Fridman (1:42:54.820)
They understand models.
Lex Fridman (1:42:57.040)
And among them,
Lex Fridman (1:42:58.800)
maybe many talented critics just analyzing this.
Lex Fridman (1:43:06.680)
And this is big science like property.
Lex Fridman (1:43:10.520)
This is blocks.
Vladimir Vapnik (1:43:13.360)
That's very smart.
Lex Fridman (1:43:17.480)
It amazes me that you are and continue to be humbled
Vladimir Vapnik (1:43:21.520)
by the brilliance of others.
Lex Fridman (1:43:22.960)
I'm very modest about myself.
Vladimir Vapnik (1:43:25.540)
I see so smart guys around.
Lex Fridman (1:43:28.960)
Well, let me be immodest for you.
Vladimir Vapnik (1:43:31.720)
You're one of the greatest mathematicians,
Lex Fridman (1:43:33.920)
statisticians of our time.
Vladimir Vapnik (1:43:35.820)
It's truly an honor.
Lex Fridman (1:43:36.960)
Thank you for talking again.
Lex Fridman (1:43:38.600)
And let's talk.
Lex Fridman (1:43:41.240)
It is not.
Vladimir Vapnik (1:43:43.440)
I know my limits.
Lex Fridman (1:43:45.720)
Let's talk again when your challenge is taken on
Lex Fridman (1:43:49.120)
and solved by grad student.
Lex Fridman (1:43:51.880)
Especially when they use it.
Vladimir Vapnik (1:43:55.200)
It happens.
Lex Fridman (1:43:57.200)
Maybe music will be involved.
Vladimir Vapnik (1:43:58.880)
Latimer, thank you so much.
Lex Fridman (1:43:59.880)
It's been an honor. Thank you very much.
Vladimir Vapnik (1:44:02.580)
Thanks for listening to this conversation
Lex Fridman (1:44:04.200)
with Latimer Vapnik.
Lex Fridman (1:44:05.480)
And thank you to our presenting sponsor, Cash App.
Lex Fridman (1:44:08.760)
Download it, use code LexPodcast.
Vladimir Vapnik (1:44:11.440)
You'll get $10 and $10 will go to FIRST,
Lex Fridman (1:44:14.320)
an organization that inspires and educates young minds
Vladimir Vapnik (1:44:17.040)
to become science and technology innovators of tomorrow.
Lex Fridman (1:44:20.760)
If you enjoy this podcast, subscribe on YouTube,
Vladimir Vapnik (1:44:23.480)
give us five stars on Apple Podcast,
Lex Fridman (1:44:25.320)
support it on Patreon,
Vladimir Vapnik (1:44:26.840)
or simply connect with me on Twitter at Lex Friedman.
Lex Fridman (1:44:31.360)
And now, let me leave you with some words
Vladimir Vapnik (1:44:33.480)
from Latimer Vapnik.
Lex Fridman (1:44:35.580)
When solving a problem of interest,
Vladimir Vapnik (1:44:37.760)
do not solve a more general problem
Lex Fridman (1:44:40.080)
as an intermediate step.
Vladimir Vapnik (1:44:43.040)
Thank you for listening.
Lex Fridman (1:44:44.360)
I hope to see you next time.
Lex Fridman (20:01.400)
What about shape in general?
Lex Fridman (20:03.320)
I mean, symmetry is a fascinating notion, but...
Vladimir Vapnik (20:06.920)
No, no, I'm talking about digit.
Lex Fridman (20:08.600)
I would like to concentrate on all I would like to know,
Vladimir Vapnik (20:12.440)
predicate for digit recognition.
Lex Fridman (20:14.440)
Yes, but symmetry is not enough for digit recognition, right?
Vladimir Vapnik (20:19.360)
It is not necessarily for digit recognition.
Lex Fridman (20:22.520)
It helps to create invariant, which you can use
Vladimir Vapnik (20:30.640)
when you will have examples for digit recognition.
Lex Fridman (20:35.000)
You have regular problem of digit recognition.
Vladimir Vapnik (20:38.240)
You have examples of the first class or second class.
Lex Fridman (20:41.600)
Plus, you know that there exists concept of symmetry.
Lex Fridman (20:45.840)
And you apply, when you're looking for decision rule,
Lex Fridman (20:50.400)
you will apply concept of symmetry,
Vladimir Vapnik (20:55.400)
of this level of symmetry, which you estimate from...
Lex Fridman (21:00.120)
So let's talk.
Vladimir Vapnik (21:01.680)
Everything comes from weak convergence.
Lex Fridman (21:06.600)
What is convergence?
Lex Fridman (21:07.840)
What is weak convergence?
Lex Fridman (21:09.280)
What is strong convergence?
Vladimir Vapnik (21:11.360)
I'm sorry, I'm gonna do this to you.
Lex Fridman (21:13.360)
What are we converging from and to?
Vladimir Vapnik (21:16.120)
You're converging, you would like to have a function.
Lex Fridman (21:20.480)
The function which, say, indicator function,
Vladimir Vapnik (21:23.600)
which indicate your digit five, for example.
Lex Fridman (21:29.920)
A classification task.
Vladimir Vapnik (21:31.480)
Let's talk only about classification.
Lex Fridman (21:33.640)
So classification means you will say
Vladimir Vapnik (21:36.840)
whether this is a five or not,
Lex Fridman (21:38.560)
or say which of the 10 digits it is.
Vladimir Vapnik (21:40.600)
Right, right.
Lex Fridman (21:42.160)
I would like to have these functions.
Vladimir Vapnik (21:46.560)
Then, I have some examples.
Lex Fridman (21:56.040)
I can consider property of these examples.
Vladimir Vapnik (22:01.120)
Say, symmetry.
Lex Fridman (22:02.720)
And I can measure level of symmetry for every digit.
Lex Fridman (22:08.040)
And then I can take average from my training data.
Lex Fridman (22:16.680)
And I will consider only functions
Vladimir Vapnik (22:20.920)
of conditional probability,
Lex Fridman (22:24.000)
which I'm looking for my decision rule.
Vladimir Vapnik (22:27.280)
Which applying to digits will give me the same average
Lex Fridman (22:38.360)
as I observe on training data.
Vladimir Vapnik (22:41.960)
So, actually, this is different level
Lex Fridman (22:45.360)
of description of what you want.
Vladimir Vapnik (22:48.480)
You want not just, you show not one digit.
Lex Fridman (22:54.080)
You show, this predicate, show general property
Vladimir Vapnik (22:59.840)
of all digits which you have in mind.
Lex Fridman (23:03.720)
If you have in mind digit three,
Vladimir Vapnik (23:06.080)
it gives you property of digit three.
Lex Fridman (23:10.360)
And you select as admissible set of function,
Vladimir Vapnik (23:13.560)
only function, which keeps this property.
Lex Fridman (23:16.960)
You will not consider other functions.
Vladimir Vapnik (23:20.760)
So, you immediately looking for smaller subset of function.
Lex Fridman (23:24.920)
That's what you mean by admissible functions.
Vladimir Vapnik (23:27.000)
Admissible function, exactly.
Lex Fridman (23:28.400)
Which is still a pretty large,
Vladimir Vapnik (23:30.920)
for the number three, is a large.
Lex Fridman (23:32.920)
It is pretty large, but if you have one predicate.
Lex Fridman (23:36.600)
But according to, there is a strong and weak convergence.
Lex Fridman (23:42.760)
Strong convergence is convergence in function.
Vladimir Vapnik (23:46.360)
You're looking for the function on one function,
Lex Fridman (23:49.200)
and you're looking for another function.
Lex Fridman (23:51.880)
And square difference from them should be small.
Lex Fridman (23:59.240)
If you take difference in any points,
Vladimir Vapnik (24:01.880)
make a square, make an integral, and it should be small.
Lex Fridman (24:05.640)
That is convergence in function.
Vladimir Vapnik (24:08.040)
Suppose you have some function, any function.
Lex Fridman (24:11.280)
So, I would say, I say that some function
Vladimir Vapnik (24:15.400)
converge to this function.
Lex Fridman (24:17.880)
If integral from square difference between them is small.
Vladimir Vapnik (24:22.880)
That's the definition of strong convergence.
Lex Fridman (24:24.760)
That definition of strong convergence.
Vladimir Vapnik (24:25.760)
Two functions, the integral, the difference, is small.
Lex Fridman (24:28.920)
Yeah, it is convergence in functions.
Vladimir Vapnik (24:31.160)
Yeah.
Lex Fridman (24:32.280)
But you have different convergence in functionals.
Vladimir Vapnik (24:36.720)
You take any function, you take some function, phi,
Lex Fridman (24:41.160)
and take inner product, this function, this f function.
Vladimir Vapnik (24:46.040)
f0 function, which you want to find.
Lex Fridman (24:50.360)
And that gives you some value.
Vladimir Vapnik (24:52.960)
So, you say that set of functions converge
Lex Fridman (24:59.960)
in inner product to this function,
Vladimir Vapnik (25:03.040)
if this value of inner product converge to value f0.
Lex Fridman (25:10.400)
That is for one phi.
Lex Fridman (25:12.480)
But weak convergence requires that it converge for any
Lex Fridman (25:16.320)
function of Hilbert space.
Vladimir Vapnik (25:20.680)
If it converge for any function of Hilbert space,
Lex Fridman (25:24.200)
then you will say that this is weak convergence.
Vladimir Vapnik (25:28.240)
You can think that when you take integral,
Lex Fridman (25:32.200)
that is integral property of function.
Vladimir Vapnik (25:35.920)
For example, if you will take sine or cosine,
Lex Fridman (25:39.120)
it is coefficient of, say, Fourier expansion.
Vladimir Vapnik (25:45.480)
So, if it converge for all coefficients of Fourier
Lex Fridman (25:51.440)
expansion, so under some condition,
Vladimir Vapnik (25:54.240)
it converge to function you're looking for.
Lex Fridman (25:58.080)
But weak convergence means any property.
Vladimir Vapnik (26:02.800)
Convergence not point wise, but integral property
Lex Fridman (26:07.640)
of function.
Vladimir Vapnik (26:09.480)
So, weak convergence means integral property of functions.
Lex Fridman (26:13.800)
When I'm talking about predicate,
Vladimir Vapnik (26:16.040)
I would like to formulate which integral properties
Lex Fridman (26:23.200)
I would like to have for convergence.
Vladimir Vapnik (26:27.840)
So, and if I will take one predicated function,
Lex Fridman (26:33.440)
which I measure property, if I will use one predicate
Lex Fridman (26:39.600)
and say, I will consider only function which give me
Lex Fridman (26:44.840)
the same value as this predicate,
Vladimir Vapnik (26:47.840)
I selecting set of functions from functions
Lex Fridman (26:53.440)
which is admissible in the sense that function which I'm
Vladimir Vapnik (26:58.000)
looking for in this set of functions
Lex Fridman (27:01.000)
because I checking in training data, it gives the same.
Vladimir Vapnik (27:08.760)
Yeah, so it always has to be connected to the training
Lex Fridman (27:10.960)
data in terms of?
Vladimir Vapnik (27:12.600)
Yeah, but property, you can know independent on training data.
Lex Fridman (27:18.720)
And this guy, prop, says that there is formal property,
Vladimir Vapnik (27:24.000)
31 property.
Lex Fridman (27:25.360)
A fairy tale, a Russian fairy tale.
Lex Fridman (27:27.640)
But Russian fairy tale is not so interesting.
Lex Fridman (27:30.560)
More interesting that people apply this to movies,
Vladimir Vapnik (27:34.880)
to theater, to different things.
Lex Fridman (27:38.000)
And the same works, they're universal.
Vladimir Vapnik (27:41.960)
Well, so I would argue that there's
Lex Fridman (27:44.400)
a little bit of a difference between the kinds of things
Vladimir Vapnik (27:48.520)
that were applied to which are essentially stories
Lex Fridman (27:51.480)
and digit recognition.
Vladimir Vapnik (27:54.240)
It is the same story.
Lex Fridman (27:55.880)
You're saying digits, there's a story within the digit.
Vladimir Vapnik (27:59.600)
Yeah.
Lex Fridman (28:00.360)
And so but my point is why I hope
Vladimir Vapnik (28:04.640)
that it possible to beat record using not 60,000,
Lex Fridman (28:11.440)
but say 100 times less.
Vladimir Vapnik (28:13.800)
Because instead, you will give predicates.
Lex Fridman (28:17.840)
And you will select your decision
Vladimir Vapnik (28:21.040)
not from wide set of functions, but from set of functions
Lex Fridman (28:25.680)
which keeps this predicates.
Lex Fridman (28:28.040)
But predicate is not related just to digit recognition.
Lex Fridman (28:32.760)
Right.
Vladimir Vapnik (28:33.800)
Like in Plato's case.
Lex Fridman (28:37.640)
Do you think it's possible to automatically discover
Lex Fridman (28:40.800)
the predicates?
Lex Fridman (28:42.120)
So you basically said that the essence of intelligence
Vladimir Vapnik (28:46.520)
is the discovery of good predicates.
Lex Fridman (28:49.560)
Yeah.
Vladimir Vapnik (28:51.240)
Now, the natural question is that's
Lex Fridman (28:55.800)
what Einstein was good at doing in physics.
Vladimir Vapnik (28:59.040)
Can we make machines do these kinds
Lex Fridman (29:02.400)
of discovery of good predicates?
Lex Fridman (29:04.480)
Or is this ultimately a human endeavor?
Lex Fridman (29:07.720)
That I don't know.
Vladimir Vapnik (29:09.080)
I don't think that machine can do.
Lex Fridman (29:11.400)
Because according to theory about weak convergence,
Vladimir Vapnik (29:18.840)
any function from Hilbert space can be predicated.
Lex Fridman (29:23.120)
So you have infinite number of predicate in upper.
Lex Fridman (29:27.560)
And before, you don't know which predicate is good and which.
Lex Fridman (29:32.800)
But whatever prop show and why people call it breakthrough,
Vladimir Vapnik (29:39.880)
that there is not too many predicate
Lex Fridman (29:44.600)
which cover most of situation happened in the world.
Vladimir Vapnik (29:48.600)
Right.
Lex Fridman (29:51.280)
So there's a sea of predicates.
Lex Fridman (29:54.200)
And most of the only a small amount
Lex Fridman (29:57.240)
are useful for the kinds of things
Vladimir Vapnik (29:58.800)
that happen in the world.
Lex Fridman (30:01.240)
I think that I would say only small part of predicate
Vladimir Vapnik (30:07.120)
very useful.
Lex Fridman (30:08.680)
Useful all of them.
Vladimir Vapnik (30:11.280)
Only very few are what we should let's call them
Lex Fridman (30:14.360)
good predicates.
Vladimir Vapnik (30:15.440)
Very good predicates.
Lex Fridman (30:16.640)
Very good predicates.
Lex Fridman (30:18.160)
So can we linger on it?
Lex Fridman (30:20.720)
What's your intuition?
Lex Fridman (30:21.680)
Why is it hard for a machine to discover good predicates?
Lex Fridman (30:27.520)
Even in my talk described how to do predicate.
Lex Fridman (30:30.680)
How to find new predicate.
Lex Fridman (30:32.640)
I'm not sure that it is very good.
Lex Fridman (30:34.960)
What did you propose in your talk?
Lex Fridman (30:36.600)
No.
Vladimir Vapnik (30:37.160)
In my talk, I gave example for diabetes.
Lex Fridman (30:42.360)
Diabetes, yeah.
Vladimir Vapnik (30:43.720)
When we achieve some percent.
Lex Fridman (30:46.160)
So then we're looking for area where
Vladimir Vapnik (30:50.760)
some sort of predicate, which I formulate,
Lex Fridman (30:54.760)
does not keeps invariant.
Lex Fridman (31:03.120)
So if it doesn't keep, I retrain my data.
Lex Fridman (31:06.920)
I select only function which keeps this invariant.
Lex Fridman (31:11.080)
And when I did it, I improved my performance.
Lex Fridman (31:14.400)
I can looking for this predicate.
Vladimir Vapnik (31:16.400)
I know technically how to do that.
Lex Fridman (31:19.440)
And you can, of course, do it using machine.
Lex Fridman (31:25.560)
But I'm not sure that we will construct the smartest
Lex Fridman (31:29.560)
predicate.
Lex Fridman (31:30.920)
But this is the, allow me to linger on it.
Lex Fridman (31:34.120)
Because that's the essence.
Vladimir Vapnik (31:35.280)
That's the challenge.
Lex Fridman (31:36.240)
That is artificial.
Vladimir Vapnik (31:37.600)
That's the human level intelligence
Lex Fridman (31:40.320)
that we seek is the discovery of these good predicates.
Vladimir Vapnik (31:43.720)
You've talked about deep learning as a way to,
Lex Fridman (31:47.560)
the predicates they use and the functions are mediocre.
Vladimir Vapnik (31:52.960)
You can find better ones.
Lex Fridman (31:55.000)
Let's talk about deep learning.
Vladimir Vapnik (31:57.280)
Sure, let's do it.
Lex Fridman (31:58.360)
I know only Jan's Likun convolutional network.
Lex Fridman (32:04.200)
And what else?
Lex Fridman (32:05.160)
I don't know.
Lex Fridman (32:05.920)
And it's a very simple convolution.
Lex Fridman (32:07.960)
There's not much else to know.
Vladimir Vapnik (32:09.120)
To pixel left and right.
Lex Fridman (32:10.400)
I can do it like that with one predicate.
Vladimir Vapnik (32:14.600)
Convolution is a single predicate.
Lex Fridman (32:16.640)
It's single.
Vladimir Vapnik (32:17.600)
It's single predicate.
Lex Fridman (32:21.120)
Yes, but that's it.
Vladimir Vapnik (32:22.680)
You know exactly.
Lex Fridman (32:23.680)
You take the derivative for translation and predicate.
Vladimir Vapnik (32:28.320)
This should be kept.
Lex Fridman (32:31.040)
So that's a single predicate.
Lex Fridman (32:32.440)
But humans discovered that one.
Lex Fridman (32:34.200)
Or at least.
Vladimir Vapnik (32:35.760)
Not it.
Lex Fridman (32:36.240)
That is a risk.
Vladimir Vapnik (32:37.120)
Not too many predicates.
Lex Fridman (32:38.960)
And that is big story because Jan did it 25 years ago
Lex Fridman (32:43.720)
and nothing so clear was added to deep network.
Lex Fridman (32:50.160)
And then I don't understand why we
Vladimir Vapnik (32:55.400)
should talk about deep network instead of talking
Lex Fridman (32:58.400)
about piecewise linear functions which keeps this predicate.
Vladimir Vapnik (33:02.840)
Well, a counter argument is that maybe the amount
Lex Fridman (33:08.720)
of predicates necessary to solve general intelligence,
Vladimir Vapnik (33:14.480)
say in the space of images, doing
Lex Fridman (33:16.720)
efficient recognition of handwritten digits
Vladimir Vapnik (33:20.640)
is very small.
Lex Fridman (33:22.400)
And so we shouldn't be so obsessed about finding.
Vladimir Vapnik (33:26.840)
We'll find other good predicates like convolution, for example.
Lex Fridman (33:30.720)
There has been other advancements
Vladimir Vapnik (33:33.880)
like if you look at the work with attention,
Lex Fridman (33:37.400)
there's intentional mechanisms in especially used
Vladimir Vapnik (33:40.720)
in natural language focusing the network's ability
Lex Fridman (33:44.160)
to learn at which part of the input to look at.
Vladimir Vapnik (33:47.640)
The thing is, there's other things besides predicates
Lex Fridman (33:51.000)
that are important for the actual engineering mechanism
Vladimir Vapnik (33:55.280)
of showing how much you can really
Lex Fridman (33:57.240)
do given these predicates.
Vladimir Vapnik (34:02.120)
I mean, that's essentially the work of deep learning
Lex Fridman (34:04.360)
is constructing architectures that are able to be,
Vladimir Vapnik (34:09.000)
given the training data, to be able to converge
Lex Fridman (34:13.720)
towards a function that can generalize well.
Vladimir Vapnik (34:22.920)
It's an engineering problem.
Lex Fridman (34:24.400)
Yeah, I understand.
Lex Fridman (34:26.120)
But let's talk not on emotional level,
Lex Fridman (34:29.840)
but on a mathematical level.
Vladimir Vapnik (34:31.920)
You have set of piecewise linear functions.
Lex Fridman (34:36.480)
It is all possible neural networks.
Vladimir Vapnik (34:42.040)
It's just piecewise linear functions.
Lex Fridman (34:44.040)
It's many, many pieces.
Vladimir Vapnik (34:45.360)
Large number of piecewise linear functions.
Lex Fridman (34:47.640)
Exactly.
Vladimir Vapnik (34:48.640)
Very large.
Lex Fridman (34:49.440)
Very large.
Vladimir Vapnik (34:50.160)
Almost feels like too large.
Lex Fridman (34:51.800)
It's still simpler than, say, convolution,
Vladimir Vapnik (34:56.160)
than reproducing kernel Hilbert space, which
Lex Fridman (34:59.040)
have a Hilbert set of functions.
Lex Fridman (35:00.920)
What's Hilbert space?
Lex Fridman (35:02.960)
It's space with infinite number of coordinates,
Vladimir Vapnik (35:07.040)
say, or function for expansion, something like that.
Lex Fridman (35:11.840)
So it's much richer.
Lex Fridman (35:14.760)
And when I'm talking about closed form solution,
Lex Fridman (35:17.520)
I'm talking about this set of function,
Vladimir Vapnik (35:20.760)
not piecewise linear set, which is particular case of it
Lex Fridman (35:29.760)
is small part.
Lex Fridman (35:31.000)
So neural networks is a small part
Lex Fridman (35:32.960)
of the space of functions you're talking about.
Vladimir Vapnik (35:35.960)
Say, small set of functions.
Lex Fridman (35:39.160)
Let me take that.
Lex Fridman (35:40.600)
But it is fine.
Lex Fridman (35:42.080)
It is fine.
Vladimir Vapnik (35:42.760)
I don't want to discuss the small or big.
Lex Fridman (35:46.560)
You take advantage.
Lex Fridman (35:47.920)
So you have some set of functions.
Lex Fridman (35:51.040)
So now, when you're trying to create architecture,
Vladimir Vapnik (35:55.320)
you would like to create admissible set of functions,
Lex Fridman (35:58.800)
which all your tricks to use not all functions,
Lex Fridman (36:03.280)
but some subset of this set of functions.
Lex Fridman (36:07.200)
Say, when you're introducing convolutional net,
Vladimir Vapnik (36:10.040)
it is way to make this subset useful for you.
Lex Fridman (36:16.440)
But from my point of view, convolutional,
Vladimir Vapnik (36:19.760)
it is something you want to keep some invariants,
Lex Fridman (36:24.800)
say, translation invariants.
Lex Fridman (36:27.920)
But now, if you understand this and you cannot explain
Lex Fridman (36:35.440)
on the level of ideas what neural network does,
Vladimir Vapnik (36:41.240)
you should agree that it is much better
Lex Fridman (36:44.360)
to have a set of functions.
Lex Fridman (36:46.640)
And they say, this set of functions should be admissible.
Lex Fridman (36:51.040)
It must keep this invariant, this invariant,
Lex Fridman (36:53.640)
and that invariant.
Lex Fridman (36:55.200)
You know that as soon as you incorporate
Vladimir Vapnik (36:58.240)
new invariant set of function, because smaller and smaller
Lex Fridman (37:01.160)
and smaller.
Lex Fridman (37:02.080)
But all the invariants are specified by you, the human.
Lex Fridman (37:06.640)
Yeah, but what I hope that there is a standard predicate,
Vladimir Vapnik (37:12.400)
like PROPSHOW, that's what I want
Lex Fridman (37:17.520)
to find for digit recognition.
Vladimir Vapnik (37:19.560)
If we start, it is completely new area,
Lex Fridman (37:22.920)
what is intelligence about on the level,
Vladimir Vapnik (37:25.800)
starting from Plato's idea, what is world of ideas.
Lex Fridman (37:32.600)
And I believe that is not too many.
Lex Fridman (37:36.640)
But it is amusing that mathematicians doing something,
Lex Fridman (37:40.680)
a neural network in general function,
Lex Fridman (37:44.000)
but people from literature, from art, they use this all
Lex Fridman (37:48.720)
the time.
Vladimir Vapnik (37:49.400)
That's right.
Lex Fridman (37:50.040)
Invariants saying, it is great how people describe music.
Vladimir Vapnik (37:57.000)
We should learn from that.
Lex Fridman (37:58.720)
And something on this level.
Lex Fridman (38:02.000)
But so why Vladimir Propp, who was just theoretical,
Lex Fridman (38:09.200)
who studied theoretical literature, he found that.
Lex Fridman (38:12.960)
You know what?
Lex Fridman (38:13.720)
Let me throw that right back at you,
Vladimir Vapnik (38:15.200)
because there's a little bit of a,
Lex Fridman (38:17.280)
that's less mathematical and more emotional, philosophical,
Vladimir Vapnik (38:21.000)
Vladimir Propp.
Lex Fridman (38:22.680)
I mean, he wasn't doing math.
Vladimir Vapnik (38:24.920)
No.
Lex Fridman (38:26.840)
And you just said another emotional statement,
Vladimir Vapnik (38:30.160)
which is you believe that this Plato world of ideas is small.
Lex Fridman (38:35.760)
I hope.
Vladimir Vapnik (38:36.920)
I hope.
Lex Fridman (38:38.680)
Do you, what's your intuition, though?
Vladimir Vapnik (38:42.160)
If we can linger on it.
Lex Fridman (38:44.600)
You know, it is not just small or big.
Vladimir Vapnik (38:48.520)
I know exactly.
Lex Fridman (38:50.520)
Then when I introducing some predicate,
Vladimir Vapnik (38:56.880)
I decrease set of functions.
Lex Fridman (38:59.760)
But my goal to decrease set of function much.
Vladimir Vapnik (39:04.040)
By as much as possible.
Lex Fridman (39:05.000)
By as much as possible.
Vladimir Vapnik (39:07.480)
Good predicate, which does this, then
Lex Fridman (39:11.400)
I should choose next predicate, which decrease set
Vladimir Vapnik (39:15.560)
as much as possible.
Lex Fridman (39:17.320)
So set of good predicate, it is such
Vladimir Vapnik (39:21.400)
that they decrease this amount of admissible function.
Lex Fridman (39:27.880)
So if each good predicate significantly
Vladimir Vapnik (39:30.520)
reduces the set of admissible functions,
Lex Fridman (39:32.640)
that there naturally should not be that many good predicates.
Vladimir Vapnik (39:35.560)
No, but if you reduce very well the VC dimension
Lex Fridman (39:43.040)
of the function, of admissible set of function, it's small.
Lex Fridman (39:46.760)
And you need not too much training data to do well.
Lex Fridman (39:52.960)
And VC dimension, by the way, is some measure of capacity
Vladimir Vapnik (39:56.760)
of this set of functions.
Lex Fridman (39:57.720)
Right.
Vladimir Vapnik (39:59.400)
Roughly speaking, how many function in this set.
Lex Fridman (40:01.960)
So you're decreasing, decreasing.
Lex Fridman (40:03.880)
And it makes easy for you to find function
Lex Fridman (40:08.160)
you're looking for.
Lex Fridman (40:10.200)
But the most important part, to create good admissible set
Lex Fridman (40:14.480)
of functions.
Lex Fridman (40:15.680)
And it probably, there are many ways.
Lex Fridman (40:18.800)
But the good predicates such that they can do that.
Lex Fridman (40:25.880)
So for this duck, you should know a little bit about duck.
Lex Fridman (40:30.520)
Because what are the three fundamental laws of ducks?
Vladimir Vapnik (40:35.280)
Looks like a duck, swims like a duck, and quacks like a duck.
Lex Fridman (40:38.360)
You should know something about ducks to be able to.
Vladimir Vapnik (40:41.160)
Not necessarily.
Lex Fridman (40:42.480)
Looks like, say, horse.
Vladimir Vapnik (40:44.920)
It's also good.
Lex Fridman (40:46.520)
So it's not, it generalizes from ducks.
Lex Fridman (40:49.840)
And talk like, and make sound like horse or something.
Lex Fridman (40:54.280)
And run like horse, and moves like horse.
Vladimir Vapnik (40:57.320)
It is general, it is general predicate
Lex Fridman (41:02.000)
that this applied to duck.
Lex Fridman (41:04.560)
But for duck, you can say, play chess like duck.
Lex Fridman (41:09.800)
You cannot say play chess like duck.
Lex Fridman (41:11.520)
Why not?
Lex Fridman (41:12.600)
So you're saying you can, but that would not be a good.
Vladimir Vapnik (41:15.680)
No, you will not reduce a lot of functions.
Lex Fridman (41:18.160)
You would not do, yeah, you would not
Vladimir Vapnik (41:19.760)
reduce the set of functions.
Lex Fridman (41:21.600)
So you can, the story is formal story, mathematical story.
Vladimir Vapnik (41:26.760)
Is that you can use any function you want as a predicate.
Lex Fridman (41:31.120)
But some of them are good, some of them are not,
Vladimir Vapnik (41:33.160)
because some of them reduce a lot of functions
Lex Fridman (41:36.880)
to admissible set of some of them.
Lex Fridman (41:39.720)
But the question is, and I'll probably
Lex Fridman (41:41.440)
keep asking this question, but how do we find such,
Lex Fridman (41:45.680)
what's your intuition?
Lex Fridman (41:47.360)
Handwritten recognition.
Lex Fridman (41:49.400)
How do we find the answer to your challenge?
Lex Fridman (41:52.600)
Yeah, I understand it like that.
Vladimir Vapnik (41:55.840)
I understand what.
Lex Fridman (41:57.800)
What defined?
Lex Fridman (41:59.160)
What it means, I knew predicate.
Lex Fridman (42:01.680)
Yeah.
Vladimir Vapnik (42:02.720)
Like guy who understand music can say this word,
Lex Fridman (42:06.160)
which he described when he listened to music.
Vladimir Vapnik (42:09.520)
He understand music.
Lex Fridman (42:11.600)
He use not too many different, oh, you can do like prop.
Vladimir Vapnik (42:15.480)
You can make collection.
Lex Fridman (42:17.280)
What he talking about music, about this, about that.
Vladimir Vapnik (42:20.920)
It's not too many different situation he described.
Lex Fridman (42:24.960)
Because we mentioned Vladimir prop a bunch.
Vladimir Vapnik (42:26.920)
Let me just mention, there's a sequence of 31
Lex Fridman (42:33.640)
structural notions that are common in stories.
Lex Fridman (42:36.880)
And I think.
Lex Fridman (42:37.720)
You call it units.
Vladimir Vapnik (42:38.560)
Units.
Lex Fridman (42:39.400)
And I think they resonate.
Vladimir Vapnik (42:40.480)
I mean, it starts just to give an example,
Lex Fridman (42:43.600)
obsession, a member of the hero's community,
Vladimir Vapnik (42:46.040)
a family leaves the security of the home environment.
Lex Fridman (42:48.920)
Then it goes to the interdiction,
Vladimir Vapnik (42:51.040)
a forbidding edict or command is passed upon the hero.
Lex Fridman (42:54.520)
Don't go there.
Vladimir Vapnik (42:55.360)
Don't do this.
Lex Fridman (42:56.640)
The hero is warned against some action.
Vladimir Vapnik (42:58.680)
Then step three, violation of interdiction.
Lex Fridman (43:05.280)
Break the rules, break out on your own.
Vladimir Vapnik (43:07.600)
Then reconnaissance.
Lex Fridman (43:09.200)
The villain makes an effort to attain knowledge,
Vladimir Vapnik (43:11.400)
needing to fulfill their plan, so on.
Lex Fridman (43:13.160)
It goes on like this, ends in a wedding, number 31.
Vladimir Vapnik (43:19.480)
Happily ever after.
Lex Fridman (43:20.640)
No, he just gave description of all situations.
Vladimir Vapnik (43:26.000)
He understands this world.
Lex Fridman (43:28.160)
Of folktales.
Vladimir Vapnik (43:29.280)
Yeah, not folktales, but stories.
Lex Fridman (43:33.160)
And these stories not in just folktales.
Vladimir Vapnik (43:36.560)
These stories in detective serials as well.
Lex Fridman (43:40.880)
And probably in our lives.
Vladimir Vapnik (43:42.200)
We probably live.
Lex Fridman (43:43.760)
Read this.
Lex Fridman (43:45.040)
And then they wrote that this predicate is good
Lex Fridman (43:52.040)
for different situation.
Vladimir Vapnik (43:54.800)
From movie, for theater.
Lex Fridman (43:57.920)
By the way, there's also criticism, right?
Vladimir Vapnik (44:00.640)
There's an other way to interpret narratives
Lex Fridman (44:03.840)
from Claude Levi Strauss.
Vladimir Vapnik (44:09.880)
I don't know.
Lex Fridman (44:10.880)
I am not in this business.
Vladimir Vapnik (44:12.520)
No, I know, it's theoretical literature,
Lex Fridman (44:14.360)
but it's looking at paradigms behind things.
Vladimir Vapnik (44:15.840)
It's always the discussion, yeah.
Lex Fridman (44:20.120)
But at least there is units.
Vladimir Vapnik (44:23.800)
It's not too many units that can describe.
Lex Fridman (44:27.160)
But this guy probably gives another units.
Vladimir Vapnik (44:30.840)
Or another way of...
Lex Fridman (44:31.680)
Exactly, another set of units.
Vladimir Vapnik (44:34.400)
Another set of predicates.
Lex Fridman (44:35.920)
It doesn't matter how.
Lex Fridman (44:37.560)
But they exist.
Lex Fridman (44:40.120)
Probably.
Vladimir Vapnik (44:40.960)
My question is, whether given those units,
Lex Fridman (44:46.240)
whether without our human brains to interpret these units,
Vladimir Vapnik (44:50.360)
they would still hold as much power as they have.
Lex Fridman (44:53.480)
Meaning, are those units enough
Lex Fridman (44:56.200)
when we give them to an alien species?
Lex Fridman (44:58.840)
Let me ask you.
Lex Fridman (45:00.320)
Do you understand digit images?
Lex Fridman (45:06.280)
No, I don't understand.
Vladimir Vapnik (45:07.600)
No, no, no.
Lex Fridman (45:08.640)
When you can recognize these digit images,
Vladimir Vapnik (45:11.160)
it means that you understand.
Lex Fridman (45:13.320)
Yes, exactly.
Vladimir Vapnik (45:14.160)
You understand characters, you understand...
Lex Fridman (45:17.280)
No, no, no, no.
Vladimir Vapnik (45:22.720)
It's the imitation versus understanding question,
Lex Fridman (45:25.480)
because I don't understand the mechanism
Vladimir Vapnik (45:28.360)
by which I understand.
Lex Fridman (45:29.200)
No, no, no.
Vladimir Vapnik (45:30.040)
I'm not talking about, I'm talking about predicates.
Lex Fridman (45:32.760)
You understand that it involves symmetry,
Vladimir Vapnik (45:35.120)
maybe structure, maybe something else.
Lex Fridman (45:37.400)
I cannot formulate.
Vladimir Vapnik (45:38.720)
I just was able to find symmetries, degree of symmetries.
Lex Fridman (45:43.640)
That's really good.
Lex Fridman (45:44.480)
So this is a good line.
Lex Fridman (45:47.200)
I feel like I understand the basic elements
Vladimir Vapnik (45:50.560)
of what makes a good hand recognition system my own.
Lex Fridman (45:54.280)
Like symmetry connects with me.
Vladimir Vapnik (45:56.440)
It seems like that's a very powerful predicate.
Lex Fridman (45:59.120)
My question is, is there a lot more going on
Lex Fridman (46:02.400)
that we're not able to introspect?
Lex Fridman (46:04.480)
Maybe I need to be able to understand
Vladimir Vapnik (46:09.600)
a huge amount in the world of ideas,
Lex Fridman (46:14.520)
thousands of predicates, millions of predicates
Vladimir Vapnik (46:18.400)
in order to do hand recognition.
Lex Fridman (46:20.600)
I don't think so.
Lex Fridman (46:23.200)
So both your hope and your intuition
Lex Fridman (46:26.560)
are such that very few predicates are enough.
Vladimir Vapnik (46:28.960)
You're using digits, you're using examples as well.
Lex Fridman (46:33.480)
Theory says that if you will use all possible functions
Vladimir Vapnik (46:43.480)
from Hilbert space, all possible predicate,
Lex Fridman (46:46.360)
you don't need training data.
Vladimir Vapnik (46:49.000)
You just will have admissible set of function
Lex Fridman (46:53.840)
which contain one function.
Vladimir Vapnik (46:56.060)
Yes.
Lex Fridman (46:57.160)
So the trade off is when you're not using all predicates,
Vladimir Vapnik (47:01.160)
you're only using a few good predicates
Lex Fridman (47:03.040)
you need to have some training data.
Vladimir Vapnik (47:05.000)
Yes, exactly.
Lex Fridman (47:06.800)
The more good predicates you have,
Vladimir Vapnik (47:08.440)
the less training data you need.
Lex Fridman (47:09.680)
Exactly.
Vladimir Vapnik (47:10.960)
That is intelligent.
Lex Fridman (47:13.280)
Still, okay, I'm gonna keep asking the same dumb question,
Vladimir Vapnik (47:17.400)
handwritten recognition to solve the challenge.
Lex Fridman (47:20.200)
You kind of propose a challenge that says
Vladimir Vapnik (47:21.920)
we should be able to get state of the art MNIST error rates
Lex Fridman (47:27.100)
by using very few, 60, maybe fewer examples per digit.
Lex Fridman (47:31.480)
What kind of predicates do you think it will look like?
Lex Fridman (47:35.920)
That is the challenge.
Lex Fridman (47:37.520)
So people who will solve this problem,
Lex Fridman (47:39.760)
they will answer.
Lex Fridman (47:41.480)
Do you think they'll be able to answer it
Lex Fridman (47:44.720)
in a human explainable way?
Vladimir Vapnik (47:47.800)
They just need to write function, that's it.
Lex Fridman (47:50.760)
But so can that function be written, I guess,
Lex Fridman (47:54.280)
by an automated reasoning system?
Lex Fridman (47:58.680)
Whether we're talking about a neural network
Lex Fridman (48:01.080)
learning a particular function or another mechanism?
Lex Fridman (48:05.040)
No, I'm not against neural network.
Vladimir Vapnik (48:08.520)
I'm against admissible set of function
Lex Fridman (48:11.600)
which create neural network.
Vladimir Vapnik (48:13.720)
You did it by hand.
Lex Fridman (48:16.360)
You don't do it by invariance, by predicate, by reason.
Lex Fridman (48:24.600)
But neural networks can then reverse,
Lex Fridman (48:26.400)
do the reverse step of helping you find a function
Vladimir Vapnik (48:29.840)
that just, the task of a neural network
Lex Fridman (48:33.600)
is to find a disentangled representation, for example,
Vladimir Vapnik (48:38.160)
that they call, is to find that one predicate function
Lex Fridman (48:42.120)
that's really capture some kind of essence.
Vladimir Vapnik (48:45.180)
One, not the entire essence, but one very useful essence
Lex Fridman (48:48.600)
of this particular visual space.
Lex Fridman (48:52.640)
Do you think that's possible?
Lex Fridman (48:53.840)
Listen, I'm grasping, hoping there's an automated way
Lex Fridman (48:58.620)
to find good predicates, right?
Lex Fridman (49:00.300)
So the question is what are the mechanisms
Vladimir Vapnik (49:03.000)
of finding good predicates, ideas
Lex Fridman (49:05.760)
that you think we should pursue?
Vladimir Vapnik (49:08.040)
A young grad student listening right now.
Lex Fridman (49:11.240)
I gave example.
Lex Fridman (49:13.360)
So find situation where predicate which you're suggesting
Lex Fridman (49:23.480)
don't create invariant.
Vladimir Vapnik (49:24.980)
It's like in physics.
Lex Fridman (49:28.820)
Find situation where existing theory cannot explain it.
Vladimir Vapnik (49:37.180)
Find situation where the existing theory
Lex Fridman (49:39.420)
can't explain it.
Lex Fridman (49:40.260)
So you're finding contradictions.
Lex Fridman (49:42.780)
Find contradiction, and then remove this contradiction.
Lex Fridman (49:46.140)
But in my case, what means contradiction,
Lex Fridman (49:48.940)
you find function which, if you will use this function,
Vladimir Vapnik (49:53.500)
you're not keeping invariants.
Lex Fridman (49:56.900)
This is really the process of discovering contradictions.
Vladimir Vapnik (50:01.300)
Yeah.
Lex Fridman (50:04.060)
It is like in physics.
Vladimir Vapnik (50:05.900)
Find situation where you have contradiction
Lex Fridman (50:09.800)
for one of the property, for one of the predicate.
Vladimir Vapnik (50:15.500)
Then include this predicate, making invariants,
Lex Fridman (50:19.020)
and solve again this problem.
Vladimir Vapnik (50:20.460)
Now you don't have contradiction.
Lex Fridman (50:22.100)
But it is not the best way, probably, I don't know,
Vladimir Vapnik (50:30.380)
to looking for predicate.
Lex Fridman (50:31.980)
That's just one way, okay.
Vladimir Vapnik (50:33.580)
That, no, no, it is brute force way.
Lex Fridman (50:35.900)
The brute force way.
Lex Fridman (50:37.300)
What about the ideas of what,
Lex Fridman (50:42.300)
big umbrella term of symbolic AI?
Vladimir Vapnik (50:45.660)
There's what in the 80s with expert systems,
Lex Fridman (50:48.540)
sort of logic reasoning based systems.
Vladimir Vapnik (50:52.380)
Is there hope there to find some,
Lex Fridman (50:57.020)
through sort of deductive reasoning,
Lex Fridman (51:00.500)
to find good predicates?
Lex Fridman (51:05.540)
I don't think so.
Vladimir Vapnik (51:08.980)
I think that just logic is not enough.
Lex Fridman (51:12.020)
It's kind of a compelling notion, though.
Vladimir Vapnik (51:14.420)
You know, that when smart people sit in a room
Lex Fridman (51:17.620)
and reason through things, it seems compelling.
Lex Fridman (51:20.360)
And making our machines do the same is also compelling.
Lex Fridman (51:24.940)
So, everything is very simple.
Vladimir Vapnik (51:29.420)
When you have infinite number of predicate,
Lex Fridman (51:34.100)
you can choose the function you want.
Vladimir Vapnik (51:38.580)
You have invariants and you can choose the function you want.
Lex Fridman (51:41.660)
But you have to have not too many invariants
Vladimir Vapnik (51:51.880)
to solve the problem.
Lex Fridman (51:56.200)
So, and have from infinite number of function
Vladimir Vapnik (51:59.940)
to select finite number
Lex Fridman (52:04.120)
and hopefully small number of functions,
Vladimir Vapnik (52:08.460)
which is good enough to extract small set
Lex Fridman (52:14.920)
of admissible functions.
Vladimir Vapnik (52:17.920)
So, they will be admissible, it's for sure,
Lex Fridman (52:19.840)
because every function just decrease set of function
Lex Fridman (52:23.880)
and leaving it admissible.
Lex Fridman (52:25.680)
But it will be small.
Lex Fridman (52:27.720)
But why do you think logic based systems don't,
Lex Fridman (52:32.560)
can't help, intuition, not?
Vladimir Vapnik (52:35.280)
Because you should know reality.
Lex Fridman (52:37.800)
You should know life.
Vladimir Vapnik (52:39.480)
This guy like Propp, he knows something.
Lex Fridman (52:44.280)
And he tried to put in invariant his understanding.
Vladimir Vapnik (52:49.400)
That's the human, yeah, but see,
Lex Fridman (52:51.600)
you're putting too much value into Vladimir Propp
Vladimir Vapnik (52:56.480)
knowing something.
Lex Fridman (52:57.920)
No, it is, in the story, what means you know life?
Lex Fridman (53:04.420)
What it means?
Lex Fridman (53:05.400)
You know common sense.
Vladimir Vapnik (53:07.040)
No, no, you know something.
Lex Fridman (53:10.400)
Common sense, it is some rules.
Lex Fridman (53:13.440)
You think so?
Lex Fridman (53:14.800)
Common sense is simply rules?
Vladimir Vapnik (53:17.180)
Common sense is every, it's mortality,
Lex Fridman (53:21.800)
it's fear of death, it's love, it's spirituality,
Vladimir Vapnik (53:27.880)
it's happiness and sadness.
Lex Fridman (53:30.840)
All of it is tied up into understanding gravity,
Vladimir Vapnik (53:34.420)
which is what we think of as common sense.
Lex Fridman (53:36.840)
I don't really need to discuss so wide.
Vladimir Vapnik (53:39.840)
I want to discuss, understand digit recognition.
Lex Fridman (53:45.440)
Anytime I bring up love and death,
Vladimir Vapnik (53:47.640)
you bring it back to digit recognition, I like it.
Lex Fridman (53:51.160)
No, you know, it is durable because there is a challenge.
Vladimir Vapnik (53:55.200)
Yeah.
Lex Fridman (53:56.040)
Which I see how to solve it.
Vladimir Vapnik (53:59.260)
If I will have a student concentrate on this work,
Lex Fridman (54:02.520)
I will suggest something to solve.
Lex Fridman (54:04.800)
You mean handwritten record?
Lex Fridman (54:07.000)
Yeah, it's a beautifully simple, elegant, and yet.
Vladimir Vapnik (54:10.800)
I think that I know invariants which will solve this.
Lex Fridman (54:13.440)
You do?
Vladimir Vapnik (54:14.280)
I think so, yes.
Lex Fridman (54:15.920)
But it is not universal, it is maybe,
Vladimir Vapnik (54:21.600)
I want some universal invariants
Lex Fridman (54:24.160)
which are good not only for digit recognition,
Vladimir Vapnik (54:27.360)
for image understanding.
Lex Fridman (54:28.760)
So let me ask, how hard do you think
Lex Fridman (54:34.160)
is 2D image understanding?
Lex Fridman (54:38.360)
So if we, we can kind of intuit handwritten recognition.
Lex Fridman (54:43.800)
How big of a step, leap, journey is it from that?
Lex Fridman (54:49.160)
If I gave you good, if I solved your challenge
Vladimir Vapnik (54:51.920)
for handwritten recognition,
Lex Fridman (54:53.600)
how long would my journey then be from that
Lex Fridman (54:56.480)
to understanding more general, natural images?
Lex Fridman (54:59.360)
Immediately, you will understand this
Vladimir Vapnik (55:01.920)
as soon as you will make a record.
Lex Fridman (55:05.400)
Because it is not for free.
Vladimir Vapnik (55:07.720)
As soon as you will create several invariants
Lex Fridman (55:13.000)
which will help you to get the same performance
Vladimir Vapnik (55:20.120)
that the best neural net did using 100,
Lex Fridman (55:23.880)
there might be more than 100 times less examples,
Vladimir Vapnik (55:27.760)
you have to have something smart to do that.
Lex Fridman (55:31.220)
And you're saying?
Vladimir Vapnik (55:32.220)
That is invariant, it is predicate.
Lex Fridman (55:35.160)
Because you should put some idea how to do that.
Lex Fridman (55:39.420)
But okay, let me just pause.
Lex Fridman (55:42.380)
Maybe it's a trivial point, maybe not.
Lex Fridman (55:44.520)
But handwritten recognition feels like a 2D,
Lex Fridman (55:48.840)
two dimensional problem.
Lex Fridman (55:50.440)
And it seems like how much complicated is the fact
Lex Fridman (55:55.360)
that most images are projection of a three dimensional world
Vladimir Vapnik (56:00.400)
onto a 2D plane.
Lex Fridman (56:03.100)
It feels like for a three dimensional world,
Vladimir Vapnik (56:05.880)
we need to start understanding common sense
Lex Fridman (56:08.660)
in order to understand an image.
Vladimir Vapnik (56:11.960)
It's no longer visual shape and symmetry.
Lex Fridman (56:17.480)
It's having to start to understand concepts
Vladimir Vapnik (56:19.920)
of, understand life.
Lex Fridman (56:22.120)
Yeah, you're talking that there are different invariant,
Vladimir Vapnik (56:27.320)
different predicate, yeah.
Lex Fridman (56:28.920)
And potentially much larger number.
Vladimir Vapnik (56:32.480)
You know, maybe, but let's start from simple.
Lex Fridman (56:36.360)
Yeah, but you said that it would be immediate.
Vladimir Vapnik (56:38.200)
No, you know, I cannot think about things
Lex Fridman (56:41.360)
which I don't understand.
Vladimir Vapnik (56:43.280)
This I understand, but I'm sure that I don't understand
Lex Fridman (56:46.920)
everything there.
Vladimir Vapnik (56:48.440)
Yeah, that's the difference.
Lex Fridman (56:50.440)
Do as simple as possible, but not simpler.
Lex Fridman (56:54.360)
And that is exact case.
Lex Fridman (56:56.520)
With handwritten.
Vladimir Vapnik (56:57.440)
With handwritten.
Lex Fridman (56:58.940)
Yeah, but that's the difference between you and I.
Vladimir Vapnik (57:04.880)
I welcome and enjoy thinking about things
Lex Fridman (57:07.920)
I completely don't understand.
Vladimir Vapnik (57:09.880)
Because to me, it's a natural extension
Lex Fridman (57:12.380)
without having solved handwritten recognition
Vladimir Vapnik (57:15.140)
to wonder how difficult is the next step
Lex Fridman (57:23.280)
of understanding 2D, 3D images.
Vladimir Vapnik (57:25.680)
Because ultimately, while the science of intelligence
Lex Fridman (57:29.240)
is fascinating, it's also fascinating to see
Lex Fridman (57:31.680)
how that maps to the engineering of intelligence.
Lex Fridman (57:34.680)
And recognizing handwritten digits is not,
Vladimir Vapnik (57:39.280)
doesn't help you, it might, it may not help you
Lex Fridman (57:43.080)
with the problem of general intelligence.
Vladimir Vapnik (57:46.560)
We don't know.
Lex Fridman (57:47.400)
It'll help you a little bit.
Vladimir Vapnik (57:48.240)
We don't know how much.
Lex Fridman (57:49.080)
It's unclear.
Vladimir Vapnik (57:49.900)
It's unclear.
Lex Fridman (57:50.740)
Yeah.
Vladimir Vapnik (57:51.580)
It might very much.
Lex Fridman (57:52.400)
But I would like to make a remark.
Vladimir Vapnik (57:53.240)
Yes.
Lex Fridman (57:54.080)
I start not from very primitive problem,
Vladimir Vapnik (57:58.760)
make a challenge problem.
Lex Fridman (58:03.120)
I start with very general problem, with PLATO.
Lex Fridman (58:07.640)
So you understand, and it comes from PLATO
Lex Fridman (58:10.640)
to digit recognition.
Lex Fridman (58:14.000)
So you basically took PLATO and the world
Lex Fridman (58:18.120)
of forms and ideas and mapped and projected
Vladimir Vapnik (58:22.080)
into the clearest, simplest formulation
Lex Fridman (58:25.380)
of that big world.
Vladimir Vapnik (58:26.820)
You know, I would say that I did not understand PLATO
Lex Fridman (58:31.560)
until recently, and until I consider
Vladimir Vapnik (58:36.560)
the convergence and then predicate,
Lex Fridman (58:40.800)
and then, oh, this is what PLATO told.
Vladimir Vapnik (58:45.520)
So.
Lex Fridman (58:46.360)
Can you linger on that?
Vladimir Vapnik (58:47.180)
Like why, how do you think about this world of ideas
Lex Fridman (58:50.200)
and world of things in PLATO?
Vladimir Vapnik (58:52.880)
No, it is metaphor.
Lex Fridman (58:54.160)
It is.
Vladimir Vapnik (58:55.000)
It's a metaphor, for sure.
Lex Fridman (58:55.840)
Yeah.
Vladimir Vapnik (58:56.680)
It's a compelling, it's a poetic
Lex Fridman (58:57.500)
and a beautiful metaphor.
Vladimir Vapnik (58:58.340)
Yeah, yeah, yeah.
Lex Fridman (58:59.180)
But what, can you?
Lex Fridman (59:00.560)
But it is a way how you should try to understand
Lex Fridman (59:04.960)
how to talk ideas in the world.
Lex Fridman (59:07.880)
So from my point of view,
Lex Fridman (59:11.240)
it is very clear, but it is lying.
Vladimir Vapnik (59:14.900)
All the time, people looking for that.
Lex Fridman (59:17.520)
Say, PLATO, then Hegel, whatever reasonable it exists,
Vladimir Vapnik (59:24.320)
whatever exists, it is reasonable.
Lex Fridman (59:26.700)
I don't know what he have in mind reasonable.
Vladimir Vapnik (59:30.240)
Right, this philosophers again,
Lex Fridman (59:31.600)
their words. No, no, no, no, no, no, no.
Vladimir Vapnik (59:33.320)
It is next stop of Wigner.
Lex Fridman (59:37.120)
That mathematics understand something of reality.
Vladimir Vapnik (59:40.760)
It is the same PLATO line.
Lex Fridman (59:43.440)
And then it comes suddenly to Vladimir Propp.
Vladimir Vapnik (59:48.160)
Look, 31 ideas, 31 units, and this corrects everything.
Lex Fridman (59:54.320)
There's abstractions, ideas that represent our world.
Vladimir Vapnik (59:59.320)
Our world, and we should always try to reach into that.
🔗 相关节目