Vijay Kumar: Flying Robots
AI 与机器学习技术与编程心理与人性政治与社会音乐与艺术
📋 章节目录
暂无章节信息
🔑 关键词
robotshumanrobotlearningautonomousflyingflydonroboticsinterestingindividualhardautonomymotorsvehiclesbuildingcomputerfourdimensionaltrue
💬 精彩语录
暂无语录
🎙️ 完整对话（1196 条）
Lex Fridman (00:00.000)
The following is a conversation with Vijay Kumar.
Lex Fridman (00:03.080)
He's one of the top roboticists in the world,
Lex Fridman (00:05.760)
a professor at the University of Pennsylvania,
Lex Fridman (00:08.760)
a dean of pen engineering, former director of Grasp Lab,
Lex Fridman (00:12.880)
or the General Robotics Automation Sensing
Lex Fridman (00:15.300)
and Perception Laboratory at Penn,
Lex Fridman (00:17.560)
that was established back in 1979, that's 40 years ago.
Vijay Kumar (00:22.600)
Vijay is perhaps best known for his work
Lex Fridman (00:25.280)
in multi robot systems, robot swarms,
Lex Fridman (00:28.520)
and micro aerial vehicles,
Lex Fridman (00:30.880)
robots that elegantly cooperate in flight
Vijay Kumar (00:34.020)
under all the uncertainty and challenges
Lex Fridman (00:36.200)
that the real world conditions present.
Vijay Kumar (00:38.760)
This is the Artificial Intelligence Podcast.
Lex Fridman (00:41.920)
If you enjoy it, subscribe on YouTube,
Vijay Kumar (00:44.320)
give it five stars on iTunes, support on Patreon,
Lex Fridman (00:47.560)
or simply connect with me on Twitter
Vijay Kumar (00:49.500)
at Lex Friedman, spelled F R I D M A N.
Lex Fridman (00:53.280)
And now, here's my conversation with Vijay Kumar.
Lex Fridman (00:58.700)
What is the first robot you've ever built
Lex Fridman (01:01.080)
or were a part of building?
Vijay Kumar (01:02.840)
Way back when I was in graduate school,
Lex Fridman (01:04.760)
I was part of a fairly big project
Vijay Kumar (01:06.760)
that involved building a very large hexapod.
Lex Fridman (01:12.040)
It's weighed close to 7,000 pounds,
Lex Fridman (01:17.520)
and it was powered by hydraulic actuation,
Lex Fridman (01:21.620)
or it was actuated by hydraulics with 18 motors,
Vijay Kumar (01:27.720)
hydraulic motors, each controlled by an Intel 8085 processor
Lex Fridman (01:34.160)
and an 8086 co processor.
Lex Fridman (01:38.120)
And so imagine this huge monster that had 18 joints,
Lex Fridman (01:44.800)
each controlled by an independent computer,
Lex Fridman (01:46.960)
and there was a 19th computer that actually did
Lex Fridman (01:49.320)
the coordination between these 18 joints.
Lex Fridman (01:52.320)
So I was part of this project,
Lex Fridman (01:53.720)
and my thesis work was how do you coordinate the 18 legs?
Lex Fridman (02:02.080)
And in particular, the pressures in the hydraulic cylinders
Lex Fridman (02:06.320)
to get efficient locomotion.
Vijay Kumar (02:09.200)
It sounds like a giant mess.
Lex Fridman (02:11.640)
So how difficult is it to make all the motors communicate?
Vijay Kumar (02:14.440)
Presumably, you have to send signals hundreds of times
Lex Fridman (02:17.600)
a second, or at least.
Lex Fridman (02:18.440)
So this was not my work,
Lex Fridman (02:19.880)
but the folks who worked on this wrote what I believe
Vijay Kumar (02:23.960)
to be the first multiprocessor operating system.
Lex Fridman (02:26.640)
This was in the 80s, and you had to make sure
Vijay Kumar (02:30.320)
that obviously messages got across
Lex Fridman (02:32.800)
from one joint to another.
Vijay Kumar (02:34.640)
You have to remember the clock speeds on those computers
Lex Fridman (02:37.960)
were about half a megahertz.
Vijay Kumar (02:39.660)
Right, the 80s.
Lex Fridman (02:42.180)
So not to romanticize the notion,
Lex Fridman (02:45.320)
but how did it make you feel to see that robot move?
Lex Fridman (02:51.080)
It was amazing.
Vijay Kumar (02:52.280)
In hindsight, it looks like, well, we built this thing
Lex Fridman (02:55.280)
which really should have been much smaller.
Lex Fridman (02:57.320)
And of course, today's robots are much smaller.
Lex Fridman (02:59.160)
You look at Boston Dynamics or Ghost Robotics,
Vijay Kumar (03:03.120)
a spinoff from Penn.
Lex Fridman (03:06.080)
But back then, you were stuck with the substrate you had,
Vijay Kumar (03:10.080)
the compute you had, so things were unnecessarily big.
Lex Fridman (03:13.720)
But at the same time, and this is just human psychology,
Vijay Kumar (03:18.040)
somehow bigger means grander.
Lex Fridman (03:21.600)
People never had the same appreciation
Vijay Kumar (03:23.640)
for nanotechnology or nanodevices
Lex Fridman (03:26.360)
as they do for the Space Shuttle or the Boeing 747.
Vijay Kumar (03:30.160)
Yeah, you've actually done quite a good job
Lex Fridman (03:32.760)
at illustrating that small is beautiful
Vijay Kumar (03:36.000)
in terms of robotics.
Lex Fridman (03:37.760)
So what is on that topic is the most beautiful
Lex Fridman (03:42.600)
or elegant robot in motion that you've ever seen?
Lex Fridman (03:46.200)
Not to pick favorites or whatever,
Lex Fridman (03:47.880)
but something that just inspires you that you remember.
Lex Fridman (03:51.000)
Well, I think the thing that I'm most proud of
Vijay Kumar (03:54.000)
that my students have done is really think about
Lex Fridman (03:57.200)
small UAVs that can maneuver in constrained spaces
Lex Fridman (04:00.360)
and in particular, their ability to coordinate
Lex Fridman (04:03.640)
with each other and form three dimensional patterns.
Lex Fridman (04:06.760)
So once you can do that,
Lex Fridman (04:08.920)
you can essentially create 3D objects in the sky
Lex Fridman (04:14.960)
and you can deform these objects on the fly.
Lex Fridman (04:17.680)
So in some sense, your toolbox of what you can create
Vijay Kumar (04:21.560)
has suddenly got enhanced.
Lex Fridman (04:25.240)
And before that, we did the two dimensional version of this.
Lex Fridman (04:27.800)
So we had ground robots forming patterns and so on.
Lex Fridman (04:31.680)
So that was not as impressive, that was not as beautiful.
Lex Fridman (04:34.960)
But if you do it in 3D,
Lex Fridman (04:36.560)
suspended in midair, and you've got to go back to 2011
Vijay Kumar (04:40.240)
when we did this, now it's actually pretty standard
Lex Fridman (04:43.040)
to do these things eight years later.
Lex Fridman (04:45.600)
But back then it was a big accomplishment.
Lex Fridman (04:47.680)
So the distributed cooperation
Lex Fridman (04:50.280)
is where beauty emerges in your eyes?
Lex Fridman (04:53.480)
Well, I think beauty to an engineer is very different
Vijay Kumar (04:55.800)
from beauty to someone who's looking at robots
Lex Fridman (04:59.400)
from the outside, if you will.
Lex Fridman (05:01.240)
But what I meant there, so before we said that grand,
Lex Fridman (05:04.800)
so before we said that grand is associated with size.
Lex Fridman (05:10.520)
And another way of thinking about this
Lex Fridman (05:13.720)
is just the physical shape
Lex Fridman (05:15.600)
and the idea that you can get physical shapes in midair
Lex Fridman (05:18.400)
and have them deform, that's beautiful.
Lex Fridman (05:21.560)
But the individual components,
Lex Fridman (05:23.040)
the agility is beautiful too, right?
Vijay Kumar (05:24.880)
That is true too.
Lex Fridman (05:25.720)
So then how quickly can you actually manipulate
Vijay Kumar (05:28.480)
these three dimensional shapes
Lex Fridman (05:29.560)
and the individual components?
Vijay Kumar (05:31.280)
Yes, you're right.
Lex Fridman (05:32.240)
But by the way, you said UAV, unmanned aerial vehicle.
Lex Fridman (05:36.760)
What's a good term for drones, UAVs, quad copters?
Lex Fridman (05:41.840)
Is there a term that's being standardized?
Vijay Kumar (05:44.560)
I don't know if there is.
Lex Fridman (05:45.440)
Everybody wants to use the word drones.
Lex Fridman (05:47.920)
And I've often said this, drones to me is a pejorative word.
Lex Fridman (05:51.080)
It signifies something that's dumb,
Vijay Kumar (05:53.960)
that's pre programmed, that does one little thing
Lex Fridman (05:56.360)
and robots are anything but drones.
Lex Fridman (05:58.600)
So I actually don't like that word,
Lex Fridman (06:00.680)
but that's what everybody uses.
Vijay Kumar (06:02.960)
You could call it unpiloted.
Lex Fridman (06:04.880)
Unpiloted.
Lex Fridman (06:05.800)
But even unpiloted could be radio controlled,
Lex Fridman (06:08.120)
could be remotely controlled in many different ways.
Lex Fridman (06:11.560)
And I think the right word is,
Lex Fridman (06:12.960)
thinking about it as an aerial robot.
Lex Fridman (06:15.040)
You also say agile, autonomous, aerial robot, right?
Lex Fridman (06:19.080)
Yeah, so agility is an attribute, but they don't have to be.
Lex Fridman (06:23.080)
So what biological system,
Lex Fridman (06:24.800)
because you've also drawn a lot of inspiration with those.
Vijay Kumar (06:27.200)
I've seen bees and ants that you've talked about.
Lex Fridman (06:30.360)
What living creatures have you found to be most inspiring
Lex Fridman (06:35.240)
as an engineer, instructive in your work in robotics?
Lex Fridman (06:38.520)
To me, so ants are really quite incredible creatures, right?
Lex Fridman (06:43.440)
So you, I mean, the individuals arguably are very simple
Lex Fridman (06:47.880)
in how they're built and yet they're incredibly resilient
Vijay Kumar (06:52.360)
as a population.
Lex Fridman (06:53.960)
And as individuals, they're incredibly robust.
Vijay Kumar (06:56.760)
So, if you take an ant, it's six legs,
Lex Fridman (07:00.600)
you remove one leg, it still works just fine.
Lex Fridman (07:04.120)
And it moves along.
Lex Fridman (07:05.760)
And I don't know that he even realizes it's lost a leg.
Lex Fridman (07:09.760)
So that's the robustness at the individual ant level.
Lex Fridman (07:13.400)
But then you look about this instinct
Vijay Kumar (07:15.360)
for self preservation of the colonies
Lex Fridman (07:17.680)
and they adapt in so many amazing ways.
Vijay Kumar (07:20.400)
You know, transcending gaps by just chaining themselves
Lex Fridman (07:26.800)
together when you have a flood,
Vijay Kumar (07:29.600)
being able to recruit other teammates
Lex Fridman (07:32.360)
to carry big morsels of food,
Lex Fridman (07:35.760)
and then going out in different directions looking for food,
Lex Fridman (07:38.760)
and then being able to demonstrate consensus,
Vijay Kumar (07:43.160)
even though they don't communicate directly with each other
Lex Fridman (07:47.040)
the way we communicate with each other.
Vijay Kumar (07:49.080)
In some sense, they also know how to do democracy,
Lex Fridman (07:51.880)
probably better than what we do.
Vijay Kumar (07:53.640)
Yeah, somehow it's even democracy is emergent.
Lex Fridman (07:57.000)
It seems like all of the phenomena that we see
Vijay Kumar (07:59.120)
is all emergent.
Lex Fridman (08:00.480)
It seems like there's no centralized communicator.
Vijay Kumar (08:03.560)
There is, so I think a lot is made about that word,
Lex Fridman (08:06.520)
emergent, and it means lots of things to different people.
Lex Fridman (08:09.640)
But you're absolutely right.
Lex Fridman (08:10.680)
I think as an engineer, you think about
Lex Fridman (08:13.040)
what element, elemental behaviors
Lex Fridman (08:17.720)
were primitives you could synthesize
Lex Fridman (08:21.320)
so that the whole looks incredibly powerful,
Lex Fridman (08:25.240)
incredibly synergistic,
Vijay Kumar (08:26.520)
the whole definitely being greater than some of the parts,
Lex Fridman (08:29.520)
and ants are living proof of that.
Lex Fridman (08:32.480)
So when you see these beautiful swarms
Lex Fridman (08:34.960)
where there's biological systems of robots,
Lex Fridman (08:38.520)
do you sometimes think of them
Lex Fridman (08:40.200)
as a single individual living intelligent organism?
Lex Fridman (08:44.640)
So it's the same as thinking of our human beings
Lex Fridman (08:47.400)
are human civilization as one organism,
Vijay Kumar (08:51.160)
or do you still, as an engineer,
Lex Fridman (08:52.960)
think about the individual components
Lex Fridman (08:54.600)
and all the engineering
Lex Fridman (08:55.440)
that went into the individual components?
Vijay Kumar (08:57.320)
Well, that's very interesting.
Lex Fridman (08:58.640)
So again, philosophically as engineers,
Lex Fridman (09:01.480)
what we wanna do is to go beyond
Lex Fridman (09:05.400)
the individual components, the individual units,
Lex Fridman (09:08.280)
and think about it as a unit, as a cohesive unit,
Lex Fridman (09:11.520)
without worrying about the individual components.
Vijay Kumar (09:15.120)
If you start obsessing about
Lex Fridman (09:17.760)
the individual building blocks and what they do,
Vijay Kumar (09:23.320)
you inevitably will find it hard to scale up.
Lex Fridman (09:27.960)
Just mathematically,
Vijay Kumar (09:29.000)
just think about individual things you wanna model,
Lex Fridman (09:31.600)
and if you want to have 10 of those,
Vijay Kumar (09:34.040)
then you essentially are taking Cartesian products
Lex Fridman (09:36.440)
of 10 things, and that makes it really complicated.
Vijay Kumar (09:39.320)
Then to do any kind of synthesis or design
Lex Fridman (09:41.840)
in that high dimension space is really hard.
Lex Fridman (09:44.200)
So the right way to do this
Lex Fridman (09:45.800)
is to think about the individuals in a clever way
Lex Fridman (09:49.040)
so that at the higher level,
Lex Fridman (09:51.120)
when you look at lots and lots of them,
Vijay Kumar (09:53.400)
abstractly, you can think of them
Lex Fridman (09:55.320)
in some low dimensional space.
Lex Fridman (09:57.120)
So what does that involve?
Lex Fridman (09:58.680)
For the individual, do you have to try to make
Lex Fridman (10:02.160)
the way they see the world as local as possible?
Lex Fridman (10:05.160)
And the other thing,
Lex Fridman (10:06.440)
do you just have to make them robust to collisions?
Lex Fridman (10:09.560)
Like you said with the ants,
Vijay Kumar (10:10.880)
if something fails, the whole swarm doesn't fail.
Lex Fridman (10:15.320)
Right, I think as engineers, we do this.
Vijay Kumar (10:17.760)
I mean, you think about, we build planes,
Lex Fridman (10:19.760)
or we build iPhones,
Lex Fridman (10:22.240)
and we know that by taking individual components,
Lex Fridman (10:26.280)
well engineered components with well specified interfaces
Vijay Kumar (10:30.080)
that behave in a predictable way,
Lex Fridman (10:31.680)
you can build complex systems.
Lex Fridman (10:34.440)
So that's ingrained, I would claim,
Lex Fridman (10:36.880)
in most engineers thinking,
Lex Fridman (10:39.400)
and it's true for computer scientists as well.
Lex Fridman (10:41.600)
I think what's different here is that you want
Vijay Kumar (10:44.760)
the individuals to be robust in some sense,
Lex Fridman (10:49.480)
as we do in these other settings,
Lex Fridman (10:52.000)
but you also want some degree of resiliency
Lex Fridman (10:54.480)
for the population.
Lex Fridman (10:56.320)
And so you really want them to be able to reestablish
Lex Fridman (11:02.040)
communication with their neighbors.
Vijay Kumar (11:03.840)
You want them to rethink their strategy for group behavior.
Lex Fridman (11:08.840)
You want them to reorganize.
Lex Fridman (11:12.200)
And that's where I think a lot of the challenges lie.
Lex Fridman (11:15.920)
So just at a high level,
Lex Fridman (11:18.160)
what does it take for a bunch of,
Lex Fridman (11:22.200)
what should we call them, flying robots,
Lex Fridman (11:24.440)
to create a formation?
Lex Fridman (11:26.680)
Just for people who are not familiar
Lex Fridman (11:28.680)
with robotics in general, how much information is needed?
Lex Fridman (11:32.760)
How do you even make it happen
Lex Fridman (11:35.840)
without a centralized controller?
Lex Fridman (11:39.520)
So, I mean, there are a couple of different ways
Vijay Kumar (11:41.080)
of looking at this.
Lex Fridman (11:43.160)
If you are a purist,
Vijay Kumar (11:45.680)
you think of it as a way of recreating what nature does.
Lex Fridman (11:53.560)
So nature forms groups for several reasons,
Lex Fridman (11:58.440)
but mostly it's because of this instinct
Lex Fridman (12:02.000)
that organisms have of preserving their colonies,
Lex Fridman (12:05.680)
their population, which means what?
Lex Fridman (12:09.520)
You need shelter, you need food, you need to procreate,
Lex Fridman (12:12.920)
and that's basically it.
Lex Fridman (12:14.760)
So the kinds of interactions you see are all organic.
Vijay Kumar (12:18.440)
They're all local.
Lex Fridman (12:20.760)
And the only information that they share,
Lex Fridman (12:24.080)
and mostly it's indirectly, is to, again,
Lex Fridman (12:27.520)
preserve the herd or the flock,
Vijay Kumar (12:30.000)
or the swarm, and either by looking for new sources of food
Lex Fridman (12:37.480)
or looking for new shelters, right?
Vijay Kumar (12:39.440)
Right.
Lex Fridman (12:41.240)
As engineers, when we build swarms, we have a mission.
Lex Fridman (12:46.560)
And when you think of a mission, and it involves mobility,
Lex Fridman (12:52.480)
most often it's described in some kind
Vijay Kumar (12:55.000)
of a global coordinate system.
Lex Fridman (12:56.880)
As a human, as an operator, as a commander,
Vijay Kumar (12:59.440)
or as a collaborator, I have my coordinate system,
Lex Fridman (13:03.560)
and I want the robots to be consistent with that.
Lex Fridman (13:07.600)
So I might think of it slightly differently.
Lex Fridman (13:11.240)
I might want the robots to recognize that coordinate system,
Vijay Kumar (13:15.440)
which means not only do they have to think locally
Lex Fridman (13:17.720)
in terms of who their immediate neighbors are,
Lex Fridman (13:19.600)
but they have to be cognizant
Lex Fridman (13:20.920)
of what the global environment is.
Vijay Kumar (13:24.040)
They have to be cognizant of what the global environment
Lex Fridman (13:27.040)
looks like.
Lex Fridman (13:28.280)
So if I say, surround this building
Lex Fridman (13:31.040)
and protect this from intruders,
Vijay Kumar (13:33.240)
well, they're immediately in a building centered
Lex Fridman (13:35.600)
coordinate system, and I have to tell them
Vijay Kumar (13:37.040)
where the building is.
Lex Fridman (13:38.680)
And they're globally collaborating
Vijay Kumar (13:40.040)
on the map of that building.
Lex Fridman (13:41.280)
They're maintaining some kind of global,
Vijay Kumar (13:44.160)
not just in the frame of the building,
Lex Fridman (13:45.480)
but there's information that's ultimately being built up
Vijay Kumar (13:49.000)
explicitly as opposed to kind of implicitly,
Lex Fridman (13:53.280)
like nature might.
Vijay Kumar (13:54.360)
Correct, correct.
Lex Fridman (13:55.200)
So in some sense, nature is very, very sophisticated,
Lex Fridman (13:57.680)
but the tasks that nature solves or needs to solve
Lex Fridman (14:01.880)
are very different from the kind of engineered tasks,
Vijay Kumar (14:05.160)
artificial tasks that we are forced to address.
Lex Fridman (14:09.760)
And again, there's nothing preventing us
Vijay Kumar (14:12.520)
from solving these other problems,
Lex Fridman (14:15.160)
but ultimately it's about impact.
Vijay Kumar (14:16.600)
You want these swarms to do something useful.
Lex Fridman (14:19.360)
And so you're kind of driven into this very unnatural,
Vijay Kumar (14:24.640)
if you will.
Lex Fridman (14:25.480)
Unnatural, meaning not like how nature does, setting.
Lex Fridman (14:29.160)
And it's probably a little bit more expensive
Lex Fridman (14:31.920)
to do it the way nature does,
Vijay Kumar (14:33.760)
because nature is less sensitive
Lex Fridman (14:37.560)
to the loss of the individual.
Lex Fridman (14:39.480)
And cost wise in robotics,
Lex Fridman (14:42.280)
I think you're more sensitive to losing individuals.
Vijay Kumar (14:45.480)
I think that's true, although if you look at the price
Lex Fridman (14:49.000)
to performance ratio of robotic components,
Lex Fridman (14:51.520)
it's coming down dramatically, right?
Lex Fridman (14:54.720)
It continues to come down.
Lex Fridman (14:56.040)
So I think we're asymptotically approaching the point
Lex Fridman (14:58.920)
where we would get, yeah,
Vijay Kumar (14:59.960)
the cost of individuals would really become insignificant.
Lex Fridman (15:05.040)
So let's step back at a high level view,
Vijay Kumar (15:07.640)
the impossible question of what kind of, as an overview,
Lex Fridman (15:12.480)
what kind of autonomous flying vehicles
Lex Fridman (15:14.400)
are there in general?
Lex Fridman (15:16.200)
I think the ones that receive a lot of notoriety
Vijay Kumar (15:19.720)
are obviously the military vehicles.
Lex Fridman (15:22.560)
Military vehicles are controlled by a base station,
Lex Fridman (15:26.280)
but have a lot of human supervision.
Lex Fridman (15:29.640)
But they have limited autonomy,
Vijay Kumar (15:31.800)
which is the ability to go from point A to point B.
Lex Fridman (15:34.760)
And even the more sophisticated now,
Vijay Kumar (15:37.080)
sophisticated vehicles can do autonomous takeoff
Lex Fridman (15:40.400)
and landing.
Lex Fridman (15:41.760)
And those usually have wings and they're heavy.
Lex Fridman (15:44.360)
Usually they're wings,
Lex Fridman (15:45.360)
but then there's nothing preventing us from doing this
Lex Fridman (15:47.440)
for helicopters as well.
Vijay Kumar (15:49.000)
There are many military organizations
Lex Fridman (15:52.480)
that have autonomous helicopters in the same vein.
Lex Fridman (15:56.560)
And by the way, you look at autopilots and airplanes
Lex Fridman (16:00.080)
and it's actually very similar.
Vijay Kumar (16:02.840)
In fact, one interesting question we can ask is,
Lex Fridman (16:07.160)
if you look at all the air safety violations,
Vijay Kumar (16:12.120)
all the crashes that occurred,
Lex Fridman (16:14.080)
would they have happened if the plane were truly autonomous?
Lex Fridman (16:18.640)
And I think you'll find that in many of the cases,
Lex Fridman (16:21.960)
because of pilot error, we made silly decisions.
Lex Fridman (16:24.600)
And so in some sense, even in air traffic,
Lex Fridman (16:26.960)
commercial air traffic, there's a lot of applications,
Vijay Kumar (16:29.800)
although we only see autonomy being enabled
Lex Fridman (16:33.960)
at very high altitudes when the plane is an autopilot.
Vijay Kumar (16:38.960)
The plane is an autopilot.
Lex Fridman (16:41.960)
There's still a role for the human
Lex Fridman (16:42.800)
and that kind of autonomy is, you're kind of implying,
Lex Fridman (16:47.640)
I don't know what the right word is,
Lex Fridman (16:48.680)
but it's a little dumber than it could be.
Lex Fridman (16:53.480)
Right, so in the lab, of course,
Vijay Kumar (16:55.720)
we can afford to be a lot more aggressive.
Lex Fridman (16:59.200)
And the question we try to ask is,
Vijay Kumar (17:04.200)
can we make robots that will be able to make decisions
Lex Fridman (17:10.360)
without any kind of external infrastructure?
Lex Fridman (17:13.680)
So what does that mean?
Lex Fridman (17:14.880)
So the most common piece of infrastructure
Vijay Kumar (17:16.960)
that airplanes use today is GPS.
Lex Fridman (17:20.560)
GPS is also the most brittle form of information.
Vijay Kumar (17:26.680)
If you have driven in a city, try to use GPS navigation,
Lex Fridman (17:30.480)
in tall buildings, you immediately lose GPS.
Lex Fridman (17:32.760)
And so that's not a very sophisticated way
Lex Fridman (17:36.280)
of building autonomy.
Vijay Kumar (17:37.840)
I think the second piece of infrastructure
Lex Fridman (17:39.560)
they rely on is communications.
Vijay Kumar (17:41.920)
Again, it's very easy to jam communications.
Lex Fridman (17:47.360)
In fact, if you use wifi, you know that wifi signals
Vijay Kumar (17:51.320)
drop out, cell signals drop out.
Lex Fridman (17:53.520)
So to rely on something like that is not good.
Vijay Kumar (17:58.560)
The third form of infrastructure we use,
Lex Fridman (18:01.200)
and I hate to call it infrastructure,
Lex Fridman (18:02.920)
but it is that, in the sense of robots, is people.
Lex Fridman (18:06.360)
So you could rely on somebody to pilot you.
Lex Fridman (18:09.960)
And so the question you wanna ask is,
Lex Fridman (18:11.600)
if there are no pilots, there's no communications
Vijay Kumar (18:14.760)
with any base station, if there's no knowledge of position,
Lex Fridman (18:18.720)
and if there's no a priori map,
Vijay Kumar (18:21.640)
a priori knowledge of what the environment looks like,
Lex Fridman (18:24.880)
a priori model of what might happen in the future,
Lex Fridman (18:28.240)
can robots navigate?
Lex Fridman (18:29.560)
So that is true autonomy.
Lex Fridman (18:31.480)
So that's true autonomy, and we're talking about,
Lex Fridman (18:34.160)
you mentioned like military application of drones.
Lex Fridman (18:36.880)
Okay, so what else is there?
Lex Fridman (18:38.320)
You talk about agile, autonomous flying robots,
Vijay Kumar (18:42.080)
aerial robots, so that's a different kind of,
Lex Fridman (18:45.680)
it's not winged, it's not big, at least it's small.
Lex Fridman (18:48.160)
So I use the word agility mostly,
Lex Fridman (18:50.840)
or at least we're motivated to do agile robots,
Vijay Kumar (18:53.520)
mostly because robots can operate
Lex Fridman (18:58.000)
and should be operating in constrained environments.
Lex Fridman (19:02.120)
And if you want to operate the way a global hawk operates,
Lex Fridman (19:06.960)
I mean, the kinds of conditions in which you operate
Vijay Kumar (19:09.120)
are very, very restrictive.
Lex Fridman (19:11.760)
If you wanna go inside a building,
Vijay Kumar (19:13.720)
for example, for search and rescue,
Lex Fridman (19:15.600)
or to locate an active shooter,
Vijay Kumar (19:18.120)
or you wanna navigate under the canopy in an orchard
Lex Fridman (19:22.120)
to look at health of plants,
Vijay Kumar (19:23.880)
or to look for, to count fruits,
Lex Fridman (19:28.240)
to measure the tree trunks.
Vijay Kumar (19:31.240)
These are things we do, by the way.
Lex Fridman (19:33.240)
There's some cool agriculture stuff you've shown
Vijay Kumar (19:35.400)
in the past, it's really awesome.
Lex Fridman (19:37.080)
So in those kinds of settings, you do need that agility.
Vijay Kumar (19:40.360)
Agility does not necessarily mean
Lex Fridman (19:42.560)
you break records for the 100 meters dash.
Lex Fridman (19:45.440)
What it really means is you see the unexpected
Lex Fridman (19:48.000)
and you're able to maneuver in a safe way,
Lex Fridman (19:51.480)
and in a way that gets you the most information
Lex Fridman (19:55.400)
about the thing you're trying to do.
Vijay Kumar (19:57.640)
By the way, you may be the only person
Lex Fridman (20:00.440)
who, in a TED Talk, has used a math equation,
Vijay Kumar (20:04.200)
which is amazing, people should go see one of your TED Talks.
Lex Fridman (20:07.600)
Actually, it's very interesting,
Vijay Kumar (20:08.800)
because the TED curator, Chris Anderson,
Lex Fridman (20:12.400)
told me, you can't show math.
Lex Fridman (20:15.360)
And I thought about it, but that's who I am.
Lex Fridman (20:18.200)
I mean, that's our work.
Lex Fridman (20:20.760)
And so I felt compelled to give the audience a taste
Lex Fridman (20:25.760)
for at least some math.
Lex Fridman (20:27.640)
So on that point, simply, what does it take
Lex Fridman (20:32.880)
to make a thing with four motors fly, a quadcopter,
Lex Fridman (20:37.360)
one of these little flying robots?
Lex Fridman (20:41.760)
How hard is it to make it fly?
Lex Fridman (20:43.960)
How do you coordinate the four motors?
Lex Fridman (20:46.560)
How do you convert those motors into actual movement?
Lex Fridman (20:52.600)
So this is an interesting question.
Lex Fridman (20:54.800)
We've been trying to do this since 2000.
Vijay Kumar (20:58.080)
It is a commentary on the sensors
Lex Fridman (21:00.560)
that were available back then,
Vijay Kumar (21:02.080)
the computers that were available back then.
Lex Fridman (21:05.560)
And a number of things happened between 2000 and 2007.
Vijay Kumar (21:11.520)
One is the advances in computing,
Lex Fridman (21:14.120)
which is, so we all know about Moore's Law,
Lex Fridman (21:16.760)
but I think 2007 was a tipping point,
Lex Fridman (21:19.680)
the year of the iPhone, the year of the cloud.
Vijay Kumar (21:22.720)
Lots of things happened in 2007.
Lex Fridman (21:25.600)
But going back even further,
Vijay Kumar (21:27.600)
inertial measurement units as a sensor really matured.
Lex Fridman (21:31.360)
Again, lots of reasons for that.
Vijay Kumar (21:33.920)
Certainly, there's a lot of federal funding,
Lex Fridman (21:35.400)
particularly DARPA in the US,
Lex Fridman (21:38.320)
but they didn't anticipate this boom in IMUs.
Lex Fridman (21:42.760)
But if you look, subsequently what happened
Vijay Kumar (21:46.560)
is that every car manufacturer had to put an airbag in,
Lex Fridman (21:50.040)
which meant you had to have an accelerometer on board.
Lex Fridman (21:52.600)
And so that drove down the price to performance ratio.
Lex Fridman (21:55.000)
Wow, I should know this.
Vijay Kumar (21:56.880)
That's very interesting.
Lex Fridman (21:57.960)
That's very interesting, the connection there.
Lex Fridman (21:59.360)
And that's why research is very,
Lex Fridman (22:01.320)
it's very hard to predict the outcomes.
Lex Fridman (22:04.840)
And again, the federal government spent a ton of money
Lex Fridman (22:07.640)
on things that they thought were useful for resonators,
Lex Fridman (22:12.280)
but it ended up enabling these small UAVs, which is great,
Lex Fridman (22:16.840)
because I could have never raised that much money
Lex Fridman (22:18.520)
and sold this project,
Lex Fridman (22:20.760)
hey, we want to build these small UAVs.
Lex Fridman (22:22.200)
Can you actually fund the development of low cost IMUs?
Lex Fridman (22:25.440)
So why do you need an IMU on an IMU?
Lex Fridman (22:27.600)
So I'll come back to that.
Lex Fridman (22:31.000)
So in 2007, 2008, we were able to build these.
Lex Fridman (22:33.320)
And then the question you're asking was a good one.
Lex Fridman (22:35.200)
How do you coordinate the motors to develop this?
Lex Fridman (22:40.240)
But over the last 10 years, everything is commoditized.
Lex Fridman (22:43.880)
A high school kid today can pick up
Vijay Kumar (22:46.240)
a Raspberry Pi kit and build this.
Lex Fridman (22:50.560)
All the low levels functionality is all automated.
Lex Fridman (22:54.160)
But basically at some level,
Lex Fridman (22:56.360)
you have to drive the motors at the right RPMs,
Vijay Kumar (23:01.360)
the right velocity,
Lex Fridman (23:04.560)
in order to generate the right amount of thrust,
Vijay Kumar (23:07.480)
in order to position it and orient it in a way
Lex Fridman (23:10.360)
that you need to in order to fly.
Vijay Kumar (23:13.800)
The feedback that you get is from onboard sensors,
Lex Fridman (23:16.680)
and the IMU is an important part of it.
Vijay Kumar (23:18.400)
The IMU tells you what the acceleration is,
Lex Fridman (23:23.840)
as well as what the angular velocity is.
Lex Fridman (23:26.400)
And those are important pieces of information.
Lex Fridman (23:30.440)
In addition to that, you need some kind of local position
Vijay Kumar (23:34.200)
or velocity information.
Lex Fridman (23:37.480)
For example, when we walk,
Vijay Kumar (23:39.360)
we implicitly have this information
Lex Fridman (23:41.560)
because we kind of know what our stride length is.
Vijay Kumar (23:46.720)
We also are looking at images fly past our retina,
Lex Fridman (23:51.480)
if you will, and so we can estimate velocity.
Vijay Kumar (23:54.280)
We also have accelerometers in our head,
Lex Fridman (23:56.360)
and we're able to integrate all these pieces of information
Vijay Kumar (23:59.160)
to determine where we are as we walk.
Lex Fridman (24:02.360)
And so robots have to do something very similar.
Vijay Kumar (24:04.320)
You need an IMU, you need some kind of a camera
Lex Fridman (24:08.160)
or other sensor that's measuring velocity,
Lex Fridman (24:12.560)
and then you need some kind of a global reference frame
Lex Fridman (24:15.800)
if you really want to think about doing something
Vijay Kumar (24:19.520)
in a world coordinate system.
Lex Fridman (24:21.280)
And so how do you estimate your position
Lex Fridman (24:23.680)
with respect to that global reference frame?
Lex Fridman (24:25.160)
That's important as well.
Lex Fridman (24:26.560)
So coordinating the RPMs of the four motors
Lex Fridman (24:29.520)
is what allows you to, first of all, fly and hover,
Lex Fridman (24:32.640)
and then you can change the orientation
Lex Fridman (24:35.600)
and the velocity and so on.
Vijay Kumar (24:37.600)
Exactly, exactly.
Lex Fridman (24:38.440)
So it's a bunch of degrees of freedom
Vijay Kumar (24:40.320)
that you're complaining about.
Lex Fridman (24:41.160)
There's six degrees of freedom,
Lex Fridman (24:42.200)
but you only have four inputs, the four motors.
Lex Fridman (24:44.920)
And it turns out to be a remarkably versatile configuration.
Vijay Kumar (24:50.920)
You think at first, well, I only have four motors,
Lex Fridman (24:53.080)
how do I go sideways?
Lex Fridman (24:55.000)
But it's not too hard to say, well, if I tilt myself,
Lex Fridman (24:57.280)
I can go sideways, and then you have four motors
Vijay Kumar (25:00.440)
pointing up, how do I rotate in place
Lex Fridman (25:03.320)
about a vertical axis?
Vijay Kumar (25:05.360)
Well, you rotate them at different speeds
Lex Fridman (25:07.800)
and that generates reaction moments
Lex Fridman (25:09.720)
and that allows you to turn.
Lex Fridman (25:11.520)
So it's actually a pretty, it's an optimal configuration
Vijay Kumar (25:14.960)
from an engineer standpoint.
Lex Fridman (25:18.360)
It's very simple, very cleverly done, and very versatile.
Lex Fridman (25:23.360)
So if you could step back to a time,
Lex Fridman (25:27.240)
so I've always known flying robots as,
Vijay Kumar (25:31.040)
to me, it was natural that a quadcopter should fly.
Lex Fridman (25:35.760)
But when you first started working with it,
Lex Fridman (25:38.800)
how surprised are you that you can make,
Lex Fridman (25:42.000)
do so much with the four motors?
Lex Fridman (25:45.520)
How surprising is it that you can make this thing fly,
Lex Fridman (25:47.600)
first of all, that you can make it hover,
Lex Fridman (25:49.760)
that you can add control to it?
Lex Fridman (25:52.000)
Firstly, this is not, the four motor configuration
Vijay Kumar (25:55.080)
is not ours.
Lex Fridman (25:56.400)
You can, it has at least a hundred year history.
Lex Fridman (26:00.320)
And various people, various people try to get quadrotors
Lex Fridman (26:04.160)
to fly without much success.
Vijay Kumar (26:08.480)
As I said, we've been working on this since 2000.
Lex Fridman (26:10.760)
Our first designs were, well, this is way too complicated.
Lex Fridman (26:14.400)
Why not we try to get an omnidirectional flying robot?
Lex Fridman (26:18.480)
So our early designs, we had eight rotors.
Lex Fridman (26:21.760)
And so these eight rotors were arranged uniformly
Lex Fridman (26:26.600)
on a sphere, if you will.
Lex Fridman (26:28.000)
So you can imagine a symmetric configuration.
Lex Fridman (26:30.440)
And so you should be able to fly anywhere.
Lex Fridman (26:33.280)
But the real challenge we had is the strength to weight ratio
Lex Fridman (26:36.240)
is not enough.
Lex Fridman (26:37.080)
And of course, we didn't have the sensors and so on.
Lex Fridman (26:40.520)
So everybody knew, or at least the people
Vijay Kumar (26:43.040)
who worked with rotorcrafts knew,
Lex Fridman (26:44.800)
four rotors will get it done.
Lex Fridman (26:47.520)
So that was not our idea.
Lex Fridman (26:49.400)
But it took a while before we could actually do
Vijay Kumar (26:52.800)
the onboard sensing and the computation that was needed
Lex Fridman (26:56.920)
for the kinds of agile maneuvering that we wanted to do
Vijay Kumar (27:01.000)
in our little aerial robots.
Lex Fridman (27:03.000)
And that only happened between 2007 and 2009 in our lab.
Vijay Kumar (27:07.560)
Yeah, and you have to send the signal
Lex Fridman (27:09.960)
maybe a hundred times a second.
Lex Fridman (27:12.480)
So the compute there, everything has to come down in price.
Lex Fridman (27:15.960)
And what are the steps of getting from point A to point B?
Lex Fridman (27:21.720)
So we just talked about like local control.
Lex Fridman (27:25.200)
But if all the kind of cool dancing in the air
Lex Fridman (27:30.840)
that I've seen you show, how do you make it happen?
Lex Fridman (27:34.520)
How do you make a trajectory?
Vijay Kumar (27:37.360)
First of all, okay, figure out a trajectory.
Lex Fridman (27:40.520)
So plan a trajectory.
Lex Fridman (27:41.680)
And then how do you make that trajectory happen?
Lex Fridman (27:44.400)
Yeah, I think planning is a very fundamental problem
Vijay Kumar (27:47.280)
in robotics.
Lex Fridman (27:48.120)
I think 10 years ago it was an esoteric thing,
Lex Fridman (27:50.800)
but today with self driving cars,
Lex Fridman (27:53.040)
everybody can understand this basic idea
Vijay Kumar (27:55.840)
that a car sees a whole bunch of things
Lex Fridman (27:57.920)
and it has to keep a lane or maybe make a right turn
Vijay Kumar (28:00.320)
or switch lanes.
Lex Fridman (28:01.280)
It has to plan a trajectory.
Vijay Kumar (28:02.680)
It has to be safe.
Lex Fridman (28:03.560)
It has to be efficient.
Lex Fridman (28:04.840)
So everybody's familiar with that.
Lex Fridman (28:06.640)
That's kind of the first step that you have to think about
Vijay Kumar (28:10.240)
when you say autonomy.
Lex Fridman (28:14.800)
And so for us, it's about finding smooth motions,
Vijay Kumar (28:19.120)
motions that are safe.
Lex Fridman (28:21.320)
So we think about these two things.
Vijay Kumar (28:22.880)
One is optimality, one is safety.
Lex Fridman (28:24.680)
Clearly you cannot compromise safety.
Lex Fridman (28:28.440)
So you're looking for safe, optimal motions.
Lex Fridman (28:31.360)
The other thing you have to think about is
Lex Fridman (28:34.480)
can you actually compute a reasonable trajectory
Lex Fridman (28:38.160)
in a small amount of time?
Vijay Kumar (28:40.760)
Cause you have a time budget.
Lex Fridman (28:42.280)
So the optimal becomes suboptimal,
Lex Fridman (28:45.160)
but in our lab we focus on synthesizing smooth trajectory
Lex Fridman (28:51.160)
that satisfy all the constraints.
Vijay Kumar (28:53.000)
In other words, don't violate any safety constraints
Lex Fridman (28:58.440)
and is as efficient as possible.
Lex Fridman (29:02.880)
And when I say efficient,
Lex Fridman (29:04.360)
it could mean I want to get from point A to point B
Vijay Kumar (29:06.600)
as quickly as possible,
Lex Fridman (29:08.360)
or I want to get to it as gracefully as possible,
Vijay Kumar (29:12.840)
or I want to consume as little energy as possible.
Lex Fridman (29:15.960)
But always staying within the safety constraints.
Lex Fridman (29:18.240)
But yes, always finding a safe trajectory.
Lex Fridman (29:22.800)
So there's a lot of excitement and progress
Vijay Kumar (29:25.040)
in the field of machine learning
Lex Fridman (29:27.360)
and reinforcement learning
Lex Fridman (29:29.360)
and the neural network variant of that
Lex Fridman (29:32.200)
with deep reinforcement learning.
Lex Fridman (29:33.920)
Do you see a role of machine learning
Lex Fridman (29:36.360)
in, so a lot of the success of flying robots
Vijay Kumar (29:40.560)
did not rely on machine learning,
Lex Fridman (29:42.320)
except for maybe a little bit of the perception
Vijay Kumar (29:45.040)
on the computer vision side.
Lex Fridman (29:46.600)
On the control side and the planning,
Lex Fridman (29:48.440)
do you see there's a role in the future
Lex Fridman (29:50.400)
for machine learning?
Lex Fridman (29:51.680)
So let me disagree a little bit with you.
Lex Fridman (29:53.800)
I think we never perhaps called out in my work,
Vijay Kumar (29:56.800)
called out learning,
Lex Fridman (29:57.720)
but even this very simple idea of being able to fly
Vijay Kumar (30:00.600)
through a constrained space.
Lex Fridman (30:02.200)
The first time you try it, you'll invariably,
Vijay Kumar (30:05.680)
you might get it wrong if the task is challenging.
Lex Fridman (30:08.440)
And the reason is to get it perfectly right,
Vijay Kumar (30:12.200)
you have to model everything in the environment.
Lex Fridman (30:15.600)
And flying is notoriously hard to model.
Vijay Kumar (30:19.960)
There are aerodynamic effects that we constantly discover.
Lex Fridman (30:26.520)
Even just before I was talking to you,
Vijay Kumar (30:29.440)
I was talking to a student about how blades flap
Lex Fridman (30:33.440)
when they fly.
Lex Fridman (30:35.320)
And that ends up changing how a rotorcraft
Lex Fridman (30:40.880)
is accelerated in the angular direction.
Lex Fridman (30:43.960)
Does he use like micro flaps or something?
Lex Fridman (30:46.360)
It's not micro flaps.
Lex Fridman (30:47.280)
So we assume that each blade is rigid,
Lex Fridman (30:49.640)
but actually it flaps a little bit.
Vijay Kumar (30:51.720)
It bends.
Lex Fridman (30:52.880)
Interesting, yeah.
Lex Fridman (30:53.720)
And so the models rely on the fact,
Lex Fridman (30:56.040)
on the assumption that they're not rigid.
Vijay Kumar (30:58.640)
On the assumption that they're actually rigid,
Lex Fridman (31:00.640)
but that's not true.
Vijay Kumar (31:02.240)
If you're flying really quickly,
Lex Fridman (31:03.720)
these effects become significant.
Vijay Kumar (31:06.920)
If you're flying close to the ground,
Lex Fridman (31:09.240)
you get pushed off by the ground, right?
Vijay Kumar (31:12.160)
Something which every pilot knows when he tries to land
Lex Fridman (31:14.920)
or she tries to land, this is called a ground effect.
Vijay Kumar (31:18.920)
Something very few pilots think about
Lex Fridman (31:21.000)
is what happens when you go close to a ceiling
Vijay Kumar (31:23.040)
or you get sucked into a ceiling.
Lex Fridman (31:25.320)
There are very few aircrafts
Vijay Kumar (31:26.880)
that fly close to any kind of ceiling.
Lex Fridman (31:29.520)
Likewise, when you go close to a wall,
Vijay Kumar (31:33.520)
there are these wall effects.
Lex Fridman (31:35.720)
And if you've gone on a train
Lex Fridman (31:37.680)
and you pass another train that's traveling
Lex Fridman (31:39.600)
in the opposite direction, you feel the buffeting.
Lex Fridman (31:42.400)
And so these kinds of microclimates
Lex Fridman (31:45.400)
affect our UAV significantly.
Lex Fridman (31:47.880)
So if you want...
Lex Fridman (31:48.720)
And they're impossible to model, essentially.
Vijay Kumar (31:50.640)
I wouldn't say they're impossible to model,
Lex Fridman (31:52.480)
but the level of sophistication you would need
Vijay Kumar (31:54.880)
in the model and the software would be tremendous.
Lex Fridman (32:00.000)
Plus, to get everything right would be awfully tedious.
Lex Fridman (32:02.920)
So the way we do this is over time,
Lex Fridman (32:05.080)
we figure out how to adapt to these conditions.
Lex Fridman (32:10.360)
So early on, we use the form of learning
Lex Fridman (32:13.160)
that we call iterative learning.
Lex Fridman (32:15.760)
So this idea, if you want to perform a task,
Lex Fridman (32:18.600)
there are a few things that you need to change
Lex Fridman (32:22.120)
and iterate over a few parameters
Lex Fridman (32:24.960)
that over time you can figure out.
Lex Fridman (32:29.280)
So I could call it policy gradient reinforcement learning,
Lex Fridman (32:33.400)
but actually it was just iterative learning.
Vijay Kumar (32:34.920)
Iterative learning.
Lex Fridman (32:36.000)
And so this was there way back.
Vijay Kumar (32:37.800)
I think what's interesting is,
Lex Fridman (32:39.440)
if you look at autonomous vehicles today,
Vijay Kumar (32:43.120)
learning occurs, could occur in two pieces.
Lex Fridman (32:45.680)
One is perception, understanding the world.
Vijay Kumar (32:47.960)
Second is action, taking actions.
Lex Fridman (32:50.080)
Everything that I've seen that is successful
Vijay Kumar (32:52.240)
is on the perception side of things.
Lex Fridman (32:54.360)
So in computer vision,
Vijay Kumar (32:55.400)
we've made amazing strides in the last 10 years.
Lex Fridman (32:57.840)
So recognizing objects, actually detecting objects,
Vijay Kumar (33:01.640)
classifying them and tagging them in some sense,
Lex Fridman (33:06.400)
annotating them.
Vijay Kumar (33:07.440)
This is all done through machine learning.
Lex Fridman (33:09.640)
On the action side, on the other hand,
Vijay Kumar (33:12.160)
I don't know of any examples
Lex Fridman (33:13.720)
where there are fielded systems
Vijay Kumar (33:15.560)
where we actually learn
Lex Fridman (33:17.560)
the right behavior.
Vijay Kumar (33:20.560)
Outside of single demonstration is successful.
Lex Fridman (33:22.760)
In the laboratory, this is the holy grail.
Lex Fridman (33:24.640)
Can you do end to end learning?
Lex Fridman (33:26.040)
Can you go from pixels to motor currents?
Vijay Kumar (33:30.200)
This is really, really hard.
Lex Fridman (33:32.800)
And I think if you go forward,
Vijay Kumar (33:35.080)
the right way to think about these things
Lex Fridman (33:37.600)
is data driven approaches,
Vijay Kumar (33:40.720)
learning based approaches,
Lex Fridman (33:42.400)
in concert with model based approaches,
Vijay Kumar (33:45.280)
which is the traditional way of doing things.
Lex Fridman (33:47.320)
So I think there's a piece,
Vijay Kumar (33:48.720)
there's a role for each of these methodologies.
Lex Fridman (33:51.400)
So what do you think,
Vijay Kumar (33:52.440)
just jumping out on topic
Lex Fridman (33:53.880)
since you mentioned autonomous vehicles,
Lex Fridman (33:56.200)
what do you think are the limits on the perception side?
Lex Fridman (33:58.480)
So I've talked to Elon Musk
Lex Fridman (34:01.080)
and there on the perception side,
Lex Fridman (34:03.320)
they're using primarily computer vision
Vijay Kumar (34:05.960)
to perceive the environment.
Lex Fridman (34:08.080)
In your work with,
Vijay Kumar (34:09.760)
because you work with the real world a lot
Lex Fridman (34:12.560)
and the physical world,
Lex Fridman (34:13.720)
what are the limits of computer vision?
Lex Fridman (34:15.800)
Do you think we can solve autonomous vehicles
Vijay Kumar (34:19.160)
on the perception side,
Lex Fridman (34:20.880)
focusing on vision alone and machine learning?
Vijay Kumar (34:24.240)
So, we also have a spinoff company,
Lex Fridman (34:27.480)
Exxon Technologies that works underground in mines.
Lex Fridman (34:31.840)
So you go into mines, they're dark, they're dirty.
Lex Fridman (34:36.480)
You fly in a dirty area,
Vijay Kumar (34:38.600)
there's stuff you kick up from by the propellers,
Lex Fridman (34:41.120)
the downwash kicks up dust.
Vijay Kumar (34:42.720)
I challenge you to get a computer vision algorithm
Lex Fridman (34:45.520)
to work there.
Lex Fridman (34:46.680)
So we use LIDARs in that setting.
Lex Fridman (34:51.200)
Indoors and even outdoors when we fly through fields,
Vijay Kumar (34:55.360)
I think there's a lot of potential
Lex Fridman (34:57.120)
for just solving the problem using computer vision alone.
Lex Fridman (35:01.240)
But I think the bigger question is,
Lex Fridman (35:02.760)
can you actually solve
Vijay Kumar (35:06.160)
or can you actually identify all the corner cases
Lex Fridman (35:09.440)
using a single sensing modality and using learning alone?
Lex Fridman (35:13.920)
So what's your intuition there?
Lex Fridman (35:15.400)
So look, if you have a corner case
Lex Fridman (35:17.920)
and your algorithm doesn't work,
Lex Fridman (35:20.000)
your instinct is to go get data about the corner case
Lex Fridman (35:23.200)
and patch it up, learn how to deal with that corner case.
Lex Fridman (35:27.640)
But at some point, this is gonna saturate,
Vijay Kumar (35:32.040)
this approach is not viable.
Lex Fridman (35:34.200)
So today, computer vision algorithms can detect
Vijay Kumar (35:38.000)
90% of the objects or can detect objects 90% of the time,
Lex Fridman (35:41.360)
classify them 90% of the time.
Vijay Kumar (35:43.920)
Cats on the internet probably can do 95%, I don't know.
Lex Fridman (35:47.960)
But to get from 90% to 99%, you need a lot more data.
Lex Fridman (35:52.520)
And then I tell you, well, that's not enough
Lex Fridman (35:54.480)
because I have a safety critical application,
Vijay Kumar (35:56.680)
I wanna go from 99% to 99.9%.
Lex Fridman (36:00.160)
That's even more data.
Lex Fridman (36:01.600)
So I think if you look at wanting accuracy on the X axis
Lex Fridman (36:09.600)
and look at the amount of data on the Y axis,
Vijay Kumar (36:14.080)
I believe that curve is an exponential curve.
Lex Fridman (36:16.440)
Wow, okay, it's even hard if it's linear.
Vijay Kumar (36:19.480)
It's hard if it's linear, totally,
Lex Fridman (36:20.800)
but I think it's exponential.
Lex Fridman (36:22.560)
And the other thing you have to think about
Lex Fridman (36:24.120)
is that this process is a very, very power hungry process
Vijay Kumar (36:29.600)
to run data farms or servers.
Lex Fridman (36:32.880)
Power, do you mean literally power?
Vijay Kumar (36:34.600)
Literally power, literally power.
Lex Fridman (36:36.600)
So in 2014, five years ago, and I don't have more recent data,
Vijay Kumar (36:41.760)
2% of US electricity consumption was from data farms.
Lex Fridman (36:48.360)
So we think about this as an information science
Lex Fridman (36:52.080)
and information processing problem.
Lex Fridman (36:54.240)
Actually, it is an energy processing problem.
Lex Fridman (36:57.840)
And so unless we figured out better ways of doing this,
Lex Fridman (37:00.440)
I don't think this is viable.
Lex Fridman (37:02.440)
So talking about driving, which is a safety critical application
Lex Fridman (37:06.600)
and some aspect of flight is safety critical,
Vijay Kumar (37:10.440)
maybe philosophical question, maybe an engineering one,
Lex Fridman (37:12.960)
what problem do you think is harder to solve,
Lex Fridman (37:15.000)
autonomous driving or autonomous flight?
Lex Fridman (37:18.120)
That's a really interesting question.
Vijay Kumar (37:19.920)
I think autonomous flight has several advantages
Lex Fridman (37:25.440)
that autonomous driving doesn't have.
Lex Fridman (37:29.360)
So look, if I want to go from point A to point B,
Lex Fridman (37:32.400)
I have a very, very safe trajectory.
Vijay Kumar (37:34.320)
Go vertically up to a maximum altitude,
Lex Fridman (37:36.800)
fly horizontally to just about the destination,
Lex Fridman (37:39.480)
and then come down vertically.
Lex Fridman (37:42.400)
This is preprogrammed.
Vijay Kumar (37:45.400)
The equivalent of that is very hard to find
Lex Fridman (37:48.040)
in the self driving car world because you're on the ground,
Vijay Kumar (37:51.560)
you're in a two dimensional surface,
Lex Fridman (37:53.560)
and the trajectories on the two dimensional surface
Vijay Kumar (37:56.680)
are more likely to encounter obstacles.
Lex Fridman (38:00.200)
I mean this in an intuitive sense, but mathematically true.
Vijay Kumar (38:03.280)
That's mathematically as well, that's true.
Lex Fridman (38:06.360)
There's other option on the 2G space of platooning,
Vijay Kumar (38:10.040)
or because there's so many obstacles,
Lex Fridman (38:11.640)
you can connect with those obstacles
Lex Fridman (38:13.280)
and all these kind of options.
Lex Fridman (38:14.560)
Sure, but those exist in the three dimensional space as well.
Lex Fridman (38:16.560)
So they do.
Lex Fridman (38:17.560)
So the question also implies how difficult are obstacles
Lex Fridman (38:21.800)
in the three dimensional space in flight?
Lex Fridman (38:23.800)
So that's the downside.
Vijay Kumar (38:25.600)
I think in three dimensional space,
Lex Fridman (38:26.920)
you're modeling three dimensional world,
Vijay Kumar (38:29.080)
not just because you want to avoid it,
Lex Fridman (38:31.280)
but you want to reason about it,
Lex Fridman (38:33.040)
and you want to work in the three dimensional environment,
Lex Fridman (38:35.360)
and that's significantly harder.
Lex Fridman (38:37.480)
So that's one disadvantage.
Lex Fridman (38:38.920)
I think the second disadvantage is of course,
Vijay Kumar (38:41.040)
anytime you fly, you have to put up
Lex Fridman (38:43.200)
with the peculiarities of aerodynamics
Lex Fridman (38:46.560)
and their complicated environments.
Lex Fridman (38:48.720)
How do you negotiate that?
Lex Fridman (38:49.800)
So that's always a problem.
Lex Fridman (38:51.880)
Do you see a time in the future where there is,
Vijay Kumar (38:55.240)
you mentioned there's agriculture applications.
Lex Fridman (38:58.720)
So there's a lot of applications of flying robots,
Lex Fridman (39:01.680)
but do you see a time in the future
Lex Fridman (39:03.040)
where there's tens of thousands,
Vijay Kumar (39:05.360)
or maybe hundreds of thousands of delivery drones
Lex Fridman (39:08.160)
that fill the sky, delivery flying robots?
Vijay Kumar (39:12.160)
I think there's a lot of potential
Lex Fridman (39:14.200)
for the last mile delivery.
Lex Fridman (39:15.920)
And so in crowded cities, I don't know,
Lex Fridman (39:19.240)
if you go to a place like Hong Kong,
Vijay Kumar (39:21.400)
just crossing the river can take half an hour,
Lex Fridman (39:24.400)
and while a drone can just do it in five minutes at most.
Vijay Kumar (39:29.400)
I think you look at delivery of supplies to remote villages.
Lex Fridman (39:35.800)
I work with a nonprofit called Weave Robotics.
Lex Fridman (39:38.680)
So they work in the Peruvian Amazon,
Lex Fridman (39:40.920)
where the only highways that are available
Vijay Kumar (39:44.680)
are the only highways or rivers.
Lex Fridman (39:47.440)
And to get from point A to point B may take five hours,
Vijay Kumar (39:52.960)
while with a drone, you can get there in 30 minutes.
Lex Fridman (39:56.680)
So just delivering drugs,
Vijay Kumar (39:59.880)
retrieving samples for testing vaccines,
Lex Fridman (40:05.160)
I think there's huge potential here.
Lex Fridman (40:07.120)
So I think the challenges are not technological,
Lex Fridman (40:09.960)
but the challenge is economical.
Vijay Kumar (40:12.040)
The one thing I'll tell you that nobody thinks about
Lex Fridman (40:15.560)
is the fact that we've not made huge strides
Vijay Kumar (40:18.920)
in battery technology.
Lex Fridman (40:20.840)
Yes, it's true, batteries are becoming less expensive
Vijay Kumar (40:23.520)
because we have these mega factories that are coming up,
Lex Fridman (40:26.240)
but they're all based on lithium based technologies.
Lex Fridman (40:28.800)
And if you look at the energy density
Lex Fridman (40:31.480)
and the power density,
Vijay Kumar (40:33.240)
those are two fundamentally limiting numbers.
Lex Fridman (40:38.000)
So power density is important
Vijay Kumar (40:39.680)
because for a UAV to take off vertically into the air,
Lex Fridman (40:42.480)
which most drones do, they don't have a runway,
Vijay Kumar (40:46.360)
you consume roughly 200 watts per kilo at the small size.
Lex Fridman (40:51.560)
That's a lot, right?
Vijay Kumar (40:53.920)
In contrast, the human brain consumes less than 80 watts,
Lex Fridman (40:57.520)
the whole of the human brain.
Lex Fridman (40:59.920)
So just imagine just lifting yourself into the air
Lex Fridman (41:03.600)
is like two or three light bulbs,
Vijay Kumar (41:06.000)
which makes no sense to me.
Lex Fridman (41:07.840)
Yeah, so you're going to have to at scale
Vijay Kumar (41:10.440)
solve the energy problem then,
Lex Fridman (41:12.880)
charging the batteries, storing the energy and so on.
Lex Fridman (41:18.920)
And then the storage is the second problem,
Lex Fridman (41:20.680)
but storage limits the range.
Lex Fridman (41:22.960)
But you have to remember that you have to burn
Lex Fridman (41:28.680)
a lot of it per given time.
Lex Fridman (41:31.600)
So the burning is another problem.
Lex Fridman (41:32.920)
Which is a power question.
Vijay Kumar (41:34.640)
Yes, and do you think just your intuition,
Lex Fridman (41:38.640)
there are breakthroughs in batteries on the horizon?
Lex Fridman (41:44.960)
How hard is that problem?
Lex Fridman (41:46.440)
Look, there are a lot of companies
Vijay Kumar (41:47.600)
that are promising flying cars that are autonomous
Lex Fridman (41:53.880)
and that are clean.
Vijay Kumar (41:59.400)
I think they're over promising.
Lex Fridman (42:01.680)
The autonomy piece is doable.
Vijay Kumar (42:04.800)
The clean piece, I don't think so.
Lex Fridman (42:08.000)
There's another company that I work with called JetOptra.
Vijay Kumar (42:11.840)
They make small jet engines.
Lex Fridman (42:15.760)
And they can get up to 50 miles an hour very easily
Lex Fridman (42:18.080)
and lift 50 kilos.
Lex Fridman (42:19.960)
But they're jet engines, they're efficient,
Vijay Kumar (42:23.920)
they're a little louder than electric vehicles,
Lex Fridman (42:26.320)
but they can build flying cars.
Lex Fridman (42:28.960)
So your sense is that there's a lot of pieces
Lex Fridman (42:32.440)
that have come together.
Lex Fridman (42:33.520)
So on this crazy question,
Lex Fridman (42:37.360)
if you look at companies like Kitty Hawk,
Vijay Kumar (42:39.720)
working on electric, so the clean,
Lex Fridman (42:43.880)
talking to Sebastian Thrun, right?
Lex Fridman (42:45.840)
It's a crazy dream, you know?
Lex Fridman (42:48.840)
But you work with flight a lot.
Vijay Kumar (42:52.080)
You've mentioned before that manned flights
Lex Fridman (42:55.760)
or carrying a human body is very difficult to do.
Lex Fridman (43:01.640)
So how crazy is flying cars?
Lex Fridman (43:04.240)
Do you think there'll be a day
Vijay Kumar (43:05.400)
when we have vertical takeoff and landing vehicles
Lex Fridman (43:11.080)
that are sufficiently affordable
Lex Fridman (43:14.960)
that we're going to see a huge amount of them?
Lex Fridman (43:17.440)
And they would look like something like we dream of
Vijay Kumar (43:19.680)
when we think about flying cars.
Lex Fridman (43:21.080)
Yeah, like the Jetsons.
Vijay Kumar (43:22.200)
The Jetsons, yeah.
Lex Fridman (43:23.160)
So look, there are a lot of smart people working on this
Lex Fridman (43:25.560)
and you never say something is not possible
Lex Fridman (43:29.640)
when you have people like Sebastian Thrun working on it.
Lex Fridman (43:32.200)
So I totally think it's viable.
Lex Fridman (43:35.160)
I question, again, the electric piece.
Vijay Kumar (43:38.240)
The electric piece, yeah.
Lex Fridman (43:39.520)
And again, for short distances, you can do it.
Lex Fridman (43:41.680)
And there's no reason to suggest
Lex Fridman (43:43.640)
that these all just have to be rotorcrafts.
Vijay Kumar (43:45.840)
You take off vertically,
Lex Fridman (43:46.920)
but then you morph into a forward flight.
Vijay Kumar (43:49.680)
I think there are a lot of interesting designs.
Lex Fridman (43:51.600)
The question to me is, are these economically viable?
Lex Fridman (43:56.040)
And if you agree to do this with fossil fuels,
Lex Fridman (43:59.160)
it instantly immediately becomes viable.
Vijay Kumar (44:01.960)
That's a real challenge.
Lex Fridman (44:03.480)
Do you think it's possible for robots and humans
Lex Fridman (44:06.560)
to collaborate successfully on tasks?
Lex Fridman (44:08.840)
So a lot of robotics folks that I talk to and work with,
Vijay Kumar (44:13.640)
I mean, humans just add a giant mess to the picture.
Lex Fridman (44:18.000)
So it's best to remove them from consideration
Vijay Kumar (44:20.320)
when solving specific tasks.
Lex Fridman (44:22.400)
It's very difficult to model.
Vijay Kumar (44:23.600)
There's just a source of uncertainty.
Lex Fridman (44:26.000)
In your work with these agile flying robots,
Lex Fridman (44:32.560)
do you think there's a role for collaboration with humans?
Lex Fridman (44:35.680)
Or is it best to model tasks in a way
Lex Fridman (44:38.600)
that doesn't have a human in the picture?
Lex Fridman (44:43.400)
Well, I don't think we should ever think about robots
Vijay Kumar (44:46.760)
without human in the picture.
Lex Fridman (44:48.120)
Ultimately, robots are there because we want them
Vijay Kumar (44:50.960)
to solve problems for humans.
Lex Fridman (44:54.360)
But there's no general solution to this problem.
Vijay Kumar (44:58.280)
I think if you look at human interaction
Lex Fridman (45:00.000)
and how humans interact with robots,
Vijay Kumar (45:02.400)
you know, we think of these in sort of three different ways.
Lex Fridman (45:05.280)
One is the human commanding the robot.
Vijay Kumar (45:08.880)
The second is the human collaborating with the robot.
Lex Fridman (45:12.880)
So for example, we work on how a robot
Vijay Kumar (45:15.520)
can actually pick up things with a human and carry things.
Lex Fridman (45:18.720)
That's like true collaboration.
Lex Fridman (45:20.880)
And third, we think about humans as bystanders,
Lex Fridman (45:25.000)
self driving cars, what's the human's role
Lex Fridman (45:27.240)
and how do self driving cars
Lex Fridman (45:30.320)
acknowledge the presence of humans?
Lex Fridman (45:32.920)
So I think all of these things are different scenarios.
Lex Fridman (45:35.840)
It depends on what kind of humans, what kind of task.
Lex Fridman (45:39.640)
And I think it's very difficult to say
Lex Fridman (45:41.840)
that there's a general theory that we all have for this.
Lex Fridman (45:45.520)
But at the same time, it's also silly to say
Lex Fridman (45:48.440)
that we should think about robots independent of humans.
Lex Fridman (45:52.000)
So to me, human robot interaction
Lex Fridman (45:55.760)
is almost a mandatory aspect of everything we do.
Vijay Kumar (45:59.760)
Yes, but to which degree, so your thoughts,
Lex Fridman (46:02.440)
if we jump to autonomous vehicles, for example,
Vijay Kumar (46:05.240)
there's a big debate between what's called
Lex Fridman (46:08.680)
level two and level four.
Lex Fridman (46:10.640)
So semi autonomous and autonomous vehicles.
Lex Fridman (46:13.680)
And so the Tesla approach currently at least
Vijay Kumar (46:16.440)
has a lot of collaboration between human and machine.
Lex Fridman (46:18.960)
So the human is supposed to actively supervise
Vijay Kumar (46:22.040)
the operation of the robot.
Lex Fridman (46:23.880)
Part of the safety definition of how safe a robot is
Vijay Kumar (46:29.160)
in that case is how effective is the human in monitoring it.
Lex Fridman (46:32.880)
Do you think that's ultimately not a good approach
Vijay Kumar (46:37.880)
in sort of having a human in the picture,
Lex Fridman (46:42.360)
not as a bystander or part of the infrastructure,
Lex Fridman (46:47.400)
but really as part of what's required
Lex Fridman (46:50.000)
to make the system safe?
Vijay Kumar (46:51.560)
This is harder than it sounds.
Lex Fridman (46:53.720)
I think, you know, if you, I mean,
Vijay Kumar (46:58.200)
I'm sure you've driven before in highways and so on.
Lex Fridman (47:01.360)
It's really very hard to have to relinquish control
Vijay Kumar (47:06.120)
to a machine and then take over when needed.
Lex Fridman (47:10.440)
So I think Tesla's approach is interesting
Vijay Kumar (47:12.280)
because it allows you to periodically establish
Lex Fridman (47:14.800)
some kind of contact with the car.
Vijay Kumar (47:18.520)
Toyota, on the other hand, is thinking about
Lex Fridman (47:20.640)
shared autonomy or collaborative autonomy as a paradigm.
Vijay Kumar (47:24.800)
If I may argue, these are very, very simple ways
Lex Fridman (47:27.480)
of human robot collaboration,
Vijay Kumar (47:29.680)
because the task is pretty boring.
Lex Fridman (47:31.880)
You sit in a vehicle, you go from point A to point B.
Vijay Kumar (47:35.000)
I think the more interesting thing to me is,
Lex Fridman (47:37.360)
for example, search and rescue.
Vijay Kumar (47:38.760)
I've got a human first responder, robot first responders.
Lex Fridman (47:43.160)
I gotta do something.
Vijay Kumar (47:45.120)
It's important.
Lex Fridman (47:46.000)
I have to do it in two minutes.
Vijay Kumar (47:47.800)
The building is burning.
Lex Fridman (47:49.240)
There's been an explosion.
Vijay Kumar (47:50.440)
It's collapsed.
Lex Fridman (47:51.360)
How do I do it?
Vijay Kumar (47:52.800)
I think to me, those are the interesting things
Lex Fridman (47:54.740)
where it's very, very unstructured.
Lex Fridman (47:57.160)
And what's the role of the human?
Lex Fridman (47:58.480)
What's the role of the robot?
Vijay Kumar (48:00.200)
Clearly, there's lots of interesting challenges
Lex Fridman (48:02.440)
and there's a field.
Vijay Kumar (48:03.440)
I think we're gonna make a lot of progress in this area.
Lex Fridman (48:05.760)
Yeah, it's an exciting form of collaboration.
Vijay Kumar (48:07.600)
You're right.
Lex Fridman (48:08.440)
In autonomous driving, the main enemy
Vijay Kumar (48:11.120)
is just boredom of the human.
Lex Fridman (48:13.120)
Yes.
Vijay Kumar (48:13.960)
As opposed to in rescue operations,
Lex Fridman (48:15.680)
it's literally life and death.
Lex Fridman (48:18.360)
And the collaboration enables
Lex Fridman (48:22.080)
the effective completion of the mission.
Lex Fridman (48:23.820)
So it's exciting.
Lex Fridman (48:24.760)
In some sense, we're also doing this.
Vijay Kumar (48:27.400)
You think about the human driving a car
Lex Fridman (48:30.520)
and almost invariably, the human's trying
Vijay Kumar (48:33.800)
to estimate the state of the car,
Lex Fridman (48:35.000)
they estimate the state of the environment and so on.
Lex Fridman (48:37.280)
But what if the car were to estimate the state of the human?
Lex Fridman (48:40.120)
So for example, I'm sure you have a smartphone
Lex Fridman (48:41.960)
and the smartphone tries to figure out what you're doing
Lex Fridman (48:44.580)
and send you reminders and oftentimes telling you
Vijay Kumar (48:48.320)
to drive to a certain place,
Lex Fridman (48:49.540)
although you have no intention of going there
Vijay Kumar (48:51.400)
because it thinks that that's where you should be
Lex Fridman (48:53.880)
because of some Gmail calendar entry
Vijay Kumar (48:57.520)
or something like that.
Lex Fridman (48:58.960)
And it's trying to constantly figure out who you are,
Lex Fridman (49:01.600)
what you're doing.
Lex Fridman (49:02.740)
If a car were to do that,
Vijay Kumar (49:04.200)
maybe that would make the driver safer
Lex Fridman (49:06.840)
because the car is trying to figure out
Vijay Kumar (49:08.160)
is the driver paying attention,
Lex Fridman (49:09.760)
looking at his or her eyes,
Vijay Kumar (49:12.480)
looking at circadian movements.
Lex Fridman (49:14.400)
So I think the potential is there,
Lex Fridman (49:16.480)
but from the reverse side,
Lex Fridman (49:18.600)
it's not robot modeling, but it's human modeling.
Vijay Kumar (49:21.640)
It's more on the human, right.
Lex Fridman (49:22.880)
And I think the robots can do a very good job
Vijay Kumar (49:25.320)
of modeling humans if you really think about the framework
Lex Fridman (49:29.120)
that you have a human sitting in a cockpit,
Vijay Kumar (49:32.640)
surrounded by sensors, all staring at him,
Lex Fridman (49:35.820)
in addition to be staring outside,
Lex Fridman (49:37.860)
but also staring at him.
Lex Fridman (49:39.160)
I think there's a real synergy there.
Vijay Kumar (49:40.960)
Yeah, I love that problem
Lex Fridman (49:42.360)
because it's the new 21st century form of psychology,
Vijay Kumar (49:45.560)
actually AI enabled psychology.
Lex Fridman (49:48.520)
A lot of people have sci fi inspired fears
Vijay Kumar (49:51.280)
of walking robots like those from Boston Dynamics.
Lex Fridman (49:54.080)
If you just look at shows on Netflix and so on,
Vijay Kumar (49:56.480)
or flying robots like those you work with,
Lex Fridman (49:59.920)
how would you, how do you think about those fears?
Lex Fridman (50:03.160)
How would you alleviate those fears?
Lex Fridman (50:05.040)
Do you have inklings, echoes of those same concerns?
Vijay Kumar (50:09.040)
You know, anytime we develop a technology
Lex Fridman (50:11.760)
meaning to have positive impact in the world,
Vijay Kumar (50:14.160)
there's always the worry that,
Lex Fridman (50:17.440)
you know, somebody could subvert those technologies
Lex Fridman (50:21.000)
and use it in an adversarial setting.
Lex Fridman (50:23.280)
And robotics is no exception, right?
Lex Fridman (50:25.280)
So I think it's very easy to weaponize robots.
Lex Fridman (50:29.280)
I think we talk about swarms.
Vijay Kumar (50:31.720)
One thing I worry a lot about is,
Lex Fridman (50:33.960)
so, you know, for us to get swarms to work
Lex Fridman (50:35.880)
and do something reliably, it's really hard.
Lex Fridman (50:38.280)
But suppose I have this challenge
Vijay Kumar (50:42.040)
of trying to destroy something,
Lex Fridman (50:44.360)
and I have a swarm of robots,
Vijay Kumar (50:45.720)
where only one out of the swarm
Lex Fridman (50:47.280)
needs to get to its destination.
Lex Fridman (50:48.920)
So that suddenly becomes a lot more doable.
Lex Fridman (50:52.640)
And so I worry about, you know,
Vijay Kumar (50:54.720)
this general idea of using autonomy
Lex Fridman (50:56.920)
with lots and lots of agents.
Vijay Kumar (51:00.040)
I mean, having said that, look,
Lex Fridman (51:01.320)
a lot of this technology is not very mature.
Vijay Kumar (51:03.760)
My favorite saying is that
Lex Fridman (51:06.560)
if somebody had to develop this technology,
Lex Fridman (51:10.520)
wouldn't you rather the good guys do it?
Lex Fridman (51:12.320)
So the good guys have a good understanding
Vijay Kumar (51:13.880)
of the technology, so they can figure out
Lex Fridman (51:15.560)
how this technology is being used in a bad way,
Vijay Kumar (51:18.320)
or could be used in a bad way and try to defend against it.
Lex Fridman (51:21.360)
So we think a lot about that.
Lex Fridman (51:22.760)
So we have, we're doing research
Lex Fridman (51:25.400)
on how to defend against swarms, for example.
Vijay Kumar (51:28.240)
That's interesting.
Lex Fridman (51:29.600)
There's in fact a report by the National Academies
Vijay Kumar (51:32.960)
on counter UAS technologies.
Lex Fridman (51:36.680)
This is a real threat,
Lex Fridman (51:38.200)
but we're also thinking about how to defend against this
Lex Fridman (51:40.320)
and knowing how swarms work.
Vijay Kumar (51:42.920)
Knowing how autonomy works is, I think, very important.
Lex Fridman (51:47.160)
So it's not just politicians?
Lex Fridman (51:49.280)
Do you think engineers have a role in this discussion?
Lex Fridman (51:51.640)
Absolutely.
Vijay Kumar (51:52.480)
I think the days where politicians
Lex Fridman (51:55.280)
can be agnostic to technology are gone.
Vijay Kumar (51:59.200)
I think every politician needs to be
Lex Fridman (52:03.840)
literate in technology.
Lex Fridman (52:05.680)
And I often say technology is the new liberal art.
Lex Fridman (52:09.800)
Understanding how technology will change your life,
Vijay Kumar (52:12.920)
I think is important.
Lex Fridman (52:14.480)
And every human being needs to understand that.
Lex Fridman (52:18.080)
And maybe we can elect some engineers
Lex Fridman (52:20.160)
to office as well on the other side.
Lex Fridman (52:22.720)
What are the biggest open problems in robotics?
Lex Fridman (52:24.840)
And you said we're in the early days in some sense.
Lex Fridman (52:27.760)
What are the problems we would like to solve in robotics?
Lex Fridman (52:31.040)
I think there are lots of problems, right?
Lex Fridman (52:32.520)
But I would phrase it in the following way.
Lex Fridman (52:36.440)
If you look at the robots we're building,
Vijay Kumar (52:39.520)
they're still very much tailored towards
Lex Fridman (52:43.160)
doing specific tasks and specific settings.
Vijay Kumar (52:46.520)
I think the question of how do you get them to operate
Lex Fridman (52:49.480)
in much broader settings
Vijay Kumar (52:53.560)
where things can change in unstructured environments
Lex Fridman (52:58.040)
is up in the air.
Lex Fridman (52:59.160)
So think of self driving cars.
Lex Fridman (53:02.920)
Today, we can build a self driving car in a parking lot.
Vijay Kumar (53:05.680)
We can do level five autonomy in a parking lot.
Lex Fridman (53:10.040)
But can you do a level five autonomy
Lex Fridman (53:13.240)
in the streets of Napoli in Italy or Mumbai in India?
Lex Fridman (53:16.840)
No.
Lex Fridman (53:17.760)
So in some sense, when we think about robotics,
Lex Fridman (53:22.400)
we have to think about where they're functioning,
Lex Fridman (53:25.120)
what kind of environment, what kind of a task.
Lex Fridman (53:27.760)
We have no understanding
Vijay Kumar (53:29.800)
of how to put both those things together.
Lex Fridman (53:32.800)
So we're in the very early days
Vijay Kumar (53:34.000)
of applying it to the physical world.
Lex Fridman (53:35.920)
And I was just in Naples actually.
Lex Fridman (53:38.800)
And there's levels of difficulty and complexity
Lex Fridman (53:42.200)
depending on which area you're applying it to.
Vijay Kumar (53:45.880)
I think so.
Lex Fridman (53:46.720)
And we don't have a systematic way of understanding that.
Vijay Kumar (53:51.040)
Everybody says, just because a computer
Lex Fridman (53:53.800)
can now beat a human at any board game,
Vijay Kumar (53:56.520)
we certainly know something about intelligence.
Lex Fridman (53:59.920)
That's not true.
Vijay Kumar (54:01.360)
A computer board game is very, very structured.
Lex Fridman (54:04.400)
It is the equivalent of working in a Henry Ford factory
Vijay Kumar (54:08.480)
where things, parts come, you assemble, move on.
Lex Fridman (54:11.680)
It's a very, very, very structured setting.
Vijay Kumar (54:14.120)
That's the easiest thing.
Lex Fridman (54:15.680)
And we know how to do that.
Lex Fridman (54:18.400)
So you've done a lot of incredible work
Lex Fridman (54:20.400)
at the UPenn, University of Pennsylvania, GraspLab.
Vijay Kumar (54:23.720)
You're now Dean of Engineering at UPenn.
Lex Fridman (54:26.560)
What advice do you have for a new bright eyed undergrad
Lex Fridman (54:31.320)
interested in robotics or AI or engineering?
Lex Fridman (54:34.640)
Well, I think there's really three things.
Vijay Kumar (54:36.560)
One is you have to get used to the idea
Lex Fridman (54:40.600)
that the world will not be the same in five years
Lex Fridman (54:42.840)
or four years whenever you graduate, right?
Lex Fridman (54:45.160)
Which is really hard to do.
Lex Fridman (54:46.120)
So this thing about predicting the future,
Lex Fridman (54:48.960)
every one of us needs to be trying
Vijay Kumar (54:50.520)
to predict the future always.
Lex Fridman (54:53.280)
Not because you'll be any good at it,
Lex Fridman (54:54.960)
but by thinking about it,
Lex Fridman (54:56.440)
I think you sharpen your senses and you become smarter.
Lex Fridman (55:00.880)
So that's number one.
Lex Fridman (55:02.080)
Number two, it's a corollary of the first piece,
Vijay Kumar (55:05.760)
which is you really don't know what's gonna be important.
Lex Fridman (55:09.360)
So this idea that I'm gonna specialize in something
Vijay Kumar (55:12.080)
which will allow me to go in a particular direction,
Lex Fridman (55:15.320)
it may be interesting,
Lex Fridman (55:16.480)
but it's important also to have this breadth
Lex Fridman (55:18.480)
so you have this jumping off point.
Vijay Kumar (55:22.000)
I think the third thing,
Lex Fridman (55:23.000)
and this is where I think Penn excels.
Vijay Kumar (55:25.360)
I mean, we teach engineering,
Lex Fridman (55:27.240)
but it's always in the context of the liberal arts.
Vijay Kumar (55:29.960)
It's always in the context of society.
Lex Fridman (55:32.360)
As engineers, we cannot afford to lose sight of that.
Lex Fridman (55:35.840)
So I think that's important.
Lex Fridman (55:37.640)
But I think one thing that people underestimate
Vijay Kumar (55:39.960)
when they do robotics
Lex Fridman (55:40.920)
is the importance of mathematical foundations,
Vijay Kumar (55:43.440)
the importance of representations.
Lex Fridman (55:47.720)
Not everything can just be solved
Vijay Kumar (55:50.040)
by looking for Ross packages on the internet
Lex Fridman (55:52.440)
or to find a deep neural network that works.
Vijay Kumar (55:56.280)
I think the representation question is key,
Lex Fridman (55:59.080)
even to machine learning,
Vijay Kumar (56:00.400)
where if you ever hope to achieve or get to explainable AI,
Lex Fridman (56:05.400)
somehow there need to be representations
Vijay Kumar (56:07.760)
that you can understand.
Lex Fridman (56:09.080)
So if you wanna do robotics,
Vijay Kumar (56:11.120)
you should also do mathematics.
Lex Fridman (56:12.680)
And you said liberal arts, a little literature.
Vijay Kumar (56:16.160)
If you wanna build a robot,
Lex Fridman (56:17.200)
it should be reading Dostoyevsky.
Vijay Kumar (56:19.320)
I agree with that.
Lex Fridman (56:20.360)
Very good.
Lex Fridman (56:21.200)
So Vijay, thank you so much for talking today.
Lex Fridman (56:23.560)
It was an honor.
Vijay Kumar (56:24.400)
Thank you.
Lex Fridman (56:25.240)
It was just a very exciting conversation.
Vijay Kumar (56:26.200)
Thank you.
🔗 相关节目