Rohit Prasad: Amazon Alexa and Conversational AI
音乐与艺术AI 与机器学习心理与人性技术与编程商业与创业
📋 章节目录
暂无章节信息
🔑 关键词
alexatermsdatacustomerhumanlearningcustomersdonsaidconversationexperiencehardvoiceperspectiveintelligencefivedeviceunderstandingteamresearch
💬 精彩语录
暂无语录
🎙️ 完整对话（2364 条）
Lex Fridman (00:00.000)
The following is a conversation with Rohit Prasad.
以下是与罗希特·普拉萨德的对话。
Lex Fridman (00:02.960)
He's the vice president and head scientist of Amazon Alexa
他是亚马逊Alexa的副总裁兼首席科学家
Lex Fridman (00:06.360)
and one of its original creators.
及其原创者之一。
Lex Fridman (00:08.880)
The Alexa team embodies some of the most challenging,
Alexa 团队体现了一些最具挑战性的团队，
Lex Fridman (00:12.120)
incredible, impactful, and inspiring work
令人难以置信的、有影响力的、鼓舞人心的工作
Rohit Prasad (00:14.960)
that is done in AI today.
今天人工智能已经做到了这一点。
Lex Fridman (00:17.040)
The team has to both solve problems
团队必须解决问题
Rohit Prasad (00:19.120)
at the cutting edge of natural language processing
处于自然语言处理的最前沿
Lex Fridman (00:21.720)
and provide a trustworthy, secure, and enjoyable experience
并提供值得信赖、安全和愉快的体验
Rohit Prasad (00:25.320)
to millions of people.
给数百万人。
Lex Fridman (00:27.440)
This is where state of the art methods
这就是最先进的方法的所在
Rohit Prasad (00:29.400)
in computer science meet the challenges
在计算机科学领域迎接挑战
Lex Fridman (00:31.840)
of real world engineering.
现实世界的工程。
Rohit Prasad (00:33.720)
In many ways, Alexa and the other voice assistants
在很多方面，Alexa 和其他语音助手
Lex Fridman (00:37.280)
are the voices of artificial intelligence
是人工智能的声音
Rohit Prasad (00:39.520)
to millions of people and an introduction to AI
向数百万人介绍人工智能
Lex Fridman (00:43.160)
for people who have only encountered it in science fiction.
对于那些只在科幻小说中遇到过它的人来说。
Rohit Prasad (00:46.940)
This is an important and exciting opportunity.
这是一个重要且令人兴奋的机会。
Lex Fridman (00:49.960)
So the work that Rohit and the Alexa team are doing
Rohit 和 Alexa 团队正在做的工作
Rohit Prasad (00:52.920)
is an inspiration to me and to many researchers
对我和许多研究人员来说都是一种启发
Lex Fridman (00:55.960)
and engineers in the AI community.
Rohit Prasad (00:58.840)
This is the Artificial Intelligence Podcast.
Lex Fridman (01:01.940)
If you enjoy it, subscribe on YouTube,
Rohit Prasad (01:04.400)
give it five stars on Apple Podcast, support it on Patreon,
Lex Fridman (01:07.720)
or simply connect with me on Twitter,
Rohit Prasad (01:09.820)
at Lex Friedman, spelled F R I D M A N.
Lex Fridman (01:13.680)
If you leave a review on Apple Podcasts especially,
Lex Fridman (01:16.960)
but also cast box or comment on YouTube,
Lex Fridman (01:20.040)
consider mentioning topics, people, ideas, questions,
Rohit Prasad (01:22.920)
quotes in science, tech, or philosophy
Lex Fridman (01:25.160)
that you find interesting,
Lex Fridman (01:26.320)
and I'll read them on this podcast.
Lex Fridman (01:28.800)
I won't call out names, but I love comments
Rohit Prasad (01:31.640)
with kindness and thoughtfulness in them,
Lex Fridman (01:33.240)
so I thought I'd share them.
Rohit Prasad (01:35.720)
Someone on YouTube highlighted a quote
Lex Fridman (01:37.480)
from the conversation with Ray Dalio,
Rohit Prasad (01:40.280)
where he said that you have to appreciate
Lex Fridman (01:41.960)
all the different ways that people can be A players.
Rohit Prasad (01:45.300)
This connected me to, on teams of engineers,
Lex Fridman (01:48.560)
it's easy to think that raw productivity
Rohit Prasad (01:50.360)
is the measure of excellence, but there are others.
Lex Fridman (01:53.480)
I've worked with people who brought a smile to my face
Rohit Prasad (01:55.760)
every time I got to work in the morning.
Lex Fridman (01:57.920)
Their contribution to the team is immeasurable.
Rohit Prasad (02:01.240)
I recently started doing podcast ads
Lex Fridman (02:03.040)
at the end of the introduction.
Rohit Prasad (02:04.660)
I'll do one or two minutes after introducing the episode,
Lex Fridman (02:07.640)
and never any ads in the middle
Rohit Prasad (02:09.160)
that break the flow of the conversation.
Lex Fridman (02:11.540)
I hope that works for you.
Rohit Prasad (02:13.000)
It doesn't hurt the listening experience.
Lex Fridman (02:15.680)
This show is presented by Cash App,
Rohit Prasad (02:17.840)
the number one finance app in the App Store.
Lex Fridman (02:20.340)
I personally use Cash App to send money to friends,
Lex Fridman (02:23.000)
but you can also use it to buy, sell,
Lex Fridman (02:24.720)
and deposit Bitcoin in just seconds.
Rohit Prasad (02:27.140)
Cash App also has a new investing feature.
Lex Fridman (02:30.360)
You can buy fractions of a stock, say $1 worth,
Rohit Prasad (02:33.640)
no matter what the stock price is.
Lex Fridman (02:35.800)
Brokerage services are provided by Cash App Investing,
Rohit Prasad (02:38.660)
a subsidiary of Square and member SIPC.
Lex Fridman (02:42.420)
I'm excited to be working with Cash App
Rohit Prasad (02:44.440)
to support one of my favorite organizations called First,
Lex Fridman (02:47.560)
best known for their FIRST Robotics and Lego competitions.
Rohit Prasad (02:50.920)
They educate and inspire hundreds of thousands of students
Lex Fridman (02:54.360)
in over 110 countries, and have a perfect rating
Rohit Prasad (02:57.360)
on Charity Navigator, which means that donated money
Lex Fridman (03:00.100)
is used to maximum effectiveness.
Rohit Prasad (03:03.480)
When you get Cash App from the App Store, Google Play,
Lex Fridman (03:06.380)
and use code LexPodcast, you'll get $10,
Lex Fridman (03:10.260)
and Cash App will also donate $10 to FIRST,
Lex Fridman (03:13.240)
which again, is an organization that I've personally seen
Rohit Prasad (03:16.140)
inspire girls and boys to dream
Lex Fridman (03:19.100)
of engineering a better world.
Rohit Prasad (03:20.740)
This podcast is also supported by ZipRecruiter.
Lex Fridman (03:24.240)
Hiring great people is hard, and to me,
Rohit Prasad (03:26.880)
is one of the most important elements
Lex Fridman (03:28.960)
of a successful mission driven team.
Rohit Prasad (03:31.400)
I've been fortunate to be a part of,
Lex Fridman (03:33.280)
and lead several great engineering teams.
Rohit Prasad (03:35.920)
The hiring I've done in the past was mostly through tools
Lex Fridman (03:38.840)
we built ourselves, but reinventing the wheel was painful.
Rohit Prasad (03:42.720)
ZipRecruiter is a tool that's already available for you.
Lex Fridman (03:45.880)
It seeks to make hiring simple, fast, and smart.
Rohit Prasad (03:49.400)
For example, Codable cofounder, Gretchen Huebner,
Lex Fridman (03:52.800)
used ZipRecruiter to find a new game artist
Rohit Prasad (03:55.160)
to join our education tech company.
Lex Fridman (03:57.320)
By using ZipRecruiter's screening questions
Rohit Prasad (03:59.440)
to filter candidates, Gretchen found it easier
Lex Fridman (04:02.080)
to focus on the best candidates,
Lex Fridman (04:03.760)
and finally, hiring the perfect person for the role,
Lex Fridman (04:06.840)
in less than two weeks, from start to finish.
Rohit Prasad (04:10.160)
ZipRecruiter, the smartest way to hire.
Lex Fridman (04:13.600)
See why ZipRecruiter is effective for businesses
Rohit Prasad (04:15.920)
of all sizes by signing up,
Lex Fridman (04:17.920)
as I did, for free, at ziprecruiter.com slash lexpod.
Rohit Prasad (04:23.160)
That's ziprecruiter.com slash lexpod.
Lex Fridman (04:27.160)
And now, here's my conversation with Rohit Prasad.
Rohit Prasad (04:33.000)
In the movie Her, I'm not sure if you've ever seen it.
Lex Fridman (04:36.120)
Human falls in love with the voice of an AI system.
Rohit Prasad (04:39.720)
Let's start at the highest philosophical level
Lex Fridman (04:42.000)
before we get to deep learning and some of the fun things.
Lex Fridman (04:45.080)
Do you think this, what the movie Her shows,
Lex Fridman (04:48.200)
is within our reach?
Rohit Prasad (04:51.160)
I think not specifically about Her,
Lex Fridman (04:54.480)
but I think what we are seeing is a massive increase
Rohit Prasad (04:59.000)
in adoption of AI assistance, or AI,
Lex Fridman (05:02.240)
in all parts of our social fabric.
Lex Fridman (05:05.320)
And I think it's, what I do believe,
Lex Fridman (05:08.880)
is that the utility these AIs provide,
Rohit Prasad (05:11.680)
some of the functionalities that are shown
Lex Fridman (05:14.680)
are absolutely within reach.
Lex Fridman (05:18.240)
So some of the functionality
Lex Fridman (05:19.600)
in terms of the interactive elements,
Lex Fridman (05:21.640)
but in terms of the deep connection,
Lex Fridman (05:24.680)
that's purely voice based.
Lex Fridman (05:26.840)
Do you think such a close connection is possible
Lex Fridman (05:29.160)
with voice alone?
Rohit Prasad (05:30.600)
It's been a while since I saw Her,
Lex Fridman (05:32.240)
but I would say in terms of interactions
Rohit Prasad (05:36.760)
which are both human like and in these AI systems,
Lex Fridman (05:40.240)
you have to value what is also superhuman.
Rohit Prasad (05:44.800)
We as humans can be in only one place.
Lex Fridman (05:47.760)
AI assistance can be in multiple places at the same time.
Rohit Prasad (05:51.240)
One with you on your mobile device,
Lex Fridman (05:53.720)
one at your home, one at work.
Lex Fridman (05:56.360)
So you have to respect these superhuman capabilities too.
Lex Fridman (06:00.280)
Plus as humans, we have certain attributes
Rohit Prasad (06:03.080)
we are very good at, very good at reasoning.
Lex Fridman (06:05.120)
AI assistance not yet there,
Lex Fridman (06:07.360)
but in the realm of AI assistance,
Lex Fridman (06:10.360)
what they're great at is computation, memory,
Rohit Prasad (06:12.680)
it's infinite and pure.
Lex Fridman (06:14.600)
These are the attributes you have to start respecting.
Lex Fridman (06:16.440)
So I think the comparison with human like
Lex Fridman (06:18.360)
versus the other aspect, which is also superhuman,
Rohit Prasad (06:21.480)
has to be taken into consideration.
Lex Fridman (06:22.920)
So I think we need to elevate the discussion
Rohit Prasad (06:25.440)
to not just human like.
Lex Fridman (06:27.240)
So there's certainly elements,
Rohit Prasad (06:28.800)
we just mentioned, Alexa is everywhere,
Lex Fridman (06:32.680)
computation speaking.
Lex Fridman (06:33.960)
So this is a much bigger infrastructure
Lex Fridman (06:35.600)
than just the thing that sits there in the room with you.
Lex Fridman (06:38.440)
But it certainly feels to us mere humans
Lex Fridman (06:43.120)
that there's just another little creature there
Rohit Prasad (06:47.320)
when you're interacting with it.
Lex Fridman (06:48.400)
You're not interacting with the entirety
Rohit Prasad (06:49.880)
of the infrastructure, you're interacting with the device.
Lex Fridman (06:52.360)
The feeling is, okay, sure, we anthropomorphize things,
Lex Fridman (06:56.560)
but that feeling is still there.
Lex Fridman (06:58.640)
So what do you think we as humans,
Rohit Prasad (07:02.240)
the purity of the interaction with a smart device,
Lex Fridman (07:04.760)
interaction with a smart assistant,
Lex Fridman (07:06.920)
what do you think we look for in that interaction?
Lex Fridman (07:10.200)
I think in the certain interactions
Rohit Prasad (07:12.240)
I think will be very much where it does feel like a human
Lex Fridman (07:15.920)
because it has a persona of its own.
Lex Fridman (07:19.080)
And in certain ones it wouldn't be.
Lex Fridman (07:20.680)
So I think a simple example to think of it
Rohit Prasad (07:23.080)
is if you're walking through the house
Lex Fridman (07:25.200)
and you just wanna turn on your lights on and off
Lex Fridman (07:27.960)
and you're issuing a command,
Lex Fridman (07:29.840)
that's not very much like a human like interaction
Lex Fridman (07:32.040)
and that's where the AI shouldn't come back
Lex Fridman (07:33.840)
and have a conversation with you,
Rohit Prasad (07:35.240)
just it should simply complete that command.
Lex Fridman (07:38.480)
So those, I think the blend of,
Rohit Prasad (07:40.200)
we have to think about this is not human, human alone.
Lex Fridman (07:43.360)
It is a human machine interaction
Lex Fridman (07:45.080)
and certain aspects of humans are needed
Lex Fridman (07:48.160)
and certain aspects are in situations
Rohit Prasad (07:49.920)
demand it to be like a machine.
Lex Fridman (07:51.640)
So I told you, it's gonna be philosophical in parts.
Rohit Prasad (07:55.040)
What's the difference between human and machine
Lex Fridman (07:57.480)
in that interaction?
Rohit Prasad (07:58.640)
When we interact to humans,
Lex Fridman (08:00.760)
especially those are friends and loved ones
Rohit Prasad (08:04.000)
versus you and a machine that you also are close with.
Lex Fridman (08:10.400)
I think the, you have to think about the roles
Lex Fridman (08:12.640)
the AI plays, right?
Lex Fridman (08:13.800)
So, and it differs from different customer to customer,
Rohit Prasad (08:16.320)
different situation to situation,
Lex Fridman (08:18.840)
especially I can speak from Alexa's perspective.
Rohit Prasad (08:21.560)
It is a companion, a friend at times,
Lex Fridman (08:25.000)
an assistant, an advisor down the line.
Lex Fridman (08:27.520)
So I think most AIs will have this kind of attributes
Lex Fridman (08:31.240)
and it will be very situational in nature.
Lex Fridman (08:33.040)
So where is the boundary?
Lex Fridman (08:34.680)
I think the boundary depends on exact context
Rohit Prasad (08:37.080)
in which you're interacting with the AI.
Lex Fridman (08:39.320)
So the depth and the richness
Rohit Prasad (08:41.240)
of natural language conversation
Lex Fridman (08:42.920)
is been by Alan Turing been used to try to define
Lex Fridman (08:48.160)
what it means to be intelligent.
Lex Fridman (08:50.480)
There's a lot of criticism of that kind of test,
Lex Fridman (08:52.280)
but what do you think is a good test of intelligence
Lex Fridman (08:55.840)
in your view, in the context of the Turing test
Lex Fridman (08:58.360)
and Alexa or the Alexa prize, this whole realm,
Lex Fridman (09:03.240)
do you think about this human intelligence,
Lex Fridman (09:07.160)
what it means to define it,
Lex Fridman (09:08.000)
what it means to reach that level?
Rohit Prasad (09:10.080)
I do think the ability to converse
Lex Fridman (09:12.480)
is a sign of an ultimate intelligence.
Rohit Prasad (09:15.160)
I think that there's no question about it.
Lex Fridman (09:18.320)
So if you think about all aspects of humans,
Rohit Prasad (09:20.560)
there are sensors we have,
Lex Fridman (09:22.840)
and those are basically a data collection mechanism.
Lex Fridman (09:26.400)
And based on that,
Lex Fridman (09:27.240)
we make some decisions with our sensory brains, right?
Lex Fridman (09:30.560)
And from that perspective,
Lex Fridman (09:32.720)
I think there are elements we have to talk about
Lex Fridman (09:35.240)
how we sense the world
Lex Fridman (09:37.080)
and then how we act based on what we sense.
Rohit Prasad (09:40.360)
Those elements clearly machines have,
Lex Fridman (09:43.640)
but then there's the other aspects of computation
Rohit Prasad (09:46.800)
that is way better.
Lex Fridman (09:48.360)
I also mentioned about memory again,
Rohit Prasad (09:50.040)
in terms of being near infinite,
Lex Fridman (09:51.880)
depending on the storage capacity you have,
Lex Fridman (09:54.200)
and the retrieval can be extremely fast and pure
Lex Fridman (09:58.200)
in terms of like, there's no ambiguity
Lex Fridman (09:59.600)
of who did I see when, right?
Lex Fridman (10:02.080)
I mean, machines can remember that quite well.
Lex Fridman (10:04.440)
So again, on a philosophical level,
Lex Fridman (10:06.840)
I do subscribe to the fact that to be able to converse
Lex Fridman (10:10.840)
and as part of that, to be able to reason
Lex Fridman (10:13.400)
based on the world knowledge you've acquired
Lex Fridman (10:15.240)
and the sensory knowledge that is there
Lex Fridman (10:18.320)
is definitely very much the essence of intelligence.
Lex Fridman (10:23.160)
But intelligence can go beyond human level intelligence
Lex Fridman (10:26.960)
based on what machines are getting capable of.
Lex Fridman (10:29.800)
So what do you think maybe stepping outside of Alexa
Lex Fridman (10:33.440)
broadly as an AI field,
Lex Fridman (10:35.760)
what do you think is a good test of intelligence?
Lex Fridman (10:38.720)
Put it another way outside of Alexa,
Rohit Prasad (10:41.200)
because so much of Alexa is a product,
Lex Fridman (10:43.040)
is an experience for the customer.
Rohit Prasad (10:44.920)
On the research side,
Lex Fridman (10:46.400)
what would impress the heck out of you if you saw,
Rohit Prasad (10:49.240)
you know, what is the test where you said,
Lex Fridman (10:50.800)
wow, this thing is now starting to encroach
Rohit Prasad (10:57.000)
into the realm of what we loosely think
Lex Fridman (10:59.040)
of as human intelligence?
Rohit Prasad (11:00.360)
So, well, we think of it as AGI
Lex Fridman (11:02.400)
and human intelligence altogether, right?
Lex Fridman (11:04.320)
So in some sense, and I think we are quite far from that.
Lex Fridman (11:08.000)
I think an unbiased view I have
Rohit Prasad (11:11.480)
is that the Alexa's intelligence capability is a great test.
Lex Fridman (11:17.760)
I think of it as there are many other true points
Rohit Prasad (11:20.600)
like self driving cars, game playing like go or chess.
Lex Fridman (11:26.320)
Let's take those two for as an example,
Rohit Prasad (11:28.680)
clearly requires a lot of data driven learning
Lex Fridman (11:31.760)
and intelligence, but it's not as hard a problem
Rohit Prasad (11:35.080)
as conversing with, as an AI is with humans
Lex Fridman (11:39.760)
to accomplish certain tasks or open domain chat,
Rohit Prasad (11:42.320)
as you mentioned, Alexa prize.
Lex Fridman (11:44.880)
In those settings, the key differences
Rohit Prasad (11:47.760)
that the end goal is not defined unlike game playing.
Lex Fridman (11:51.920)
You also do not know exactly what state you are in
Rohit Prasad (11:55.720)
in a particular goal completion scenario.
Lex Fridman (11:58.960)
In certain sense, sometimes you can,
Rohit Prasad (12:00.760)
if it's a simple goal, but if you're even certain examples
Lex Fridman (12:04.440)
like planning a weekend or you can imagine
Lex Fridman (12:07.120)
how many things change along the way,
Lex Fridman (12:09.920)
you look for whether you may change your mind
Lex Fridman (12:11.920)
and you change the destination,
Lex Fridman (12:14.840)
or you want to catch a particular event
Lex Fridman (12:17.040)
and then you decide, no, I want this other event
Lex Fridman (12:19.400)
I want to go to.
Lex Fridman (12:20.520)
So these dimensions of how many different steps
Lex Fridman (12:24.000)
are possible when you're conversing as a human
Rohit Prasad (12:26.360)
with a machine makes it an extremely daunting problem.
Lex Fridman (12:29.120)
And I think it is the ultimate test for intelligence.
Lex Fridman (12:32.360)
And don't you think that natural language is enough to prove
Lex Fridman (12:37.440)
that conversation, just pure conversation?
Rohit Prasad (12:40.360)
From a scientific standpoint,
Lex Fridman (12:42.280)
natural language is a great test,
Lex Fridman (12:45.000)
but I would go beyond, I don't want to limit it
Lex Fridman (12:47.800)
to as natural language as simply understanding an intent
Rohit Prasad (12:51.040)
or parsing for entities and so forth.
Lex Fridman (12:52.760)
We are really talking about dialogue.
Rohit Prasad (12:54.880)
Dialogue, yeah.
Lex Fridman (12:55.720)
So I would say human machine dialogue
Rohit Prasad (12:58.480)
is definitely one of the best tests of intelligence.
Lex Fridman (13:02.960)
So can you briefly speak to the Alexa Prize
Rohit Prasad (13:06.680)
for people who are not familiar with it,
Lex Fridman (13:08.640)
and also just maybe where things stand
Lex Fridman (13:12.640)
and what have you learned and what's surprising?
Lex Fridman (13:15.440)
What have you seen that's surprising
Lex Fridman (13:16.920)
from this incredible competition?
Lex Fridman (13:18.440)
Absolutely, it's a very exciting competition.
Rohit Prasad (13:20.960)
Alexa Prize is essentially a grand challenge
Lex Fridman (13:24.040)
in conversational artificial intelligence,
Rohit Prasad (13:26.880)
where we threw the gauntlet to the universities
Lex Fridman (13:29.440)
who do active research in the field,
Rohit Prasad (13:31.960)
to say, can you build what we call a social bot
Lex Fridman (13:35.360)
that can converse with you coherently
Lex Fridman (13:37.320)
and engagingly for 20 minutes?
Lex Fridman (13:39.800)
That is an extremely hard challenge,
Rohit Prasad (13:42.480)
talking to someone who you're meeting for the first time,
Lex Fridman (13:46.480)
or even if you've met them quite often,
Rohit Prasad (13:49.640)
to speak at 20 minutes on any topic,
Lex Fridman (13:53.560)
an evolving nature of topics is super hard.
Rohit Prasad (13:57.760)
We have completed two successful years of the competition.
Lex Fridman (14:01.600)
The first was won with the University of Washington,
Rohit Prasad (14:03.400)
second, the University of California.
Lex Fridman (14:05.560)
We are in our third instance.
Rohit Prasad (14:06.880)
We have an extremely strong team of 10 cohorts,
Lex Fridman (14:09.640)
and the third instance of the Alexa Prize is underway now.
Lex Fridman (14:14.840)
And we are seeing a constant evolution.
Lex Fridman (14:17.480)
First year was definitely a learning.
Rohit Prasad (14:18.920)
It was a lot of things to be put together.
Lex Fridman (14:21.200)
We had to build a lot of infrastructure
Rohit Prasad (14:23.640)
to enable these universities
Lex Fridman (14:25.960)
to be able to build magical experiences
Lex Fridman (14:28.280)
and do high quality research.
Lex Fridman (14:31.560)
Just a few quick questions, sorry for the interruption.
Lex Fridman (14:33.880)
What does failure look like in the 20 minute session?
Lex Fridman (14:37.240)
So what does it mean to fail,
Lex Fridman (14:38.720)
not to reach the 20 minute mark?
Lex Fridman (14:39.960)
Oh, awesome question.
Lex Fridman (14:41.200)
So there are one, first of all,
Lex Fridman (14:43.360)
I forgot to mention one more detail.
Rohit Prasad (14:45.360)
It's not just 20 minutes,
Lex Fridman (14:46.560)
but the quality of the conversation too that matters.
Lex Fridman (14:49.320)
And the beauty of this competition
Lex Fridman (14:51.480)
before I answer that question on what failure means
Rohit Prasad (14:53.800)
is first that you actually converse
Lex Fridman (14:56.600)
with millions and millions of customers
Rohit Prasad (14:59.000)
as the social bots.
Lex Fridman (15:00.840)
So during the judging phases, there are multiple phases,
Rohit Prasad (15:05.000)
before we get to the finals,
Lex Fridman (15:06.320)
which is a very controlled judging in a situation
Rohit Prasad (15:08.640)
where we bring in judges
Lex Fridman (15:10.400)
and we have interactors who interact with these social bots,
Rohit Prasad (15:14.400)
that is a much more controlled setting.
Lex Fridman (15:15.920)
But till the point we get to the finals,
Rohit Prasad (15:18.960)
all the judging is essentially by the customers of Alexa.
Lex Fridman (15:22.680)
And there you basically rate on a simple question,
Lex Fridman (15:26.160)
how good your experience was.
Lex Fridman (15:28.400)
So that's where we are not testing
Rohit Prasad (15:29.840)
for a 20 minute boundary being crossed,
Lex Fridman (15:32.760)
because you do want it to be very much like a clear cut,
Rohit Prasad (15:36.600)
winner, be chosen, and it's an absolute bar.
Lex Fridman (15:40.040)
So did you really break that 20 minute barrier
Rohit Prasad (15:42.760)
is why we have to test it in a more controlled setting
Lex Fridman (15:45.880)
with actors, essentially interactors.
Lex Fridman (15:48.640)
And see how the conversation goes.
Lex Fridman (15:50.800)
So this is why it's a subtle difference
Rohit Prasad (15:54.160)
between how it's being tested in the field
Lex Fridman (15:57.000)
with real customers versus in the lab to award the prize.
Lex Fridman (16:00.480)
So on the latter one, what it means is that
Lex Fridman (16:03.520)
essentially there are three judges
Lex Fridman (16:08.000)
and two of them have to say
Lex Fridman (16:09.520)
this conversation has stalled, essentially.
Rohit Prasad (16:13.080)
Got it.
Lex Fridman (16:13.920)
And the judges are human experts.
Rohit Prasad (16:15.720)
Judges are human experts.
Lex Fridman (16:16.920)
Okay, great.
Lex Fridman (16:17.760)
So this is in the third year.
Lex Fridman (16:19.120)
So what's been the evolution?
Lex Fridman (16:20.920)
How far, so the DARPA challenge in the first year,
Lex Fridman (16:24.640)
the autonomous vehicles, nobody finished.
Rohit Prasad (16:26.560)
In the second year, a few more finished in the desert.
Lex Fridman (16:30.640)
So how far along in this,
Lex Fridman (16:33.280)
I would say much harder challenge are we?
Lex Fridman (16:36.360)
This challenge has come a long way
Rohit Prasad (16:37.720)
to the extent that we're definitely not close
Lex Fridman (16:40.480)
to the 20 minute barrier being with coherence
Lex Fridman (16:42.760)
and engaging conversation.
Lex Fridman (16:44.760)
I think we are still five to 10 years away
Rohit Prasad (16:46.880)
in that horizon to complete that.
Lex Fridman (16:49.480)
But the progress is immense.
Rohit Prasad (16:51.360)
Like what you're finding is the accuracy
Lex Fridman (16:54.080)
and what kind of responses these social bots generate
Rohit Prasad (16:57.360)
is getting better and better.
Lex Fridman (16:59.520)
What's even amazing to see that now there's humor coming in.
Rohit Prasad (17:03.360)
The bots are quite...
Lex Fridman (17:04.880)
Awesome.
Rohit Prasad (17:05.720)
You know, you're talking about
Lex Fridman (17:07.360)
ultimate science of intelligence.
Rohit Prasad (17:09.440)
I think humor is a very high bar
Lex Fridman (17:11.840)
in terms of what it takes to create humor.
Lex Fridman (17:14.880)
And I don't mean just being goofy.
Lex Fridman (17:16.520)
I really mean good sense of humor
Rohit Prasad (17:19.480)
is also a sign of intelligence in my mind
Lex Fridman (17:21.600)
and something very hard to do.
Lex Fridman (17:23.120)
So these social bots are now exploring
Lex Fridman (17:25.040)
not only what we think of natural language abilities,
Lex Fridman (17:28.560)
but also personality attributes
Lex Fridman (17:30.400)
and aspects of when to inject an appropriate joke,
Rohit Prasad (17:34.120)
when you don't know the domain,
Lex Fridman (17:38.440)
how you come back with something more intelligible
Lex Fridman (17:41.400)
so that you can continue the conversation.
Lex Fridman (17:43.200)
If you and I are talking about AI
Lex Fridman (17:45.200)
and we are domain experts, we can speak to it.
Lex Fridman (17:47.480)
But if you suddenly switch a topic to that I don't know of,
Lex Fridman (17:50.480)
how do I change the conversation?
Lex Fridman (17:52.160)
So you're starting to notice these elements as well.
Lex Fridman (17:55.240)
And that's coming from partly by the nature
Lex Fridman (17:58.560)
of the 20 minute challenge
Rohit Prasad (18:00.120)
that people are getting quite clever
Lex Fridman (18:02.520)
on how to really converse
Lex Fridman (18:05.600)
and essentially mask some of the understanding defects
Lex Fridman (18:08.600)
if they exist.
Lex Fridman (18:09.840)
So some of this, this is not Alexa, the product.
Lex Fridman (18:12.680)
This is somewhat for fun, for research,
Rohit Prasad (18:16.240)
for innovation and so on.
Lex Fridman (18:17.800)
I have a question sort of in this modern era,
Rohit Prasad (18:20.280)
there's a lot of, if you look at Twitter and Facebook
Lex Fridman (18:24.280)
and so on, there's discourse, public discourse going on
Lex Fridman (18:27.160)
and some things that are a little bit too edgy,
Lex Fridman (18:28.800)
people get blocked and so on.
Rohit Prasad (18:30.640)
I'm just out of curiosity,
Lex Fridman (18:32.280)
are people in this context pushing the limits?
Lex Fridman (18:35.960)
Is anyone using the F word?
Lex Fridman (18:37.760)
Is anyone sort of pushing back
Rohit Prasad (18:41.440)
sort of arguing, I guess I should say,
Lex Fridman (18:45.960)
as part of the dialogue to really draw people in?
Rohit Prasad (18:48.280)
First of all, let me just back up a bit
Lex Fridman (18:50.320)
in terms of why we are doing this, right?
Lex Fridman (18:52.120)
So you said it's fun.
Lex Fridman (18:54.280)
I think fun is more part of the engaging part for customers.
Rohit Prasad (18:59.920)
It is one of the most used skills as well
Lex Fridman (19:02.480)
in our skill store.
Lex Fridman (19:04.360)
But up that apart, the real goal was essentially
Lex Fridman (19:07.200)
what was happening is with a lot of AI research
Rohit Prasad (19:10.400)
moving to industry, we felt that academia has the risk
Lex Fridman (19:14.200)
of not being able to have the same resources
Rohit Prasad (19:16.800)
at disposal that we have, which is lots of data,
Lex Fridman (19:20.480)
massive computing power, and a clear ways
Rohit Prasad (19:24.640)
to test these AI advances with real customer benefits.
Lex Fridman (19:28.520)
So we brought all these three together in the Alexa price.
Rohit Prasad (19:30.880)
That's why it's one of my favorite projects in Amazon.
Lex Fridman (19:33.880)
And with that, the secondary effect is yes,
Rohit Prasad (19:37.800)
it has become engaging for our customers as well.
Lex Fridman (19:40.920)
We're not there in terms of where we want it to be, right?
Lex Fridman (19:43.880)
But it's a huge progress.
Lex Fridman (19:45.040)
But coming back to your question on
Lex Fridman (19:47.080)
how do the conversations evolve?
Lex Fridman (19:48.800)
Yes, there is some natural attributes of what you said
Rohit Prasad (19:51.880)
in terms of argument and some amount of swearing.
Lex Fridman (19:54.160)
The way we take care of that is that there is
Rohit Prasad (19:57.160)
a sensitive filter we have built that sees keywords.
Lex Fridman (1:00:01.400)
in terms of the amount of data.
Lex Fridman (1:00:03.960)
So that was quite important work
Lex Fridman (1:00:06.200)
where it was algorithmic improvements
Rohit Prasad (1:00:07.840)
as well as a lot of engineering improvements
Lex Fridman (1:00:09.920)
to be able to train on thousands and thousands of speech.
Lex Fridman (1:00:14.000)
And that was an important factor.
Lex Fridman (1:00:15.600)
So if you ask me like back in 2013 and 2014,
Rohit Prasad (1:00:19.320)
when we launched Echo,
Lex Fridman (1:00:22.440)
the combination of large scale data,
Rohit Prasad (1:00:25.680)
deep learning progress, near infinite GPUs
Lex Fridman (1:00:29.680)
we had available on AWS even then,
Rohit Prasad (1:00:33.120)
was all came together for us to be able
Lex Fridman (1:00:35.320)
to solve the far field speech recognition
Rohit Prasad (1:00:38.400)
to the extent it could be useful to the customers.
Lex Fridman (1:00:40.640)
It's still not solved.
Rohit Prasad (1:00:41.480)
Like, I mean, it's not that we are perfect
Lex Fridman (1:00:43.000)
at recognizing speech, but we are great at it
Lex Fridman (1:00:45.520)
in terms of the settings that are in homes, right?
Lex Fridman (1:00:48.360)
So, and that was important even in the early stages.
Lex Fridman (1:00:50.920)
So first of all, just even,
Lex Fridman (1:00:51.960)
I'm trying to look back at that time.
Rohit Prasad (1:00:54.240)
If I remember correctly,
Lex Fridman (1:00:57.120)
it was, it seems like the task would be pretty daunting.
Lex Fridman (1:01:01.160)
So like, so we kind of take it for granted
Lex Fridman (1:01:04.480)
that it works now.
Rohit Prasad (1:01:06.400)
Yes, you're right.
Lex Fridman (1:01:07.720)
So let me, like how, first of all, you mentioned startup.
Rohit Prasad (1:01:10.880)
I wasn't familiar how big the team was.
Lex Fridman (1:01:12.880)
I kind of, cause I know there's a lot
Rohit Prasad (1:01:14.200)
of really smart people working on it.
Lex Fridman (1:01:16.040)
So now it's a very, very large team.
Lex Fridman (1:01:19.120)
How big was the team?
Lex Fridman (1:01:20.840)
How likely were you to fail in the eyes of everyone else?
Lex Fridman (1:01:24.120)
And ourselves?
Lex Fridman (1:01:26.120)
And yourself?
Lex Fridman (1:01:27.760)
So like what?
Lex Fridman (1:01:28.600)
I'll give you a very interesting anecdote on that.
Rohit Prasad (1:01:31.600)
When I joined the team,
Lex Fridman (1:01:33.880)
the speech recognition team was six people.
Rohit Prasad (1:01:37.680)
My first meeting, and we had hired a few more people,
Lex Fridman (1:01:40.520)
it was 10 people.
Rohit Prasad (1:01:42.960)
Nine out of 10 people thought it can't be done.
Lex Fridman (1:01:48.040)
Who was the one?
Rohit Prasad (1:01:50.080)
The one was me, say, actually I should say,
Lex Fridman (1:01:52.960)
and one was semi optimistic.
Lex Fridman (1:01:56.000)
And eight were trying to convince,
Lex Fridman (1:01:59.120)
let's go to the management and say,
Rohit Prasad (1:02:01.720)
let's not work on this problem.
Lex Fridman (1:02:03.600)
Let's work on some other problem,
Rohit Prasad (1:02:05.240)
like either telephony speech for customer service calls
Lex Fridman (1:02:09.000)
and so forth.
Lex Fridman (1:02:10.160)
But this was the kind of belief you must have.
Lex Fridman (1:02:12.040)
And I had experience with far field speech recognition
Lex Fridman (1:02:14.360)
and my eyes lit up when I saw a problem like that saying,
Lex Fridman (1:02:17.720)
okay, we have been in speech recognition,
Rohit Prasad (1:02:20.840)
always looking for that killer app.
Lex Fridman (1:02:23.400)
And this was a killer use case
Rohit Prasad (1:02:25.840)
to bring something delightful in the hands of customers.
Lex Fridman (1:02:28.840)
So you mentioned the way you kind of think of it
Rohit Prasad (1:02:31.200)
in the product way in the future,
Lex Fridman (1:02:32.680)
have a press release and an FAQ and you think backwards.
Lex Fridman (1:02:35.760)
Did you have, did the team have the echo in mind?
Lex Fridman (1:02:41.000)
So this far field speech recognition,
Rohit Prasad (1:02:43.040)
actually putting a thing in the home that works,
Lex Fridman (1:02:45.360)
that it's able to interact with,
Lex Fridman (1:02:46.640)
was that the press release?
Lex Fridman (1:02:48.160)
What was the?
Rohit Prasad (1:02:49.000)
The way close, I would say, in terms of the,
Lex Fridman (1:02:51.440)
as I said, the vision was start a computer, right?
Rohit Prasad (1:02:55.520)
Or the inspiration.
Lex Fridman (1:02:56.880)
And from there, I can't divulge
Rohit Prasad (1:02:59.120)
all the exact specifications,
Lex Fridman (1:03:00.600)
but one of the first things that was magical on Alexa
Rohit Prasad (1:03:07.200)
was music.
Lex Fridman (1:03:08.800)
It brought me to back to music
Rohit Prasad (1:03:11.160)
because my taste was still in when I was an undergrad.
Lex Fridman (1:03:14.200)
So I still listened to those songs and I,
Lex Fridman (1:03:17.400)
it was too hard for me to be a music fan with a phone, right?
Lex Fridman (1:03:21.400)
So I, and I don't, I hate things in my ears.
Lex Fridman (1:03:24.200)
So from that perspective, it was quite hard
Lex Fridman (1:03:28.120)
and music was part of the,
Lex Fridman (1:03:32.040)
at least the documents I have seen, right?
Lex Fridman (1:03:33.640)
So from that perspective, I think, yes,
Lex Fridman (1:03:36.120)
in terms of how far are we from the original vision?
Lex Fridman (1:03:40.920)
I can't reveal that, but it's,
Rohit Prasad (1:03:42.400)
that's why I have done a fun at work
Lex Fridman (1:03:44.520)
because every day we go in and thinking like,
Rohit Prasad (1:03:47.200)
these are the new set of challenges to solve.
Lex Fridman (1:03:49.080)
Yeah, that's a great way to do great engineering
Rohit Prasad (1:03:51.920)
as you think of the press release.
Lex Fridman (1:03:53.640)
I like that idea actually.
Rohit Prasad (1:03:55.040)
Maybe we'll talk about it a bit later,
Lex Fridman (1:03:56.840)
but it's just a super nice way to have a focus.
Rohit Prasad (1:03:59.280)
I'll tell you this, you're a scientist
Lex Fridman (1:04:01.400)
and a lot of my scientists have adopted that.
Rohit Prasad (1:04:03.760)
They have now, they love it as a process
Lex Fridman (1:04:07.000)
because it was very, as scientists,
Rohit Prasad (1:04:09.000)
you're trained to write great papers,
Lex Fridman (1:04:10.960)
but they are all after you've done the research
Rohit Prasad (1:04:13.520)
or you've proven that and your PhD dissertation proposal
Lex Fridman (1:04:16.640)
is something that comes closest
Rohit Prasad (1:04:18.480)
or a DARPA proposal or a NSF proposal
Lex Fridman (1:04:21.200)
is the closest that comes to a press release.
Lex Fridman (1:04:23.640)
But that process is now ingrained in our scientists,
Lex Fridman (1:04:27.040)
which is like delightful for me to see.
Rohit Prasad (1:04:30.960)
You write the paper first and then make it happen.
Lex Fridman (1:04:33.080)
That's right.
Rohit Prasad (1:04:33.920)
In fact, it's not.
Lex Fridman (1:04:34.760)
State of the art results.
Rohit Prasad (1:04:36.320)
Or you leave the results section open
Lex Fridman (1:04:38.480)
where you have a thesis about here's what I expect, right?
Lex Fridman (1:04:41.680)
And here's what it will change, right?
Lex Fridman (1:04:44.960)
So I think it is a great thing.
Rohit Prasad (1:04:46.560)
It works for researchers as well.
Lex Fridman (1:04:48.280)
Yeah.
Lex Fridman (1:04:49.120)
So far field recognition.
Lex Fridman (1:04:50.760)
Yeah.
Lex Fridman (1:04:52.400)
What was the big leap?
Lex Fridman (1:04:53.920)
What were the breakthroughs
Lex Fridman (1:04:55.520)
and what was that journey like to today?
Lex Fridman (1:04:58.440)
Yeah, I think the, as you said first,
Rohit Prasad (1:05:00.240)
there was a lot of skepticism
Lex Fridman (1:05:01.640)
on whether far field speech recognition
Lex Fridman (1:05:03.400)
will ever work to be good enough, right?
Lex Fridman (1:05:06.560)
And what we first did was got a lot of training data
Rohit Prasad (1:05:10.040)
in a far field setting.
Lex Fridman (1:05:11.520)
And that was extremely hard to get
Rohit Prasad (1:05:14.080)
because none of it existed.
Lex Fridman (1:05:16.240)
So how do you collect data in far field setup, right?
Rohit Prasad (1:05:20.120)
With no customer base at this time.
Lex Fridman (1:05:21.400)
With no customer base, right?
Lex Fridman (1:05:22.720)
So that was first innovation.
Lex Fridman (1:05:24.840)
And once we had that, the next thing was,
Rohit Prasad (1:05:27.040)
okay, if you have the data,
Lex Fridman (1:05:29.760)
first of all, we didn't talk about like,
Lex Fridman (1:05:31.920)
what would magical mean in this kind of a setting?
Lex Fridman (1:05:35.320)
What is good enough for customers, right?
Rohit Prasad (1:05:37.520)
That's always, since you've never done this before,
Lex Fridman (1:05:40.480)
what would be magical?
Lex Fridman (1:05:41.680)
So it wasn't just a research problem.
Lex Fridman (1:05:44.280)
You had to put some in terms of accuracy
Lex Fridman (1:05:47.720)
and customer experience features,
Lex Fridman (1:05:49.960)
some stakes on the ground saying,
Rohit Prasad (1:05:51.560)
here's where I think it should get to.
Lex Fridman (1:05:55.000)
So you established a bar
Lex Fridman (1:05:56.120)
and then how do you measure progress
Lex Fridman (1:05:57.520)
towards given you have no customer right now.
Lex Fridman (1:06:01.800)
So from that perspective, we went,
Lex Fridman (1:06:04.240)
so first was the data without customers.
Rohit Prasad (1:06:07.600)
Second was doubling down on deep learning
Lex Fridman (1:06:10.600)
as a way to learn.
Lex Fridman (1:06:11.960)
And I can just tell you that the combination of the two
Lex Fridman (1:06:16.200)
got our error rates by a factor of five.
Rohit Prasad (1:06:19.240)
From where we were when I started
Lex Fridman (1:06:21.440)
to within six months of having that data,
Rohit Prasad (1:06:24.360)
we, at that point, I got the conviction
Lex Fridman (1:06:28.440)
that this will work, right?
Rohit Prasad (1:06:29.960)
So, because that was magical
Lex Fridman (1:06:31.680)
in terms of when it started working and.
Rohit Prasad (1:06:34.760)
That reached the magical bar.
Lex Fridman (1:06:36.280)
That came close to the magical bar.
Lex Fridman (1:06:38.000)
To the bar, right?
Lex Fridman (1:06:39.560)
That we felt would be where people will use it.
Rohit Prasad (1:06:44.280)
That was critical.
Lex Fridman (1:06:45.360)
Because you really have one chance at this.
Rohit Prasad (1:06:48.880)
If we had launched in November 2014 is when we launched,
Lex Fridman (1:06:51.920)
if it was below the bar,
Rohit Prasad (1:06:53.160)
I don't think this category exists
Lex Fridman (1:06:56.520)
if you don't meet the bar.
Rohit Prasad (1:06:58.120)
Yeah, and just having looked at voice based interactions
Lex Fridman (1:07:02.080)
like in the car or earlier systems,
Rohit Prasad (1:07:06.120)
it's a source of huge frustration for people.
Lex Fridman (1:07:08.320)
In fact, we use voice based interaction
Rohit Prasad (1:07:10.280)
for collecting data on subjects to measure frustration.
Lex Fridman (1:07:14.600)
So, as a training set for computer vision,
Rohit Prasad (1:07:16.560)
for face data, so we can get a data set
Lex Fridman (1:07:19.360)
of frustrated people.
Rohit Prasad (1:07:20.600)
That's the best way to get frustrated people
Lex Fridman (1:07:22.240)
is having them interact with a voice based system
Rohit Prasad (1:07:24.840)
in the car.
Lex Fridman (1:07:25.680)
So, that bar I imagine is pretty high.
Rohit Prasad (1:07:28.520)
It was very high.
Lex Fridman (1:07:29.480)
And we talked about how also errors are perceived
Rohit Prasad (1:07:32.720)
from AIs versus errors by humans.
Lex Fridman (1:07:35.400)
But we are not done with the problems that ended up,
Rohit Prasad (1:07:38.320)
we had to solve to get it to launch.
Lex Fridman (1:07:39.800)
So, do you want the next one?
Rohit Prasad (1:07:41.280)
Yeah, the next one.
Lex Fridman (1:07:42.680)
So, the next one was what I think of as
Rohit Prasad (1:07:47.680)
multi domain natural language understanding.
Lex Fridman (1:07:50.960)
It's very, I wouldn't say easy,
Lex Fridman (1:07:53.200)
but it is during those days,
Lex Fridman (1:07:56.160)
solving it, understanding in one domain,
Rohit Prasad (1:07:59.720)
a narrow domain was doable,
Lex Fridman (1:08:02.880)
but for these multiple domains like music,
Rohit Prasad (1:08:06.880)
like information, other kinds of household productivity,
Lex Fridman (1:08:10.680)
alarms, timers, even though it wasn't as big as it is
Rohit Prasad (1:08:14.160)
in terms of the number of skills Alexa has
Lex Fridman (1:08:15.640)
and the confusion space has like grown
Rohit Prasad (1:08:17.480)
by three orders of magnitude,
Lex Fridman (1:08:20.680)
it was still daunting even those days.
Lex Fridman (1:08:22.680)
And again, no customer base yet.
Lex Fridman (1:08:24.640)
Again, no customer base.
Rohit Prasad (1:08:26.200)
So, now you're looking at meaning understanding
Lex Fridman (1:08:28.200)
and intent understanding and taking actions
Rohit Prasad (1:08:30.120)
on behalf of customers.
Lex Fridman (1:08:31.640)
Based on their requests.
Lex Fridman (1:08:33.440)
And that is the next hard problem.
Lex Fridman (1:08:36.440)
Even if you have gotten the words recognized,
Lex Fridman (1:08:39.960)
how do you make sense of them?
Lex Fridman (1:08:42.520)
In those days, there was still a lot of emphasis
Rohit Prasad (1:08:47.520)
on rule based systems for writing grammar patterns
Lex Fridman (1:08:50.760)
to understand the intent.
Lex Fridman (1:08:52.360)
But we had a statistical first approach even then,
Lex Fridman (1:08:55.560)
where for our language understanding we had,
Lex Fridman (1:08:58.240)
and even those starting days,
Lex Fridman (1:09:00.200)
an entity recognizer and an intent classifier,
Rohit Prasad (1:09:03.520)
which was all trained statistically.
Lex Fridman (1:09:06.080)
In fact, we had to build the deterministic matching
Rohit Prasad (1:09:09.400)
as a follow up to fix bugs that statistical models have.
Lex Fridman (1:09:14.400)
So, it was just a different mindset
Rohit Prasad (1:09:16.320)
where we focused on data driven statistical understanding.
Lex Fridman (1:09:20.080)
It wins in the end if you have a huge data set.
Rohit Prasad (1:09:22.720)
Yes, it is contingent on that.
Lex Fridman (1:09:24.520)
And that's why it came back to how do you get the data.
Rohit Prasad (1:09:27.120)
Before customers, the fact that this is why data
Lex Fridman (1:09:30.360)
becomes crucial to get to the point
Rohit Prasad (1:09:33.280)
that you have the understanding system built up.
Lex Fridman (1:09:37.840)
And notice that for you,
Rohit Prasad (1:09:40.680)
we were talking about human machine dialogue,
Lex Fridman (1:09:42.480)
and even those early days,
Rohit Prasad (1:09:44.800)
even it was very much transactional,
Lex Fridman (1:09:47.120)
do one thing, one shot utterances in great way.
Rohit Prasad (1:09:50.560)
There was a lot of debate on how much should Alexa talk back
Lex Fridman (1:09:52.840)
in terms of if you misunderstood it.
Rohit Prasad (1:09:55.680)
If you misunderstood you or you said play songs by the stones,
Lex Fridman (1:10:01.440)
and let's say it doesn't know early days,
Lex Fridman (1:10:04.760)
knowledge can be sparse, who are the stones?
Lex Fridman (1:10:09.240)
It's the Rolling Stones.
Lex Fridman (1:10:12.760)
And you don't want the match to be Stone Temple Pilots
Lex Fridman (1:10:16.280)
or Rolling Stones.
Rohit Prasad (1:10:17.200)
So, you don't know which one it is.
Lex Fridman (1:10:18.840)
So, these kind of other signals,
Rohit Prasad (1:10:22.480)
now there we had great assets from Amazon in terms of...
Lex Fridman (1:10:27.040)
UX, like what is it, what kind of...
Lex Fridman (1:10:29.560)
Yeah, how do you solve that problem?
Lex Fridman (1:10:31.200)
In terms of what we think of it
Lex Fridman (1:10:32.280)
as an entity resolution problem, right?
Lex Fridman (1:10:34.000)
So, because which one is it, right?
Rohit Prasad (1:10:36.200)
I mean, even if you figured out the stones as an entity,
Lex Fridman (1:10:40.160)
you have to resolve it to whether it's the stones
Rohit Prasad (1:10:42.200)
or the Stone Temple Pilots or some other stones.
Lex Fridman (1:10:44.840)
Maybe I misunderstood, is the resolution
Rohit Prasad (1:10:47.080)
the job of the algorithm or is the job of UX
Lex Fridman (1:10:50.520)
communicating with the human to help the resolution?
Lex Fridman (1:10:52.320)
Well, there is both, right?
Lex Fridman (1:10:54.240)
It is, you want 90% or high 90s to be done
Lex Fridman (1:10:58.760)
without any further questioning or UX, right?
Lex Fridman (1:11:01.200)
So, but it's absolutely okay, just like as humans,
Rohit Prasad (1:11:05.560)
we ask the question, I didn't understand you, Lex.
Lex Fridman (1:11:09.000)
It's fine for Alexa to occasionally say,
Lex Fridman (1:11:10.640)
I did not understand you, right?
Lex Fridman (1:11:12.080)
And that's an important way to learn.
Lex Fridman (1:11:14.640)
And I'll talk about where we have come
Lex Fridman (1:11:16.240)
with more self learning with these kind of feedback signals.
Lex Fridman (1:11:20.080)
But in those days, just solving the ability
Lex Fridman (1:11:23.240)
of understanding the intent and resolving to an action
Rohit Prasad (1:11:26.480)
where action could be play a particular artist
Lex Fridman (1:11:28.760)
or a particular song was super hard.
Lex Fridman (1:11:31.960)
Again, the bar was high as we were talking about, right?
Lex Fridman (1:11:35.400)
So, while we launched it in sort of 13 big domains,
Rohit Prasad (1:11:40.240)
I would say in terms of,
Lex Fridman (1:11:42.360)
we think of it as 13, the big skills we had,
Rohit Prasad (1:11:44.760)
like music is a massive one when we launched it.
Lex Fridman (1:11:47.720)
And now we have 90,000 plus skills on Alexa.
Lex Fridman (1:11:51.480)
So, what are the big skills?
Lex Fridman (1:11:52.640)
Can you just go over them?
Rohit Prasad (1:11:53.480)
Because the only thing I use it for
Lex Fridman (1:11:55.480)
is music, weather and shopping.
Lex Fridman (1:11:58.840)
So, we think of it as music information, right?
Lex Fridman (1:12:02.520)
So, weather is a part of information, right?
Rohit Prasad (1:12:05.360)
So, when we launched, we didn't have smart home,
Lex Fridman (1:12:08.000)
but within, by smart home I mean,
Rohit Prasad (1:12:10.360)
you connect your smart devices,
Lex Fridman (1:12:12.040)
you control them with voice.
Rohit Prasad (1:12:13.080)
If you haven't done it, it's worth,
Lex Fridman (1:12:15.000)
it will change your life.
Rohit Prasad (1:12:15.840)
Like turning on the lights and so on.
Lex Fridman (1:12:16.680)
Turning on your light to anything that's connected
Lex Fridman (1:12:20.200)
and has a, it's just that.
Lex Fridman (1:12:21.480)
What's your favorite smart device for you?
Rohit Prasad (1:12:23.160)
My light.
Lex Fridman (1:12:24.000)
Light.
Lex Fridman (1:12:24.840)
And now you have the smart plug with,
Lex Fridman (1:12:26.320)
and you don't, we also have this echo plug, which is.
Rohit Prasad (1:12:29.880)
Oh yeah, you can plug in anything.
Lex Fridman (1:12:30.720)
You can plug in anything
Lex Fridman (1:12:31.560)
and now you can turn that one on and off.
Lex Fridman (1:12:33.560)
I use this conversation motivation to get one.
Rohit Prasad (1:12:35.680)
Garage door, you can check your status of the garage door
Lex Fridman (1:12:39.560)
and things like, and we have gone,
Rohit Prasad (1:12:41.200)
make Alexa more and more proactive,
Lex Fridman (1:12:43.200)
where it even has hunches now,
Rohit Prasad (1:12:45.120)
that, oh, looks, hunches, like you left your light on.
Lex Fridman (1:12:50.520)
Let's say you've gone to your bed
Lex Fridman (1:12:51.640)
and you left the garage light on.
Lex Fridman (1:12:52.880)
So it will help you out in these settings, right?
Rohit Prasad (1:12:56.600)
That's smart devices, information, smart devices.
Lex Fridman (1:13:00.160)
You said music.
Rohit Prasad (1:13:01.120)
Yeah, so I don't remember everything we had,
Lex Fridman (1:13:02.960)
but alarms, timers were the big ones.
Rohit Prasad (1:13:05.040)
Like that was, you know,
Lex Fridman (1:13:06.680)
the timers were very popular right away.
Rohit Prasad (1:13:09.520)
Music also, like you could play song, artist, album,
Lex Fridman (1:13:13.440)
everything, and so that was like a clear win
Rohit Prasad (1:13:17.000)
in terms of the customer experience.
Lex Fridman (1:13:19.440)
So that's, again, this is language understanding.
Lex Fridman (1:13:22.760)
Now things have evolved, right?
Lex Fridman (1:13:24.080)
So where we want Alexa definitely to be more accurate,
Rohit Prasad (1:13:28.360)
competent, trustworthy,
Lex Fridman (1:13:29.800)
based on how well it does these core things,
Lex Fridman (1:13:33.080)
but we have evolved in many different dimensions.
Lex Fridman (1:13:35.240)
First is what I think of are doing more conversational
Lex Fridman (1:13:38.360)
for high utility, not just for chat, right?
Lex Fridman (1:13:40.920)
And there at Remars this year, which is our AI conference,
Rohit Prasad (1:13:44.920)
we launched what is called Alexa Conversations.
Lex Fridman (1:13:48.560)
That is providing the ability for developers
Rohit Prasad (1:13:51.800)
to author multi turn experiences on Alexa
Lex Fridman (1:13:55.040)
with no code, essentially,
Rohit Prasad (1:13:57.080)
in terms of the dialogue code.
Lex Fridman (1:13:58.880)
Initially it was like, you know, all these IVR systems,
Rohit Prasad (1:14:02.600)
you have to fully author if the customer says this,
Lex Fridman (1:14:06.560)
do that, right?
Lex Fridman (1:14:07.560)
So the whole dialogue flow is hand authored.
Lex Fridman (1:14:11.440)
And with Alexa Conversations,
Rohit Prasad (1:14:13.640)
the way it is that you just provide
Lex Fridman (1:14:15.440)
a sample interaction data with your service or your API,
Rohit Prasad (1:14:18.040)
let's say your Atom tickets that provides a service
Lex Fridman (1:14:21.400)
for buying movie tickets.
Rohit Prasad (1:14:23.400)
You provide a few examples of how your customers
Lex Fridman (1:14:25.840)
will interact with your APIs.
Lex Fridman (1:14:27.840)
And then the dialogue flow is automatically constructed
Lex Fridman (1:14:29.960)
using a record neural network trained on that data.
Lex Fridman (1:14:33.360)
So that simplifies the developer experience.
Lex Fridman (1:14:35.920)
We just launched our preview for the developers
Rohit Prasad (1:14:38.440)
to try this capability out.
Lex Fridman (1:14:40.600)
And then the second part of it,
Rohit Prasad (1:14:42.120)
which shows even increased utility for customers
Lex Fridman (1:14:45.680)
is you and I, when we interact with Alexa or any customer,
Rohit Prasad (1:14:50.920)
as I'm coming back to our initial part of the conversation,
Lex Fridman (1:14:53.160)
the goal is often unclear or unknown to the AI.
Lex Fridman (1:14:58.960)
If I say, Alexa, what movies are playing nearby?
Lex Fridman (1:15:02.680)
Am I trying to just buy movie tickets?
Rohit Prasad (1:15:07.080)
Am I actually even,
Lex Fridman (1:15:09.120)
do you think I'm looking for just movies for curiosity,
Lex Fridman (1:15:12.040)
whether the Avengers is still in theater or when is it?
Lex Fridman (1:15:15.120)
Maybe it's gone and maybe it will come on my missed it.
Lex Fridman (1:15:17.640)
So I may watch it on Prime, right?
Lex Fridman (1:15:20.680)
Which happened to me.
Lex Fridman (1:15:21.920)
So from that perspective now,
Lex Fridman (1:15:24.680)
you're looking into what is my goal?
Lex Fridman (1:15:27.680)
And let's say I now complete the movie ticket purchase.
Lex Fridman (1:15:31.480)
Maybe I would like to get dinner nearby.
Lex Fridman (1:15:35.760)
So what is really the goal here?
Lex Fridman (1:15:38.680)
Is it night out or is it movies?
Lex Fridman (1:15:41.920)
As in just go watch a movie?
Lex Fridman (1:15:44.040)
The answer is, we don't know.
Lex Fridman (1:15:46.240)
So can Alexa now figuratively have the intelligence
Lex Fridman (1:15:50.720)
that I think this meta goal is really night out
Rohit Prasad (1:15:53.760)
or at least say to the customer
Lex Fridman (1:15:55.800)
when you've completed the purchase of movie tickets
Rohit Prasad (1:15:58.200)
from Atom tickets or Fandango,
Lex Fridman (1:16:00.320)
or pick your anyone.
Rohit Prasad (1:16:01.840)
Then the next thing is,
Lex Fridman (1:16:02.880)
do you want to get an Uber to the theater, right?
Lex Fridman (1:16:09.360)
Or do you want to book a restaurant next to it?
Lex Fridman (1:16:12.880)
And then not ask the same information over and over again,
Lex Fridman (1:16:17.560)
what time, how many people in your party, right?
Lex Fridman (1:16:22.560)
So this is where you shift the cognitive burden
Rohit Prasad (1:16:26.560)
from the customer to the AI.
Lex Fridman (1:16:29.000)
Where it's thinking of what is your,
Rohit Prasad (1:16:32.120)
it anticipates your goal
Lex Fridman (1:16:34.200)
and takes the next best action to complete it.
Rohit Prasad (1:16:37.480)
Now that's the machine learning problem.
Lex Fridman (1:16:40.760)
But essentially the way we solve this first instance,
Lex Fridman (1:16:43.760)
and we have a long way to go to make it scale
Lex Fridman (1:16:46.800)
to everything possible in the world.
Lex Fridman (1:16:48.720)
But at least for this situation,
Lex Fridman (1:16:50.160)
it is from at every instance,
Rohit Prasad (1:16:53.000)
Alexa is making the determination,
Lex Fridman (1:16:54.600)
whether it should stick with the experience
Rohit Prasad (1:16:56.240)
with Atom tickets or not.
Lex Fridman (1:16:58.600)
Or offer you based on what you say,
Rohit Prasad (1:17:03.800)
whether either you have completed the interaction,
Lex Fridman (1:17:06.280)
or you said, no, get me an Uber now.
Lex Fridman (1:17:07.760)
So it will shift context into another experience or skill
Lex Fridman (1:17:12.080)
or another service.
Lex Fridman (1:17:12.920)
So that's a dynamic decision making.
Lex Fridman (1:17:15.360)
That's making Alexa, you can say more conversational
Rohit Prasad (1:17:18.160)
for the benefit of the customer,
Lex Fridman (1:17:20.200)
rather than simply complete transactions,
Rohit Prasad (1:17:22.520)
which are well thought through.
Lex Fridman (1:17:24.360)
You as a customer has fully specified
Lex Fridman (1:17:27.840)
what you want to be accomplished.
Lex Fridman (1:17:29.680)
It's accomplishing that.
Lex Fridman (1:17:30.840)
So it's kind of as we do this with pedestrians,
Lex Fridman (1:17:34.080)
like intent modeling is predicting
Lex Fridman (1:17:36.840)
what your possible goals are and what's the most likely goal
Lex Fridman (1:17:40.040)
and switching that depending on the things you say.
Lex Fridman (1:17:42.440)
So my question is there,
Lex Fridman (1:17:44.440)
it seems maybe it's a dumb question,
Lex Fridman (1:17:46.520)
but it would help a lot if Alexa remembered me,
Lex Fridman (1:17:51.400)
what I said previously.
Rohit Prasad (1:17:53.040)
Right.
Lex Fridman (1:17:53.880)
Is it trying to use some memories for the customer?
Rohit Prasad (1:17:58.360)
Yeah, it is using a lot of memory within that.
Lex Fridman (1:18:00.680)
So right now, not so much in terms of,
Lex Fridman (1:18:02.560)
okay, which restaurant do you prefer, right?
Lex Fridman (1:18:05.280)
That is a more longterm memory,
Lex Fridman (1:18:06.680)
but within the short term memory, within the session,
Lex Fridman (1:18:09.720)
it is remembering how many people did you,
Lex Fridman (1:18:11.720)
so if you said buy four tickets,
Lex Fridman (1:18:13.720)
now it has made an implicit assumption
Rohit Prasad (1:18:15.560)
that you were gonna have,
Lex Fridman (1:18:18.200)
you need at least four seats at a restaurant, right?
Lex Fridman (1:18:21.640)
So these are the kind of context it's preserving
Lex Fridman (1:18:24.200)
between these skills, but within that session.
Lex Fridman (1:18:26.720)
But you're asking the right question
Lex Fridman (1:18:28.000)
in terms of for it to be more and more useful,
Rohit Prasad (1:18:32.040)
it has to have more longterm memory
Lex Fridman (1:18:33.680)
and that's also an open question
Lex Fridman (1:18:35.120)
and again, these are still early days.
Lex Fridman (1:18:37.400)
So for me, I mean, everybody's different,
Lex Fridman (1:18:40.240)
but yeah, I'm definitely not representative
Lex Fridman (1:18:43.920)
of the general population in the sense
Rohit Prasad (1:18:45.240)
that I do the same thing every day.
Lex Fridman (1:18:47.800)
Like I eat the same,
Rohit Prasad (1:18:48.640)
I do everything the same, the same thing,
Lex Fridman (1:18:51.760)
wear the same thing clearly, this or the black shirt.
Lex Fridman (1:18:55.360)
So it's frustrating when Alexa doesn't get what I'm saying
Lex Fridman (1:18:59.000)
because I have to correct her every time
Rohit Prasad (1:19:01.920)
in the exact same way.
Lex Fridman (1:19:02.800)
This has to do with certain songs,
Rohit Prasad (1:19:05.480)
like she doesn't know certain weird songs I like
Lex Fridman (1:19:08.240)
and doesn't know, I've complained to Spotify about this,
Rohit Prasad (1:19:11.240)
talked to the RD, head of RD at Spotify,
Lex Fridman (1:19:13.840)
it's their way to heaven.
Rohit Prasad (1:19:15.040)
I have to correct it every time.
Lex Fridman (1:19:16.280)
It doesn't play Led Zeppelin correctly.
Rohit Prasad (1:19:18.720)
It plays cover of Led's of Stairway to Heaven.
Lex Fridman (1:19:22.080)
So I'm.
Rohit Prasad (1:19:22.920)
You should figure, you should send me your,
Lex Fridman (1:19:24.920)
next time it fails, feel free to send it to me,
Rohit Prasad (1:19:27.480)
we'll take care of it.
Lex Fridman (1:19:28.400)
Okay, well.
Rohit Prasad (1:19:29.240)
Because Led Zeppelin is one of my favorite brands,
Lex Fridman (1:19:31.720)
it works for me, so I'm like shocked it doesn't work for you.
Rohit Prasad (1:19:34.120)
This is an official bug report.
Lex Fridman (1:19:35.440)
I'll put it, I'll make it public,
Rohit Prasad (1:19:37.480)
I'll make everybody retweet it.
Lex Fridman (1:19:39.000)
We're gonna fix the Stairway to Heaven problem.
Rohit Prasad (1:19:40.960)
Anyway, but the point is,
Lex Fridman (1:19:43.200)
you know, I'm pretty boring and do the same things,
Lex Fridman (1:19:45.120)
but I'm sure most people do the same set of things.
Lex Fridman (1:19:48.320)
Do you see Alexa sort of utilizing that in the future
Lex Fridman (1:19:51.360)
for improving the experience?
Lex Fridman (1:19:52.760)
Yes, and not only utilizing,
Rohit Prasad (1:19:54.680)
it's already doing some of it.
Lex Fridman (1:19:56.200)
We call it, where Alexa is becoming more self learning.
Rohit Prasad (1:19:59.520)
So, Alexa is now auto correcting millions and millions
Lex Fridman (1:20:04.360)
of utterances in the US
Rohit Prasad (1:20:06.360)
without any human supervision involved.
Lex Fridman (1:20:08.720)
The way it does it is,
Rohit Prasad (1:20:10.840)
let's take an example of a particular song
Lex Fridman (1:20:13.320)
didn't work for you.
Lex Fridman (1:20:14.720)
What do you do next?
Lex Fridman (1:20:15.680)
You either it played the wrong song
Lex Fridman (1:20:17.840)
and you said, Alexa, no, that's not the song I want.
Lex Fridman (1:20:20.720)
Or you say, Alexa play that, you try it again.
Lex Fridman (1:20:25.160)
And that is a signal to Alexa
Lex Fridman (1:20:27.440)
that she may have done something wrong.
Lex Fridman (1:20:30.080)
And from that perspective,
Lex Fridman (1:20:31.840)
we can learn if there's that failure pattern
Rohit Prasad (1:20:35.200)
or that action of song A was played
Lex Fridman (1:20:38.480)
when song B was requested.
Lex Fridman (1:20:41.000)
And it's very common with station names
Lex Fridman (1:20:43.040)
because play NPR, you can have N be confused as an M.
Lex Fridman (1:20:47.160)
And then you, for a certain accent like mine,
Lex Fridman (1:20:51.840)
people confuse my N and M all the time.
Lex Fridman (1:20:54.720)
And because I have a Indian accent,
Lex Fridman (1:20:57.640)
they're confusable to humans.
Rohit Prasad (1:20:59.600)
It is for Alexa too.
Lex Fridman (1:21:01.600)
And in that part, but it starts auto correcting
Lex Fridman (1:21:05.080)
and we collect, we correct a lot of these automatically
Lex Fridman (1:21:09.680)
without a human looking at the failures.
Lex Fridman (1:21:12.680)
So one of the things that's for me missing in Alexa,
Lex Fridman (1:21:17.360)
I don't know if I'm a representative customer,
Lex Fridman (1:21:19.720)
but every time I correct it,
Lex Fridman (1:21:22.920)
it would be nice to know that that made a difference.
Rohit Prasad (1:21:26.120)
Yes.
Lex Fridman (1:21:26.960)
You know what I mean?
Rohit Prasad (1:21:27.800)
Like the sort of like, I heard you like a sort of.
Lex Fridman (1:21:31.880)
Some acknowledgement of that.
Rohit Prasad (1:21:33.840)
We work a lot with Tesla, we study autopilot and so on.
Lex Fridman (1:21:37.440)
And a large amount of the customers
Rohit Prasad (1:21:39.240)
that use Tesla autopilot,
Lex Fridman (1:21:40.720)
they feel like they're always teaching the system.
Rohit Prasad (1:21:43.000)
They're almost excited
Lex Fridman (1:21:43.840)
by the possibility that they're teaching.
Rohit Prasad (1:21:45.080)
I don't know if Alexa customers generally think of it
Lex Fridman (1:21:48.440)
as they're teaching to improve the system.
Lex Fridman (1:21:51.160)
And that's a really powerful thing.
Lex Fridman (1:21:52.680)
Again, I would say it's a spectrum.
Rohit Prasad (1:21:55.200)
Some customers do think that way
Lex Fridman (1:21:57.320)
and some would be annoyed by Alexa acknowledging that.
Lex Fridman (1:22:02.320)
So there's, again, no one,
Lex Fridman (1:22:04.360)
while there are certain patterns,
Rohit Prasad (1:22:05.760)
not everyone is the same in this way.
Lex Fridman (1:22:08.280)
But we believe that, again, customers helping Alexa
Rohit Prasad (1:22:13.680)
is a tenet for us in terms of improving it.
Lex Fridman (1:22:15.720)
And some more self learning is by, again,
Lex Fridman (1:22:18.280)
this is like fully unsupervised, right?
Lex Fridman (1:22:20.120)
There is no human in the loop and no labeling happening.
Lex Fridman (1:22:23.600)
And based on your actions as a customer,
Lex Fridman (1:22:27.120)
Alexa becomes smarter.
Rohit Prasad (1:22:29.080)
Again, it's early days,
Lex Fridman (1:22:31.160)
but I think this whole area of teachable AI
Rohit Prasad (1:22:35.840)
is gonna get bigger and bigger in the whole space,
Lex Fridman (1:22:38.680)
especially in the AI assistant space.
Lex Fridman (1:22:40.760)
So that's the second part
Lex Fridman (1:22:41.920)
where I mentioned more conversational.
Rohit Prasad (1:22:44.800)
This is more self learning.
Lex Fridman (1:22:46.520)
The third is more natural.
Lex Fridman (1:22:48.320)
And the way I think of more natural
Lex Fridman (1:22:50.240)
is we talked about how Alexa sounds.
Lex Fridman (1:22:53.240)
And we have done a lot of advances in our text to speech
Lex Fridman (1:22:58.080)
by using, again, neural network technology
Rohit Prasad (1:23:00.480)
for it to sound very humanlike.
Lex Fridman (1:23:03.520)
From the individual texture of the sound to the timing,
Rohit Prasad (1:23:07.520)
the tonality, the tone, everything, the whole thing.
Lex Fridman (1:23:09.240)
I would think in terms of,
Rohit Prasad (1:23:11.000)
there's a lot of controls in each of the places
Lex Fridman (1:23:13.360)
for how, I mean, the speed of the voice,
Rohit Prasad (1:23:16.640)
the prosthetic patterns,
Lex Fridman (1:23:19.520)
the actual smoothness of how it sounds,
Rohit Prasad (1:23:23.360)
all of those are factored
Lex Fridman (1:23:24.360)
and we do a ton of listening tests to make sure.
Lex Fridman (1:23:27.120)
But naturalness, how it sounds should be very natural.
Lex Fridman (1:23:30.720)
How it understands requests is also very important.
Lex Fridman (1:23:33.920)
And in terms of, we have 95,000 skills.
Lex Fridman (1:23:37.120)
And if we have, imagine that in many of these skills,
Rohit Prasad (1:23:41.440)
you have to remember the skill name
Lex Fridman (1:23:43.440)
and say, Alexa, ask the tide skill to tell me X.
Rohit Prasad (1:23:51.120)
Now, if you have to remember the skill name,
Lex Fridman (1:23:52.960)
that means the discovery and the interaction is unnatural.
Lex Fridman (1:23:56.640)
And we are trying to solve that
Lex Fridman (1:23:58.120)
by what we think of as, again,
Rohit Prasad (1:24:03.960)
you don't have to have the app metaphor here.
Lex Fridman (1:24:05.680)
These are not individual apps, right?
Rohit Prasad (1:24:07.400)
Even though they're,
Lex Fridman (1:24:08.360)
so you're not sort of opening one at a time and interacting.
Lex Fridman (1:24:11.400)
So it should be seamless because it's voice.
Lex Fridman (1:24:14.000)
And when it's voice,
Rohit Prasad (1:24:15.160)
you have to be able to understand these requests
Lex Fridman (1:24:17.560)
independent of the specificity, like a skill name.
Lex Fridman (1:24:20.600)
And to do that,
Lex Fridman (1:24:21.640)
what we have done is again,
Rohit Prasad (1:24:22.840)
built a deep learning based capability
Lex Fridman (1:24:24.440)
where we shortlist a bunch of skills
Rohit Prasad (1:24:27.040)
when you say, Alexa, get me a car.
Lex Fridman (1:24:28.880)
And then we figure it out, okay,
Rohit Prasad (1:24:30.080)
it's meant for an Uber skill versus a Lyft
Lex Fridman (1:24:33.320)
or based on your preferences.
Lex Fridman (1:24:34.880)
And then you can rank the responses from the skill
Lex Fridman (1:24:38.320)
and then choose the best response for the customer.
Lex Fridman (1:24:41.280)
So that's on the more natural,
Lex Fridman (1:24:43.240)
other examples of more natural is like,
Rohit Prasad (1:24:46.360)
we were talking about lists, for instance,
Lex Fridman (1:24:49.120)
and you don't wanna say, Alexa, add milk,
Rohit Prasad (1:24:51.720)
Alexa, add eggs, Alexa, add cookies.
Lex Fridman (1:24:55.160)
No, Alexa, add cookies, milk, and eggs
Lex Fridman (1:24:57.280)
and that in one shot, right?
Lex Fridman (1:24:59.240)
So that works, that helps with the naturalness.
Rohit Prasad (1:25:01.760)
We talked about memory, like if you said,
Lex Fridman (1:25:05.400)
you can say, Alexa, remember I have to go to mom's house,
Rohit Prasad (1:25:09.040)
or you may have entered a calendar event
Lex Fridman (1:25:11.160)
through your calendar that's linked to Alexa.
Rohit Prasad (1:25:13.520)
You don't wanna remember whether it's in my calendar
Lex Fridman (1:25:15.800)
or did I tell you to remember something
Lex Fridman (1:25:18.360)
or some other reminder, right?
Lex Fridman (1:25:20.960)
So you have to now, independent of how customers
Rohit Prasad (1:25:25.320)
create these events, it should just say,
Lex Fridman (1:25:28.120)
Alexa, when do I have to go to mom's house?
Lex Fridman (1:25:29.840)
And it tells you when you have to go to mom's house.
Lex Fridman (1:25:32.320)
Now that's a fascinating problem.
Lex Fridman (1:25:33.720)
Who's that problem on?
Lex Fridman (1:25:35.280)
So there's people who create skills.
Rohit Prasad (1:25:38.520)
Who's tasked with integrating all of that knowledge together
Lex Fridman (1:25:42.840)
so the skills become seamless?
Rohit Prasad (1:25:44.640)
Is it the creators of the skills
Lex Fridman (1:25:46.840)
or is it an infrastructure that Alexa provides problem?
Rohit Prasad (1:25:51.280)
It's both.
Lex Fridman (1:25:52.120)
I think the large problem in terms of making sure
Rohit Prasad (1:25:54.960)
your skill quality is high,
Lex Fridman (1:25:58.560)
that has to be done by our tools,
Rohit Prasad (1:26:01.240)
because it's just, so these skills,
Lex Fridman (1:26:03.160)
just to put the context,
Rohit Prasad (1:26:04.720)
they are built through Alexa Skills Kit,
Lex Fridman (1:26:06.360)
which is a self serve way of building
Rohit Prasad (1:26:09.160)
an experience on Alexa.
Lex Fridman (1:26:11.320)
This is like any developer in the world
Rohit Prasad (1:26:13.000)
could go to Alexa Skills Kit
Lex Fridman (1:26:14.880)
and build an experience on Alexa.
Rohit Prasad (1:26:16.840)
Like if you're a Domino's, you can build a Domino's Skills.
Lex Fridman (1:26:20.160)
For instance, that does pizza ordering.
Rohit Prasad (1:26:22.560)
When you have authored that,
Lex Fridman (1:26:25.320)
you do want to now,
Rohit Prasad (1:26:28.280)
if people say, Alexa, open Domino's
Lex Fridman (1:26:30.120)
or Alexa, ask Domino's to get a particular type of pizza,
Rohit Prasad (1:26:35.360)
that will work, but the discovery is hard.
Lex Fridman (1:26:37.800)
You can't just say, Alexa, get me a pizza.
Lex Fridman (1:26:39.360)
And then Alexa figures out what to do.
Lex Fridman (1:26:42.440)
That latter part is definitely our responsibility
Rohit Prasad (1:26:45.000)
in terms of when the request is not fully specific,
Lex Fridman (1:26:48.960)
how do you figure out what's the best skill
Lex Fridman (1:26:51.560)
or a service that can fulfill the customer's request?
Lex Fridman (1:26:56.120)
And it can keep evolving.
Rohit Prasad (1:26:57.280)
Imagine going to the situation I said,
Lex Fridman (1:26:59.280)
which was the night out planning,
Rohit Prasad (1:27:00.360)
that the goal could be more than that individual request
Lex Fridman (1:27:03.520)
that came up.
Rohit Prasad (1:27:05.600)
A pizza ordering could mean a night in,
Lex Fridman (1:27:08.600)
where you're having an event with your kids
Rohit Prasad (1:27:10.520)
in their house, and you're, so this is,
Lex Fridman (1:27:12.920)
welcome to the world of conversational AI.
Rohit Prasad (1:27:16.720)
This is super exciting because it's not
Lex Fridman (1:27:18.920)
the academic problem of NLP,
Rohit Prasad (1:27:20.760)
of natural language processing, understanding, dialogue.
Lex Fridman (1:27:23.080)
This is like real world.
Lex Fridman (1:27:24.640)
And the stakes are high in the sense
Lex Fridman (1:27:27.120)
that customers get frustrated quickly,
Rohit Prasad (1:27:30.000)
people get frustrated quickly.
Lex Fridman (1:27:31.800)
So you have to get it right,
Rohit Prasad (1:27:33.120)
you have to get that interaction right.
Lex Fridman (1:27:35.280)
So it's, I love it.
Lex Fridman (1:27:36.880)
But so from that perspective,
Lex Fridman (1:27:39.200)
what are the challenges today?
Lex Fridman (1:27:41.920)
What are the problems that really need to be solved
Lex Fridman (1:27:45.040)
in the next few years?
Lex Fridman (1:27:45.880)
What's the focus?
Lex Fridman (1:27:46.840)
First and foremost, as I mentioned,
Rohit Prasad (1:27:48.720)
that get the basics right is still true.
Lex Fridman (1:27:53.080)
Basically, even the one shot requests,
Rohit Prasad (1:27:57.000)
which we think of as transactional requests,
Lex Fridman (1:27:58.840)
needs to work magically, no question about that.
Rohit Prasad (1:28:01.680)
If it doesn't turn your light on and off,
Lex Fridman (1:28:03.600)
you'll be super frustrated.
Rohit Prasad (1:28:05.200)
Even if I can complete the night out for you
Lex Fridman (1:28:07.080)
and not do that, that is unacceptable as a customer, right?
Lex Fridman (1:28:10.720)
So that you have to get the foundational understanding
Lex Fridman (1:28:14.120)
going very well.
Rohit Prasad (1:28:15.440)
The second aspect when I said more conversational
Lex Fridman (1:28:17.760)
is as you imagine is more about reasoning.
Rohit Prasad (1:28:20.120)
It is really about figuring out what the latent goal is
Lex Fridman (1:28:24.360)
of the customer based on what I have the information now
Lex Fridman (1:28:28.520)
and the history, what's the next best thing to do.
Lex Fridman (1:28:31.360)
So that's a complete reasoning and decision making problem.
Rohit Prasad (1:28:35.400)
Just like your self driving car,
Lex Fridman (1:28:37.040)
but the goal is still more finite.
Rohit Prasad (1:28:38.680)
Here it evolves, your environment is super hard
Lex Fridman (1:28:41.960)
and self driving and the cost of a mistake is huge here,
Lex Fridman (1:28:46.880)
but there are certain similarities.
Lex Fridman (1:28:48.520)
But if you think about how many decisions Alexa is making
Rohit Prasad (1:28:52.640)
or evaluating at any given time,
Lex Fridman (1:28:54.280)
it's a huge hypothesis space.
Lex Fridman (1:28:56.480)
And we're only talked about so far
Lex Fridman (1:28:59.760)
about what I think of reactive decision
Rohit Prasad (1:29:02.080)
in terms of you asked for something
Lex Fridman (1:29:03.640)
and Alexa is reacting to it.
Rohit Prasad (1:29:05.920)
If you bring the proactive part,
Lex Fridman (1:29:07.760)
which is Alexa having hunches.
Lex Fridman (1:29:10.040)
So any given instance then it's really a decision
Lex Fridman (1:29:14.440)
at any given point based on the information.
Rohit Prasad (1:29:17.240)
Alexa has to determine what's the best thing it needs to do.
Lex Fridman (1:29:20.120)
So these are the ultimate AI problem
Rohit Prasad (1:29:22.520)
about decisions based on the information you have.
Lex Fridman (1:29:25.080)
Do you think, just from my perspective,
Rohit Prasad (1:29:27.880)
I work a lot with sensing of the human face.
Lex Fridman (1:29:31.120)
Do you think they'll, and we touched this topic
Rohit Prasad (1:29:33.680)
a little bit earlier, but do you think it'll be a day soon
Lex Fridman (1:29:36.560)
when Alexa can also look at you to help improve the quality
Rohit Prasad (1:29:41.360)
of the hunch it has, or at least detect frustration
Lex Fridman (1:29:46.360)
or detect, improve the quality of its perception
Lex Fridman (1:29:51.600)
of what you're trying to do?
Lex Fridman (1:29:54.360)
I mean, let me again bring back to what it already does.
Rohit Prasad (1:29:57.160)
We talked about how based on you barge in over Alexa,
Lex Fridman (1:30:01.800)
clearly it's a very high probability
Rohit Prasad (1:30:04.960)
it must have done something wrong.
Lex Fridman (1:30:06.560)
That's why you barged in.
Rohit Prasad (1:30:08.520)
The next extension of whether frustration is a signal or not,
Lex Fridman (1:30:13.240)
of course, is a natural thought
Rohit Prasad (1:30:15.320)
in terms of how that should be in a signal to it.
Lex Fridman (1:30:18.200)
You can get that from voice.
Rohit Prasad (1:30:19.520)
You can get from voice, but it's very hard.
Lex Fridman (1:30:21.280)
Like, I mean, frustration as a signal historically,
Rohit Prasad (1:30:25.920)
if you think about emotions of different kinds,
Lex Fridman (1:30:29.640)
there's a whole field of affective computing,
Rohit Prasad (1:30:31.440)
something that MIT has also done a lot of research in,
Lex Fridman (1:30:34.520)
is super hard.
Lex Fridman (1:30:35.600)
And you are now talking about a far field device,
Lex Fridman (1:30:39.040)
as in you're talking to a distance noisy environment.
Lex Fridman (1:30:41.920)
And in that environment,
Lex Fridman (1:30:44.080)
it needs to have a good sense for your emotions.
Rohit Prasad (1:30:47.520)
This is a very, very hard problem.
Lex Fridman (1:30:49.440)
Very hard problem, but you haven't shied away
Rohit Prasad (1:30:50.960)
from hard problems.
Lex Fridman (1:30:51.800)
So, Deep Learning has been at the core
Rohit Prasad (1:30:55.240)
of a lot of this technology.
Lex Fridman (1:30:57.360)
Are you optimistic
Rohit Prasad (1:30:58.200)
about the current Deep Learning approaches
Lex Fridman (1:30:59.680)
to solving the hardest aspects of what we're talking about?
Rohit Prasad (1:31:03.200)
Or do you think there will come a time
Lex Fridman (1:31:05.320)
where new ideas need to further,
Rohit Prasad (1:31:07.960)
if we look at reasoning,
Lex Fridman (1:31:09.320)
so OpenAI, DeepMind,
Rohit Prasad (1:31:10.640)
a lot of folks are now starting to work in reasoning,
Lex Fridman (1:31:13.840)
trying to see how we can make neural networks reason.
Lex Fridman (1:31:16.560)
Do you see that new approaches need to be invented
Lex Fridman (1:31:20.480)
to take the next big leap?
Rohit Prasad (1:31:23.280)
Absolutely, I think there has to be a lot more investment.
Lex Fridman (1:31:27.160)
And I think in many different ways,
Lex Fridman (1:31:29.360)
and there are these, I would say,
Lex Fridman (1:31:31.160)
nuggets of research forming in a good way,
Rohit Prasad (1:31:33.520)
like learning with less data
Lex Fridman (1:31:36.040)
or like zero short learning, one short learning.
Lex Fridman (1:31:39.640)
And the active learning stuff you've talked about
Lex Fridman (1:31:41.360)
is incredible stuff.
Rohit Prasad (1:31:43.200)
So, transfer learning is also super critical,
Lex Fridman (1:31:45.640)
especially when you're thinking about applying knowledge
Rohit Prasad (1:31:48.560)
from one task to another,
Lex Fridman (1:31:49.840)
or one language to another, right?
Rohit Prasad (1:31:52.000)
It's really ripe.
Lex Fridman (1:31:52.960)
So, these are great pieces.
Rohit Prasad (1:31:55.280)
Deep learning has been useful too.
Lex Fridman (1:31:56.760)
And now we are sort of marrying deep learning
Rohit Prasad (1:31:58.840)
with transfer learning and active learning.
Lex Fridman (1:32:02.440)
Of course, that's more straightforward
Rohit Prasad (1:32:04.480)
in terms of applying deep learning
Lex Fridman (1:32:05.840)
and an active learning setup.
Lex Fridman (1:32:06.960)
But I do think in terms of now looking
Lex Fridman (1:32:12.120)
into more reasoning based approaches
Rohit Prasad (1:32:14.200)
is going to be key for our next wave of the technology.
Lex Fridman (1:32:19.440)
But there is a good news.
Rohit Prasad (1:32:20.840)
The good news is that I think for keeping on
Lex Fridman (1:32:23.280)
to delight customers, that a lot of it
Rohit Prasad (1:32:25.200)
can be done by prediction tasks.
Lex Fridman (1:32:27.880)
So, we haven't exhausted that.
Rohit Prasad (1:32:30.640)
So, we don't need to give up
Lex Fridman (1:32:34.440)
on the deep learning approaches for that.
Rohit Prasad (1:32:37.280)
So, that's just I wanted to sort of point that out.
Lex Fridman (1:32:39.520)
Creating a rich, fulfilling, amazing experience
Rohit Prasad (1:32:42.560)
that makes Amazon a lot of money
Lex Fridman (1:32:44.200)
and a lot of everybody a lot of money
Rohit Prasad (1:32:46.360)
because it does awesome things, deep learning is enough.
Lex Fridman (1:32:49.840)
The point.
Rohit Prasad (1:32:51.080)
I don't think, I wouldn't say deep learning is enough.
Lex Fridman (1:32:54.160)
I think for the purposes of Alexa
Rohit Prasad (1:32:56.680)
accomplished the task for customers.
Lex Fridman (1:32:58.400)
I'm saying there are still a lot of things we can do
Rohit Prasad (1:33:02.160)
with prediction based approaches that do not reason.
Lex Fridman (1:33:05.280)
I'm not saying that and we haven't exhausted those.
Lex Fridman (1:33:08.600)
But for the kind of high utility experiences
Lex Fridman (1:33:12.440)
that I'm personally passionate about
Rohit Prasad (1:33:14.240)
of what Alexa needs to do, reasoning has to be solved
Lex Fridman (1:33:18.760)
to the same extent as you can think
Rohit Prasad (1:33:21.000)
of natural language understanding and speech recognition
Lex Fridman (1:33:24.720)
to the extent of understanding intents
Rohit Prasad (1:33:27.600)
has been how accurate it has become.
Lex Fridman (1:33:30.120)
But reasoning, we have very, very early days.
Rohit Prasad (1:33:32.760)
Let me ask it another way.
Lex Fridman (1:33:34.000)
How hard of a problem do you think that is?
Rohit Prasad (1:33:36.760)
Hardest of them.
Lex Fridman (1:33:39.160)
I would say hardest of them because again,
Rohit Prasad (1:33:42.560)
the hypothesis space is really, really large.
Lex Fridman (1:33:47.560)
And when you go back in time, like you were saying,
Rohit Prasad (1:33:50.000)
I wanna, I want Alexa to remember more things
Lex Fridman (1:33:53.000)
that once you go beyond a session of interaction,
Rohit Prasad (1:33:56.280)
which is by session, I mean a time span,
Lex Fridman (1:33:59.200)
which is today to versus remembering which restaurant I like.
Lex Fridman (1:34:03.120)
And then when I'm planning a night out to say,
Lex Fridman (1:34:05.440)
do you wanna go to the same restaurant?
Rohit Prasad (1:34:07.480)
Now you're up the stakes big time.
Lex Fridman (1:34:09.720)
And this is where the reasoning dimension
Rohit Prasad (1:34:12.800)
also goes way, way bigger.
Lex Fridman (1:34:14.680)
So you think the space, we'll be elaborating that
Rohit Prasad (1:34:17.760)
a little bit, just philosophically speaking,
Lex Fridman (1:34:20.480)
do you think when you reason about trying to model
Lex Fridman (1:34:24.480)
what the goal of a person is in the context
Lex Fridman (1:34:28.040)
of interacting with Alexa, you think that space is huge?
Rohit Prasad (1:34:31.080)
It's huge, absolutely huge.
Lex Fridman (1:34:32.840)
Do you think, so like another sort of devil's advocate
Rohit Prasad (1:34:35.840)
would be that we human beings are really simple
Lex Fridman (1:34:38.520)
and we all want like just a small set of things.
Lex Fridman (1:34:41.360)
And so do you think it's possible?
Lex Fridman (1:34:44.720)
Cause we're not talking about
Rohit Prasad (1:34:47.000)
a fulfilling general conversation.
Lex Fridman (1:34:49.240)
Perhaps actually the Alexa prize is a little bit after that.
Rohit Prasad (1:34:53.320)
Creating a customer, like there's so many
Lex Fridman (1:34:56.080)
of the interactions, it feels like are clustered
Rohit Prasad (1:35:01.040)
in groups that are, don't require general reasoning.
Lex Fridman (1:35:06.520)
I think you're right in terms of the head
Rohit Prasad (1:35:09.320)
of the distribution of all the possible things
Lex Fridman (1:35:11.800)
customers may wanna accomplish.
Lex Fridman (1:35:13.720)
But the tail is long and it's diverse, right?
Lex Fridman (1:35:18.200)
So from that.
Rohit Prasad (1:35:19.040)
There's many, many long tails.
Lex Fridman (1:35:21.280)
So from that perspective, I think you have
Rohit Prasad (1:35:24.880)
to solve that problem otherwise,
Lex Fridman (1:35:27.640)
and everyone's very different.
Rohit Prasad (1:35:28.800)
Like, I mean, we see this already
Lex Fridman (1:35:30.440)
in terms of the skills, right?
Lex Fridman (1:35:32.320)
I mean, if you're an average surfer, which I am not, right?
Lex Fridman (1:35:36.960)
But somebody is asking Alexa about surfing conditions, right?
Lex Fridman (1:35:41.640)
And there's a skill that is there for them to get to, right?
Lex Fridman (1:35:45.480)
That tells you that the tail is massive.
Rohit Prasad (1:35:47.840)
Like in terms of like what kind of skills
Lex Fridman (1:35:50.720)
people have created, it's humongous in terms of it.
Lex Fridman (1:35:54.200)
And which means there are these diverse needs.
Lex Fridman (1:35:56.960)
And when you start looking at the combinations
Lex Fridman (1:36:00.040)
of these, right?
Lex Fridman (1:36:00.960)
Even if you had pairs of skills and 90,000 choose two,
Rohit Prasad (1:36:05.400)
it's still a big set of combinations.
Lex Fridman (1:36:07.920)
So I'm saying there's a huge to do here now.
Lex Fridman (1:36:11.720)
And I think customers are, you know,
Lex Fridman (1:36:14.760)
wonderfully frustrated with things.
Lex Fridman (1:36:18.080)
And they have to keep getting to do better things for them.
Lex Fridman (1:36:20.880)
So.
Lex Fridman (1:36:21.720)
And they're not known to be super patient.
Lex Fridman (1:36:23.920)
So you have to.
Rohit Prasad (1:36:24.760)
Do it fast.
Lex Fridman (1:36:25.600)
You have to do it fast.
Lex Fridman (1:36:26.960)
So you've mentioned the idea of a press release,
Lex Fridman (1:36:29.840)
the research and development, Amazon Alexa
Lex Fridman (1:36:33.880)
and Amazon general, you kind of think of what
Lex Fridman (1:36:35.960)
the future product will look like.
Lex Fridman (1:36:37.240)
And you kind of make it happen.
Lex Fridman (1:36:38.360)
You work backwards.
Lex Fridman (1:36:40.040)
So can you draft for me, you probably already have one,
Lex Fridman (1:36:43.920)
but can you make up one for 10, 20, 30, 40 years out
Rohit Prasad (1:36:48.880)
that you see the Alexa team putting out
Lex Fridman (1:36:52.800)
just in broad strokes, something that you dream about?
Lex Fridman (1:36:56.520)
I think let's start with the five years first, right?
Lex Fridman (1:37:00.920)
So, and I'll get to the 40 years too.
Rohit Prasad (1:37:03.600)
Cause I'm pretty sure you have a real five year one.
Lex Fridman (1:37:06.000)
That's why I didn't want to, but yeah,
Rohit Prasad (1:37:08.720)
in broad strokes, let's start with five years.
Lex Fridman (1:37:10.120)
I think the five year is where, I mean,
Rohit Prasad (1:37:11.800)
I think of in these spaces, it's hard,
Lex Fridman (1:37:14.800)
especially if you're in the thick of things
Rohit Prasad (1:37:16.160)
to think beyond the five year space,
Lex Fridman (1:37:17.960)
because a lot of things change, right?
Rohit Prasad (1:37:20.280)
I mean, if you ask me five years back,
Lex Fridman (1:37:22.200)
will Alexa will be here?
Rohit Prasad (1:37:24.200)
I wouldn't have, I think it has surpassed
Lex Fridman (1:37:26.360)
my imagination of that time, right?
Lex Fridman (1:37:29.040)
So I think from the next five years perspective,
Lex Fridman (1:37:33.160)
from a AI perspective, what we're gonna see
Rohit Prasad (1:37:37.120)
is that notion, which you said goal oriented dialogues
Lex Fridman (1:37:40.400)
and open domain like Alexa prize.
Rohit Prasad (1:37:42.400)
I think that bridge is gonna get closed.
Lex Fridman (1:37:45.200)
They won't be different.
Lex Fridman (1:37:46.400)
And I'll give you why that's the case.
Lex Fridman (1:37:48.520)
You mentioned shopping.
Lex Fridman (1:37:50.200)
How do you shop?
Lex Fridman (1:37:52.240)
Do you shop in one shot?
Rohit Prasad (1:37:55.680)
Sure, your double A batteries, paper towels.
Lex Fridman (1:37:59.400)
Yes, how long does it take for you to buy a camera?
Rohit Prasad (1:38:04.160)
You do ton of research, then you make a decision.
Lex Fridman (1:38:07.480)
So is that a goal oriented dialogue
Lex Fridman (1:38:11.440)
when somebody says, Alexa, find me a camera?
Lex Fridman (1:38:15.480)
Is it simply inquisitiveness, right?
Lex Fridman (1:38:18.640)
So even in the something that you think of it as shopping,
Lex Fridman (1:38:20.880)
which you said you yourself use a lot of,
Rohit Prasad (1:38:23.960)
if you go beyond where it's reorders
Lex Fridman (1:38:27.360)
or items where you sort of are not brand conscious
Lex Fridman (1:38:32.440)
and so forth.
Lex Fridman (1:38:33.520)
So that was just in shopping.
Rohit Prasad (1:38:35.040)
Just to comment quickly,
Lex Fridman (1:38:36.120)
I've never bought anything through Alexa
Rohit Prasad (1:38:38.040)
that I haven't bought before on Amazon on the desktop
Lex Fridman (1:38:41.160)
after I clicked in a bunch of read a bunch of reviews,
Rohit Prasad (1:38:44.000)
that kind of stuff.
Lex Fridman (1:38:44.840)
So it's repurchase.
Lex Fridman (1:38:45.800)
So now you think in,
Lex Fridman (1:38:47.480)
even for something that you felt like is a finite goal,
Rohit Prasad (1:38:51.280)
I think the space is huge because even products,
Lex Fridman (1:38:54.680)
the attributes are many,
Lex Fridman (1:38:56.640)
and you wanna look at reviews,
Lex Fridman (1:38:58.240)
some on Amazon, some outside,
Rohit Prasad (1:39:00.000)
some you wanna look at what CNET is saying
Lex Fridman (1:39:01.960)
or another consumer forum is saying
Lex Fridman (1:39:05.200)
about even a product for instance, right?
Lex Fridman (1:39:06.880)
So that's just shopping where you could argue
Rohit Prasad (1:39:11.640)
the ultimate goal is sort of known.
Lex Fridman (1:39:13.960)
And we haven't talked about Alexa,
Lex Fridman (1:39:15.680)
what's the weather in Cape Cod this weekend, right?
Lex Fridman (1:39:18.880)
So why am I asking that weather question, right?
Lex Fridman (1:39:22.480)
So I think of it as how do you complete goals
Lex Fridman (1:39:27.480)
with minimum steps for our customers, right?
Lex Fridman (1:39:30.040)
And when you think of it that way,
Lex Fridman (1:39:32.400)
the distinction between goal oriented and conversations
Rohit Prasad (1:39:35.960)
for open domain say goes away.
Lex Fridman (1:39:38.640)
I may wanna know what happened
Lex Fridman (1:39:41.680)
in the presidential debate, right?
Lex Fridman (1:39:43.520)
And is it I'm seeking just information
Lex Fridman (1:39:45.800)
or I'm looking at who's winning the debates, right?
Lex Fridman (1:39:49.560)
So these are all quite hard problems.
Lex Fridman (1:39:53.360)
So even the five year horizon problem,
Lex Fridman (1:39:55.560)
I'm like, I sure hope we'll solve these.
Lex Fridman (1:39:59.840)
And you're optimistic because that's a hard problem.
Lex Fridman (1:40:03.440)
Which part?
Rohit Prasad (1:40:04.280)
The reasoning enough to be able to help explore
Lex Fridman (1:40:09.600)
complex goals that are beyond something simplistic.
Rohit Prasad (1:40:12.400)
That feels like it could be, well, five years is a nice.
Lex Fridman (1:40:16.560)
Is a nice bar for it, right?
Rohit Prasad (1:40:18.280)
I think you will, it's a nice ambition
Lex Fridman (1:40:21.240)
and do we have press releases for that?
Rohit Prasad (1:40:23.760)
Absolutely, can I tell you what specifically
Lex Fridman (1:40:25.880)
the roadmap will be?
Lex Fridman (1:40:26.720)
No, right?
Lex Fridman (1:40:28.080)
And what, and will we solve all of it
Lex Fridman (1:40:30.760)
in the five year space?
Lex Fridman (1:40:31.760)
No, this is, we'll work on this forever actually.
Rohit Prasad (1:40:35.560)
This is the hardest of the AI problems
Lex Fridman (1:40:37.960)
and I don't see that being solved even in a 40 year horizon
Rohit Prasad (1:40:42.240)
because even if you limit to the human intelligence,
Lex Fridman (1:40:45.200)
we know we are quite far from that.
Rohit Prasad (1:40:47.640)
In fact, every aspects of our sensing to neural processing,
Lex Fridman (1:40:52.640)
to how brain stores information and how it processes it,
Lex Fridman (1:40:56.320)
we don't yet know how to represent knowledge, right?
Lex Fridman (1:40:59.000)
So we are still in those early stages.
Lex Fridman (1:41:02.920)
So I wanted to start, that's why at the five year,
Lex Fridman (1:41:06.360)
because the five year success would look like that
Rohit Prasad (1:41:09.120)
in solving these complex goals.
Lex Fridman (1:41:11.240)
And the 40 year would be where it's just natural
Rohit Prasad (1:41:14.560)
to talk to these in terms of more of these complex goals.
Lex Fridman (1:41:18.720)
Right now, we've already come to the point
Rohit Prasad (1:41:20.000)
where these transactions you mentioned
Lex Fridman (1:41:22.840)
of asking for weather or reordering something
Rohit Prasad (1:41:25.720)
or listening to your favorite tune,
Lex Fridman (1:41:28.560)
it's natural for you to ask Alexa.
Lex Fridman (1:41:30.840)
It's now unnatural to pick up your phone, right?
Lex Fridman (1:41:33.880)
And that I think is the first five year transformation.
Rohit Prasad (1:41:36.600)
The next five year transformation would be,
Lex Fridman (1:41:38.800)
okay, I can plan my weekend with Alexa
Rohit Prasad (1:41:40.960)
or I can plan my next meal with Alexa
Lex Fridman (1:41:43.640)
or my next night out with seamless effort.
Lex Fridman (1:41:47.840)
So just to pause and look back at the big picture of it all.
Lex Fridman (1:41:51.200)
It's a, you're a part of a large team
Rohit Prasad (1:41:55.560)
that's creating a system that's in the home
Lex Fridman (1:41:58.680)
that's not human, that gets to interact with human beings.
Lex Fridman (1:42:02.760)
So we human beings, we these descendants of apes
Lex Fridman (1:42:06.120)
have created an artificial intelligence system
Rohit Prasad (1:42:09.000)
that's able to have conversations.
Lex Fridman (1:42:10.960)
I mean, that to me, the two most transformative robots
Rohit Prasad (1:42:18.800)
of this century, I think will be autonomous vehicles,
Lex Fridman (1:42:23.200)
but they're a little bit transformative
Rohit Prasad (1:42:24.760)
in a more boring way.
Lex Fridman (1:42:26.360)
It's like a tool.
Rohit Prasad (1:42:28.120)
I think conversational agents in the home
Lex Fridman (1:42:32.840)
is like an experience.
Lex Fridman (1:42:34.640)
How does that make you feel?
Lex Fridman (1:42:36.120)
That you're at the center of creating that?
Lex Fridman (1:42:38.560)
Do you sit back in awe sometimes?
Lex Fridman (1:42:42.800)
What is your feeling about the whole mess of it?
Lex Fridman (1:42:47.320)
Can you even believe that we're able
Lex Fridman (1:42:49.000)
to create something like this?
Rohit Prasad (1:42:50.840)
I think it's a privilege.
Lex Fridman (1:42:52.440)
I'm so fortunate like where I ended up, right?
Lex Fridman (1:42:57.640)
And it's been a long journey.
Lex Fridman (1:43:00.800)
Like I've been in this space for a long time in Cambridge,
Rohit Prasad (1:43:03.480)
right, and it's so heartwarming to see
Lex Fridman (1:43:07.080)
the kind of adoption conversational agents are having now.
Rohit Prasad (1:43:12.440)
Five years back, it was almost like,
Lex Fridman (1:43:14.480)
should I move out of this because we are unable
Rohit Prasad (1:43:17.120)
to find this killer application that customers would love
Lex Fridman (1:43:21.360)
that would not simply be a good to have thing
Rohit Prasad (1:43:24.440)
in research labs.
Lex Fridman (1:43:26.080)
And it's so fulfilling to see it make a difference
Rohit Prasad (1:43:29.160)
to millions and billions of people worldwide.
Lex Fridman (1:43:32.240)
The good thing is that it's still very early.
Lex Fridman (1:43:34.400)
So I have another 20 years of job security
Lex Fridman (1:43:37.360)
doing what I love.
Rohit Prasad (1:43:38.200)
Like, so I think from that perspective,
Lex Fridman (1:43:42.000)
I tell every researcher that joins
Rohit Prasad (1:43:44.280)
or every member of my team,
Lex Fridman (1:43:46.240)
that this is a unique privilege.
Rohit Prasad (1:43:47.640)
Like I think, and we have,
Lex Fridman (1:43:49.560)
and I would say not just launching Alexa in 2014,
Rohit Prasad (1:43:52.760)
which was first of its kind.
Lex Fridman (1:43:54.360)
Along the way we have, when we launched Alexa Skills Kit,
Rohit Prasad (1:43:57.360)
it became democratizing AI.
Lex Fridman (1:43:59.680)
When before that there was no good evidence
Rohit Prasad (1:44:02.440)
of an SDK for speech and language.
Lex Fridman (1:44:04.960)
Now we are coming to this where you and I
Rohit Prasad (1:44:06.640)
are having this conversation where I'm not saying,
Lex Fridman (1:44:10.320)
oh, Lex, planning a night out with an AI agent, impossible.
Rohit Prasad (1:44:14.560)
I'm saying it's in the realm of possibility
Lex Fridman (1:44:17.120)
and not only possibility, we'll be launching this, right?
Lex Fridman (1:44:19.480)
So some elements of that, it will keep getting better.
Lex Fridman (1:44:23.800)
We know that is a universal truth.
Rohit Prasad (1:44:25.640)
Once you have these kinds of agents out there being used,
Lex Fridman (1:44:30.160)
they get better for your customers.
Lex Fridman (1:44:32.080)
And I think that's where,
Lex Fridman (1:44:34.240)
I think the amount of research topics
Rohit Prasad (1:44:36.560)
we are throwing out at our budding researchers
Lex Fridman (1:44:39.480)
is just gonna be exponentially hard.
Lex Fridman (1:44:41.840)
And the great thing is you can now get immense satisfaction
Lex Fridman (1:44:45.600)
by having customers use it,
Rohit Prasad (1:44:47.280)
not just a paper in NeurIPS or another conference.
Lex Fridman (1:44:51.120)
I think everyone, myself included,
Rohit Prasad (1:44:53.120)
are deeply excited about that future.
Lex Fridman (1:44:54.840)
So I don't think there's a better place to end, Rohit.
Rohit Prasad (1:44:58.040)
Thank you so much for talking to us.
Lex Fridman (1:44:58.880)
Thank you so much.
Rohit Prasad (1:44:59.720)
This was fun.
Lex Fridman (1:45:00.560)
Thank you, same here.
Rohit Prasad (1:45:02.240)
Thanks for listening to this conversation
Lex Fridman (1:45:04.240)
with Rohit Prasad.
Lex Fridman (1:45:05.760)
And thank you to our presenting sponsor, Cash App.
Lex Fridman (1:45:08.880)
Download it, use code LEGSPodcast,
Rohit Prasad (1:45:11.600)
you'll get $10 and $10 will go to FIRST,
Lex Fridman (1:45:14.720)
a STEM education nonprofit
Rohit Prasad (1:45:16.520)
that inspires hundreds of thousands of young minds
Lex Fridman (1:45:19.760)
to learn and to dream of engineering our future.
Rohit Prasad (1:45:23.320)
If you enjoy this podcast, subscribe on YouTube,
Lex Fridman (1:45:26.220)
give it five stars on Apple Podcast,
Rohit Prasad (1:45:28.200)
support it on Patreon, or connect with me on Twitter.
Lex Fridman (1:45:31.720)
And now let me leave you with some words of wisdom
Rohit Prasad (1:45:34.960)
from the great Alan Turing.
Lex Fridman (1:45:37.500)
Sometimes it is the people no one can imagine anything of
Rohit Prasad (1:45:41.680)
who do the things no one can imagine.
Lex Fridman (1:45:44.180)
Thank you for listening and hope to see you next time.
Rohit Prasad (20:00.400)
It's more than keywords, a little more in terms of,
Lex Fridman (20:03.480)
of course, there's keyword based too,
Lex Fridman (20:04.880)
but there's more in terms of these words can be
Lex Fridman (20:07.920)
very contextual, as you can see,
Lex Fridman (20:09.440)
and also the topic can be something
Lex Fridman (20:12.600)
that you don't want a conversation to happen
Rohit Prasad (20:15.400)
because this is a communal device as well.
Lex Fridman (20:17.320)
A lot of people use these devices.
Lex Fridman (20:19.240)
So we have put a lot of guardrails for the conversation
Lex Fridman (20:22.600)
to be more useful for advancing AI
Lex Fridman (20:25.920)
and not so much of these other issues you attributed
Lex Fridman (20:31.080)
what's happening in the AI field as well.
Rohit Prasad (20:32.880)
Right, so this is actually a serious opportunity.
Lex Fridman (20:35.280)
I didn't use the right word, fun.
Rohit Prasad (20:36.880)
I think it's an open opportunity to do
Lex Fridman (20:39.960)
some of the best innovation
Rohit Prasad (20:42.000)
in conversational agents in the world.
Lex Fridman (20:44.760)
Absolutely.
Lex Fridman (20:45.920)
Why just universities?
Lex Fridman (20:49.000)
Why just universities?
Rohit Prasad (20:49.880)
Because as I said, I really felt
Lex Fridman (20:51.560)
Young minds.
Rohit Prasad (20:52.400)
Young minds, it's also to,
Lex Fridman (20:55.080)
if you think about the other aspect
Rohit Prasad (20:57.920)
of where the whole industry is moving with AI,
Lex Fridman (21:01.400)
there's a dearth of talent given the demands.
Lex Fridman (21:04.880)
So you do want universities to have a clear place
Lex Fridman (21:09.880)
where they can invent and research
Lex Fridman (21:11.440)
and not fall behind that they can't motivate students.
Lex Fridman (21:13.920)
Imagine all grad students left to industry like us
Rohit Prasad (21:19.600)
or faculty members, which has happened too.
Lex Fridman (21:22.880)
So this is a way that if you're so passionate
Rohit Prasad (21:25.200)
about the field where you feel industry and academia
Lex Fridman (21:28.640)
need to work well, this is a great example
Lex Fridman (21:31.360)
and a great way for universities to participate.
Lex Fridman (21:35.360)
So what do you think it takes to build a system
Lex Fridman (21:37.280)
that wins the Alexa Prize?
Lex Fridman (21:39.600)
I think you have to start focusing on aspects of reasoning
Rohit Prasad (21:46.200)
that it is, there are still more lookups
Lex Fridman (21:50.760)
of what intents customers asking for
Lex Fridman (21:54.160)
and responding to those rather than really reasoning
Lex Fridman (21:58.920)
about the elements of the conversation.
Rohit Prasad (22:02.480)
For instance, if you're playing,
Lex Fridman (22:06.240)
if the conversation is about games
Lex Fridman (22:08.120)
and it's about a recent sports event,
Lex Fridman (22:11.240)
there's so much context involved
Lex Fridman (22:13.320)
and you have to understand the entities
Lex Fridman (22:15.800)
that are being mentioned
Lex Fridman (22:17.320)
so that the conversation is coherent
Lex Fridman (22:19.640)
rather than you suddenly just switch to knowing some fact
Rohit Prasad (22:23.200)
about a sports entity and you're just relaying that
Lex Fridman (22:26.280)
rather than understanding the true context of the game.
Rohit Prasad (22:28.680)
Like if you just said, I learned this fun fact
Lex Fridman (22:32.280)
about Tom Brady rather than really say
Lex Fridman (22:36.000)
how he played the game the previous night,
Lex Fridman (22:39.280)
then the conversation is not really that intelligent.
Lex Fridman (22:42.800)
So you have to go to more reasoning elements
Lex Fridman (22:46.200)
of understanding the context of the dialogue
Lex Fridman (22:49.120)
and giving more appropriate responses,
Lex Fridman (22:51.240)
which tells you that we are still quite far
Rohit Prasad (22:53.680)
because a lot of times it's more facts being looked up
Lex Fridman (22:57.400)
and something that's close enough as an answer,
Lex Fridman (22:59.920)
but not really the answer.
Lex Fridman (23:02.080)
So that is where the research needs to go more
Lex Fridman (23:05.040)
and actual true understanding and reasoning.
Lex Fridman (23:08.360)
And that's why I feel it's a great way to do it
Rohit Prasad (23:10.440)
because you have an engaged set of users
Lex Fridman (23:13.520)
working to help these AI advances happen in this case.
Rohit Prasad (23:18.200)
You mentioned customers, they're quite a bit,
Lex Fridman (23:20.640)
and there's a skill.
Lex Fridman (23:22.120)
What is the experience for the user that's helping?
Lex Fridman (23:26.520)
So just to clarify, this isn't, as far as I understand,
Rohit Prasad (23:30.120)
the Alexa, so this skill is a standalone
Lex Fridman (23:32.560)
for the Alexa Prize.
Rohit Prasad (23:33.560)
I mean, it's focused on the Alexa Prize.
Lex Fridman (23:35.360)
It's not you ordering certain things on Amazon.
Rohit Prasad (23:37.720)
Like, oh, we're checking the weather
Lex Fridman (23:39.200)
or playing Spotify, right?
Rohit Prasad (23:40.720)
This is a separate skill.
Lex Fridman (23:42.520)
And so you're focused on helping that,
Lex Fridman (23:45.600)
I don't know, how do people, how do customers think of it?
Lex Fridman (23:48.520)
Are they having fun?
Lex Fridman (23:49.800)
Are they helping teach the system?
Lex Fridman (23:52.040)
What's the experience like?
Rohit Prasad (23:53.040)
I think it's both actually.
Lex Fridman (23:54.640)
And let me tell you how you invoke this skill.
Lex Fridman (23:57.800)
So all you have to say, Alexa, let's chat.
Lex Fridman (24:00.200)
And then the first time you say, Alexa, let's chat,
Rohit Prasad (24:03.320)
it comes back with a clear message
Lex Fridman (24:04.720)
that you're interacting with one of those
Rohit Prasad (24:06.240)
university social bots.
Lex Fridman (24:08.000)
And there's a clear,
Lex Fridman (24:09.320)
so you know exactly how you interact, right?
Lex Fridman (24:11.800)
And that is why it's very transparent.
Lex Fridman (24:14.080)
You are being asked to help, right?
Lex Fridman (24:16.240)
And we have a lot of mechanisms
Rohit Prasad (24:18.800)
where as we are in the first phase of feedback phase,
Lex Fridman (24:23.680)
then you send a lot of emails to our customers
Lex Fridman (24:26.720)
and then they know that the team needs a lot of interactions
Lex Fridman (24:31.760)
to improve the accuracy of the system.
Lex Fridman (24:33.920)
So we know we have a lot of customers
Lex Fridman (24:35.880)
who really want to help these university bots
Lex Fridman (24:38.920)
and they're conversing with that.
Lex Fridman (24:40.400)
And some are just having fun with just saying,
Rohit Prasad (24:42.680)
Alexa, let's chat.
Lex Fridman (24:44.000)
And also some adversarial behavior to see whether,
Lex Fridman (24:47.320)
how much do you understand as a social bot?
Lex Fridman (24:50.240)
So I think we have a good,
Rohit Prasad (24:51.480)
healthy mix of all three situations.
Lex Fridman (24:53.920)
So what is the,
Rohit Prasad (24:55.280)
if we talk about solving the Alexa challenge,
Lex Fridman (24:58.040)
the Alexa prize,
Rohit Prasad (25:00.720)
what's the data set of really engaging,
Lex Fridman (25:05.480)
pleasant conversations look like?
Rohit Prasad (25:07.520)
Because if we think of this
Lex Fridman (25:08.360)
as a supervised learning problem,
Rohit Prasad (25:10.600)
I don't know if it has to be,
Lex Fridman (25:12.200)
but if it does, maybe you can comment on that.
Lex Fridman (25:15.400)
Do you think there needs to be a data set
Lex Fridman (25:17.480)
of what it means to be an engaging, successful,
Lex Fridman (25:21.880)
fulfilling conversation?
Lex Fridman (25:22.720)
I think that's part of the research question here.
Rohit Prasad (25:24.760)
This was, I think, we at least got the first part right,
Lex Fridman (25:29.200)
which is have a way for universities to build
Lex Fridman (25:33.360)
and test in a real world setting.
Lex Fridman (25:35.680)
Now you're asking in terms of the next phase of questions,
Rohit Prasad (25:38.640)
which we are still, we're also asking, by the way,
Lex Fridman (25:41.120)
what does success look like from a optimization function?
Rohit Prasad (25:45.400)
That's what you're asking in terms of,
Lex Fridman (25:47.200)
we as researchers are used to having a great corpus
Rohit Prasad (25:49.560)
of annotated data and then making,
Lex Fridman (25:53.480)
then sort of tune our algorithms on those, right?
Lex Fridman (25:57.600)
And fortunately and unfortunately,
Lex Fridman (26:00.640)
in this world of Alexa prize,
Rohit Prasad (26:02.920)
that is not the way we are going after it.
Lex Fridman (26:05.400)
So you have to focus more on learning
Rohit Prasad (26:07.720)
based on life feedback.
Lex Fridman (26:10.920)
That is another element that's unique,
Rohit Prasad (26:12.960)
where just not to,
Lex Fridman (26:15.080)
I started with giving you how you ingress
Lex Fridman (26:17.280)
and experience this capability as a customer.
Lex Fridman (26:21.520)
What happens when you're done?
Lex Fridman (26:23.600)
So they ask you a simple question on a scale of one to five,
Lex Fridman (26:27.560)
how likely are you to interact with this social bot again?
Rohit Prasad (26:31.880)
That is a good feedback
Lex Fridman (26:33.840)
and customers can also leave more open ended feedback.
Lex Fridman (26:37.440)
And I think partly that to me
Lex Fridman (26:40.840)
is one part of the question you're asking,
Rohit Prasad (26:42.640)
which I'm saying is a mental model shift
Lex Fridman (26:44.600)
that as researchers also,
Rohit Prasad (26:47.120)
you have to change your mindset
Lex Fridman (26:48.560)
that this is not a DARPA evaluation or NSF funded study
Lex Fridman (26:52.680)
and you have a nice corpus.
Lex Fridman (26:54.960)
This is where it's real world.
Rohit Prasad (26:56.960)
You have real data.
Lex Fridman (26:58.720)
The scale is amazing and that's a beautiful thing.
Lex Fridman (27:01.560)
And then the customer,
Lex Fridman (27:02.960)
the user can quit the conversation at any time.
Rohit Prasad (27:06.160)
Exactly, the user can,
Lex Fridman (27:07.200)
that is also a signal for how good you were at that point.
Rohit Prasad (27:11.720)
So, and then on a scale one to five, one to three,
Lex Fridman (27:15.000)
do they say how likely are you
Lex Fridman (27:16.360)
or is it just a binary?
Lex Fridman (27:18.040)
One to five.
Rohit Prasad (27:18.880)
One to five.
Lex Fridman (27:20.040)
Wow, okay, that's such a beautifully constructed challenge.
Rohit Prasad (27:22.680)
Okay.
Lex Fridman (27:24.720)
You said the only way to make a smart assistant really smart
Rohit Prasad (27:30.040)
is to give it eyes and let it explore the world.
Lex Fridman (27:34.560)
I'm not sure it might've been taken out of context,
Lex Fridman (27:36.840)
but can you comment on that?
Lex Fridman (27:38.240)
Can you elaborate on that idea?
Rohit Prasad (27:40.080)
Is that I personally also find that idea super exciting
Lex Fridman (27:43.120)
from a social robotics, personal robotics perspective.
Rohit Prasad (27:46.240)
Yeah, a lot of things do get taken out of context.
Lex Fridman (27:48.840)
This particular one was just
Rohit Prasad (27:50.600)
as philosophical discussion we were having
Lex Fridman (27:53.000)
on terms of what does intelligence look like?
Lex Fridman (27:55.520)
And the context was in terms of learning,
Lex Fridman (27:59.200)
I think just we said we as humans are empowered
Rohit Prasad (28:03.040)
with many different sensory abilities.
Lex Fridman (28:05.480)
I do believe that eyes are an important aspect of it
Rohit Prasad (28:09.560)
in terms of if you think about how we as humans learn,
Lex Fridman (28:14.640)
it is quite complex and it's also not unimodal
Rohit Prasad (28:18.320)
that you are fed a ton of text or audio
Lex Fridman (28:22.040)
and you just learn that way.
Rohit Prasad (28:23.360)
No, you learn by experience, you learn by seeing,
Lex Fridman (28:27.240)
you're taught by humans
Lex Fridman (28:30.320)
and we are very efficient in how we learn.
Lex Fridman (28:33.240)
Machines on the contrary are very inefficient
Rohit Prasad (28:35.320)
on how they learn, especially these AIs.
Lex Fridman (28:38.480)
I think the next wave of research is going to be
Rohit Prasad (28:42.640)
with less data, not just less human,
Lex Fridman (28:46.000)
not just with less labeled data,
Lex Fridman (28:48.240)
but also with a lot of weak supervision
Lex Fridman (28:51.080)
and where you can increase the learning rate.
Rohit Prasad (28:55.160)
I don't mean less data
Lex Fridman (28:56.120)
in terms of not having a lot of data to learn from
Rohit Prasad (28:58.640)
that we are generating so much data,
Lex Fridman (29:00.360)
but it is more about from a aspect
Lex Fridman (29:02.640)
of how fast can you learn?
Lex Fridman (29:04.880)
So improving the quality of the data,
Rohit Prasad (29:07.880)
the quality of data and the learning process.
Lex Fridman (29:09.920)
I think more on the learning process.
Rohit Prasad (29:11.440)
I think we have to, we as humans learn
Lex Fridman (29:13.560)
with a lot of noisy data, right?
Lex Fridman (29:15.720)
And I think that's the part
Lex Fridman (29:18.480)
that I don't think should change.
Lex Fridman (29:21.440)
What should change is how we learn, right?
Lex Fridman (29:23.880)
So if you look at, you mentioned supervised learning,
Rohit Prasad (29:26.080)
we have making transformative shifts
Lex Fridman (29:27.960)
from moving to more unsupervised, more weak supervision.
Rohit Prasad (29:31.160)
Those are the key aspects of how to learn.
Lex Fridman (29:34.840)
And I think in that setting, I hope you agree with me
Rohit Prasad (29:37.760)
that having other senses is very crucial
Lex Fridman (29:41.680)
in terms of how you learn.
Lex Fridman (29:43.480)
So absolutely.
Lex Fridman (29:44.640)
And from a machine learning perspective,
Rohit Prasad (29:46.680)
which I hope we get a chance to talk to a few aspects
Lex Fridman (29:49.680)
that are fascinating there,
Lex Fridman (29:51.080)
but to stick on the point of sort of a body,
Lex Fridman (29:55.600)
an embodiment.
Lex Fridman (29:56.440)
So Alexa has a body.
Lex Fridman (29:57.520)
It has a very minimalistic, beautiful interface
Rohit Prasad (30:01.600)
where there's a ring and so on.
Lex Fridman (30:02.840)
I mean, I'm not sure of all the flavors
Rohit Prasad (30:04.480)
of the devices that Alexa lives on,
Lex Fridman (30:07.560)
but there's a minimalistic basic interface.
Lex Fridman (30:13.280)
And nevertheless, we humans, so I have a Roomba,
Lex Fridman (30:15.640)
I have all kinds of robots all over everywhere.
Lex Fridman (30:18.240)
So what do you think the Alexa of the future looks like
Lex Fridman (30:24.680)
if it begins to shift what his body looks like?
Rohit Prasad (30:29.240)
Maybe beyond the Alexa,
Lex Fridman (30:30.640)
what do you think are the different devices in the home
Lex Fridman (30:33.720)
as they start to embody their intelligence more and more?
Lex Fridman (30:36.880)
What do you think that looks like?
Lex Fridman (30:38.080)
Philosophically, a future, what do you think that looks like?
Lex Fridman (30:41.200)
I think let's look at what's happening today.
Rohit Prasad (30:43.600)
You mentioned, I think our devices as an Amazon devices,
Lex Fridman (30:46.840)
but I also wanted to point out Alexa is already integrated
Rohit Prasad (30:49.840)
a lot of third party devices,
Lex Fridman (30:51.360)
which also come in lots of forms and shapes,
Rohit Prasad (30:54.840)
some in robots, some in microwaves,
Lex Fridman (30:58.960)
some in appliances that you use in everyday life.
Lex Fridman (31:02.600)
So I think it's not just the shape Alexa takes
Lex Fridman (31:07.720)
in terms of form factors,
Lex Fridman (31:09.200)
but it's also where all it's available.
Lex Fridman (31:13.000)
And it's getting in cars,
Rohit Prasad (31:14.240)
it's getting in different appliances in homes,
Lex Fridman (31:16.760)
even toothbrushes, right?
Lex Fridman (31:18.720)
So I think you have to think about it
Lex Fridman (31:20.760)
as not a physical assistant.
Rohit Prasad (31:25.440)
It will be in some embodiment, as you said,
Lex Fridman (31:28.480)
we already have these nice devices,
Lex Fridman (31:31.120)
but I think it's also important to think of it,
Lex Fridman (31:33.800)
it is a virtual assistant.
Rohit Prasad (31:35.640)
It is superhuman in the sense that it is in multiple places
Lex Fridman (31:38.520)
at the same time.
Lex Fridman (31:40.280)
So I think the actual embodiment in some sense,
Lex Fridman (31:45.200)
to me doesn't matter.
Rohit Prasad (31:47.600)
I think you have to think of it as not as human like
Lex Fridman (31:52.800)
and more of what its capabilities are
Rohit Prasad (31:56.080)
that derive a lot of benefit for customers
Lex Fridman (31:58.840)
and how there are different ways to delight it
Lex Fridman (32:00.680)
and delight customers and different experiences.
Lex Fridman (32:03.960)
And I think I'm a big fan of it not being just human like,
Rohit Prasad (32:09.240)
it should be human like in certain situations.
Lex Fridman (32:11.120)
Alexa price social bot in terms of conversation
Rohit Prasad (32:13.360)
is a great way to look at it,
Lex Fridman (32:14.920)
but there are other scenarios where human like,
Rohit Prasad (32:18.800)
I think is underselling the abilities of this AI.
Lex Fridman (32:22.080)
So if I could trivialize what we're talking about.
Lex Fridman (32:26.120)
So if you look at the way Steve Jobs thought
Lex Fridman (32:29.400)
about the interaction with the device that Apple produced,
Rohit Prasad (32:33.440)
there was a extreme focus on controlling the experience
Lex Fridman (32:36.760)
by making sure there's only this Apple produced devices.
Rohit Prasad (32:40.200)
You see the voice of Alexa being taking all kinds of forms
Lex Fridman (32:45.600)
depending on what the customers want.
Lex Fridman (32:47.080)
And that means it could be anywhere
Lex Fridman (32:49.920)
from the microwave to a vacuum cleaner to the home
Lex Fridman (32:53.760)
and so on the voice is the essential element
Lex Fridman (32:56.960)
of the interaction.
Rohit Prasad (32:57.800)
I think voice is an essence, it's not all,
Lex Fridman (33:01.160)
but it's a key aspect.
Rohit Prasad (33:02.240)
I think to your question in terms of,
Lex Fridman (33:05.720)
you should be able to recognize Alexa
Lex Fridman (33:08.280)
and that's a huge problem.
Lex Fridman (33:10.000)
I think in terms of a huge scientific problem,
Lex Fridman (33:12.080)
I should say like, what are the traits?
Lex Fridman (33:13.800)
What makes it look like Alexa,
Rohit Prasad (33:16.200)
especially in different settings
Lex Fridman (33:17.600)
and especially if it's primarily voice, what it is,
Lex Fridman (33:20.440)
but Alexa is not just voice either, right?
Lex Fridman (33:22.320)
I mean, we have devices with a screen.
Rohit Prasad (33:25.080)
Now you're seeing just other behaviors of Alexa.
Lex Fridman (33:28.520)
So I think we're in very early stages of what that means
Lex Fridman (33:31.400)
and this will be an important topic for the following years.
Lex Fridman (33:34.960)
But I do believe that being able to recognize
Lex Fridman (33:38.240)
and tell when it's Alexa versus it's not
Lex Fridman (33:40.520)
is going to be important from an Alexa perspective.
Rohit Prasad (33:43.400)
I'm not speaking for the entire AI community,
Lex Fridman (33:46.040)
but I think attribution and as we go into more
Rohit Prasad (33:51.040)
of understanding who did what,
Lex Fridman (33:54.400)
that identity of the AI is crucial in the coming world.
Rohit Prasad (33:58.000)
I think from the broad AI community perspective,
Lex Fridman (34:00.320)
that's also a fascinating problem.
Lex Fridman (34:02.120)
So basically if I close my eyes and listen to the voice,
Lex Fridman (34:05.480)
what would it take for me to recognize that this is Alexa?
Rohit Prasad (34:08.040)
Exactly.
Lex Fridman (34:08.880)
Or at least the Alexa that I've come to know
Rohit Prasad (34:10.600)
from my personal experience in my home
Lex Fridman (34:13.000)
through my interactions that come through.
Rohit Prasad (34:14.400)
Yeah, and the Alexa here in the US is very different
Lex Fridman (34:16.920)
than Alexa in UK and the Alexa in India,
Rohit Prasad (34:19.440)
even though they are all speaking English
Lex Fridman (34:21.640)
or the Australian version.
Lex Fridman (34:23.280)
So again, so now think about when you go
Lex Fridman (34:26.680)
into a different culture, a different community,
Lex Fridman (34:28.400)
but you travel there, what do you recognize Alexa?
Lex Fridman (34:31.800)
I think these are super hard questions actually.
Lex Fridman (34:34.160)
So there's a team that works on personality.
Lex Fridman (34:36.840)
So if we talk about those different flavors
Rohit Prasad (34:39.360)
of what it means culturally speaking,
Lex Fridman (34:41.040)
India, UK, US, what does it mean to add?
Lex Fridman (34:44.680)
So the problem that we just stated,
Lex Fridman (34:46.440)
it's just fascinating, how do we make it purely recognizable
Rohit Prasad (34:51.080)
that it's Alexa, assuming that the qualities
Lex Fridman (34:55.000)
of the voice are not sufficient?
Rohit Prasad (34:58.040)
It's also the content of what is being said.
Lex Fridman (35:01.000)
How do we do that?
Lex Fridman (35:02.160)
How does the personality come into play?
Lex Fridman (35:04.320)
What's that research gonna look like?
Rohit Prasad (35:06.800)
I mean, it's such a fascinating subject.
Lex Fridman (35:08.120)
We have some very fascinating folks
Rohit Prasad (35:11.080)
who from both the UX background and human factors
Lex Fridman (35:13.560)
are looking at these aspects and these exact questions.
Lex Fridman (35:16.360)
But I'll definitely say it's not just how it sounds,
Lex Fridman (35:21.600)
the choice of words, the tone, not just, I mean,
Rohit Prasad (35:25.320)
the voice identity of it, but the tone matters,
Lex Fridman (35:28.040)
the speed matters, how you speak,
Lex Fridman (35:30.720)
how you enunciate words, what choice of words
Lex Fridman (35:34.880)
are you using, how terse are you,
Rohit Prasad (35:37.320)
or how lengthy in your explanations you are.
Lex Fridman (35:40.720)
All of these are factors.
Lex Fridman (35:42.920)
And you also, you mentioned something crucial
Lex Fridman (35:45.440)
that you may have personalized it, Alexa,
Rohit Prasad (35:49.160)
to some extent in your homes
Lex Fridman (35:51.400)
or in the devices you are interacting with.
Lex Fridman (35:53.440)
So you, as your individual, how you prefer Alexa sounds
Lex Fridman (35:59.240)
can be different than how I prefer.
Lex Fridman (36:01.240)
And the amount of customizability you want to give
Lex Fridman (36:04.440)
is also a key debate we always have.
Lex Fridman (36:07.640)
But I do want to point out it's more than the voice actor
Lex Fridman (36:10.720)
that recorded and it sounds like that actor.
Rohit Prasad (36:14.000)
It is more about the choices of words,
Lex Fridman (36:16.920)
the attributes of tonality, the volume
Rohit Prasad (36:19.800)
in terms of how you raise your pitch and so forth.
Lex Fridman (36:22.600)
All of that matters.
Rohit Prasad (36:23.880)
This is such a fascinating problem
Lex Fridman (36:25.440)
from a product perspective.
Rohit Prasad (36:27.600)
I could see those debates just happening
Lex Fridman (36:29.480)
inside of the Alexa team of how much personalization
Lex Fridman (36:32.440)
do you do for the specific customer?
Lex Fridman (36:34.440)
Because you're taking a risk if you over personalize.
Rohit Prasad (36:38.240)
Because you don't, if you create a personality
Lex Fridman (36:42.080)
for a million people, you can test that better.
Rohit Prasad (36:46.040)
You can create a rich, fulfilling experience
Lex Fridman (36:48.640)
that will do well.
Lex Fridman (36:50.040)
But the more you personalize it, the less you can test it,
Lex Fridman (36:53.480)
the less you can know that it's a great experience.
Lex Fridman (36:56.320)
So how much personalization, what's the right balance?
Lex Fridman (36:59.720)
I think the right balance depends on the customer.
Rohit Prasad (37:01.600)
Give them the control.
Lex Fridman (37:02.800)
So I'll say, I think the more control you give customers,
Rohit Prasad (37:07.400)
the better it is for everyone.
Lex Fridman (37:09.600)
And I'll give you some key personalization features.
Rohit Prasad (37:13.880)
I think we have a feature called Remember This,
Lex Fridman (37:15.840)
which is where you can tell Alexa to remember something.
Rohit Prasad (37:19.440)
There you have an explicit sort of control
Lex Fridman (37:23.080)
in customer's hand because they have to say,
Rohit Prasad (37:24.600)
Alexa, remember X, Y, Z.
Lex Fridman (37:26.520)
What kind of things would that be used for?
Lex Fridman (37:28.000)
So you can like you, I have stored my tire specs
Lex Fridman (37:32.200)
for my car because it's so hard to go and find
Lex Fridman (37:34.800)
and see what it is, right?
Lex Fridman (37:36.760)
When you're having some issues.
Rohit Prasad (37:38.320)
I store my mileage plan numbers
Lex Fridman (37:41.440)
for all the frequent flyer ones
Rohit Prasad (37:43.120)
where I'm sometimes just looking at it and it's not handy.
Lex Fridman (37:46.520)
So those are my own personal choices I've made
Lex Fridman (37:49.960)
for Alexa to remember something on my behalf, right?
Lex Fridman (37:52.320)
So again, I think the choice was be explicit
Rohit Prasad (37:56.000)
about how you provide that to a customer as a control.
Lex Fridman (38:00.000)
So I think these are the aspects of what you do.
Rohit Prasad (38:03.440)
Like think about where we can use speaker recognition
Lex Fridman (38:07.360)
capabilities that it's, if you taught Alexa
Rohit Prasad (38:11.000)
that you are Lex and this person in your household
Lex Fridman (38:14.440)
is person two, then you can personalize the experiences.
Rohit Prasad (38:17.920)
Again, these are very in the CX customer experience patterns
Lex Fridman (38:22.840)
are very clear about and transparent
Rohit Prasad (38:26.520)
when a personalization action is happening.
Lex Fridman (38:30.040)
And then you have other ways like you go
Rohit Prasad (38:32.240)
through explicit control right now through your app
Lex Fridman (38:34.640)
that your multiple service providers,
Rohit Prasad (38:36.920)
let's say for music, which one is your preferred one.
Lex Fridman (38:39.480)
So when you say play sting, depend on your
Rohit Prasad (38:42.000)
whether you have preferred Spotify or Amazon music
Lex Fridman (38:44.880)
or Apple music, that the decision is made
Rohit Prasad (38:47.240)
where to play it from.
Lex Fridman (38:49.480)
So what's Alexa's backstory from her perspective?
Rohit Prasad (38:52.720)
Is there, I remember just asking as probably a lot
Lex Fridman (38:58.120)
of us are just the basic questions about love
Lex Fridman (39:00.600)
and so on of Alexa, just to see what the answer would be.
Lex Fridman (39:03.800)
It feels like there's a little bit of a personality
Lex Fridman (39:10.280)
but not too much.
Lex Fridman (39:12.840)
Is Alexa have a metaphysical presence
Rohit Prasad (39:18.360)
in this human universe we live in
Lex Fridman (39:21.880)
or is it something more ambiguous?
Lex Fridman (39:23.720)
Is there a past?
Lex Fridman (39:25.080)
Is there a birth?
Rohit Prasad (39:26.240)
Is there a family kind of idea
Lex Fridman (39:28.920)
even for joking purposes and so on?
Rohit Prasad (39:31.120)
I think, well, it does tell you if I think you,
Lex Fridman (39:34.800)
I should double check this but if you said
Rohit Prasad (39:36.320)
when were you born, I think we do respond.
Lex Fridman (39:39.000)
I need to double check that
Lex Fridman (39:40.120)
but I'm pretty positive about it.
Lex Fridman (39:41.480)
I think you do actually because I think I've tested that.
Lex Fridman (39:44.000)
But that's like how I was born in your brand of champagne
Lex Fridman (39:49.120)
and whatever the year kind of thing, yeah.
Lex Fridman (39:51.240)
So in terms of the metaphysical, I think it's early.
Lex Fridman (39:55.720)
Does it have the historic knowledge about herself
Lex Fridman (40:00.360)
to be able to do that?
Lex Fridman (40:01.440)
Maybe, have we crossed that boundary?
Lex Fridman (40:03.720)
Not yet, right?
Lex Fridman (40:04.560)
In terms of being, thank you.
Rohit Prasad (40:06.520)
Have we thought about it quite a bit
Lex Fridman (40:08.600)
but I wouldn't say that we have come to a clear decision
Rohit Prasad (40:11.480)
in terms of what it should look like.
Lex Fridman (40:13.000)
But you can imagine though, and I bring this back
Rohit Prasad (40:16.920)
to the Alexa Prize social bot one,
Lex Fridman (40:19.200)
there you will start seeing some of that.
Rohit Prasad (40:21.200)
Like these bots have their identity
Lex Fridman (40:23.440)
and in terms of that, you may find,
Rohit Prasad (40:26.800)
this is such a great research topic
Lex Fridman (40:28.400)
that some academia team may think of these problems
Lex Fridman (40:32.120)
and start solving them too.
Lex Fridman (40:35.080)
So let me ask a question.
Rohit Prasad (40:38.840)
It's kind of difficult, I think,
Lex Fridman (40:41.160)
but it feels, and fascinating to me
Rohit Prasad (40:43.280)
because I'm fascinated with psychology.
Lex Fridman (40:45.320)
It feels that the more personality you have,
Rohit Prasad (40:48.200)
the more dangerous it is
Lex Fridman (40:50.400)
in terms of a customer perspective of product.
Rohit Prasad (40:54.480)
If you want to create a product that's useful.
Lex Fridman (40:57.080)
By dangerous, I mean creating an experience that upsets me.
Lex Fridman (41:02.360)
And so how do you get that right?
Lex Fridman (41:06.680)
Because if you look at the relationships,
Rohit Prasad (41:10.040)
maybe I'm just a screwed up Russian,
Lex Fridman (41:11.800)
but if you look at the human to human relationship,
Rohit Prasad (41:15.040)
some of our deepest relationships have fights,
Lex Fridman (41:18.120)
have tension, have the push and pull,
Rohit Prasad (41:21.200)
have a little flavor in them.
Lex Fridman (41:22.800)
Do you want to have such flavor in an interaction with Alexa?
Lex Fridman (41:26.800)
How do you think about that?
Lex Fridman (41:28.200)
So there's one other common thing that you didn't say,
Lex Fridman (41:31.280)
but we think of it as paramount for any deep relationship.
Lex Fridman (41:35.000)
That's trust.
Rohit Prasad (41:36.680)
Trust, yeah.
Lex Fridman (41:37.520)
So I think if you trust every attribute you said,
Rohit Prasad (41:40.960)
a fight, some tension, is all healthy.
Lex Fridman (41:44.880)
But what is sort of unnegotiable in this instance is trust.
Lex Fridman (41:49.880)
And I think the bar to earn customer trust for AI
Lex Fridman (41:52.920)
is very high, in some sense, more than a human.
Rohit Prasad (41:56.880)
It's not just about personal information or your data.
Lex Fridman (42:02.040)
It's also about your actions on a daily basis.
Lex Fridman (42:05.120)
How trustworthy are you in terms of consistency,
Lex Fridman (42:07.920)
in terms of how accurate are you in understanding me?
Rohit Prasad (42:11.200)
Like if you're talking to a person on the phone,
Lex Fridman (42:13.680)
if you have a problem with your,
Rohit Prasad (42:14.880)
let's say your internet or something,
Lex Fridman (42:16.360)
if the person's not understanding,
Rohit Prasad (42:17.720)
you lose trust right away.
Lex Fridman (42:19.040)
You don't want to talk to that person.
Rohit Prasad (42:20.960)
That whole example gets amplified by a factor of 10,
Lex Fridman (42:24.360)
because when you're a human interacting with an AI,
Rohit Prasad (42:28.200)
you have a certain expectation.
Lex Fridman (42:29.720)
Either you expect it to be very intelligent
Lex Fridman (42:31.960)
and then you get upset, why is it behaving this way?
Lex Fridman (42:34.400)
Or you expect it to be not so intelligent
Lex Fridman (42:37.640)
and when it surprises you, you're like,
Lex Fridman (42:38.800)
really, you're trying to be too smart?
Lex Fridman (42:40.960)
So I think we grapple with these hard questions as well.
Lex Fridman (42:43.680)
But I think the key is actions need to be trustworthy.
Rohit Prasad (42:47.720)
From these AIs, not just about data protection,
Lex Fridman (42:50.840)
your personal information protection,
Lex Fridman (42:53.400)
but also from how accurately it accomplishes
Lex Fridman (42:57.200)
all commands or all interactions.
Rohit Prasad (42:59.760)
Well, it's tough to hear because trust,
Lex Fridman (43:02.200)
you're absolutely right,
Lex Fridman (43:03.080)
but trust is such a high bar with AI systems
Lex Fridman (43:05.560)
because people, and I see this
Rohit Prasad (43:07.400)
because I work with autonomous vehicles.
Lex Fridman (43:08.880)
I mean, the bar that's placed on AI system
Rohit Prasad (43:11.720)
is unreasonably high.
Lex Fridman (43:13.440)
Yeah, that is going to be, I agree with you.
Lex Fridman (43:16.120)
And I think of it as it's a challenge
Lex Fridman (43:19.920)
and it's also keeps my job, right?
Lex Fridman (43:23.120)
So from that perspective, I totally,
Lex Fridman (43:26.360)
I think of it at both sides as a customer
Lex Fridman (43:28.720)
and as a researcher.
Lex Fridman (43:30.240)
I think as a researcher, yes, occasionally it will frustrate
Lex Fridman (43:33.400)
me that why is the bar so high for these AIs?
Lex Fridman (43:36.920)
And as a customer, then I say,
Lex Fridman (43:38.640)
absolutely, it has to be that high, right?
Lex Fridman (43:40.920)
So I think that's the trade off we have to balance,
Lex Fridman (43:44.120)
but it doesn't change the fundamentals.
Lex Fridman (43:46.760)
That trust has to be earned and the question then becomes
Rohit Prasad (43:50.520)
is are we holding the AIs to a different bar
Lex Fridman (43:53.520)
in accuracy and mistakes than we hold humans?
Rohit Prasad (43:56.320)
That's going to be a great societal questions
Lex Fridman (43:58.280)
for years to come, I think for us.
Rohit Prasad (44:00.320)
Well, one of the questions that we grapple as a society now
Lex Fridman (44:04.000)
that I think about a lot,
Rohit Prasad (44:05.480)
I think a lot of people in the AI think about a lot
Lex Fridman (44:07.840)
and Alexis taking on head on is privacy.
Rohit Prasad (44:11.640)
The reality is us giving over data to any AI system
Lex Fridman (44:20.760)
can be used to enrich our lives in profound ways.
Lex Fridman (44:25.800)
So if basically any product that does anything awesome
Lex Fridman (44:28.520)
for you, the more data it has,
Rohit Prasad (44:31.680)
the more awesome things it can do.
Lex Fridman (44:34.040)
And yet on the other side,
Rohit Prasad (44:37.040)
people imagine the worst case possible scenario
Lex Fridman (44:39.400)
of what can you possibly do with that data?
Rohit Prasad (44:42.240)
People, it's goes down to trust, as you said before.
Lex Fridman (44:45.680)
There's a fundamental distrust of,
Rohit Prasad (44:48.200)
in certain groups of governments and so on.
Lex Fridman (44:50.440)
And depending on the government,
Rohit Prasad (44:51.560)
depending on who's in power,
Lex Fridman (44:52.880)
depending on all these kinds of factors.
Lex Fridman (44:55.400)
And so here's Alexa in the middle of all of it in the home,
Lex Fridman (44:59.600)
trying to do good things for the customers.
Lex Fridman (45:02.320)
So how do you think about privacy in this context,
Lex Fridman (45:05.040)
the smart assistance in the home?
Lex Fridman (45:06.720)
How do you maintain, how do you earn trust?
Lex Fridman (45:08.680)
Absolutely.
Lex Fridman (45:09.520)
So as you said, trust is the key here.
Lex Fridman (45:12.400)
So you start with trust
Lex Fridman (45:13.560)
and then privacy is a key aspect of it.
Lex Fridman (45:16.760)
It has to be designed from very beginning about that.
Lex Fridman (45:20.240)
And we believe in two fundamental principles.
Lex Fridman (45:23.920)
One is transparency and second is control.
Lex Fridman (45:26.840)
So by transparency, I mean,
Lex Fridman (45:28.920)
when we build what is now called smart speaker
Rohit Prasad (45:32.120)
or the first echo,
Lex Fridman (45:34.320)
we were quite judicious about making these right trade offs
Rohit Prasad (45:38.400)
on customer's behalf,
Lex Fridman (45:40.160)
that it is pretty clear
Rohit Prasad (45:41.920)
when the audio is being sent to cloud,
Lex Fridman (45:44.200)
the light ring comes on
Rohit Prasad (45:45.280)
when it has heard you say the word wake word,
Lex Fridman (45:48.280)
and then the streaming happens, right?
Lex Fridman (45:49.760)
So when the light ring comes up,
Lex Fridman (45:51.360)
we also had, we put a physical mute button on it,
Rohit Prasad (45:55.520)
just so if you didn't want it to be listening,
Lex Fridman (45:57.920)
even for the wake word,
Rohit Prasad (45:58.760)
then you turn the power button or the mute button on,
Lex Fridman (46:01.800)
and that disables the microphones.
Rohit Prasad (46:04.960)
That's just the first decision on essentially transparency
Lex Fridman (46:08.040)
and control.
Rohit Prasad (46:09.720)
Oh, then even when we launched,
Lex Fridman (46:11.720)
we gave the control in the hands of the customers
Rohit Prasad (46:13.840)
that you can go and look at any of your individual utterances
Lex Fridman (46:16.400)
that is recorded and delete them anytime.
Lex Fridman (46:19.560)
And we've got to true to that promise, right?
Lex Fridman (46:22.520)
So, and that is super, again,
Rohit Prasad (46:25.000)
a great instance of showing how you have the control.
Lex Fridman (46:29.080)
Then we made it even easier.
Rohit Prasad (46:30.440)
You can say, like I said, delete what I said today.
Lex Fridman (46:33.080)
So that is now making it even just more control
Rohit Prasad (46:36.880)
in your hands with what's most convenient
Lex Fridman (46:39.360)
about this technology is voice.
Rohit Prasad (46:42.000)
You delete it with your voice now.
Lex Fridman (46:44.400)
So these are the types of decisions we continually make.
Rohit Prasad (46:48.080)
We just recently launched this feature called,
Lex Fridman (46:51.240)
what we think of it as,
Rohit Prasad (46:52.360)
if you wanted humans not to review your data,
Lex Fridman (46:56.680)
because you've mentioned supervised learning, right?
Lex Fridman (46:59.160)
So in supervised learning,
Lex Fridman (47:01.160)
humans have to give some annotation.
Lex Fridman (47:03.760)
And that also is now a feature
Lex Fridman (47:06.200)
where you can essentially, if you've selected that flag,
Rohit Prasad (47:09.320)
your data will not be reviewed by a human.
Lex Fridman (47:11.320)
So these are the types of controls
Rohit Prasad (47:13.640)
that we have to constantly offer with customers.
Lex Fridman (47:18.440)
So why do you think it bothers people so much that,
Lex Fridman (47:23.840)
so everything you just said is really powerful.
Lex Fridman (47:26.920)
So the control, the ability to delete,
Rohit Prasad (47:28.400)
cause we collect, we have studies here running at MIT
Lex Fridman (47:31.120)
that collects huge amounts of data
Lex Fridman (47:32.760)
and people consent and so on.
Lex Fridman (47:34.880)
The ability to delete that data is really empowering
Lex Fridman (47:38.040)
and almost nobody ever asked to delete it,
Lex Fridman (47:40.000)
but the ability to have that control is really powerful.
Lex Fridman (47:44.200)
But still, there's these popular anecdote,
Lex Fridman (47:47.040)
anecdotal evidence that people say,
Rohit Prasad (47:49.280)
they like to tell that,
Lex Fridman (47:51.000)
them and a friend were talking about something,
Rohit Prasad (47:53.160)
I don't know, sweaters for cats.
Lex Fridman (47:56.120)
And all of a sudden they'll have advertisements
Rohit Prasad (47:58.200)
for cat sweaters on Amazon.
Lex Fridman (48:01.400)
That's a popular anecdote
Rohit Prasad (48:02.680)
as if something is always listening.
Lex Fridman (48:05.040)
What, can you explain that anecdote,
Lex Fridman (48:07.800)
that experience that people have?
Lex Fridman (48:09.120)
What's the psychology of that?
Lex Fridman (48:11.000)
What's that experience?
Lex Fridman (48:13.080)
And can you, you've answered it,
Lex Fridman (48:15.080)
but let me just ask, is Alexa listening?
Lex Fridman (48:18.280)
No, Alexa listens only for the wake word on the device.
Lex Fridman (48:22.560)
And the wake word is?
Lex Fridman (48:23.920)
The words like Alexa, Amazon, Echo,
Lex Fridman (48:28.080)
but you only choose one at a time.
Lex Fridman (48:29.640)
So you choose one and it listens only
Rohit Prasad (48:31.640)
for that on our devices.
Lex Fridman (48:34.040)
So that's first.
Rohit Prasad (48:35.160)
From a listening perspective,
Lex Fridman (48:36.480)
we have to be very clear that it's just the wake word.
Lex Fridman (48:38.360)
So you said, why is there this anxiety, if you may?
Lex Fridman (48:41.280)
Yeah, exactly.
Rohit Prasad (48:42.120)
It's because there's a lot of confusion,
Lex Fridman (48:43.560)
what it really listens to, right?
Lex Fridman (48:45.360)
And I think it's partly on us to keep educating
Lex Fridman (48:49.680)
our customers and the general media more
Rohit Prasad (48:52.240)
in terms of like how, what really happens.
Lex Fridman (48:54.080)
And we've done a lot of it.
Lex Fridman (48:56.560)
And our pages on information are clear,
Lex Fridman (49:00.840)
but still people have to have more,
Rohit Prasad (49:04.040)
there's always a hunger for information and clarity.
Lex Fridman (49:06.680)
And we'll constantly look at how best to communicate.
Rohit Prasad (49:09.120)
If you go back and read everything,
Lex Fridman (49:10.560)
yes, it states exactly that.
Lex Fridman (49:13.120)
And then people could still question it.
Lex Fridman (49:15.360)
And I think that's absolutely okay to question.
Lex Fridman (49:17.760)
What we have to make sure is that we are,
Lex Fridman (49:21.760)
because our fundamental philosophy is customer first,
Rohit Prasad (49:24.880)
customer obsession is our leadership principle.
Lex Fridman (49:27.280)
If you put, as researchers, I put myself
Rohit Prasad (49:31.040)
in the shoes of the customer,
Lex Fridman (49:33.200)
and all decisions in Amazon are made with that.
Lex Fridman (49:35.880)
And trust has to be earned,
Lex Fridman (49:38.040)
and we have to keep earning the trust
Rohit Prasad (49:39.440)
of our customers in this setting.
Lex Fridman (49:41.800)
And to your other point on like,
Rohit Prasad (49:44.080)
is there something showing up
Lex Fridman (49:45.560)
based on your conversations?
Rohit Prasad (49:46.680)
No, I think the answer is like,
Lex Fridman (49:49.640)
a lot of times when those experiences happen,
Rohit Prasad (49:51.400)
you have to also know that, okay,
Lex Fridman (49:52.840)
it may be a winter season,
Lex Fridman (49:54.600)
people are looking for sweaters, right?
Lex Fridman (49:56.480)
And it shows up on your amazon.com because it is popular.
Lex Fridman (49:59.640)
So there are many of these,
Lex Fridman (50:02.720)
you mentioned that personality or personalization,
Lex Fridman (50:06.320)
turns out we are not that unique either, right?
Lex Fridman (50:09.120)
So those things we as humans start thinking,
Rohit Prasad (50:12.080)
oh, must be because something was heard,
Lex Fridman (50:14.120)
and that's why this other thing showed up.
Rohit Prasad (50:16.720)
The answer is no,
Lex Fridman (50:17.760)
probably it is just the season for sweaters.
Rohit Prasad (50:21.520)
I'm not gonna ask you this question
Lex Fridman (50:23.800)
because people have so much paranoia.
Lex Fridman (50:27.160)
But let me just say from my perspective,
Lex Fridman (50:29.200)
I hope there's a day when customer can ask Alexa
Rohit Prasad (50:33.160)
to listen all the time,
Lex Fridman (50:35.200)
to improve the experience,
Rohit Prasad (50:36.640)
to improve because I personally don't see the negative
Lex Fridman (50:40.760)
because if you have the control and if you have the trust,
Rohit Prasad (50:43.920)
there's no reason why I shouldn't be listening
Lex Fridman (50:45.640)
all the time to the conversations to learn more about you.
Rohit Prasad (50:48.280)
Because ultimately,
Lex Fridman (50:49.640)
as long as you have control and trust,
Rohit Prasad (50:52.560)
every data you provide to the device,
Lex Fridman (50:55.680)
that the device wants is going to be useful.
Lex Fridman (51:00.200)
And so to me, as a machine learning person,
Lex Fridman (51:03.880)
I think it worries me how sensitive people are
Rohit Prasad (51:08.200)
about their data relative to how empowering it could be
Lex Fridman (51:13.200)
relative to how empowering it could be
Rohit Prasad (51:19.320)
for the devices around them,
Lex Fridman (51:21.160)
how enriching it could be for their own life
Rohit Prasad (51:23.720)
to improve the product.
Lex Fridman (51:25.440)
So I just, it's something I think about sort of a lot,
Lex Fridman (51:28.320)
how do we make that devices,
Lex Fridman (51:29.520)
obviously Alexa thinks about a lot as well.
Rohit Prasad (51:32.200)
I don't know if you wanna comment on that,
Lex Fridman (51:34.200)
sort of, okay, have you seen,
Rohit Prasad (51:35.360)
let me ask it in the form of a question, okay.
Lex Fridman (51:38.680)
Have you seen an evolution in the way people think about
Lex Fridman (51:42.240)
their private data in the previous several years?
Lex Fridman (51:46.400)
So as we as a society get more and more comfortable
Rohit Prasad (51:48.680)
to the benefits we get by sharing more data.
Lex Fridman (51:53.520)
First, let me answer that part
Lex Fridman (51:55.040)
and then I'll wanna go back
Lex Fridman (51:55.960)
to the other aspect you were mentioning.
Lex Fridman (51:58.440)
So as a society, on a general,
Lex Fridman (52:01.160)
we are getting more comfortable as a society.
Rohit Prasad (52:03.120)
Doesn't mean that everyone is,
Lex Fridman (52:05.840)
and I think we have to respect that.
Rohit Prasad (52:07.600)
I don't think one size fits all
Lex Fridman (52:10.320)
is always gonna be the answer for all, right?
Rohit Prasad (52:13.520)
By definition.
Lex Fridman (52:14.360)
So I think that's something to keep in mind in these.
Rohit Prasad (52:17.160)
Going back to your, on what more
Lex Fridman (52:21.400)
magical experiences can be launched
Rohit Prasad (52:23.640)
in these kinds of AI settings.
Lex Fridman (52:26.040)
I think again, if you give the control,
Rohit Prasad (52:29.200)
we, it's possible certain parts of it.
Lex Fridman (52:32.080)
So we have a feature called follow up mode
Rohit Prasad (52:33.960)
where you, if you turn it on
Lex Fridman (52:37.000)
and Alexa, after you've spoken to it,
Rohit Prasad (52:40.400)
will open the mics again,
Lex Fridman (52:42.000)
thinking you will answer something again.
Rohit Prasad (52:44.680)
Like if you're adding lists to your shopping item,
Lex Fridman (52:47.880)
so right, or a shopping list or to do list,
Rohit Prasad (52:50.360)
you're not done.
Lex Fridman (52:51.440)
You want to keep, so in that setting,
Rohit Prasad (52:53.000)
it's awesome that it opens the mic
Lex Fridman (52:54.520)
for you to say eggs and milk and then bread, right?
Lex Fridman (52:57.160)
So these are the kinds of things which you can empower.
Lex Fridman (52:59.920)
So, and then another feature we have,
Rohit Prasad (53:02.320)
which is called Alexa Guard.
Lex Fridman (53:04.960)
I said it only listens for the wake word, right?
Lex Fridman (53:07.800)
But if you have, let's say you're going to say,
Lex Fridman (53:10.480)
like you leave your home and you want Alexa to listen
Rohit Prasad (53:13.440)
for a couple of sound events like smoke alarm going off
Lex Fridman (53:17.200)
or someone breaking your glass, right?
Lex Fridman (53:19.280)
So it's like just to keep your peace of mind.
Lex Fridman (53:22.160)
So you can say Alexa on guard or I'm away
Lex Fridman (53:26.480)
and then it can be listening for these sound events.
Lex Fridman (53:29.200)
And when you're home, you come out of that mode, right?
Lex Fridman (53:33.040)
So this is another one where you again gave controls
Lex Fridman (53:35.560)
in the hands of the user or the customer
Lex Fridman (53:38.040)
and to enable some experience that is high utility
Lex Fridman (53:42.440)
and maybe even more delightful in the certain settings
Rohit Prasad (53:44.600)
like follow up mode and so forth.
Lex Fridman (53:46.480)
And again, this general principle is the same,
Rohit Prasad (53:48.880)
control in the hands of the customer.
Lex Fridman (53:52.640)
So I know we kind of started with a lot of philosophy
Lex Fridman (53:55.480)
and a lot of interesting topics
Lex Fridman (53:56.840)
and we're just jumping all over the place,
Lex Fridman (53:58.280)
but really some of the fascinating things
Lex Fridman (54:00.280)
that the Alexa team and Amazon is doing
Rohit Prasad (54:03.040)
is in the algorithm side, the data side,
Lex Fridman (54:05.480)
the technology, the deep learning, machine learning
Lex Fridman (54:07.520)
and so on.
Lex Fridman (54:08.880)
So can you give a brief history of Alexa
Rohit Prasad (54:13.040)
from the perspective of just innovation,
Lex Fridman (54:15.440)
the algorithms, the data of how it was born,
Lex Fridman (54:18.640)
how it came to be, how it's grown, where it is today?
Lex Fridman (54:22.280)
Yeah, it start with in Amazon,
Rohit Prasad (54:24.360)
everything starts with the customer
Lex Fridman (54:27.000)
and we have a process called working backwards.
Rohit Prasad (54:30.320)
Alexa and more specifically than the product Echo,
Lex Fridman (54:35.040)
there was a working backwards document essentially
Rohit Prasad (54:37.320)
that reflected what it would be,
Lex Fridman (54:38.880)
started with a very simple vision statement for instance
Rohit Prasad (54:44.320)
that morphed into a full fledged document
Lex Fridman (54:47.160)
along the way changed into what all it can do, right?
Lex Fridman (54:51.720)
But the inspiration was the Star Trek computer.
Lex Fridman (54:54.160)
So when you think of it that way,
Rohit Prasad (54:56.240)
everything is possible, but when you launch a product,
Lex Fridman (54:58.360)
you have to start with some place.
Lex Fridman (55:01.040)
And when I joined, the product was already in conception
Lex Fridman (55:05.520)
and we started working on the far field speech recognition
Rohit Prasad (55:08.960)
because that was the first thing to solve.
Lex Fridman (55:10.960)
By that we mean that you should be able to speak
Rohit Prasad (55:12.880)
to the device from a distance.
Lex Fridman (55:15.280)
And in those days, that wasn't a common practice.
Lex Fridman (55:18.840)
And even in the previous research world I was in
Lex Fridman (55:22.360)
was considered to an unsolvable problem then
Rohit Prasad (55:24.640)
in terms of whether you can converse from a length.
Lex Fridman (55:28.320)
And here I'm still talking about the first part
Rohit Prasad (55:30.360)
of the problem where you say,
Lex Fridman (55:32.440)
get the attention of the device
Rohit Prasad (55:34.080)
as in by saying what we call the wake word,
Lex Fridman (55:37.120)
which means the word Alexa has to be detected
Rohit Prasad (55:40.400)
with a very high accuracy because it is a very common word.
Lex Fridman (55:44.880)
It has sound units that map with words like I like you
Lex Fridman (55:48.240)
or Alec, Alex, right?
Lex Fridman (55:51.160)
So it's a undoubtedly hard problem to detect
Rohit Prasad (55:56.160)
the right mentions of Alexa's address to the device
Lex Fridman (56:00.520)
versus I like Alexa.
Lex Fridman (56:02.800)
So you have to pick up that signal
Lex Fridman (56:04.240)
when there's a lot of noise.
Rohit Prasad (56:06.040)
Not only noise but a lot of conversation in the house,
Lex Fridman (56:09.120)
right?
Rohit Prasad (56:09.960)
You remember on the device,
Lex Fridman (56:10.800)
you're simply listening for the wake word, Alexa.
Lex Fridman (56:13.160)
And there's a lot of words being spoken in the house.
Lex Fridman (56:15.760)
How do you know it's Alexa and directed at Alexa?
Rohit Prasad (56:21.720)
Because I could say, I love my Alexa, I hate my Alexa.
Lex Fridman (56:25.320)
I want Alexa to do this.
Lex Fridman (56:27.000)
And in all these three sentences, I said, Alexa,
Lex Fridman (56:29.280)
I didn't want it to wake up.
Lex Fridman (56:32.120)
Can I just pause on that second?
Lex Fridman (56:33.720)
What would be your device that I should probably
Rohit Prasad (56:36.680)
in the introduction of this conversation give to people
Lex Fridman (56:39.920)
in terms of them turning off their Alexa device
Lex Fridman (56:43.440)
if they're listening to this podcast conversation out loud?
Lex Fridman (56:49.240)
Like what's the probability that an Alexa device
Rohit Prasad (56:51.640)
will go off because we mentioned Alexa like a million times.
Lex Fridman (56:55.160)
So it will, we have done a lot of different things
Rohit Prasad (56:58.120)
where we can figure out that there is the device,
Lex Fridman (57:03.720)
the speech is coming from a human versus over the air.
Rohit Prasad (57:08.200)
Also, I mean, in terms of like, also it is think about ads
Lex Fridman (57:11.720)
or so we have also launched a technology
Rohit Prasad (57:14.240)
for watermarking kind of approaches
Lex Fridman (57:16.280)
in terms of filtering it out.
Lex Fridman (57:18.800)
But yes, if this kind of a podcast is happening,
Lex Fridman (57:21.600)
it's possible your device will wake up a few times.
Rohit Prasad (57:24.360)
It's an unsolved problem,
Lex Fridman (57:25.440)
but it is definitely something we care very much about.
Lex Fridman (57:31.040)
But the idea is you wanna detect Alexa.
Lex Fridman (57:33.880)
Meant for the device.
Rohit Prasad (57:36.080)
First of all, just even hearing Alexa versus I like something.
Lex Fridman (57:40.040)
I mean, that's a fascinating part.
Lex Fridman (57:41.040)
So that was the first relief.
Lex Fridman (57:43.040)
That's the first.
Rohit Prasad (57:43.880)
The world's best detector of Alexa.
Lex Fridman (57:45.960)
Yeah, the world's best wake word detector
Rohit Prasad (57:48.720)
in a far field setting,
Lex Fridman (57:49.920)
not like something where the phone is sitting on the table.
Rohit Prasad (57:53.840)
This is like people have devices 40 feet away
Lex Fridman (57:56.680)
like in my house or 20 feet away and you still get an answer.
Lex Fridman (58:00.640)
So that was the first part.
Lex Fridman (58:02.480)
The next is, okay, you're speaking to the device.
Rohit Prasad (58:05.880)
Of course, you're gonna issue many different requests.
Lex Fridman (58:09.000)
Some may be simple, some may be extremely hard,
Lex Fridman (58:11.560)
but it's a large vocabulary speech recognition problem
Lex Fridman (58:13.720)
essentially, where the audio is now not coming
Rohit Prasad (58:17.600)
onto your phone or a handheld mic like this
Lex Fridman (58:20.360)
or a close talking mic, but it's from 20 feet away
Rohit Prasad (58:23.880)
where if you're in a busy household,
Lex Fridman (58:26.240)
your son may be listening to music,
Rohit Prasad (58:28.840)
your daughter may be running around with something
Lex Fridman (58:31.600)
and asking your mom something and so forth, right?
Lex Fridman (58:33.800)
So this is like a common household setting
Lex Fridman (58:36.360)
where the words you're speaking to Alexa
Lex Fridman (58:40.160)
need to be recognized with very high accuracy, right?
Lex Fridman (58:43.400)
Now we are still just in the recognition problem.
Lex Fridman (58:45.800)
We haven't yet come to the understanding one, right?
Lex Fridman (58:48.160)
And if I pause them, sorry, once again,
Lex Fridman (58:50.160)
what year was this?
Lex Fridman (58:51.160)
Is this before neural networks began to start
Lex Fridman (58:56.440)
to seriously prove themselves in the audio space?
Lex Fridman (59:00.480)
Yeah, this is around, so I joined in 2013 in April, right?
Lex Fridman (59:05.480)
So the early research and neural networks coming back
Lex Fridman (59:08.800)
and showing some promising results
Rohit Prasad (59:11.240)
in speech recognition space had started happening,
Lex Fridman (59:13.560)
but it was very early.
Lex Fridman (59:15.360)
But we just now build on that
Lex Fridman (59:17.800)
on the very first thing we did when I joined with the team.
Lex Fridman (59:23.240)
And remember, it was a very much of a startup environment,
Lex Fridman (59:25.960)
which is great about Amazon.
Lex Fridman (59:28.080)
And we doubled down on deep learning right away.
Lex Fridman (59:31.240)
And we knew we'll have to improve accuracy fast.
Lex Fridman (59:36.600)
And because of that, we worked on,
Lex Fridman (59:38.960)
and the scale of data, once you have a device like this,
Rohit Prasad (59:41.640)
if it is successful, will improve big time.
Lex Fridman (59:44.920)
Like you'll suddenly have large volumes of data
Rohit Prasad (59:48.040)
to learn from to make the customer experience better.
Lex Fridman (59:51.080)
So how do you scale deep learning?
Lex Fridman (59:52.480)
So we did one of the first works
Lex Fridman (59:54.560)
in training with distributed GPUs
Lex Fridman (59:57.600)
and where the training time was linear
🔗 相关节目