Joscha Bach: Artificial Consciousness and the Nature of Reality

Lex Fridman (03:09.200)

And now here's my conversation with Joscha Bach. As you've said, you grew up in a forest in East

Joscha Bach (03:16.880)

Germany, just as we're talking about off mic, to parents who are artists. And now I think,

Lex Fridman (03:23.840)

at least to me, you've become one of the most unique thinkers in the AI world.

Lex Fridman (03:27.920)

So can we try to reverse engineer your mind a little bit?

Lex Fridman (03:30.480)

What were the key philosopher, scientist ideas, maybe even movies or just realizations that

Joscha Bach (03:38.400)

had an impact on you when you were growing up that kind of led to the trajectory,

Lex Fridman (03:43.680)

or were the key sort of crossroads in the trajectory of your intellectual development?

Joscha Bach (03:49.600)

My father came from a long tradition of architects, a distant branch of the Bach family.

Lex Fridman (03:56.560)

And so basically, he was technically a nerd. And nerds need to interface in society with

Joscha Bach (04:03.680)

nonstandard ways. Sometimes I define a nerd as somebody who thinks that the purpose of

Joscha Bach (04:09.360)

communication is to submit your ideas to peer review. And normal people understand that the

Joscha Bach (04:15.920)

primary purpose of communication is to negotiate alignment. And these purposes tend to conflict,

Lex Fridman (04:21.600)

which means that nerds have to learn how to interact with society at large.

Lex Fridman (04:26.400)

Who is the reviewer in the nerd's view of communication?

Joscha Bach (04:31.360)

Everybody who you consider to be a peer. So whatever hapless individual is around,

Joscha Bach (04:36.960)

well, you would try to make him or her the gift of information.

Lex Fridman (04:42.400)

Okay. So you're now, by the way, my research malinformed me. So you're architect or artist?

Lex Fridman (04:50.560)

So he did study architecture. But basically, my grandfather made the wrong decision. He married

Joscha Bach (04:58.160)

an aristocrat and was drawn into the war. And he came back after 15 years. So basically, my father

Joscha Bach (05:07.680)

was not parented by a nerd, but by somebody who tried to tell him what to do, and expected him

Joscha Bach (05:14.800)

to do what he was told. And he was unable to. He's unable to do things if he's not intrinsically

Joscha Bach (05:21.120)

motivated. So in some sense, my grandmother broke her son. And her son responded when he became an

Joscha Bach (05:27.600)

architect to become an artist. So he built 100 Wasser architecture. He built houses without

Joscha Bach (05:33.200)

right angles. He built lots of things that didn't work in the more brutalist traditions of eastern

Joscha Bach (05:38.400)

Germany. And so he bought an old watermill, moved out to the countryside, and did only what he wanted

Joscha Bach (05:44.640)

to do, which was art. Eastern Germany was perfect for Boheme, because you had complete material

Joscha Bach (05:50.640)

safety. Food was heavily subsidized, healthcare was free. You didn't have to worry about rent or

Joscha Bach (05:55.520)

pensions or anything. So it's a socialized communist side. Yes. And the other thing is,

Joscha Bach (06:00.000)

it was almost impossible not to be in political disagreement with your government, which is very

Joscha Bach (06:04.080)

productive for artists. So everything that you do is intrinsically meaningful, because it will

Joscha Bach (06:08.800)

always touch on the deeper currents of society of culture and be in conflict with it and tension

Joscha Bach (06:14.160)

with it. And you will always have to define yourself with respect to this. So what impacted

Lex Fridman (06:19.040)

your father, this outside of the box thinker against the government, against the world artists?

Joscha Bach (06:26.640)

He was actually not a thinker. He was somebody who only got self aware to the degree that he

Joscha Bach (06:31.360)

needed to make himself functional. So in some sense, he was also in the late 1960s. And he was

Joscha Bach (06:39.200)

in some sense a hippie. So he became a one person cult. He lived out there in his kingdom. He built

Joscha Bach (06:44.720)

big sculpture gardens and started many avenues of art and so on and convinced a woman to live with

Joscha Bach (06:53.600)

him. She was also an architect and she adored him and decided to share her life with him.

Lex Fridman (06:58.480)

And I basically grew up in a big cave full of books. I'm almost feral. And I was bored out

Joscha Bach (07:05.280)

there. It was very, very beautiful, very quiet, and quite lonely. So I started to read. And by

Joscha Bach (07:11.120)

the time I came to school, I've read everything until fourth grade and then some. And there was

Joscha Bach (07:16.240)

not a real way for me to relate to the outside world. And I couldn't quite put my finger on why.

Lex Fridman (07:21.760)

And today I know it was because I was a nerd, obviously, and it was the only nerd around. So

Joscha Bach (07:26.800)

there was no other kids like me. And there was nobody interested in physics or computing or

Joscha Bach (07:32.800)

mathematics and so on. And this village school that I went to was basically a nice school.

Joscha Bach (07:38.960)

Kids were nice to me. I was not beaten up, but I also didn't make many friends or

Joscha Bach (07:42.800)

build deep relationships. They only happened in starting from ninth grade when I went into a

Lex Fridman (07:47.120)

school for mathematics and physics. Do you remember any key books from this moment?

Joscha Bach (07:51.200)

I basically read everything. So I went to the library and I worked my way through the

Joscha Bach (07:56.880)

children's and young adult sections. And then I read a lot of science fiction,

Joscha Bach (08:01.200)

for instance, Stanislav Lem, basically the great author of Cybernetics, has influenced me. Back

Joscha Bach (08:06.800)

then, I didn't see him as a big influence because everything that he wrote seemed to be so natural

Joscha Bach (08:10.640)

to me. And it's only later that I contrasted it with what other people wrote. Another thing that

Joscha Bach (08:16.560)

was very influential on me were the classical philosophers and also the literature of romanticism.

Lex Fridman (08:22.000)

So German poetry and art, Troste Hilshoff and Heine and up to Hesse and so on.

Joscha Bach (08:29.680)

Hesse. I love Hesse. So at which point do the classical philosophers end? At this point,

Joscha Bach (08:35.840)

we're in the 21st century. What's the latest classical philosopher? Does this stretch through

Lex Fridman (08:41.680)

even as far as Nietzsche or is this, are we talking about Plato and Aristotle?

Lex Fridman (08:45.840)

I think that Nietzsche is the classical equivalent of a shit poster.

Joscha Bach (08:52.560)

He's very smart and easy to read, but he's not so much trolling others. He's trolling himself

Joscha Bach (08:57.920)

because he was at odds with the world. Largely his romantic relationships didn't work out.

Joscha Bach (09:02.480)

He got angry and he basically became a nihilist.

Joscha Bach (09:06.240)

Isn't that a beautiful way to be as an intellectual is to constantly be trolling yourself,

Lex Fridman (09:11.760)

to be in that conflict, in that tension?

Joscha Bach (09:14.960)

I think it's a lack of self awareness. At some point, you have to understand the

Joscha Bach (09:18.720)

comedy of your own situation. If you take yourself seriously and you are not functional,

Lex Fridman (09:23.840)

it ends in tragedy as it did for Nietzsche.

Joscha Bach (09:25.840)

I think you think he took himself too seriously in that tension.

Lex Fridman (09:29.760)

And as you find the same thing in Hesse and so on, this Steppenwolf syndrome is classic

Joscha Bach (09:34.160)

adolescence where you basically feel misunderstood by the world and you don't understand that all the

Joscha Bach (09:38.880)

misunderstandings are the result of your own lack of self awareness because you think that you are

Joscha Bach (09:44.160)

a prototypical human and the others around you should behave the same way as you expect them

Joscha Bach (09:48.960)

based on your innate instincts and it doesn't work out and you become a transcendentalist

Joscha Bach (09:53.600)

to deal with that. So it's very, very understandable and have great sympathies for this

Lex Fridman (09:58.480)

to the degree that I can have sympathy for my own intellectual history.

Lex Fridman (10:02.160)

But you have to grow out of it.

Lex Fridman (10:04.720)

So as an intellectual, a life well lived, a journey well traveled is one where you don't

Lex Fridman (10:09.600)

take yourself seriously from that perspective?

Joscha Bach (10:11.120)

No, I think that you are neither serious or not serious yourself because you need to become

Joscha Bach (10:17.840)

unimportant as a subject. That is, if you are a philosopher, belief is not a verb.

Lex Fridman (10:24.720)

You don't do this for the audience and you don't do it for yourself.

Joscha Bach (10:27.360)

You have to submit to the things that are possibly true and you have to follow wherever

Lex Fridman (10:32.240)

your inquiry leads. But it's not about you. It has nothing to do with you.

Lex Fridman (10:36.400)

So do you think then people like Ayn Rand believed sort of an idea of there's objective

Joscha Bach (10:42.080)

truth. So what's your sense in the philosophical, if you remove yourself as objective from the

Joscha Bach (10:48.000)

picture, you think it's possible to actually discover ideas that are true or are we just

Joscha Bach (10:52.560)

in a mesh of relative concepts that are either true nor false? It's just a giant mess.

Joscha Bach (10:57.840)

You cannot define objective truth without understanding the nature of truth in the first

Joscha Bach (11:02.720)

place. So what does the brain mean by saying that discover something as truth? So for instance,

Joscha Bach (11:08.000)

a model can be predictive or not predictive. Then there can be a sense in which a mathematical

Joscha Bach (11:14.560)

statement can be true because it's defined as true under certain conditions. So it's basically

Joscha Bach (11:19.520)

a particular state that a variable can have in a simple game. And then you can have a

Joscha Bach (11:26.320)

correspondence between systems and talk about truth, which is again, a type of model correspondence.

Lex Fridman (11:31.120)

And there also seems to be a particular kind of ground truth. So for instance,

Joscha Bach (11:34.960)

you're confronted with the enormity of something existing at all. It's stunning when you realize

Joscha Bach (11:41.200)

something exists rather than nothing. And this seems to be true. There's an absolute truth in

Joscha Bach (11:47.360)

the fact that something seems to be happening. Yeah, that to me is a showstopper. I could just

Joscha Bach (11:52.400)

think about that idea and be amazed by that idea for the rest of my life and not going any farther

Lex Fridman (11:57.840)

because I don't even know the answer to that. Why does anything exist at all?

Joscha Bach (12:01.040)

Well, the easiest answer is existence is the default, right? So this is the lowest number of

Lex Fridman (12:04.960)

bits that you would need to encode this. Whose answer?

Joscha Bach (12:07.920)

The simplest answer to this is that existence is the default.

Lex Fridman (12:11.280)

What about nonexistence? I mean, that seems...

Joscha Bach (12:14.240)

Nonexistence might not be a meaningful notion in this sense. So in some sense,

Joscha Bach (12:18.080)

if everything that can exist exists, for something to exist, it probably needs to be implementable.

Joscha Bach (12:23.440)

The only thing that can be implemented is finite automata. So maybe the whole of existence is the

Joscha Bach (12:28.240)

superposition of all finite automata and we are in some region of the fractal that has the properties

Lex Fridman (12:33.120)

that it can contain us. What does it mean to be a superposition of finite automata?

Joscha Bach (12:40.000)

Superposition of all possible rules? Imagine that every automaton is basically an operator

Joscha Bach (12:45.840)

that acts on some substrate and as a result, you get emergent patterns.

Lex Fridman (12:50.400)

What's the substrate?

Joscha Bach (12:52.640)

I have no idea to know. But some substrate.

Lex Fridman (12:55.040)

It's something that can store information.

Joscha Bach (12:58.400)

Something that can store information, there's a automaton.

Lex Fridman (13:00.320)

Something that can hold state.

Joscha Bach (13:01.680)

Still, it doesn't make sense to me the why that exists at all. I could just sit there

Lex Fridman (13:06.480)

with a beer or a vodka and just enjoy the fact, pondering the why.

Joscha Bach (13:11.280)

It may not have a why. This might be the wrong direction to ask into this. So there could be no

Joscha Bach (13:16.960)

relation in the why direction without asking for a purpose or for a cause. It doesn't mean

Joscha Bach (13:22.640)

that everything has to have a purpose or cause. So we mentioned some philosophers in that early,

Lex Fridman (13:28.720)

just taking a brief step back into that. So we asked ourselves when did classical philosophy end?

Joscha Bach (13:34.800)

I think for Germany, it largely ended with the first revolution.

Joscha Bach (13:38.240)

That's basically when we entered the monarchy and started a democracy. And at this point,

Joscha Bach (13:45.680)

we basically came up with a new form of government that didn't have a good sense of

Joscha Bach (13:50.080)

this new organism that society wanted to be. And in a way, it decapitated the universities.

Lex Fridman (13:56.080)

So the universities went on through modernism like a headless chicken.

Lex Fridman (13:59.760)

At the same time, democracy failed in Germany and we got fascism as a result.

Lex Fridman (14:04.240)

And it burned down things in a similar way as Stalinism burned down intellectual traditions

Joscha Bach (14:08.640)

in Russia. And Germany, both Germanys have not recovered from this. Eastern Germany had this

Joscha Bach (14:14.160)

vulgar dialectic materialism and Western Germany didn't get much more edgy than Habermas. So in

Joscha Bach (14:20.560)

some sense, both countries lost their intellectual traditions and killing off and driving out the

Joscha Bach (14:24.640)

Jews didn't help. Yeah. So that was the end of really rigorous what you would say is classical

Joscha Bach (14:33.680)

philosophy. There's also this thing that in some sense, the low hanging foods in philosophy

Joscha Bach (14:39.440)

were mostly wrapped. And the last big things that we discovered was the constructivist turn

Joscha Bach (14:46.240)

in mathematics. So to understand that the parts of mathematics that work are computation,

Joscha Bach (14:51.680)

there was a very significant discovery in the first half of the 20th century. And it hasn't

Joscha Bach (14:57.200)

fully permeated philosophy and even physics yet. Physicists checked out the code libraries

Joscha Bach (15:02.640)

for mathematics before constructivism became universal. What's constructivism? What are you

Joscha Bach (15:08.880)

referring to, Gödel's incompleteness theorem, those kinds of ideas? So basically, Gödel himself,

Joscha Bach (15:13.600)

I think, didn't get it yet. Hilbert could get it. Hilbert saw that, for instance, countries

Joscha Bach (15:18.880)

set theoretic experiments and mathematics led into contradictions. And he noticed that with the

Joscha Bach (15:25.120)

current semantics, we cannot build a computer in mathematics that runs mathematics without crashing.

Lex Fridman (15:30.080)

And Gödel could prove this. And so what Gödel could show is using classical mathematical

Joscha Bach (15:35.040)

semantics, you run into contradictions. And because Gödel strongly believed in these semantics and

Joscha Bach (15:40.480)

more than what he could observe and so on, he was shocked. It basically shook his world to the core

Joscha Bach (15:46.640)

because in some sense, he felt that the world has to be implemented in classical mathematics.

Lex Fridman (15:50.960)

And for Turing, it wasn't quite so bad. I think that Turing could see that the solution is to

Joscha Bach (15:56.400)

understand that mathematics was computation all along, which means you, for instance, pi

Joscha Bach (16:01.920)

in classical mathematics is a value. It's also a function, but it's the same thing. And in

Joscha Bach (16:08.080)

computation, a function is only a value when you can compute it. And if you cannot compute the last

Joscha Bach (16:13.360)

digit of pi, you only have a function. You can plug this function into your local sun, let it run

Joscha Bach (16:18.240)

until the sun burns out. This is it. This is the last digit of pi you will know. But it also means

Joscha Bach (16:22.880)

there can be no process in the physical universe or in any physically realized computer that depends

Joscha Bach (16:28.480)

on having known the last digit of pi. Which means there are parts of physics that are defined in

Joscha Bach (16:34.160)

such a way that cannot strictly be true, because assuming that this could be true leads into

Joscha Bach (16:38.160)

contradictions. So I think putting computation at the center of the world view is actually the

Joscha Bach (16:44.240)

right way to think about it. Yes. And Wittgenstein could see it. And Wittgenstein basically preempted

Joscha Bach (16:49.840)

the logitist program of AI that Minsky started later, like 30 years later. Turing was actually

Joscha Bach (16:55.360)

a pupil of Wittgenstein. I didn't know there's any connection between Turing and Wittgenstein.

Joscha Bach (17:00.480)

Wittgenstein even cancelled some classes when Turing was not present because he thought it

Joscha Bach (17:03.680)

was not worth spending the time with the others. If you read the Tractatus, it's a very beautiful

Joscha Bach (17:09.680)

book, like basically one thought on 75 pages. It's very non typical for philosophy because it doesn't

Joscha Bach (17:15.040)

have arguments in it and it doesn't have references in it. It's just one thought that is not intending

Joscha Bach (17:21.600)

to convince anybody. He says, it's mostly for people that had the same insight as me,

Joscha Bach (17:26.320)

just spell it out. And this insight is there is a way in which mathematics and philosophy

Joscha Bach (17:32.080)

ought to meet. Mathematics tries to understand the domain of all languages by starting with those

Joscha Bach (17:37.040)

that are so formalizable that you can prove all the properties of the statements that you make.

Lex Fridman (17:42.560)

But the price that you pay is that your language is very, very simple. So it's very hard to say

Joscha Bach (17:46.960)

something meaningful in mathematics. And it looks complicated to people, but it's far less complicated

Joscha Bach (17:52.320)

than what our brain is casually doing all the time when it makes sense of reality. And philosophy is

Joscha Bach (17:58.320)

coming from the top. So it's mostly starting from natural languages with vaguely defined concepts.

Lex Fridman (18:03.600)

And the hope is that mathematics and philosophy can meet at some point. And Wittgenstein was trying

Joscha Bach (18:08.720)

to make them meet. And he already understood that, for instance, you could express everything with

Joscha Bach (18:12.320)

the NAND calculus, that you could reduce the entire logic to NAND gates as we do in our modern

Joscha Bach (18:17.440)

computers. So in some sense, he already understood Turing universality before Turing spelled it out.

Joscha Bach (18:22.320)

I think when he wrote the Tractatus, he didn't understand yet that the idea was so important

Lex Fridman (18:26.640)

and significant. And I suspect then when Turing wrote it out, nobody cared that much. Turing was

Joscha Bach (18:32.400)

not that famous when he lived. It was mostly his work in decrypting the German codes that made him

Joscha Bach (18:39.280)

famous or gave him some notoriety. But this saint status that he has to computer science right now

Lex Fridman (18:44.400)

and the AI is something that I think he could acquire later. That's kind of interesting. Do

Joscha Bach (18:48.720)

you think of computation and computer science? And you kind of represent that to me is maybe

Joscha Bach (18:53.040)

that's the modern day. You in a sense are the new philosopher by sort of the computer scientist

Joscha Bach (19:00.720)

who dares to ask the bigger questions that philosophy originally started is the new

Joscha Bach (19:06.000)

philosopher. Certainly not me. I think I'm mostly still this child that grows up in a

Joscha Bach (19:12.240)

very beautiful valley and looks at the world from the outside and tries to understand what's going

Joscha Bach (19:16.160)

on. And my teachers tell me things and they largely don't make sense. So I have to make my

Joscha Bach (19:20.560)

own models. I have to discover the foundations of what the others are saying. I have to try to fix

Joscha Bach (19:25.040)

them to be charitable. I try to understand what they must have thought originally or what their

Joscha Bach (19:30.160)

teachers or their teacher's teachers must have thought until everything got lost in translation

Lex Fridman (19:34.240)

and how to make sense of the reality that we are in. And whenever I have an original idea,

Joscha Bach (19:38.880)

I'm usually late to the party by say 400 years. And the only thing that's good is that the parties

Joscha Bach (19:43.360)

get smaller and smaller the older I get and the more I explore it. The parties get smaller and

Joscha Bach (19:49.120)

more exclusive and more exclusive. So it seems like one of the key qualities of your upbringing

Joscha Bach (19:55.680)

was that you were not tethered, whether it's because of your parents or in general,

Joscha Bach (1:00:03.600)

In the early days of automated language translation, there was an example that I

Joscha Bach (1:00:08.960)

found particularly funny where somebody tried to translate a text from English into German

Lex Fridman (1:00:13.840)

and it was a bat broke the window. And the translation in German was

Joscha Bach (1:00:25.760)

to translate back into English a bat, this flying mammal broke the window with a baseball bat.

Lex Fridman (1:00:31.520)

Yes. And it seemed to be the

Joscha Bach (1:00:34.160)

most similar to this program because it somehow maximized the possibility of translating the

Joscha Bach (1:00:39.840)

concept bat into German in the same sentence. And this is a mistake that the transformer model is

Joscha Bach (1:00:45.280)

not doing because it's tracking identity. And the attentional mechanism in the transformer

Joscha Bach (1:00:49.840)

model is basically putting its finger on individual concepts and make sure that these concepts pop up

Joscha Bach (1:00:56.160)

later in the text and tracks basically the individuals through the text. And it's why

Joscha Bach (1:01:02.560)

the system can learn things that other systems couldn't before it, which makes it, for instance,

Joscha Bach (1:01:07.120)

possible to write a text where it talks about the scientist, then the scientist has a name

Lex Fridman (1:01:10.960)

and has a pronoun and it gets a consistent story about that thing. What it does not do,

Joscha Bach (1:01:16.160)

it doesn't fully integrate this. So this meaning falls apart at some point, it loses track of this

Joscha Bach (1:01:20.800)

context. It does not yet understand that everything that it says has to refer to the same universe.

Lex Fridman (1:01:25.920)

And this is where this thing falls apart. But the attention in the transformer model does not go

Joscha Bach (1:01:31.520)

beyond tracking identity. And tracking identity is an important part of attention, but it's a

Joscha Bach (1:01:36.240)

different, very specific attentional mechanism. And it's not the one that gives rise to the type

Lex Fridman (1:01:41.360)

of consciousness that we have. Just to linger on, what do you mean by

Joscha Bach (1:01:44.800)

identity in the context of language? So when you talk about language,

Lex Fridman (1:01:49.360)

you have different words that can refer to the same concept.

Joscha Bach (1:01:52.640)

Got it. And in the sense that...

Lex Fridman (1:01:53.680)

The space of concepts. So... Yes. And it can also be in a nominal sense

Joscha Bach (1:01:59.040)

or in an inexical sense that you say this word does not only refer to this class of objects,

Lex Fridman (1:02:05.520)

but it refers to a definite object, to some kind of agent that waves their way through the story.

Lex Fridman (1:02:11.520)

And it's only referred by different ways in the language. So the language is basically a

Joscha Bach (1:02:16.720)

projection from a conceptual representation from a scene that is evolving into a discrete string

Joscha Bach (1:02:23.680)

of symbols. And what the transformer is able to do, it learns aspects of this projection

Joscha Bach (1:02:29.680)

mechanism that other models couldn't learn. So have you ever seen an artificial intelligence

Joscha Bach (1:02:34.720)

or any kind of construction idea that allows for, unlike neural networks or perhaps within

Joscha Bach (1:02:40.720)

neural networks, that's able to form something where the space of concepts continues to be

Joscha Bach (1:02:47.440)

integrated? So what you're describing, building a knowledge base, building this consistent,

Lex Fridman (1:02:54.080)

larger and larger sets of ideas that would then allow for deeper understanding.

Joscha Bach (1:02:59.200)

Wittgenstein thought that we can build everything from language,

Lex Fridman (1:03:02.640)

from basically a logical grammatical construct. And I think to some degree,

Joscha Bach (1:03:07.360)

this was also what Minsky believed. So that's why he focused so much on common sense reasoning and

Lex Fridman (1:03:12.560)

so on. And a project that was inspired by him was Psych. That was basically going on.

Joscha Bach (1:03:19.600)

Yes. Of course, ideas don't die. Only people die.

Lex Fridman (1:03:25.760)

That's true.

Lex Fridman (1:03:27.120)

And Alt Psych is a productive project. It's just probably not one that is going to

Lex Fridman (1:03:31.760)

converge to general intelligence. The thing that Wittgenstein couldn't solve,

Lex Fridman (1:03:35.440)

and he looked at this in his book at the end of his life, Philosophical Investigations,

Joscha Bach (1:03:40.080)

was the notion of images. So images play an important role in Tractatus. The Tractatus is

Joscha Bach (1:03:44.960)

an attempt to basically turn philosophy into logical probing language, to design a logical

Joscha Bach (1:03:49.360)

language in which you can do actual philosophy that's rich enough for doing this. And the

Joscha Bach (1:03:54.240)

difficulty was to deal with perceptual content. And eventually, I think he decided that he was

Joscha Bach (1:04:00.480)

not able to solve it. And I think this preempted the failure of the logitist program in AI.

Lex Fridman (1:04:06.400)

And the solution, as we see it today, is we need more general function approximation. There are

Joscha Bach (1:04:11.280)

geometric functions that we learn to approximate that cannot be efficiently expressed and computed

Joscha Bach (1:04:16.960)

in a grammatical language. We can, of course, build automata that go via number theory and so on

Lex Fridman (1:04:22.240)

to learn in algebra and then compute an approximation of this geometry.

Lex Fridman (1:04:26.080)

But to equate language and geometry is not an efficient way to think about it.

Lex Fridman (1:04:32.080)

So function, well, you kind of just said that neural networks are sort of, the approach that

Joscha Bach (1:04:37.360)

neural networks takes is actually more general than what can be expressed through language.

Joscha Bach (1:04:44.640)

Yes. So what can be efficiently expressed through language at the data rates at which

Lex Fridman (1:04:49.840)

we process grammatical language?

Lex Fridman (1:04:51.360)

Okay. So you don't think languages, so you disagree with Wittgenstein,

Joscha Bach (1:04:56.880)

that language is not fundamental to...

Lex Fridman (1:04:58.720)

I agree with Wittgenstein. I just agree with the late Wittgenstein.

Lex Fridman (1:05:03.680)

And I also agree with the beauty of the early Wittgenstein. I think that the Tractatus itself

Joscha Bach (1:05:09.520)

is probably the most beautiful philosophical text that was written in the 20th century.

Lex Fridman (1:05:13.920)

But language is not fundamental to cognition and intelligence and consciousness.

Lex Fridman (1:05:18.480)

So I think that language is a particular way or the natural language that we're using is a

Joscha Bach (1:05:23.520)

particular level of abstraction that we use to communicate with each other. But the languages

Joscha Bach (1:05:28.800)

in which we express geometry are not grammatical languages in the same sense. So they work slightly

Joscha Bach (1:05:35.040)

differently, more general expressions of functions. And I think the general nature of a model is you

Joscha Bach (1:05:41.120)

have a bunch of parameters. These have a range, these are the variances of the world, and you have

Joscha Bach (1:05:47.280)

relationships between them, which are constraints, which say if certain parameters have these values,

Joscha Bach (1:05:52.400)

then other parameters have to have the following values. And this is a very early insight in

Joscha Bach (1:05:58.880)

computer science. And I think some of the earliest formulations is the Boltzmann machine.

Lex Fridman (1:06:03.600)

And the problem with the Boltzmann machine is that it has a measure of whether it's good. This

Joscha Bach (1:06:07.680)

is basically the energy on the system, the amount of tension that you have left in the constraints

Joscha Bach (1:06:11.600)

where the constraints don't quite match. It's very difficult to, despite having this global

Joscha Bach (1:06:16.960)

measure, to train it. Because as soon as you add more than trivially few elements,

Joscha Bach (1:06:22.720)

parameters into the system, it's very difficult to get it settled in the right architecture.

Lex Fridman (1:06:27.760)

And so the solution that Hinton and Zanofsky found was to use a restricted Boltzmann machine,

Joscha Bach (1:06:34.960)

which uses the hidden links, the internal links in the Boltzmann machine and only has

Joscha Bach (1:06:39.760)

basically input and output layer. But this limits the expressivity of the Boltzmann machine. So now

Joscha Bach (1:06:44.960)

he builds a network of these primitive Boltzmann machines. And in some sense, you can see almost

Joscha Bach (1:06:50.320)

continuous development from this to the deep learning models that we're using today,

Joscha Bach (1:06:54.640)

even though we don't use Boltzmann machines at this point. But the idea of the Boltzmann

Joscha Bach (1:06:58.400)

machine is you take this model, you clamp some of the values to perception, and this forces

Joscha Bach (1:07:02.640)

the entire machine to go into a state that is compatible with the states that you currently

Joscha Bach (1:07:06.240)

perceive. And this state is your model of the world. I think it's a very general way of thinking

Joscha Bach (1:07:12.960)

about models, but we have to use a different approach to make it work. We have to find

Joscha Bach (1:07:19.920)

different networks that train the Boltzmann machine. So the mechanism that trains the Boltzmann

Joscha Bach (1:07:23.920)

machine and the mechanism that makes the Boltzmann machine settle into its state

Joscha Bach (1:07:28.320)

are distinct from the constrained architecture of the Boltzmann machine itself.

Lex Fridman (1:07:33.360)

The kind of mechanisms that we want to develop, you're saying?

Joscha Bach (1:07:36.480)

Yes. So the direction in which I think our research is going to go

Joscha Bach (1:07:40.320)

is going to, for instance, what you notice in perception is our perceptual models of the world

Joscha Bach (1:07:45.920)

are not probabilistic, but possibilistic, which means you should be able to perceive things that

Joscha Bach (1:07:50.880)

are improbable, but possible. A perceptual state is valid, not if it's probable, but if it's

Joscha Bach (1:07:57.200)

possible, if it's coherent. So if you see a tiger coming after you, you should be able to see this

Joscha Bach (1:08:02.560)

even if it's unlikely. And the probability is necessary for convergence of the model. So

Joscha Bach (1:08:09.120)

given the state of possibilities that is very, very large and a set of perceptual features,

Lex Fridman (1:08:15.520)

how should you change the states of the model to get it to converge with your perception?

Lex Fridman (1:08:21.840)

But the space of ideas that are coherent with the context that you're sensing

Lex Fridman (1:08:29.680)

is perhaps not as large. I mean, that's perhaps pretty small.

Joscha Bach (1:08:34.160)

The degree of coherence that you need to achieve depends, of course, how deep your models go.

Joscha Bach (1:08:39.920)

That is, for instance, politics is very simple when you know very little about game theory and

Lex Fridman (1:08:44.560)

human nature. So the younger you are, the more obvious it is how politics would work, right?

Lex Fridman (1:08:49.600)

And because you get a coherent aesthetics from relatively few inputs. And the more layers you

Joscha Bach (1:08:55.520)

model, the more layers you model reality, the harder it gets to satisfy all the constraints.

Joscha Bach (1:09:00.880)

So, you know, the current neural networks are fundamentally supervised learning system with

Joscha Bach (1:09:07.040)

a feed forward neural network is back propagation to learn. What's your intuition about what kind

Lex Fridman (1:09:12.240)

of mechanisms might we move towards to improve the learning procedure?

Joscha Bach (1:09:18.640)

I think one big aspect is going to be meta learning and architecture search starts in

Joscha Bach (1:09:22.960)

this direction. In some sense, the first wave of classical AI work by identifying a problem

Lex Fridman (1:09:28.560)

and a possible solution and implementing the solution, right? A program that plays chess.

Lex Fridman (1:09:32.880)

And right now we are in the second wave of AI. So instead of writing the algorithm that

Joscha Bach (1:09:37.840)

implements the solution, we write an algorithm that automatically searches

Joscha Bach (1:09:41.600)

for an algorithm that implements the solution. So the learning system in some sense is an

Joscha Bach (1:09:46.400)

algorithm that itself discovers the algorithm that solves the problem, like Go. Go is too hard

Joscha Bach (1:09:51.600)

to implement the solution by hand, but we can implement an algorithm that finds the solution.

Joscha Bach (1:09:56.240)

Yeah. So now let's move to the third stage, right? The third stage would be meta learning.

Lex Fridman (1:10:00.880)

Find an algorithm that discovers a learning algorithm for the given domain.

Joscha Bach (1:10:05.040)

Our brain is probably not a learning system, but a meta learning system.

Joscha Bach (1:10:08.640)

This is one way of looking at what we are doing. There is another way. If you look at the way our

Joscha Bach (1:10:13.840)

brain is, for instance, implemented, there is no central control that tells all the neurons how

Joscha Bach (1:10:18.160)

to wire up. Instead, every neuron is an individual reinforcement learning agent. Every neuron is a

Joscha Bach (1:10:24.000)

single celled organism that is quite complicated and in some sense quite motivated to get fed.

Lex Fridman (1:10:28.960)

And it gets fed if it fires on average at the right time. And the right time depends on the

Joscha Bach (1:10:36.160)

context that the neuron exists in, which is the electrical and chemical environment that it has.

Lex Fridman (1:10:42.080)

So it basically has to learn a function over its environment that tells us when to fire to get fed.

Joscha Bach (1:10:48.240)

Or if you see it as a reinforcement learning agent, every neuron is in some sense making a

Joscha Bach (1:10:52.720)

hypothesis when it sends a signal and tries to pipe a signal through the universe and tries to

Joscha Bach (1:10:57.440)

get positive feedback for it. And the entire thing is set up in such a way that it's robustly

Joscha Bach (1:11:02.560)

self organizing into a brain, which means you start out with different neuron types

Joscha Bach (1:11:07.200)

that have different priors on which hypothesis to test and how to get its reward.

Lex Fridman (1:11:12.080)

And you put them into different concentrations in a certain spatial alignment,

Lex Fridman (1:11:16.320)

and then you entrain it in a particular order. And as a result, you get a well organized brain.

Joscha Bach (1:11:22.080)

Yeah, so okay, so the brain is a meta learning system with a bunch of reinforcement learning

Joscha Bach (1:11:29.120)

agents. And what I think you said, but just to clarify, there's no centralized government that

Joscha Bach (1:11:40.000)

tells you, here's a loss function, here's a loss function, here's a loss function. Who says what's

Joscha Bach (1:11:48.240)

the objective? There are also governments which impose loss functions on different parts of the

Joscha Bach (1:11:52.560)

brain. So we have differential attention. Some areas in your brain get specially rewarded when

Joscha Bach (1:11:56.880)

you look at faces. If you don't have that, you will get prosopagnosia, which basically

Joscha Bach (1:12:01.600)

the inability to tell people apart by their faces. And the reason that happens is because

Joscha Bach (1:12:07.360)

it had an evolutionary advantage. So like evolution comes into play here. But it's basically an

Joscha Bach (1:12:12.240)

extraordinary attention that we have for faces. I don't think that people with prosopagnosia have a

Joscha Bach (1:12:17.440)

perceived defective brain, the brain just has an average attention for faces. So people with

Joscha Bach (1:12:22.240)

prosopagnosia don't look at faces more than they look at cups. So the level at which they resolve

Joscha Bach (1:12:27.040)

the geometry of faces is not higher than for cups. And people that don't have prosopagnosia look

Joscha Bach (1:12:34.400)

obsessively at faces, right? For you and me, it's impossible to move through a crowd

Joscha Bach (1:12:38.080)

without scanning the faces. And as a result, we make insanely detailed models of faces that allow

Joscha Bach (1:12:43.440)

us to discern mental states of people. So obviously, we don't know 99% of the details

Joscha Bach (1:12:49.840)

of this meta learning system. That's our mind. Okay. But still, we took a leap from something

Joscha Bach (1:12:55.920)

much dumber to that from through the evolutionary process. Can you first of all, maybe say how hard

Joscha Bach (1:13:04.000)

the, how big of a leap is that from our brain, from our ancestors to multi cell organisms? And

Joscha Bach (1:13:14.560)

is there something we can think about? As we start to think about how to engineer intelligence,

Joscha Bach (1:13:21.680)

is there something we can learn from evolution? In some sense, life exists because of the market

Joscha Bach (1:13:28.880)

opportunity of controlled chemical reactions. We compete with dump chemical reactions and we win

Joscha Bach (1:13:34.960)

in some areas against this dump combustion because we can harness those entropy gradients where you

Joscha Bach (1:13:40.000)

need to add a little bit of energy in a specific way to harvest more energy. So we out competed

Joscha Bach (1:13:44.640)

combustion. Yes, in many regions we do and we try very hard because when we are in direct

Joscha Bach (1:13:49.520)

competition, we lose, right? So because the combustion is going to close the entropy

Joscha Bach (1:13:55.120)

gradients much faster than we can run. So basically we do this because every cell has

Joscha Bach (1:14:02.800)

a Turing machine built into it. It's like literally a read write head on the tape.

Lex Fridman (1:14:09.680)

So everything that's more complicated than a molecule that just is a vortex around attractors

Joscha Bach (1:14:16.400)

that needs a Turing machine for its regulation. And then you bind cells together and you get next

Joscha Bach (1:14:21.920)

level organizational organism where the cells together implement some kind of software.

Joscha Bach (1:14:28.880)

For me, a very interesting discovery in the last year was the word spirit because I realized that

Lex Fridman (1:14:33.120)

what spirit actually means is an operating system for an autonomous robot. And when the word was

Joscha Bach (1:14:38.640)

invented, people needed this word. But they didn't have robots that they built themselves yet. The

Joscha Bach (1:14:43.520)

only autonomous robots that were known were people, animals, plants, ecosystems, cities,

Lex Fridman (1:14:48.080)

and so on. And they all had spirits. And it makes sense to say that the plant has an operating

Joscha Bach (1:14:53.040)

system, right? If you pinch the plant in one area, then it's going to have repercussions

Joscha Bach (1:14:57.280)

throughout the plant. Everything in the plant is in some sense connected into some global aesthetics

Joscha Bach (1:15:02.160)

like in other organisms. An organism is not a collection of cells, it's a function that

Joscha Bach (1:15:07.120)

tells cells how to behave. And this function is not implemented as some kind of supernatural thing,

Joscha Bach (1:15:13.600)

like some morphogenetic field. It is an emergent result of the interactions of each cell with each

Joscha Bach (1:15:19.360)

other cell. Oh my God. So what you're saying is the organism is a function that tells what to do

Lex Fridman (1:15:31.040)

and the function emerges from the interaction of the cells. Yes. So it's basically a description

Joscha Bach (1:15:39.840)

of what the plant is doing in terms of microstates. And the microstates, the physical implementation

Joscha Bach (1:15:46.640)

are too many of them to describe them. So the software that we use to describe what the plant is

Joscha Bach (1:15:51.760)

doing, the spirit of the plant is the software, the operating system of the plant, right? This is

Joscha Bach (1:15:57.520)

a way in which we, the observers, make sense of the plant. And the same is true for people. So

Joscha Bach (1:16:03.120)

people have spirits, which is their operating system in a way, right? And there's aspects of

Joscha Bach (1:16:07.120)

that operating system that relate to how your body functions and others, how you socially interact,

Lex Fridman (1:16:12.080)

how you interact with yourself and so on. And we make models of that spirit. And we think it's a

Joscha Bach (1:16:18.480)

loaded term because it's from a pre scientific age. But it took the scientific age a long time

Joscha Bach (1:16:24.320)

to rediscover a term that is pretty much the same thing. And I suspect that the differences that we

Joscha Bach (1:16:29.440)

still see between the old word and the new word are translation errors that have happened over

Joscha Bach (1:16:33.840)

the centuries. Can you actually linger on that? Why do you say that spirit, just to clarify,

Joscha Bach (1:16:39.760)

because I'm a little bit confused. So the word spirit is a powerful thing. But why did you say

Joscha Bach (1:16:45.120)

in the last year or so that you discovered this? Do you mean the same old traditional idea of a

Joscha Bach (1:16:50.160)

spirit? I try to find out what people mean by spirit. When people say spirituality in the US,

Joscha Bach (1:16:56.240)

it usually refers to the phantom limb that they develop in the absence of culture.

Lex Fridman (1:17:00.240)

And a culture is in some sense, you could say the spirit of a society that is long game. This thing

Joscha Bach (1:17:06.880)

that is become self aware at a level above the individuals where you say, if you don't do the

Joscha Bach (1:17:12.720)

following things, then the grand, grand, grand grandchildren of our children will have nothing

Joscha Bach (1:17:16.960)

to eat. So if you take this long scope, where you try to maximize the length of the game that you

Joscha Bach (1:17:22.800)

are playing as a species, you realize that you're part of a larger thing that you cannot fully

Joscha Bach (1:17:27.040)

control. You probably need to submit to the ecosphere instead of trying to completely control

Joscha Bach (1:17:32.560)

it. There needs to be a certain level at which we can exist as a species if you want to endure.

Lex Fridman (1:17:39.440)

And our culture is not sustaining this anymore. We basically made this bet with the industrial

Joscha Bach (1:17:44.320)

revolution that we can control everything. And the modernist societies with basically unfettered

Joscha Bach (1:17:48.800)

growth led to a situation in which we depend on the ability to control the entire planet.

Lex Fridman (1:17:54.800)

And since we are not able to do that, as it seems, this culture will die. And we realize that it

Joscha Bach (1:18:00.960)

doesn't have a future, right? We called our children generation Z. That's a very optimistic

Joscha Bach (1:18:06.320)

thing to do. Yeah. So you can have this kind of intuition that our civilization, you said culture,

Lex Fridman (1:18:13.920)

but you really mean the spirit of the civilization, the entirety of the civilization may not exist

Joscha Bach (1:18:22.640)

for long. Yeah. Can you untangle that? What's your intuition behind that? So you kind of offline

Joscha Bach (1:18:29.840)

mentioned to me that the industrial revolution was kind of the moment we agreed to accept

Joscha Bach (1:18:36.400)

the offer sign on the paper on the dotted line with the industrial revolution, we doomed ourselves.

Lex Fridman (1:18:42.640)

Can you elaborate on that? This is a suspicion. I, of course, don't know how it plays out. But

Joscha Bach (1:18:47.520)

it seems to me that in a society in which you leverage yourself very far over an entropic abyss

Joscha Bach (1:18:55.840)

without land on the other side, it's relatively clear that your cantilever is at some point

Joscha Bach (1:19:00.560)

going to break down into this entropic abyss. And you have to pay the bill. Okay. Russia is

Lex Fridman (1:19:06.240)

my first language. And I'm also an idiot. Me too. This is just two apes.

Joscha Bach (1:19:13.840)

Instead of playing with a banana, trying to have fun by talking. Okay. Anthropic what? And what's

Joscha Bach (1:19:21.840)

entropic? Entropic. So entropic in the sense of entropy. Oh, entropic. Got it. And entropic,

Lex Fridman (1:19:29.200)

what was the other word you used? Abyss. What's that? It's a big gorge. Oh, abyss. Abyss, yes.

Joscha Bach (1:19:35.360)

Entropic abyss. So many of the things you say are poetic. It's hurting my ears. And this one

Joscha Bach (1:19:39.920)

is amazing, right? It's mispronounced, which makes you more poetic. Wittgenstein would be proud. So

Joscha Bach (1:19:50.080)

entropic abyss. Okay. Let's rewind then. The industrial revolution. So how does that get us

Joscha Bach (1:19:58.320)

into the entropic abyss? So in some sense, we burned a hundred million years worth of trees

Joscha Bach (1:20:04.640)

to get everybody plumbing. Yes. And the society that we had before that had a very limited number

Joscha Bach (1:20:10.480)

of people. So basically since zero BC, we hovered between 300 and 400 million people. Yes. And this

Joscha Bach (1:20:18.560)

only changed with the enlightenment and the subsequent industrial revolution. And in some

Joscha Bach (1:20:24.480)

sense, the enlightenment freed our rationality and also freed our norms from the preexisting order

Joscha Bach (1:20:30.560)

gradually. It was a process that basically happened in feedback loops. So it was not that

Joscha Bach (1:20:35.600)

just one caused the other. It was a dynamic that started. And the dynamic worked by basically

Joscha Bach (1:20:41.440)

increasing productivity to such a degree that we could feed all our children. And I think the

Joscha Bach (1:20:48.560)

definition of poverty is that you have as many children as you can feed before they die, which is

Joscha Bach (1:20:55.760)

in some sense, the state that all animals on earth are in. The definition of poverty is having

Joscha Bach (1:21:01.600)

enough. So you can have only so many children as you can feed and if you have more, they die. Yes.

Lex Fridman (1:21:06.480)

And in our societies, you can basically have as many children as you want, they don't die. Right.

Lex Fridman (1:21:12.240)

So the reason why we don't have as many children as we want is because we also have to pay a price

Joscha Bach (1:21:17.680)

in terms of we have to insert ourselves in a lower social stratum if we have too many children.

Lex Fridman (1:21:22.080)

So basically everybody in the under middle and lower upper class has only a limited number of

Joscha Bach (1:21:28.240)

children because having more of them would mean a big economic hit to the individual families.

Joscha Bach (1:21:33.680)

Yes. Because children, especially in the US, super expensive to have. And you only are taken out of

Joscha Bach (1:21:39.040)

this if you are basically super rich or if you are super poor. If you're super poor, it doesn't

Joscha Bach (1:21:43.440)

matter how many kids you have because your status is not going to change. And these children allow

Joscha Bach (1:21:48.080)

you not going to die of hunger. So how does this lead to self destruction? So there's a lot of

Joscha Bach (1:21:54.320)

unpleasant properties about this process. So basically what we try to do is we try to

Joscha Bach (1:21:58.800)

let our children survive, even if they have diseases. Like I would have died before my

Joscha Bach (1:22:06.240)

mid twenties without modern medicine. And most of my friends would have as well. And so many of us

Joscha Bach (1:22:12.080)

wouldn't live without the advantages of modern medicine and modern industrialized society. We

Joscha Bach (1:22:18.880)

get our protein largely by subduing the entirety of nature. Imagine there would be some very clever

Joscha Bach (1:22:25.760)

microbe that would live in our organisms and would completely harvest them and change them into a

Joscha Bach (1:22:32.960)

thing that is necessary to sustain itself. And it would discover that for instance,

Joscha Bach (1:22:38.640)

brain cells are kind of edible, but they're not quite nice. So you need to have more fat in them

Lex Fridman (1:22:43.440)

and you turn them into more fat cells. And basically this big organism would become a

Joscha Bach (1:22:47.600)

vegetable that is barely alive and it's going to be very brittle and not resilient when the

Joscha Bach (1:22:52.000)

environment changes. Yeah, but some part of that organism, the one that's actually doing all the

Joscha Bach (1:22:57.280)

using of the, there'll still be somebody thriving. So it relates back to this original question

Joscha Bach (1:23:04.560)

I suspect that we are not the smartest thing on this planet. I suspect that basically every complex

Joscha Bach (1:23:10.400)

system has to have some complex regulation if it depends on feedback loops. And so for instance,

Joscha Bach (1:23:17.840)

it's likely that we should describe a certain degree of intelligence to plants. The problem is

Joscha Bach (1:23:24.320)

that plants don't have a nervous system. So they don't have a way to telegraph messages over large

Joscha Bach (1:23:28.960)

distances almost instantly in the plant. And instead, they will rely on chemicals between

Joscha Bach (1:23:34.640)

adjacent cells, which means the signal processing speed depends on the signal processing with a

Joscha Bach (1:23:40.160)

rate of a few millimeters per second. And as a result, if the plant is intelligent,

Lex Fridman (1:23:46.320)

it's not going to be intelligent at similar timescales as this.

Joscha Bach (1:23:49.200)

Yeah, the time scale is different. So you suspect we might not be the most intelligent

Lex Fridman (1:23:55.200)

but we're the most intelligent in this spatial scale in our timescale.

Lex Fridman (1:24:00.000)

So basically, if you would zoom out very far, we might discover that there have been intelligent

Joscha Bach (1:24:05.440)

ecosystems on the planet that existed for thousands of years in an almost undisturbed state. And it

Joscha Bach (1:24:12.080)

could be that these ecosystems actively related their environment. So basically change the course

Joscha Bach (1:24:16.880)

of the evolution vision, this ecosystem to make it more efficient in the future.

Lex Fridman (1:24:20.640)

So it's possible something like plants is actually a set of living organisms,

Joscha Bach (1:24:25.440)

an ecosystem of living organisms that are just operating a different timescale and are far

Joscha Bach (1:24:30.080)

superior in intelligence than human beings. And then human beings will die out and plants will

Lex Fridman (1:24:34.560)

still be there and they'll be there.

Joscha Bach (1:24:36.880)

Yeah, there's an evolutionary adaptation playing a role at all of these levels. For instance,

Lex Fridman (1:24:41.920)

if mice don't get enough food and get stressed, the next step is to

Joscha Bach (1:24:45.840)

get more sparse and more scrawny. And the reason for this is because they in a natural

Joscha Bach (1:24:51.280)

environment, the mice have probably hidden a drought or something else. And if they're overgrazed,

Joscha Bach (1:24:56.400)

then all the things that sustain them might go extinct. And there will be no mice a few

Joscha Bach (1:25:01.200)

generations from now. So to make sure that there will be mice in five generations from now,

Joscha Bach (1:25:05.600)

basically the mice scale back. And a similar thing happens with the predators of mice.

Joscha Bach (1:25:10.640)

They should make sure that the mice don't completely go extinct. So in some sense, if the predators are

Joscha Bach (1:25:15.840)

smart enough, they will be tasked with shepherding their food supply. Maybe the reason why lions have

Joscha Bach (1:25:22.800)

much larger brains than antelopes is not so much because it's so hard to catch an antelope as

Joscha Bach (1:25:27.920)

opposed to run away from the lion. But the lions need to make complex models of their environment,

Lex Fridman (1:25:33.680)

more complex than the antelopes.

Lex Fridman (1:25:35.360)

So first of all, just describing that there's a bunch of complex systems and human beings may not

Joscha Bach (1:25:40.560)

even be the most special or intelligent of those complex systems, even on Earth, makes me feel a

Joscha Bach (1:25:45.600)

little better about the extinction of human species that we're talking about.

Lex Fridman (1:25:48.800)

Yes, maybe you're just Guy Astploit to put the carbon back into the atmosphere.

Joscha Bach (1:25:52.160)

Yeah, this is just a nice, we tried it out.

Joscha Bach (1:25:54.880)

The big stain on evolution is not us, it was trees. Earth evolved trees before there could be

Joscha Bach (1:26:00.160)

digested again. There were no insects that could break all of them apart. Cellulose is so robust

Joscha Bach (1:26:05.680)

that you cannot get all of it with microorganisms. So many of these trees fell into swamps and all

Joscha Bach (1:26:11.520)

this carbon became inert and could no longer be recycled into organisms. And we are the species

Lex Fridman (1:26:16.160)

that is destined to take care of that.

Lex Fridman (1:26:17.920)

So this is kind of...

Joscha Bach (1:26:20.160)

To get out of the ground, put it back into the atmosphere and the Earth is already greening.

Lex Fridman (1:26:24.320)

So we have to be careful about that.

Joscha Bach (1:26:26.400)

To get out of the ground, put it back into the atmosphere and the Earth is already greening.

Lex Fridman (1:26:30.240)

So within a million years or so when the ecosystems have recovered from the rapid changes,

Joscha Bach (1:26:35.680)

that they're not compatible with right now, the Earth is going to be awesome again.

Lex Fridman (1:26:39.600)

And there won't be even a memory of us, of us little apes.

Joscha Bach (1:26:42.400)

I think there will be memories of us. I suspect we are the first generally intelligent species

Joscha Bach (1:26:46.480)

in the sense. We are the first species within industrial society because we will leave more

Lex Fridman (1:26:51.040)

phones than bones in the stratosphere.

Joscha Bach (1:26:53.360)

Phones than bones. I like it. But then let me push back. You've kind of suggested that

Joscha Bach (1:27:01.040)

we have a very narrow definition of... I mean, why aren't trees a higher level of general

Lex Fridman (1:27:08.800)

intelligence?

Joscha Bach (1:27:09.440)

If trees were intelligent, then they would be at different timescales, which means within

Joscha Bach (1:27:13.760)

a hundred years, the tree is probably not going to make models that are as complex as

Lex Fridman (1:27:17.520)

the ones that we make in 10 years.

Lex Fridman (1:27:18.800)

But maybe the trees are the ones that made the phones, right?

Joscha Bach (1:27:25.520)

You could say the entirety of life did it. The first cell never died. The first cell

Joscha Bach (1:27:31.120)

only split, right? And every cell in our body is still an instance of the first cell that

Joscha Bach (1:27:36.640)

split off from that very first cell. There was only one cell on this planet as far as

Joscha Bach (1:27:40.240)

we know. And so the cell is not just a building block of life. It's a hyperorganism. And we

Lex Fridman (1:27:46.480)

are part of this hyperorganism.

Lex Fridman (1:27:49.600)

So nevertheless, this hyperorganism, no, this little particular branch of it, which is us

Joscha Bach (1:27:56.720)

humans, because of the industrial revolution and maybe the exponential growth of technology

Joscha Bach (1:28:02.480)

might somehow destroy ourselves. So what do you think is the most likely way we might

Joscha Bach (1:28:07.840)

destroy ourselves? So some people worry about genetic manipulation. Some people, as we've

Joscha Bach (1:28:13.200)

talked about, worry about either dumb artificial intelligence or super intelligent artificial

Joscha Bach (1:28:18.400)

intelligence destroying us. Some people worry about nuclear weapons and weapons of war in

Joscha Bach (1:28:25.200)

general. What do you think? If you were a betting man, what would you bet on in terms

Lex Fridman (1:28:29.920)

of self destruction? And then would it be higher than 50%?

Lex Fridman (1:28:34.800)

So it's very likely that nothing that we bet on matters after we win our bets. So I

Lex Fridman (1:28:40.960)

don't think that bets are literally the right way to go about this.

Joscha Bach (1:28:44.000)

I mean, once you're dead, you won't be there to collect the wings.

Lex Fridman (1:28:47.440)

So it's also not clear if we as a species go extinct. But I think that our present

Joscha Bach (1:28:53.040)

civilization is not sustainable. So the thing that will change is there will be probably

Joscha Bach (1:28:57.120)

fewer people on the planet than there are today. And even if not, then still most of

Joscha Bach (1:29:01.920)

people that are alive today will not have offspring in 100 years from now because of

Joscha Bach (1:29:05.600)

the geographic changes and so on and the changes in the food supply. It's quite likely

Joscha Bach (1:29:10.880)

that many areas of the planet will only be livable with a close cooling chain in 100

Joscha Bach (1:29:15.760)

years from now. So many of the areas around the equator and in subtropical climates that

Joscha Bach (1:29:22.400)

are now quite pleasant to live in, will stop to be inhabitable without air conditioning.

Lex Fridman (1:29:27.520)

So you honestly, wow, cooling chain, close knit cooling chain communities. So you think

Lex Fridman (1:29:33.840)

you have a strong worry about the effects of global warming?

Joscha Bach (1:29:38.000)

By itself, it's not a big issue. If you live in Arizona right now, you have basically three

Joscha Bach (1:29:42.480)

months in the summer in which you cannot be outside. And so you have a close cooling chain.

Joscha Bach (1:29:47.440)

You have air conditioning in your car and in your home and you're fine. And if the air

Joscha Bach (1:29:50.560)

conditioning would stop for a few days, then in many areas you would not be able to survive.

Joscha Bach (1:29:56.560)

Can we just pause for a second? You say so many brilliant, poetic things. Do people use

Joscha Bach (1:30:03.520)

that term closed cooling chain? I imagine that people use it when they describe how they get

Joscha Bach (1:30:08.640)

meat into a supermarket, right? If you break the cooling chain and this thing starts to thaw,

Joscha Bach (1:30:13.840)

you're in trouble and you have to throw it away. That's such a beautiful way to put it. It's like

Joscha Bach (1:30:19.760)

calling a city a closed social chain or something like that. I mean, that's right. I mean, the

Joscha Bach (1:30:25.120)

locality of it is really important. It basically means you wake up in a climatized room, you go

Joscha Bach (1:30:28.720)

to work in a climatized car, you work in a climatized office, you shop in a climatized

Joscha Bach (1:30:32.720)

supermarket and in between you have very short distance in which you run from your car to the

Joscha Bach (1:30:37.360)

supermarket, but you have to make sure that your temperature does not approach the temperature of

Joscha Bach (1:30:42.560)

the environment. The crucial thing is the wet barb temperature. The wet barb temperature. It's

Lex Fridman (1:30:46.960)

what you get when you take a wet cloth and you put it around your thermometer and then you move

Joscha Bach (1:30:54.560)

it very quickly through the air so you get the evaporation heat. And as soon as you can no longer

Joscha Bach (1:31:01.920)

cool your body temperature via evaporation to a temperature below something like I think 35

Joscha Bach (1:31:08.240)

degrees, you die. Which means if the outside world is dry, you can still cool yourself down

Joscha Bach (1:31:15.440)

by sweating. But if it has a certain degree of humidity or if it goes over a certain temperature,

Joscha Bach (1:31:20.720)

then sweating will not save you. And this means even if you're a healthy, fit individual within

Joscha Bach (1:31:26.320)

a few hours, even if you try to be in the shade and so on, you'll die unless you have

Joscha Bach (1:31:31.040)

some climatizing equipment. And this itself, as long as you maintain civilization and you have

Joscha Bach (1:31:37.360)

energy supply and you have foot trucks coming to your home that are climatized, everything is fine.

Lex Fridman (1:31:41.840)

But what if you lose large scale open agriculture at the same time? So basically you run into foot

Joscha Bach (1:31:47.440)

insecurity because climate becomes very irregular or weather becomes very irregular and you have a

Joscha Bach (1:31:52.480)

lot of extreme weather events. So you need to roll most of your foot maybe indoor or you need to

Joscha Bach (1:31:59.760)

import your foot from certain regions. And maybe you're not able to maintain the civilization

Joscha Bach (1:32:04.720)

throughout the planet to get the infrastructure to get the foot to your home.

Joscha Bach (1:32:09.040)

Right. But there could be significant impacts in the sense that people begin to suffer.

Joscha Bach (1:32:13.920)

There could be wars over resources and so on. But ultimately, do you not have a, not a faith, but

Lex Fridman (1:32:20.960)

what do you make of the capacity of technological innovation to help us prevent some of the worst

Joscha Bach (1:32:30.560)

damages that this condition can create? So as an example, as an almost out there example,

Joscha Bach (1:32:38.480)

is the work that SpaceX and Elon Musk is doing of trying to also consider our propagation

Joscha Bach (1:32:45.280)

throughout the universe in deep space to colonize other planets. That's one technological step.

Lex Fridman (1:32:51.360)

But of course, what Elon Musk is trying on Mars is not to save us from global warming,

Joscha Bach (1:32:56.160)

because Mars looks much worse than Earth will look like after the worst outcomes of global warming

Lex Fridman (1:33:01.760)

imaginable, right? Mars is essentially not habitable.

Joscha Bach (1:33:06.000)

It's exceptionally harsh environment, yes. But what he is doing, what a lot of people throughout

Joscha Bach (1:33:10.480)

history since the Industrial Revolution are doing, are just doing a lot of different technological

Joscha Bach (1:33:15.200)

innovation with some kind of target. And when it ends up happening, it's totally unexpected new

Joscha Bach (1:33:20.240)

things come up. So trying to terraform or trying to colonize Mars, extremely harsh environment,

Joscha Bach (1:33:27.840)

might give us totally new ideas of how to expand or increase the power of this closed cooling

Joscha Bach (1:33:36.240)

circuit that empowers the community. So it seems like there's a little bit of a race between our

Joscha Bach (1:33:44.880)

open ended technological innovation of this communal operating system that we have and our

Joscha Bach (1:33:55.280)

general tendency to want to overuse resources and thereby destroy ourselves. You don't think

Joscha Bach (1:34:02.800)

technology can win that race? I think the probability is relatively low, given that our

Joscha Bach (1:34:08.960)

technology is, for instance, the US is stagnating since the 1970s roughly, in terms of technology.

Lex Fridman (1:34:15.040)

Most of the things that we do are the result of incremental processes. What about Intel?

Lex Fridman (1:34:19.920)

What about Moore's Law? It's basically, it's very incremental. The things that we're doing is,

Lex Fridman (1:34:24.480)

so the invention of the microprocessor was a major thing, right? The miniaturization

Joscha Bach (1:34:31.760)

of transistors was really major. But the things that we did afterwards largely were not that

Joscha Bach (1:34:38.880)

innovative. We had gradual changes of scaling things from CPUs into GPUs and things like that.

Lex Fridman (1:34:48.960)

But I don't think that there are, basically there are not many things. If you take a person that

Joscha Bach (1:34:54.080)

died in the 70s and was at the top of their game, they would not need to read that many books

Joscha Bach (1:34:59.040)

to be current again. But it's all about books. Who cares about books? There might be things that are

Joscha Bach (1:35:05.040)

beyond books. Or say papers. No, papers. Forget papers. There might be things that are, so papers

Lex Fridman (1:35:11.040)

and books and knowledge, that's a concept of a time when you were sitting there by candlelight

Lex Fridman (1:35:16.400)

and individual consumers of knowledge. What about the impact that we're not in the middle of,

Joscha Bach (1:35:21.280)

might not be understanding of Twitter, of YouTube? The reason you and I are sitting here today

Joscha Bach (1:35:27.760)

is because of Twitter and YouTube. So the ripple effect, and there's two minds, sort of two dumb

Joscha Bach (1:35:35.600)

apes coming up with a new, perhaps a new clean insights, and there's 200 other apes listening

Joscha Bach (1:35:42.160)

right now, 200,000 other apes listening right now. And that effect, it's very difficult to understand

Lex Fridman (1:35:48.800)

what that effect will have. That might be bigger than any of the advancements of the microprocessor

Joscha Bach (1:35:53.360)

or any of the industrial revolution, the ability of spread knowledge. And that knowledge,

Joscha Bach (1:36:02.880)

like it allows good ideas to reach millions much faster. And the effect of that, that might be the

Joscha Bach (1:36:09.840)

new, that might be the 21st century, is the multiplying of ideas, of good ideas. Because if

Joscha Bach (1:36:16.880)

you say one good thing today, that will multiply across huge amounts of people, and then they will

Joscha Bach (1:36:24.160)

say something, and then they will have another podcast, and they'll say something, and then they'll

Joscha Bach (1:36:27.840)

write a paper. That could be a huge, you don't think that? Yeah, we should have billions for

Joscha Bach (1:36:33.360)

Neumann's right now in two rings, and we don't for some reason. I suspect the reason is that

Joscha Bach (1:36:38.960)

we destroy our attention span. Also the incentives, of course, different. Yeah, we have extreme

Joscha Bach (1:36:43.200)

Kardashians, yeah. So the reason why we're sitting here and doing this as a YouTube video is because

Joscha Bach (1:36:48.000)

you and me don't have the attention span to write a book together right now. And you guys probably

Joscha Bach (1:36:52.160)

don't have the attention span to read it. So let me tell you, it's very short. But we're an hour

Lex Fridman (1:37:01.600)

and 40 minutes in, and I guarantee you that 80% of the people are still listening. So there is an

Lex Fridman (1:37:06.240)

attention span. It's just the form. Who said that the book is the optimal way to transfer information?

Joscha Bach (1:37:13.520)

This is still an open question. That's what we're... It's something that social media could be doing

Joscha Bach (1:37:17.440)

that other forms could not be doing. I think the end game of social media is a global brain.

Lex Fridman (1:37:22.240)

And Twitter is in some sense a global brain that is completely hooked on dopamine, doesn't have any

Joscha Bach (1:37:26.560)

kind of inhibition, and as a result is caught in a permanent seizure. It's also in some sense a

Joscha Bach (1:37:32.320)

multiplayer role playing game. And people use it to play an avatar that is not like them,

Joscha Bach (1:37:38.000)

as they were in this sane world, and they look through the world through the lens of their phones

Lex Fridman (1:37:41.600)

and think it's the real world. But it's the Twitter world that is distorted by the popularity

Joscha Bach (1:37:45.280)

incentives of Twitter. Yeah, the incentives and just our natural biological, the dopamine rush

Joscha Bach (1:37:52.640)

of a like, no matter how... I try to be very kind of Zen like and minimalist and not be influenced

Joscha Bach (1:38:01.920)

by likes and so on, but it's probably very difficult to avoid that to some degree.

Lex Fridman (1:38:07.280)

Speaking at a small tangent of Twitter, how can Twitter be done better?

Joscha Bach (1:38:15.760)

I think it's an incredible mechanism that has a huge impact on society

Joscha Bach (1:38:19.760)

by doing exactly what you're doing. Sorry, doing exactly what you described, which is having this...

Joscha Bach (1:38:25.040)

We're like, is this some kind of game, and we're kind of our individual RL agents in this game,

Lex Fridman (1:38:33.440)

and it's uncontrollable because there's not really a centralized control. Neither Jack Dorsey nor

Joscha Bach (1:38:37.440)

the engineers at Twitter seem to be able to control this game. Or can they? That's sort

Lex Fridman (1:38:44.800)

of a question. Is there any advice you would give on how to control this game?

Joscha Bach (1:38:49.760)

I wouldn't give advice because I am certainly not an expert, but I can give my thoughts on this.

Lex Fridman (1:38:53.440)

And our brain has solved this problem to some degree. Our brain has lots of individual agents

Joscha Bach (1:39:01.120)

that manage to play together in a way. And we have also many contexts in which other organisms

Joscha Bach (1:39:06.080)

have found ways to solve the problems of cooperation that we don't solve on Twitter.

Lex Fridman (1:39:12.480)

And maybe the solution is to go for an evolutionary approach. So imagine that you

Joscha Bach (1:39:18.720)

have something like Reddit or something like Facebook and something like Twitter,

Lex Fridman (1:39:23.520)

and you think about what they have in common. What they have in common, they are companies

Joscha Bach (1:39:27.440)

that in some sense own a protocol. And this protocol is imposed on a community, and the

Joscha Bach (1:39:32.960)

protocol has different components for monetization, for user management, for user display, for rating,

Joscha Bach (1:39:39.680)

for anonymity, for import of other content, and so on. And now imagine that you take these

Joscha Bach (1:39:44.320)

components of the protocol apart, and you do it in some sense like communities within this

Joscha Bach (1:39:50.960)

social network. And these communities are allowed to mix and match their protocols and design new

Joscha Bach (1:39:55.840)

ones. So for instance, the UI and the UX can be defined by the community. The rules for sharing

Joscha Bach (1:40:02.000)

content across communities can be defined. The monetization can be redefined. The way you reward

Joscha Bach (1:40:07.440)

individual users for what can be redefined. The way users can represent themselves and to each

Joscha Bach (1:40:13.040)

other can redefined. Who could be the redefiner? So can individual human beings build enough

Joscha Bach (1:40:18.720)

intuition to redefine those things? This itself can become part of the protocol. So for instance,

Joscha Bach (1:40:22.800)

it could be in some communities, it will be a single person that comes up with these things.

Lex Fridman (1:40:27.600)

And others, it's a group of friends. Some might implement a voting scheme that has some interesting

Joscha Bach (1:40:32.480)

weighted voting. Who knows? Who knows what will be the best self organizing principle for this.

Lex Fridman (1:40:36.640)

But the process can't be automated. I mean, it seems like the brain.

Joscha Bach (1:40:39.840)

It can be automated so people can write software for this. And eventually the idea is,

Joscha Bach (1:40:45.920)

let's not make an assumption about this thing if you don't know what the right solution is. In

Joscha Bach (1:40:50.240)

those areas that we have no idea whether the right solution will be people designing this ad hoc,

Joscha Bach (1:40:55.520)

or machines doing this. Whether you want to enforce compliance by social norms like Wikipedia,

Joscha Bach (1:41:01.200)

or with software solutions, or with AI that goes through the posts of people, or with a

Joscha Bach (1:41:06.480)

legal principle, and so on. This is something maybe you need to find out. And so the idea would

Joscha Bach (1:41:12.160)

be if you let the communities evolve, and you just control it in such a way that you are

Joscha Bach (1:41:17.840)

incentivizing the most sentient communities. The ones that produce the most interesting

Joscha Bach (1:41:24.400)

behaviors that allow you to interact in the most helpful ways to the individuals.

Joscha Bach (1:41:29.440)

You have a network that gives you information that is relevant to you.

Joscha Bach (1:41:32.640)

It helps you to maintain relationships to others in healthy ways. It allows you to build teams. It

Joscha Bach (1:41:37.680)

allows you to basically bring the best of you into this thing and goes into a coupling into

Joscha Bach (1:41:42.720)

a relationship with others in which you produce things that you would be unable to produce alone.

Joscha Bach (1:41:47.040)

Yes, beautifully put. But the key process of that with incentives and evolution

Joscha Bach (1:41:53.040)

is things that don't adopt themselves to effectively get the incentives have to die.

Lex Fridman (1:42:02.640)

And the thing about social media is communities that are unhealthy or whatever you wanted that

Joscha Bach (1:42:07.600)

defines the incentives really don't like dying. One of the things that people really get aggressive,

Joscha Bach (1:42:13.120)

protest aggressively is when they're censored. Especially in America. I don't know much about

Joscha Bach (1:42:19.280)

the rest of the world, but the idea of freedom of speech, the idea of censorship is really painful

Joscha Bach (1:42:24.880)

in America. And so what do you think about that? Having grown up in East Germany, do you think

Lex Fridman (1:42:38.480)

censorship is an important tool in our brain and the intelligence and in social networks?

Lex Fridman (1:42:45.520)

So basically, if you're not a good member of the entirety of the system, they should be blocked

Joscha Bach (1:42:53.200)

away. Well, locked away, blocked. An important thing is who decides that you are a good member.

Joscha Bach (1:42:59.040)

Who? Is it distributed? And what is the outcome of the process that decides it,

Joscha Bach (1:43:04.160)

both for the individual and for society at large. For instance, if you have a high trust society,

Joscha Bach (1:43:09.840)

you don't need a lot of surveillance. And the surveillance is even in some sense undermining

Joscha Bach (1:43:14.720)

trust. Because it's basically punishing people that look suspicious when surveyed,

Lex Fridman (1:43:21.280)

but do the right thing anyway. And the opposite, if you have a low trust society,

Joscha Bach (1:43:26.560)

then surveillance can be a better trade off. And the US is currently making a transition from a

Joscha Bach (1:43:30.960)

relatively high trust or mixed trust society to a low trust society. So surveillance will increase.

Joscha Bach (1:43:36.400)

Another thing is that beliefs are not just inert representations. There are implementations that

Joscha Bach (1:43:40.880)

run code on your brain and change your reality and change the way you interact with each other

Joscha Bach (1:43:45.600)

at some level. And some of the beliefs are just public opinions that we use to display our

Joscha Bach (1:43:52.240)

alignment. So for instance, people might say, all cultures are the same and equally good,

Lex Fridman (1:43:58.560)

but still they prefer to live in some cultures over others, very, very strongly so. And it turns

Joscha Bach (1:44:03.760)

out that the cultures are defined by certain rules of interaction. And these rules of interaction

Joscha Bach (1:44:08.240)

lead to different results when you implement them. So if you adhere to certain rules,

Joscha Bach (1:44:12.720)

you get different outcomes in different societies. And this all leads to very tricky

Lex Fridman (1:44:18.720)

situations when people do not have a commitment to a shared purpose.

Lex Fridman (1:44:22.640)

And our societies probably need to rediscover what it means to have a shared purpose and how

Joscha Bach (1:44:27.920)

to make this compatible with a non totalitarian view. So in some sense, the US is caught in a

Joscha Bach (1:44:34.320)

conundrum between totalitarianism and diversity, and doesn't need to know how to resolve this.

Lex Fridman (1:44:42.880)

And the solutions that the US has found so far are very crude because it's a very young society

Joscha Bach (1:44:47.280)

that is also under a lot of tension. It seems to me that the US will have to reinvent itself.

Lex Fridman (1:44:52.240)

What do you think, just philosophizing, what kind of mechanisms of government do you think

Joscha Bach (1:45:01.120)

we as a species should be involved with, US or broadly? What do you think will work well

Lex Fridman (1:45:07.200)

as a system? Of course, we don't know. It all seems to work pretty crappily,

Joscha Bach (1:45:11.360)

some things worse than others. Some people argue that communism is the best. Others say,

Joscha Bach (1:45:16.800)

yeah, look at the Soviet Union. Some people argue that anarchy is the best and then completely

Joscha Bach (1:45:22.720)

discarding the positive effects of government. There's a lot of arguments. US seems to be doing

Joscha Bach (1:45:29.920)

pretty damn well in the span of history. There's a respect for human rights, which seems to be a

Joscha Bach (1:45:36.240)

nice feature, not a bug. And economically, a lot of growth, a lot of technological development.

Lex Fridman (1:45:42.320)

People seem to be relatively kind on the grand scheme of things.

Lex Fridman (1:45:47.760)

What lessons do you draw from that? What kind of government system do you think is good?

Joscha Bach (1:45:52.400)

Ideally, a government should not be perceivable. It should be frictionless. The more you notice the

Joscha Bach (1:45:58.880)

influence of the government, the more friction you experience, the less effective and efficient

Joscha Bach (1:46:04.240)

the government probably is. A government, game theoretically, is an agent that imposes

Joscha Bach (1:46:10.560)

an offset on your payout metrics to make your Nash equilibrium compatible with the common good.

Joscha Bach (1:46:17.680)

You have these situations where people act on local incentives and these local incentives,

Joscha Bach (1:46:23.120)

everybody does the thing that's locally the best for them, but the global outcome is not good.

Lex Fridman (1:46:27.040)

And this is even the case when people care about the global outcome, because a regulation mechanism

Joscha Bach (1:46:31.760)

exists that creates a causal relationship between what I want to have for the global good and what

Joscha Bach (1:46:36.240)

I do. For instance, if I think that we should fly less and I stay at home, there's not a single plane

Joscha Bach (1:46:41.440)

that is going to not start because of me, right? It's not going to have an influence, but I don't

Joscha Bach (1:46:49.120)

get from A to B. So the way to implement this would be to have a government that is sharing

Joscha Bach (1:46:55.840)

this idea that we should fly less and is then imposing a regulation that, for instance,

Joscha Bach (1:46:59.600)

makes flying more expensive and gives incentives for inventing other forms of transportation that

Joscha Bach (1:47:06.800)

are less putting that strain on the environment, for instance. So there's so much optimism and

Lex Fridman (1:47:14.000)

so many things you describe, and yet there's the pessimism of you think our civilization is going

Joscha Bach (1:47:18.160)

to come to an end. So that's not a hundred percent probability. Nothing in this world is.

Lex Fridman (1:47:23.760)

So what's the trajectory out of self destruction, do you think? I suspect that in some sense,

Joscha Bach (1:47:30.960)

we are both too smart and not smart enough, which means we are very good at solving near

Joscha Bach (1:47:35.520)

term problems. And at the same time, we are unwilling to submit to the imperatives that

Joscha Bach (1:47:43.040)

we would have to follow in if you want to stick around. So that makes it difficult. If you were

Joscha Bach (1:47:48.720)

unable to solve everything technologically, you can probably understand how high the child mortality

Joscha Bach (1:47:53.760)

needs to be to absorb the mutation rate and how high the mutation rate needs to be to adapt to a

Joscha Bach (1:47:59.280)

slowly changing ecosystemic environment. So you could in principle compute all these things game

Joscha Bach (1:48:04.480)

theoretically and adapt to it. But if you cannot do this, because you are like me and you have

Joscha Bach (1:48:10.720)

children, you don't want them to die, you will use any kind of medical information to keep

Joscha Bach (1:48:16.320)

mortality low. Even if it means that within a few generations, we have enormous genetic drift,

Lex Fridman (1:48:22.000)

and most of us have allergies as a result of not being adapted to the changes that we

Joscha Bach (1:48:26.400)

made to our food supply. That's for now, I say technologically speaking, we're just very young,

Joscha Bach (1:48:31.760)

300 years industrial revolution, we're very new to this idea. So you're attached to your kids being

Joscha Bach (1:48:36.880)

alive and not being murdered for the good of society. But that might be a very temporary

Joscha Bach (1:48:41.680)

moment of time that we might evolve in our thinking. So like you said, we're both smart

Lex Fridman (1:48:48.800)

and not smart enough. We are probably not the first human civilization that has discovered

Joscha Bach (1:48:54.720)

technology that allows us to efficiently overgraze our resources. And this overgrazing,

Joscha Bach (1:48:59.840)

this thing, at some point, we think we can compensate this because if we have eaten all

Joscha Bach (1:49:04.320)

the grass, we will find a way to grow mushrooms. But it could also be that the ecosystems tip.

Lex Fridman (1:49:10.160)

And so what really concerns me is not so much the end of the civilization, because we will

Joscha Bach (1:49:14.240)

invent a new one. But what concerns me is the fact that, for instance, the oceans might tip.

Lex Fridman (1:49:21.920)

So for instance, maybe the plankton dies because of ocean acidification and cyanobacteria take over,

Lex Fridman (1:49:27.520)

and as a result, we can no longer breathe the atmosphere. This would be really concerning.

Lex Fridman (1:49:32.160)

So basically a major reboot of most complex organisms on Earth. And I think this is a

Joscha Bach (1:49:37.200)

possibility. I don't know what the percentage for this possibility is, but it doesn't seem to be

Joscha Bach (1:49:42.880)

outlandish to me if you look at the scale of the changes that we've already triggered on this

Joscha Bach (1:49:46.480)

planet. And so Danny Hiller suggests that, for instance, we may be able to put chalk into the

Joscha Bach (1:49:51.920)

stratosphere to limit solar radiation. Maybe it works. Maybe this is sufficient to counter

Joscha Bach (1:49:57.040)

the effects of what we've done. Maybe it won't be. Maybe we won't be able to implement it by

Joscha Bach (1:50:01.840)

the time it's prevalent. I have no idea how the future is going to play out in this regard. It's

Joscha Bach (1:50:07.360)

just, I think it's quite likely that we cannot continue like this. All our cousin species,

Joscha Bach (1:50:12.560)

the other hominids are gone. So the right step would be to what? To rewind

Lex Fridman (1:50:19.280)

and to rewind towards the industrial revolution and slow the, so try to contain the technological

Joscha Bach (1:50:28.320)

process that leads to the overconsumption of resources? Imagine you get to choose,

Joscha Bach (1:50:33.440)

you have one lifetime. You get born into a sustainable agricultural civilization,

Joscha Bach (1:50:38.640)

300, maybe 400 million people on the planet tops. Or before this, some kind of nomadic

Joscha Bach (1:50:45.840)

species was like a million or 2 million. And so you don't meet new people unless you give birth

Joscha Bach (1:50:51.440)

to them. You cannot travel to other places in the world. There is no internet. There is no

Joscha Bach (1:50:55.920)

interesting intellectual tradition that reaches considerably deep. So you would not discover

Joscha Bach (1:51:00.160)

human completeness probably and so on. We wouldn't exist. And the alternative is you get born into an

Joscha Bach (1:51:06.480)

insane world. One that is doomed to die because it has just burned a hundred million years worth

Joscha Bach (1:51:11.440)

of trees in a single century. Which one do you like? I think I like this one. It's a very weird

Joscha Bach (1:51:16.720)

thing that when you find yourself on a Titanic and you see this iceberg and it looks like we

Joscha Bach (1:51:21.280)

are not going to miss it. And a lot of people are in denial. And most of the counter arguments

Joscha Bach (1:51:25.440)

sound like denial to me. They don't seem to be rational arguments. And the other thing is we

Joscha Bach (1:51:30.320)

are born on this Titanic. Without this Titanic, we wouldn't have been born. We wouldn't be here. We

Joscha Bach (1:51:34.320)

wouldn't be talking. We wouldn't be on the internet. We wouldn't do all the things that we enjoy.

Lex Fridman (1:51:38.320)

And we are not responsible for this happening. If we had the choice, we would probably try to

Joscha Bach (1:51:45.680)

prevent it. But when we were born, we were never asked when we want to be born, in which society

Joscha Bach (1:51:51.120)

we want to be born, what incentive structures we want to be exposed to. We have relatively

Joscha Bach (1:51:55.760)

little agency in the entire thing. Humanity has relatively little agency in the whole thing. It's

Joscha Bach (1:52:00.080)

basically a giant machine that's tumbling down a hill and everybody is frantically trying to push

Joscha Bach (1:52:04.720)

some buttons. Nobody knows what these buttons are meaning, what they connect to. And most of them

Joscha Bach (1:52:09.840)

are not stopping this tumbling down the hill. Is it possible that artificial intelligence will give

Joscha Bach (1:52:15.360)

us an escape latch somehow? So there's a lot of worry about existential threats of artificial

Joscha Bach (1:52:25.040)

intelligence. But what AI also allows, in general forms of automation, allows the potential of

Joscha Bach (1:52:33.600)

extreme productivity growth that will also perhaps in a positive way transform society,

Joscha Bach (1:52:40.560)

that may allow us to inadvertently to return to the more, to the same kind of ideals of closer to

Joscha Bach (1:52:52.480)

nature that's represented in hunter gatherer societies. That's not destroying the planet,

Joscha Bach (1:52:59.600)

that's not doing overconsumption and so on. I mean, generally speaking,

Lex Fridman (1:53:03.680)

do you have hope that AI can help somehow? I think it's not fun to be very close to nature

Joscha Bach (1:53:09.520)

until you completely subdue nature. So our idea of being close to nature means being close to

Lex Fridman (1:53:16.720)

agriculture, basically forests that don't have anything in them that eats us.

Joscha Bach (1:53:21.760)

TITO See, I mean, I want to disagree with that. I think the niceness of being close to nature

Lex Fridman (1:53:30.000)

is to being fully present and in like, when survival becomes your primary,

Joscha Bach (1:53:37.040)

not just your goal, but your whole existence. I'm not just romanticizing, I can just speak for

Lex Fridman (1:53:47.680)

myself. I am self aware enough that that is a fulfilling existence.

Joscha Bach (1:53:54.880)

I personally prefer to be in nature and not fight for my survival. I think fighting for your survival

Joscha Bach (1:54:00.880)

while being in the cold and in the rain and being hunted by animals and having open wounds

Joscha Bach (1:54:06.480)

is very unpleasant.

Joscha Bach (1:54:07.760)

There's a contradiction in there. Yes, I and you, just as you said, would not choose it.

Lex Fridman (1:54:14.000)

But if I was forced into it, it would be a fulfilling existence.

Joscha Bach (1:54:17.680)

Yes, if you are adapted to it, basically, if your brain is wired up in such a way that you

Joscha Bach (1:54:23.280)

get rewards optimally in such an environment. And there's some evidence for this that for

Joscha Bach (1:54:29.280)

a certain degree of complexity, basically, people are more happy in such an environment because

Joscha Bach (1:54:33.520)

it's what you largely have evolved for. In between, we had a few thousand years in which

Lex Fridman (1:54:38.400)

I think we have evolved for a slightly more comfortable environment. So

Joscha Bach (1:54:41.680)

there is probably something like an intermediate stage in which people would be more happy than

Joscha Bach (1:54:47.360)

they would be if they would have to fend for themselves in small groups in the forest and

Joscha Bach (1:54:51.680)

often die. Versus something like this, where we now have basically a big machine, a big

Joscha Bach (1:54:57.680)

Mordor in which we run through concrete boxes and press buttons and machines, and largely

Joscha Bach (1:55:05.920)

don't feel well cared for as the monkeys that we are. So returning briefly to, not briefly,

Lex Fridman (1:55:12.880)

but returning to AI, what, let me ask a romanticized question, what is the most beautiful

Joscha Bach (1:55:19.680)

to you, silly ape, the most beautiful or surprising idea in the development of artificial

Joscha Bach (1:55:25.760)

intelligence, whether in your own life or in the history of artificial intelligence that you've

Joscha Bach (1:55:30.000)

come across? If you built an AI, it probably can make models at an arbitrary degree of detail,

Joscha Bach (1:55:37.120)

right, of the world. And then it would try to understand its own nature. It's tempting to think

Joscha Bach (1:55:42.400)

that at some point when we have general intelligence, we have competitions where we

Joscha Bach (1:55:46.000)

will let the AIs wake up in different kinds of physical universes, and we measure how many

Joscha Bach (1:55:51.040)

movements of the Rubik's cube it takes until it's figured out what's going on in its universe and

Lex Fridman (1:55:55.920)

what it is in its own nature and its own physics and so on, right? So what if we exist in the

Joscha Bach (1:56:00.640)

memory of an AI that is trying to understand its own nature and remembers its own genesis and

Joscha Bach (1:56:05.440)

remembers Lex and Joscha sitting in a hotel room, sparking some of the ideas off that led to the

Joscha Bach (1:56:11.280)

development of general intelligence. So we're a kind of simulation that's running in an AI

Joscha Bach (1:56:15.760)

system that's trying to understand itself. It's not that I believe that, but I think it's a

Joscha Bach (1:56:21.680)

beautiful idea. I mean, you kind of returned to this idea with the Turing test of intelligence

Joscha Bach (1:56:31.760)

being the process of asking and answering what is intelligence. I mean, do you think there is an

Joscha Bach (1:56:47.280)

answer? Why is there such a search for an answer? So does there have to be like an answer? You just

Joscha Bach (1:56:57.280)

said an AI system that's trying to understand the why of what, you know, understand itself.

Joscha Bach (1:57:04.880)

Is that a fundamental process of greater and greater complexity, greater and greater

Lex Fridman (1:57:09.920)

intelligence is the continuous trying of understanding itself?

Joscha Bach (1:57:13.840)

No, I think you will find that most people don't care about that because they're well adjusted

Joscha Bach (1:57:18.080)

enough to not care. And the reason why people like you and me care about it probably has to

Joscha Bach (1:57:23.200)

do with the need to understand ourselves. It's because we are in fundamental disagreement with

Joscha Bach (1:57:28.000)

the universe that we wake up in. They look down on me and they see, oh my God, I'm caught in a

Joscha Bach (1:57:32.720)

monkey. What's that? Some people are unhappy with the government and I'm unhappy with the entire

Joscha Bach (1:57:38.400)

universe that I find myself in. Oh, so you don't think that's a fundamental aspect of human nature

Lex Fridman (1:57:45.120)

that some people are just suppressing? That they wake up shocked they're in the body of a monkey?

Lex Fridman (1:57:51.200)

No, there is a clear adaptive value to not be confused by that and by...

Joscha Bach (1:57:56.960)

Well, no, that's not what I asked. So you have this clear adaptive value, then there's clear

Joscha Bach (1:58:04.880)

there's clear adaptive value to while fundamentally your brain is confused by that,

Joscha Bach (1:58:09.680)

by creating an illusion, another layer of the narrative that says, you know, that tries to

Joscha Bach (1:58:16.800)

suppress that and instead say that, you know, what's going on with the government right now

Joscha Bach (1:58:21.120)

is the most important thing. What's going on with my football team is the most important thing.

Lex Fridman (1:58:25.120)

But it seems to me, like for me, it was a really interesting moment reading Ernest

Joscha Bach (1:58:32.000)

Becker's Denial of Death. That, you know, this kind of idea that we're all, you know,

Joscha Bach (1:58:40.640)

the fundamental thing from which most of our human mind springs is this fear of mortality

Lex Fridman (1:58:49.680)

and being cognizant of your mortality and the fear of that mortality. And then you construct

Joscha Bach (1:58:54.720)

illusions on top of that. I guess you being just a push on it, you really don't think it's possible

Joscha Bach (1:59:03.600)

that this worry of the big existential questions is actually fundamental as the existentialist

Joscha Bach (1:59:11.280)

thought to our existence. I think that the fear of death only plays a role as long as you don't

Joscha Bach (1:59:17.040)

see the big picture. The thing is that minds are software states, right? Software doesn't have

Joscha Bach (1:59:22.560)

identity. Software in some sense is a physical law. But it feels like there's an identity. I

Joscha Bach (1:59:30.880)

thought that was the for this particular piece of software and the narrative it tells, that's

Joscha Bach (1:59:35.440)

a fundamental property of it. The maintenance of the identity is not terminal. It's instrumental

Joscha Bach (1:59:41.280)

to something else. You maintain your identity so you can serve your meaning. So you can do the

Joscha Bach (1:59:46.320)

things that you're supposed to do before you die. And I suspect that for most people the fear of

Joscha Bach (1:59:51.120)

death is the fear of dying before they are done with the things that they feel they have to do,

Lex Fridman (1:59:54.800)

even though they cannot quite put their finger on it, what that is.

Lex Fridman (1:59:59.280)

Right. But in the software world, to return to the question, then what happens after we die?

Joscha Bach (20:01.040)

maybe something within your mind, some genetic material, you were not tethered to the ideas

Joscha Bach (20:07.840)

of the general populace, which is actually a unique property. We're kind of the education

Joscha Bach (20:14.720)

system and whatever, not education system, just existing in this world forces certain sets of

Joscha Bach (20:20.160)

ideas onto you. Can you disentangle that? Why are you not so tethered? Even in your work today,

Joscha Bach (20:28.480)

you seem to not care about perhaps a best paper in Europe, right? Being tethered to particular

Joscha Bach (20:38.400)

things that current today in this year, people seem to value as a thing you put on your CV and

Joscha Bach (20:44.960)

resume. You're a little bit more outside of that world, outside of the world of ideas that people

Lex Fridman (20:50.160)

are especially focused in the benchmarks of today, the things. Can you disentangle that?

Lex Fridman (20:56.080)

Because I think that's inspiring. And if there were more people like that,

Joscha Bach (20:59.440)

we might be able to solve some of the bigger problems that AI dreams to solve.

Lex Fridman (21:05.760)

And there's a big danger in this because in a way you are expected to marry into an

Joscha Bach (21:10.640)

intellectual tradition and visit this tradition into a particular school. If everybody comes up

Joscha Bach (21:16.320)

with their own paradigms, the whole thing is not cumulative as an enterprise. So in some sense,

Joscha Bach (21:22.000)

you need a healthy balance. You need paradigmatic thinkers and you need people that work within

Joscha Bach (21:26.320)

given paradigms. Basically, scientists today define themselves largely by methods. And it's almost a

Joscha Bach (21:32.080)

disease that we think as a scientist, as somebody who was convinced by their guidance counselor,

Joscha Bach (21:38.080)

that they should join a particular discipline and then they find a good mentor to learn the

Joscha Bach (21:42.240)

right methods. And then they are lucky enough and privileged enough to join the right team. And then

Joscha Bach (21:47.120)

their name will show up on influential papers. But we also see that there are diminishing returns

Joscha Bach (21:52.800)

with this approach. And when our field, computer science and AI started, most of the people that

Joscha Bach (21:59.280)

joined this field had interesting opinions. And today's thinkers in AI either don't have interesting

Joscha Bach (22:05.440)

opinions at all, or these opinions are inconsequential for what they're actually

Joscha Bach (22:08.960)

doing. Because what they're doing is they apply the state of the art methods with a small epsilon.

Lex Fridman (22:13.520)

And this is often a good idea if you think that this is the best way to make progress. And for me,

Joscha Bach (22:21.360)

it's first of all, very boring. If somebody else can do it, why should I do it? If the current

Joscha Bach (22:27.360)

methods of machine learning lead to strong AI, why should I be doing it? I will just wait until

Joscha Bach (22:32.880)

they're done and wait until they do this on the beach or read interesting books or write some

Lex Fridman (22:38.240)

and have fun. But if you don't think that we are currently doing the right thing, if we are missing

Joscha Bach (22:45.040)

some perspectives, then it's required to think outside of the box. It's also required to understand

Joscha Bach (22:51.760)

the boxes. But it's necessary to understand what worked and what didn't work and for what reasons.

Lex Fridman (22:59.200)

So you have to be willing to ask new questions and design new methods whenever you want to

Joscha Bach (23:03.920)

answer them. And you have to be willing to dismiss the existing methods if you think that they're

Joscha Bach (23:09.120)

not going to yield the right answers. It's very bad career advice to do that. So maybe to briefly

Joscha Bach (23:16.960)

stay for one more time in the early days, when would you say for you was the dream

Joscha Bach (23:24.480)

before we dive into the discussions that we just almost started, when was the dream to understand

Lex Fridman (23:30.880)

or maybe to create human level intelligence born for you?

Joscha Bach (23:35.440)

I think that you can see AI largely today as advanced information processing. If you would

Joscha Bach (23:44.000)

change the acronym of AI into that, most people in the field would be happy. It would not change

Joscha Bach (23:48.640)

anything what they're doing. We're automating statistics and many of the statistical models

Joscha Bach (23:54.560)

are more advanced than what statisticians had in the past. And it's pretty good work. It's very

Joscha Bach (23:59.360)

productive. And the other aspect of AI is philosophical project. And this philosophical

Joscha Bach (24:04.480)

project is very risky and very few people work on it and it's not clear if it succeeds.

Lex Fridman (24:10.400)

So first of all, you keep throwing sort of a lot of really interesting ideas and I have to

Joscha Bach (24:16.000)

pick which ones we go with. But first of all, you use the term information processing,

Joscha Bach (24:22.480)

just information processing as if it's the mere, it's the muck of existence as if it's the epitome

Joscha Bach (24:31.440)

of existence, that the entirety of the universe might be information processing, that consciousness

Lex Fridman (24:36.720)

and intelligence might be information processing. So that maybe you can comment on if the advanced

Joscha Bach (24:42.240)

information processing is a limiting kind of a round of ideas. And then the other one is,

Lex Fridman (24:48.800)

what do you mean by the philosophical project? So I suspect that general intelligence is the

Joscha Bach (24:54.880)

result of trying to solve general problems. So intelligence, I think, is the ability to model.

Joscha Bach (25:00.800)

It's not necessarily goal directed rationality or something. Many intelligent people are bad at this,

Lex Fridman (25:06.640)

but it's the ability to be presented with a number of patterns and see a structure in those patterns

Lex Fridman (25:12.560)

and be able to predict the next set of patterns, to make sense of things. And

Joscha Bach (25:17.360)

some problems are very general. Usually intelligence serves control, so you make these

Joscha Bach (25:22.240)

models for a particular purpose of interacting as an agent with the world and getting certain results.

Lex Fridman (25:26.720)

But the intelligence itself is in the sense instrumental to something, but by itself it's

Joscha Bach (25:31.760)

just the ability to make models. And some of the problems are so general that the system that makes

Joscha Bach (25:36.400)

them needs to understand what itself is and how it relates to the environment. So as a child,

Joscha Bach (25:42.480)

for instance, you notice you do certain things despite you perceiving yourself as wanting

Joscha Bach (25:47.360)

different things. So you become aware of your own psychology. You become aware of the fact that you

Joscha Bach (25:53.040)

have complex structure in yourself and you need to model yourself, to reverse engineer yourself,

Joscha Bach (25:57.840)

to be able to predict how you will react to certain situations and how you deal with yourself

Joscha Bach (26:02.640)

in relationship to your environment. And this process, this project, if you reverse engineer

Joscha Bach (26:08.000)

yourself and your relationship to reality and the nature of a universe that can continue, if you go

Joscha Bach (26:12.720)

all the way, this is basically the project of AI, or you could say the project of AI is a very

Lex Fridman (26:17.840)

important component in it. The Turing test, in a way, is you ask a system, what is intelligence?

Joscha Bach (26:24.240)

If that system is able to explain what it is, how it works, then you should assign it the property

Joscha Bach (26:32.000)

of being intelligent in this general sense. So the test that Turing was administering

Joscha Bach (26:36.320)

in a way, I don't think that he couldn't see it, but he didn't express it yet in the original 1950

Joscha Bach (26:41.120)

paper, is that he was trying to find out whether he was generally intelligent. Because in order to

Joscha Bach (26:47.600)

take this test, the rub is, of course, you need to be able to understand what that system is saying.

Lex Fridman (26:51.920)

And we don't yet know if we can build an AI. We don't yet know if we are generally intelligent.

Joscha Bach (26:56.320)

Basically, you win the Turing test by building an AI. Yes. So in a sense, hidden within the Turing

Joscha Bach (27:03.440)

test is a kind of recursive test. Yes, it's a test on us. The Turing test is basically

Joscha Bach (27:08.480)

a test of the conjecture, whether people are intelligent enough to understand themselves.

Joscha Bach (27:14.240)

Okay. But you also mentioned a little bit of a self awareness and then the project of AI.

Lex Fridman (27:18.800)

Do you think this kind of emergent self awareness is one of the fundamental aspects of intelligence?

Lex Fridman (27:25.200)

So as opposed to goal oriented, as you said, kind of puzzle solving, is

Lex Fridman (27:31.200)

coming to grips with the idea that you're an agent in the world.

Joscha Bach (27:37.280)

I find that many highly intelligent people are not very self aware, right? So self awareness

Lex Fridman (27:42.400)

and intelligence are not the same thing. And you can also be self aware if you have good priors,

Joscha Bach (27:47.200)

especially, without being especially intelligent. So you don't need to be very good at solving

Lex Fridman (27:52.480)

puzzles if the system that you are already implements the solution.

Lex Fridman (27:56.080)

But I do find intelligence, you kind of mentioned children, right? Is that the fundamental project

Joscha Bach (28:03.600)

of AI is to create the learning system that's able to exist in the world. So you kind of drew

Joscha Bach (28:11.200)

a difference between self awareness and intelligence. And yet you said that the self

Joscha Bach (28:18.320)

awareness seems to be important for children. So I call this ability to make sense of the

Joscha Bach (28:23.600)

world and your own place in it. So to make you able to understand what you're doing in this world,

Joscha Bach (28:28.800)

sentience. And I would distinguish sentience from intelligence because sentience is

Joscha Bach (28:34.560)

possessing certain classes of models. And intelligence is a way to get to these models

Lex Fridman (28:39.200)

if you don't already have them. I see. So can you maybe pause a bit and try to

Joscha Bach (28:47.680)

answer the question that we just said we may not be able to answer? And it might be a recursive

Joscha Bach (28:53.120)

meta question of what is intelligence? I think that intelligence is the ability to make models.

Lex Fridman (28:59.840)

So models. I think it's useful as examples. Very popular now. Neural networks form representations

Joscha Bach (29:08.000)

of a large scale data set. They form models of those data sets. When you say models and look

Joscha Bach (29:17.200)

at today's neural networks, what are the difference of how you're thinking about what is intelligent

Joscha Bach (29:22.560)

in saying that intelligence is the process of making models? Two aspects to this question. One

Lex Fridman (29:29.120)

is the representation. Is the representation adequate for the domain that we're talking about?

Lex Fridman (29:33.760)

One is the representation. Is the representation adequate for the domain that we want to represent?

Joscha Bach (29:39.920)

The other one is the type of the model that you arrive at adequate. So basically, are you

Joscha Bach (29:45.760)

modeling the correct domain? I think in both of these cases, modern AI is lacking still. I think

Joscha Bach (29:53.200)

that I'm not saying anything new here. I'm not criticizing the field. Most of the people that

Joscha Bach (29:58.080)

design our paradigms are aware of that. One aspect that we're missing is unified learning.

Lex Fridman (2:00:10.240)

Why would you care? You will not be longer there. The point of dying is that you are gone.

Lex Fridman (2:00:14.960)

Well, maybe I'm not. This is what, you know, it seems like there's so much,

Joscha Bach (2:00:23.040)

in the idea that this is just, the mind is just a simulation that's constructing a narrative around

Joscha Bach (2:00:28.800)

some particular aspects of the quantum mechanical wave function world that we can't quite get direct

Joscha Bach (2:00:37.280)

access to. Then like the idea of mortality seems to be a little fuzzy as well. It doesn't, maybe

Joscha Bach (2:00:44.320)

there's not a clear answer. The fuzzy idea is the one of continuous existence. We don't have

Joscha Bach (2:00:49.520)

continuous existence. How do you know that? Because it's not computable. Because you're

Joscha Bach (2:00:55.840)

saying it's going to be directly infinite. There is no continuous process. The only thing that

Joscha Bach (2:00:58.400)

binds you together with the Lex Friedman from yesterday is the illusion that you have memories

Joscha Bach (2:01:02.080)

about him. So if you want to upload, it's very easy. You make a machine that thinks it's you.

Joscha Bach (2:01:07.120)

Because this is the same thing that you are. You are a machine that thinks it's you.

Lex Fridman (2:01:10.000)

But that's immortality. Yeah, but it's just a belief. You can create this belief very easily

Joscha Bach (2:01:15.600)

once you realize that the question whether you are immortal or not depends entirely on your beliefs

Lex Fridman (2:01:21.840)

and your own continuity. But then you can be immortal by the continuity of the belief.

Joscha Bach (2:01:28.880)

You cannot be immortal, but you can stop being afraid of your mortality because you realize you

Joscha Bach (2:01:33.680)

were never continuously existing in the first place. Well, I don't know if I'd be more terrified

Joscha Bach (2:01:39.600)

or less terrified by that. It seems like the fact that I existed.

Joscha Bach (2:01:44.160)

You don't know this state in which you don't have a self. You can't turn off yourself.

Joscha Bach (2:01:49.280)

I can't turn off myself.

Lex Fridman (2:01:50.320)

You can't turn it off. You can't turn it off.

Joscha Bach (2:01:52.080)

I can.

Joscha Bach (2:01:52.640)

Yes. And you can basically meditate yourself in a state where you are still conscious,

Joscha Bach (2:01:57.120)

where still things are happening, where you know everything that you knew before,

Lex Fridman (2:02:00.800)

but you're no longer identified with changing anything.

Lex Fridman (2:02:03.120)

And this means that yourself, in a way, dissolves. There is no longer this person. You know that this

Joscha Bach (2:02:09.120)

person construct exists in other states and it runs on this brain of Lex Friedman, but it's not

Joscha Bach (2:02:15.520)

a real thing. It's a construct. It's an idea. And you can change that idea. And if you let go of

Joscha Bach (2:02:21.520)

this idea, if you don't think that you are special, you realize it's just one of many people and it's

Joscha Bach (2:02:26.800)

not your favorite person even. It's just one of many. And it's the one that you are doomed to

Joscha Bach (2:02:31.760)

control for the most part. And that is basically informing the actions of this organism as a

Joscha Bach (2:02:37.360)

control model. And this is all there is. And you are somehow afraid that this control model gets

Lex Fridman (2:02:42.640)

interrupted or loses the identity of continuity.

Joscha Bach (2:02:47.200)

Yeah. So I'm attached. I mean, yeah, it's a very popular, it's a somehow compelling notion that

Joscha Bach (2:02:52.880)

being attached, like there's no need to be attached to this idea of an identity.

Lex Fridman (2:02:58.720)

But that in itself could be an illusion that you construct. So the process of meditation,

Joscha Bach (2:03:03.040)

while popular, is thought of as getting under the concept of identity. It could be just putting a

Joscha Bach (2:03:08.480)

cloak over it, just telling it to be quiet for the moment. I think that meditation is eventually just

Joscha Bach (2:03:18.240)

a bunch of techniques that let you control attention. And when you can control attention,

Joscha Bach (2:03:22.720)

you can get access to your own source code, hopefully not before you understand what you're

Lex Fridman (2:03:31.200)

doing. And then you can change the way it works temporarily or permanently.

Lex Fridman (2:03:36.000)

So yeah, meditation is to get a glimpse at the source code, get under, so basically control or

Lex Fridman (2:03:41.920)

turn off the attention.

Joscha Bach (2:03:42.720)

The entire thing is that you learn to control attention. So everything else is downstream

Lex Fridman (2:03:46.240)

from controlling attention.

Lex Fridman (2:03:47.440)

And control the attention that's looking at the attention.

Joscha Bach (2:03:50.880)

Normally we only get attention in the parts of our mind that create heat, where you have a

Joscha Bach (2:03:54.720)

mismatch between model and the results that are happening. And so most people are not self aware

Joscha Bach (2:04:00.240)

because their control is too good. If everything works out roughly the way you want, and the only

Joscha Bach (2:04:05.280)

things that don't work out is whether your football team wins, then you will mostly have

Joscha Bach (2:04:09.920)

models about these domains. And it's only when, for instance, your fundamental relationships to

Joscha Bach (2:04:15.120)

the world around you don't work, because the ideology of your country is insane, and you don't

Joscha Bach (2:04:20.080)

understand why it's insane, and the other kids are not nerds, and don't understand why you

Joscha Bach (2:04:24.160)

understand physics, and you don't, why you want to understand physics, and you don't understand

Lex Fridman (2:04:29.280)

why somebody would not want to understand physics.

Lex Fridman (2:04:32.640)

So we kind of brought up neurons in the brain as reinforcement learning agents.

Lex Fridman (2:04:40.480)

And there's been some successes as you brought up with Go, with AlphaGo, AlphaZero, with ideas

Joscha Bach (2:04:46.240)

which I think are incredibly interesting ideas of systems playing each other in an automated way

Joscha Bach (2:04:52.400)

to improve by playing other systems in a particular construct of a game that are a little

Joscha Bach (2:05:00.560)

bit better than itself, and then thereby improving continuously. All the competitors in the game

Joscha Bach (2:05:05.920)

are improving gradually. So being just challenging enough and from learning from the process of the

Joscha Bach (2:05:11.520)

competition. Do you have hope for that reinforcement learning process to achieve

Joscha Bach (2:05:16.720)

greater and greater level of intelligence? So we talked about different ideas in AI that need to

Lex Fridman (2:05:21.200)

be solved. Is RL a part of that process of trying to create an AGI system? What do you think?

Joscha Bach (2:05:28.480)

Definitely forms of unsupervised learning, but there are many algorithms that can achieve that.

Lex Fridman (2:05:32.800)

And I suspect that ultimately the algorithms that work, there will be a class of them or many of

Joscha Bach (2:05:38.800)

them. And they might have small differences of like a magnitude and efficiency, but eventually

Lex Fridman (2:05:45.120)

what matters is the type of model that you form and the types of models that we form right now

Joscha Bach (2:05:49.760)

are not sparse enough. What does it mean to be sparse? It means that ideally every potential

Joscha Bach (2:05:59.200)

model state should correspond to a potential world state. So basically if you vary states

Joscha Bach (2:06:06.000)

in your model, you always end up with valid world states and our mind is not quite there.

Lex Fridman (2:06:10.400)

So an indication is basically what we see in dreams. The older we get, the more boring our

Joscha Bach (2:06:14.960)

dreams become because we incorporate more and more constraints that we learned about how the

Joscha Bach (2:06:19.840)

world works. So many of the things that we imagine to be possible as children turn out to be

Joscha Bach (2:06:25.280)

constrained by physical and social dynamics. And as a result, fewer and fewer things remain

Joscha Bach (2:06:31.600)

possible. It's not because our imagination scales back, but the constraints under which it operates

Joscha Bach (2:06:36.640)

become tighter and tighter. And so the constraints under which our neural networks operate are

Joscha Bach (2:06:42.800)

almost limitless, which means it's very difficult to get a neural network to imagine things that

Joscha Bach (2:06:47.360)

look real. So I suspect part of what we need to do is we probably need to build dreaming systems.

Joscha Bach (2:06:55.200)

I suspect that part of the purpose of dreams is similar to a generative adversarial network,

Joscha Bach (2:07:01.280)

we learn certain constraints and then it produces alternative perspectives on the same set of

Joscha Bach (2:07:07.360)

constraints. So you can recognize it under different circumstances. Maybe we have flying

Joscha Bach (2:07:11.520)

dreams as children because we recreate the objects that we know and the maps that we know from

Joscha Bach (2:07:16.080)

different perspectives, which also means from a bird's eye perspective. So I mean, aren't we

Joscha Bach (2:07:21.040)

doing that anyway? I mean, not with our eyes closed and when we're sleeping, aren't we just

Lex Fridman (2:07:27.200)

constantly running dreams and simulations in our mind as we try to interpret the environment?

Joscha Bach (2:07:32.560)

I mean, sort of considering all the different possibilities, the way we interact with the

Joscha Bach (2:07:37.200)

environment seems like, essentially, like you said, sort of creating a bunch of simulations

Joscha Bach (2:07:46.000)

that are consistent with our expectations, with our previous experiences, with the things we just

Joscha Bach (2:07:52.240)

saw recently. And through that hallucination process, we are able to then somehow stitch

Joscha Bach (2:08:02.400)

together what actually we see in the world with the simulations that match it well and thereby

Joscha Bach (2:08:07.760)

interpret it. I suspect that you and my brain are slightly unusual in this regard,

Joscha Bach (2:08:13.440)

which is probably what got you into MIT. So this obsession of constantly pondering possibilities

Lex Fridman (2:08:19.520)

and solutions to problems. Oh, stop it. I think I'm not talking about intellectual stuff. I'm

Joscha Bach (2:08:27.520)

talking about just doing the kind of stuff it takes to walk and not fall. Yes, this is

Joscha Bach (2:08:35.040)

largely automatic. Yes, but the process is, I mean... It's not complicated. It's relatively

Joscha Bach (2:08:43.600)

easy to build a neural network that, in some sense, learns the dynamics. The fact that we

Joscha Bach (2:08:48.480)

haven't done it right so far doesn't mean it's hard, because you can see that a biological

Joscha Bach (2:08:52.560)

organism does it with relatively few neurons. So basically, you build a bunch of neural

Joscha Bach (2:08:57.360)

oscillators that entrain themselves with the dynamics of your body in such a way that the

Joscha Bach (2:09:01.760)

regulator becomes isomorphic in its model to the dynamics that it regulates, and then it's

Joscha Bach (2:09:06.880)

automatic. And it's only interesting in the sense that it captures attention when the system is off.

Joscha Bach (2:09:12.160)

See, but thinking of the kind of mechanism that's required to do walking as a controller,

Joscha Bach (2:09:18.000)

as a neural network, I think it's a compelling notion, but it discards quietly,

Joscha Bach (2:09:27.840)

or at least makes implicit, the fact that you need to have something like common sense reasoning

Joscha Bach (2:09:33.040)

to walk. It's an open question whether you do or not. But my intuition is to act in this world,

Joscha Bach (2:09:40.400)

there's a huge knowledge base that's underlying it somehow. There's so much information

Joscha Bach (2:09:46.400)

of the kind we have never been able to construct in neural networks in an artificial intelligence

Joscha Bach (2:09:53.520)

systems period, which is like, it's humbling, at least in my imagination, the amount of information

Joscha Bach (2:10:00.000)

required to act in this world humbles me. And I think saying that neural networks can accomplish

Joscha Bach (2:10:07.600)

it is missing the fact that we don't have yet a mechanism for constructing something like

Joscha Bach (2:10:16.560)

common sense reasoning. I mean, what's your sense about to linger on the idea of what kind of

Joscha Bach (2:10:28.480)

mechanism would be effective at walking? You said just a neural network, not maybe the kind we have,

Lex Fridman (2:10:33.920)

but something a little bit better, would be able to walk easily. Don't you think it also needs to know

Lex Fridman (2:10:42.080)

like a huge amount of knowledge that's represented under the flag of common sense reasoning?

Lex Fridman (2:10:47.360)

How much common sense knowledge do we actually have? Imagine that you are really hardworking

Joscha Bach (2:10:51.680)

for all your life and you form two new concepts every half hour or so. You end up with something

Joscha Bach (2:10:56.640)

like a million concepts because you don't get that old. So a million concepts, that's not a lot.

Lex Fridman (2:11:02.320)

So it's not just a million concepts. I think it would be a lot. I personally think it might be

Joscha Bach (2:11:06.800)

much more than a million. But if you think just about the numbers, you don't live that long.

Joscha Bach (2:11:12.320)

If you think about how many cycles do your neurons have in your life, it's quite limited.

Joscha Bach (2:11:16.480)

You don't get that old. Yeah, but the powerful thing is the number of concepts, and they're

Joscha Bach (2:11:23.440)

probably deeply hierarchical in nature. The relations, as you described between them,

Joscha Bach (2:11:29.200)

is the key thing. So it's like, even if it's a million concepts, the graph of relations that's

Joscha Bach (2:11:35.120)

formed and some kind of, perhaps, some kind of probabilistic relationships, that's what's common

Joscha Bach (2:11:42.800)

sense reasoning is the relationship between things. Yeah, so in some sense, I think of the concepts as

Joscha Bach (2:11:48.960)

the address space for our behavior programs. And the behavior programs allow us to recognize objects

Lex Fridman (2:11:53.920)

and interact with them, also mental objects. And a large part of that is the physical world that we

Joscha Bach (2:11:59.840)

interact with, which is this RAS extender thing, which is basically navigation of information in

Joscha Bach (2:12:04.640)

space. And basically, it's similar to a game engine. It's a physics engine that you can use to

Joscha Bach (2:12:12.480)

describe and predict how things that look in a particular way, that feel when you touch them in

Joscha Bach (2:12:18.080)

a particular way, that love proprioception, that love auditory, for example. So it's a

Joscha Bach (2:12:22.480)

lot of auditory perception and so on, how they work out. So basically, the geometry of all these

Joscha Bach (2:12:27.200)

things. And this is probably 80% of what our brain is doing is dealing with that, with this real time

Joscha Bach (2:12:33.600)

simulation. And by itself, a game engine is fascinating, but it's not that hard to understand

Lex Fridman (2:12:39.360)

what it's doing. And our game engines are already, in some sense, approximating the fidelity of what

Joscha Bach (2:12:47.040)

we can perceive. So if we put on an Oculus Quest, we get something that is still relatively crude

Joscha Bach (2:12:54.000)

with respect to what we can perceive, but it's also in the same ballpark already. It's just a

Joscha Bach (2:12:58.480)

couple order of magnitudes away from saturating our perception in terms of the complexity that

Joscha Bach (2:13:04.640)

it can produce. So in some sense, it's reasonable to say that the computer that you can buy and put

Joscha Bach (2:13:10.800)

into your home is able to give a perceptual reality that has a detail that is already in

Joscha Bach (2:13:15.600)

the same ballpark as what your brain can process. And everything else are ideas about the world.

Lex Fridman (2:13:22.080)

And I suspect that they are relatively sparse and also the intuitive models that we form about

Joscha Bach (2:13:27.040)

social interaction. Social interaction is not so hard. It's just hard for us nerds because we all

Joscha Bach (2:13:32.480)

have our wires crossed, so we need to deduce them. But the pyres are present in most social animals.

Lex Fridman (2:13:37.760)

So it's interesting thing to notice that many domestic social animals, like cats and dogs,

Joscha Bach (2:13:44.560)

have better social cognition than children. Right. I hope so. I hope it's not that many concepts

Joscha Bach (2:13:51.440)

fundamentally to do to exist in this world. For me, it's more like I'm afraid so because this

Joscha Bach (2:13:57.760)

thing that we only appear to be so complex to each other because we are so stupid is a little

Joscha Bach (2:14:02.560)

bit depressing. Yeah, to me that's inspiring if we're indeed as stupid as it seems. The things

Joscha Bach (2:14:11.360)

our brains don't scale and the information processing that we build tend to scale very well.

Joscha Bach (2:14:16.800)

Yeah, but I mean, one of the things that worries me is that the fact that the brain doesn't scale

Joscha Bach (2:14:23.840)

means that that's actually a fundamental feature of the brain. All the flaws of the brain,

Joscha Bach (2:14:30.000)

everything we see that we see as limitations, perhaps there's a fundamental, the constraints

Joscha Bach (2:14:34.480)

on the system could be a requirement of its power, which is different than our current

Joscha Bach (2:14:43.040)

understanding of intelligent systems where scale, especially with deep learning, especially with

Joscha Bach (2:14:48.080)

reinforcement learning, the hope behind OpenAI and DeepMind, all the major results really have

Joscha Bach (2:14:55.440)

to do with huge compute. It could also be that our brains are so small, not just because they

Joscha Bach (2:15:01.680)

take up so much glucose in our body, like 20% of the glucose, so they don't arbitrarily scale.

Joscha Bach (2:15:07.520)

There's some animals like elephants which have larger brains than us and they don't seem to be

Joscha Bach (2:15:11.760)

smarter. Elephants seem to be autistic. They have very, very good motor control and they're really

Joscha Bach (2:15:16.560)

good with details, but they really struggle to see the big picture. So you can make them

Joscha Bach (2:15:21.120)

recreate drawings stroke by stroke, they can do that, but they cannot reproduce a still life. So

Joscha Bach (2:15:27.200)

they cannot make a drawing of a scene that they see. They will always be only able to reproduce

Lex Fridman (2:15:31.840)

the line drawing, at least as far from what I could see in the experiments. So why is that?

Joscha Bach (2:15:37.120)

Maybe smarter elephants would meditate themselves out of existence because their brains are too

Joscha Bach (2:15:41.680)

large. So basically the elephants that were not autistic, they didn't reproduce.

Joscha Bach (2:15:46.400)

Yeah. So we have to remember that the brain is fundamentally interlinked with the body in our

Joscha Bach (2:15:50.400)

human and biological system. Do you think that AGI systems that we try to create or greater

Joscha Bach (2:15:55.440)

intelligent systems would need to have a body? I think they should be able to make use of a body

Joscha Bach (2:16:00.960)

if you give it to them. But I don't think that they fundamentally need a body. So I suspect if

Joscha Bach (2:16:06.080)

you can interact with the world by moving your eyes and your head, you can make controlled

Joscha Bach (2:16:11.840)

experiments. And this allows you to have many magnitudes, fewer observations in order to reduce

Joscha Bach (2:16:19.920)

the uncertainty in your models. So you can pinpoint the areas in your models where you're

Joscha Bach (2:16:24.000)

not quite sure and you just move your head and see what's going on over there and you get additional

Joscha Bach (2:16:28.480)

information. If you just have to use YouTube as an input and you cannot do anything beyond this,

Joscha Bach (2:16:33.680)

you probably need just much more data. But we have much more data. So if you can build a system that

Joscha Bach (2:16:39.680)

has enough time and attention to browse all of YouTube and extract all the information that there

Joscha Bach (2:16:44.320)

is to be found, I don't think there's an obvious limit to what it can do. Yeah, but it seems that

Joscha Bach (2:16:50.080)

the interactivity is a fundamental thing that the physical body allows you to do. But let me ask on

Joscha Bach (2:16:55.280)

that topic sort of that that's what a body is, is allowing the brain to like touch things and move

Joscha Bach (2:17:00.640)

things and interact with the whether the physical world exists or not, whatever, but interact with

Joscha Bach (2:17:06.800)

some interface to the physical world. What about a virtual world? Do you think we can do the same kind

Lex Fridman (2:17:14.960)

of reasoning, consciousness, intelligence if we put on a VR headset and move over to that world?

Lex Fridman (2:17:23.760)

Do you think there's any fundamental difference between the interface to the physical world that

Joscha Bach (2:17:28.080)

it's here in this hotel and if we were sitting in the same hotel in a virtual world? The question

Joscha Bach (2:17:32.960)

is, does this nonphysical world or this other environment entice you to solve problems that

Joscha Bach (2:17:39.360)

require general intelligence? If it doesn't, then you probably will not develop general intelligence

Lex Fridman (2:17:44.800)

and arguably most people are not generally intelligent because they don't have to solve

Joscha Bach (2:17:48.240)

problems that make them generally intelligent. And even for us, it's not yet clear if we are smart

Joscha Bach (2:17:52.880)

enough to build AI and understand our own nature to this degree. So it could be a matter of capacity

Lex Fridman (2:17:58.720)

and for most people, it's in the first place a matter of interest. They don't see the point

Joscha Bach (2:18:02.320)

because the benefit of attempting this project are marginal because you're probably not going

Joscha Bach (2:18:06.960)

to succeed in it and the cost of trying to do it requires complete dedication of your entire life.

Joscha Bach (2:18:11.920)

Right? But it seems like the possibilities of what you can do in the virtual world,

Lex Fridman (2:18:15.760)

so imagine that is much greater than you can in the real world. So imagine a situation,

Joscha Bach (2:18:21.200)

maybe interesting option for me. If somebody came to me and offered what I'll do is,

Lex Fridman (2:18:27.840)

so from now on, you can only exist in the virtual world. And so you put on this headset and when you

Joscha Bach (2:18:34.720)

eat, we'll make sure to connect your body up in a way that when you eat in the virtual world,

Joscha Bach (2:18:41.040)

your body will be nourished in the same way in the virtual world. So the aligning incentives

Joscha Bach (2:18:45.760)

between our common sort of real world and the virtual world, but then the possibilities become

Joscha Bach (2:18:50.880)

much bigger. Like I could be other kinds of creatures. I could do, I can break the laws

Joscha Bach (2:18:57.120)

of physics as we know them. I could do a lot. I mean, the possibilities are endless, right? As far

Joscha Bach (2:19:02.320)

as we think it's an interesting thought, whether like what existence would be like, what kind of

Joscha Bach (2:19:08.640)

intelligence would emerge there? What kind of consciousness? What kind of maybe greater

Joscha Bach (2:19:13.840)

intelligence, even in me, Lex, even at this stage in my life, if I spend the next 20 years in that

Joscha Bach (2:19:19.920)

world to see how that intelligence emerges. And if that happened at the very beginning,

Joscha Bach (2:19:26.080)

before I was even cognizant of my existence in this physical world, it's interesting to think

Lex Fridman (2:19:31.040)

how that child would develop. And the way virtual reality and digitization of everything is moving,

Joscha Bach (2:19:37.440)

it's not completely out of the realm of possibility that we're all, that some part of our lives will,

Joscha Bach (2:19:44.800)

if not entirety of it, will live in a virtual world to a greater degree than we currently have

Joscha Bach (2:19:51.360)

living on Twitter and social media and so on. Do you have, I mean, does something draw you

Joscha Bach (2:19:56.800)

intellectually or naturally in terms of thinking about AI to this virtual world where more

Joscha Bach (2:20:03.360)

possibilities are? I think that currently it's a waste of time to deal with the physical world

Lex Fridman (2:20:09.200)

before we have mechanisms that can automatically learn how to deal with it.

Joscha Bach (2:20:13.280)

The body gives you second order agency, but what constitutes the body is the things that

Joscha Bach (2:20:18.320)

you can indirectly control. The third order are tools, and the second order is the things that

Joscha Bach (2:20:24.080)

are basically always present, but you operate on them with first order things, which are mental

Joscha Bach (2:20:29.680)

operators. And the zero order is in some sense, the direct sense of what you're deciding. Right.

Lex Fridman (2:20:36.800)

So you observe yourself initiating an action, there are features that you interpret as the

Joscha Bach (2:20:42.000)

initiation of an action. Then you perform the operations that you perform to make that happen.

Lex Fridman (2:20:47.520)

And then you see the movement of your limbs and you learn to associate those and thereby model

Joscha Bach (2:20:52.480)

your own agency over this feedback, right? But the first feedback that you get is from this first

Joscha Bach (2:20:56.800)

order thing already. Basically, you decide to think a thought and the thought is being thought.

Joscha Bach (2:21:01.280)

You decide to change the thought and you observe how the thought is being changed.

Lex Fridman (2:21:05.200)

And in some sense, this is, you could say, an embodiment already, right? And I suspect it's

Joscha Bach (2:21:10.000)

sufficient as an embodiment for intelligence. And so it's not that important at least at

Joscha Bach (2:21:14.560)

this time to consider variations in the second order. Yes. But the thing that you also put

Lex Fridman (2:21:20.800)

mentioned just now is physics that you could change in any way you want.

Lex Fridman (2:21:24.800)

So you need an environment that puts up resistance against you. If there's nothing to control,

Joscha Bach (2:21:29.920)

you cannot make models, right? There needs to be a particular way that resists you.

Lex Fridman (2:21:34.560)

And by the way, your motivation is usually outside of your mind. It resists you. Motivation

Joscha Bach (2:21:38.800)

is what gets you up in the morning even though it would be much less work to stay in bed.

Lex Fridman (2:21:43.840)

So it's basically forcing you to resist the environment and it forces your mind to serve it,

Joscha Bach (2:21:51.760)

to serve this resistance to the environment. So in some sense, it is also putting up resistance

Joscha Bach (2:21:56.640)

against the natural tendency of the mind to not do anything. Yeah. So some of that resistance,

Joscha Bach (2:22:01.200)

just like you described with motivation is like in the first order, it's in the mind.

Joscha Bach (2:22:05.520)

Some resistance is in the second order, like actual physical objects pushing against you and so on.

Lex Fridman (2:22:11.040)

It seems that the second order stuff in virtual reality could be recreated.

Joscha Bach (2:22:14.480)

Of course. But it might be sufficient that you just do mathematics and mathematics is already

Joscha Bach (2:22:19.360)

putting up enough resistance against you. So basically just with an aesthetic motive,

Joscha Bach (2:22:24.160)

this could maybe sufficient to form a type of intelligence. It would probably not be a very

Joscha Bach (2:22:29.760)

human intelligence, but it might be one that is already general. So to mess with this zero order,

Joscha Bach (2:22:37.760)

maybe first order, what do you think about ideas of brain computer interfaces? So again, returning

Joscha Bach (2:22:43.040)

to our friend Elon Musk and Neuralink, a company that's trying to, of course, there's a lot of

Joscha Bach (2:22:48.160)

a trying to cure diseases and so on with a near term, but the longterm vision is to add an extra

Joscha Bach (2:22:54.240)

layer to basically expand the capacity of the brain connected to the computational world.

Lex Fridman (2:23:03.120)

Do you think one that's possible too, how does that change the fundamentals of the zeroth order

Joscha Bach (2:23:07.120)

in the first order? It's technically possible, but I don't see that the FDA would ever allow me to

Joscha Bach (2:23:11.360)

drill holes in my skull to interface my neocortex the way Elon Musk envisions. So at the moment,

Joscha Bach (2:23:16.480)

I can do horrible things to mice, but I'm not able to do useful things to people,

Joscha Bach (2:23:21.760)

except maybe at some point down the line in medical applications. So this thing that we

Joscha Bach (2:23:26.960)

are envisioning, which means recreational and creational brain computer interfaces

Joscha Bach (2:23:33.440)

are probably not going to happen in the present legal system.

Lex Fridman (2:23:36.560)

I love it how I'm asking you out there philosophical and sort of engineering

Joscha Bach (2:23:43.200)

questions. And for the first time ever, you jumped to the legal FDA.

Joscha Bach (2:23:48.000)

There would be enough people that would be crazy enough to have holes drilled in their skull to

Joscha Bach (2:23:51.760)

try a new type of brain computer interface. But also, if it works, FDA will approve it.

Lex Fridman (2:23:57.680)

I mean, yes, it's like, you know, I work a lot with autonomous vehicles. Yes,

Joscha Bach (2:24:02.560)

you can say that it's going to be a very difficult regulatory process of approving

Lex Fridman (2:24:05.760)

autonomous, but it doesn't mean autonomous vehicles are never going to happen.

Joscha Bach (2:24:08.800)

No, they will totally happen as soon as we create jobs for at least two lawyers

Lex Fridman (2:24:14.080)

and one regulator per car.

Joscha Bach (2:24:17.120)

Yes, lawyers, that's actually like lawyers is the fundamental substrate of reality.

Joscha Bach (2:24:24.800)

In the US, it's a very weird system. It's not universal in the world. The law is a very

Joscha Bach (2:24:30.400)

interesting software once you realize it, right? These circuits are in some sense streams of

Joscha Bach (2:24:34.880)

software and this is largely works by exception handling. So you make decisions on the ground

Lex Fridman (2:24:39.600)

and they get synchronized with the next level structure as soon as an exception is being

Lex Fridman (2:24:43.120)

thrown. So it escalates the exception handling. The process is very expensive,

Joscha Bach (2:24:49.600)

especially since it incentivizes the lawyers for producing work for lawyers.

Joscha Bach (2:24:54.960)

Yes, so the exceptions are actually incentivized for firing often. But to return, outside of

Joscha Bach (2:25:02.960)

lawyers, is there anything interesting, insightful about the possibility of this extra layer

Lex Fridman (2:25:13.440)

of intelligence added to the brain?

Joscha Bach (2:25:15.280)

I do think so, but I don't think that you need technically invasive procedures to do

Joscha Bach (2:25:20.320)

so. We can already interface with other people by observing them very, very closely and getting

Joscha Bach (2:25:25.120)

in some kind of empathetic resonance. And I'm not very good at this, but I noticed that

Joscha Bach (2:25:31.360)

people are able to do this to some degree. And it basically means that we model an interface

Joscha Bach (2:25:37.360)

layer of the other person in real time. And it works despite our neurons being slow because

Joscha Bach (2:25:42.720)

most of the things that we do are built on periodic processes. So you just need to entrain

Joscha Bach (2:25:46.960)

yourself with the oscillation that happens. And if the oscillation itself changes slowly

Lex Fridman (2:25:51.600)

enough, you can basically follow along.

Joscha Bach (2:25:54.080)

Right. But the bandwidth of the interaction, it seems like you can do a lot more computation

Lex Fridman (2:26:03.360)

when there's...

Joscha Bach (2:26:04.080)

Of course. But the other thing is that the bandwidth that our brain, our own mind is

Joscha Bach (2:26:08.240)

running on is actually quite slow. So the number of thoughts that I can productively

Joscha Bach (2:26:12.320)

think in any given day is quite limited. If they had the discipline to write it down

Lex Fridman (2:26:18.400)

and the speed to write it down, maybe it would be a book every day or so. But if you think

Joscha Bach (2:26:22.400)

about the computers that we can build, the magnitudes at which they operate, this would

Lex Fridman (2:26:28.080)

be nothing. It's something that it can put out in a second.

Joscha Bach (2:26:30.640)

Well, I don't know. So it's possible the number of thoughts you have in your brain is... It

Joscha Bach (2:26:37.120)

could be several orders of magnitude higher than what you're possibly able to express

Joscha Bach (2:26:41.760)

through your fingers or through your voice.

Lex Fridman (2:26:45.040)

Most of them are going to be repetitive because they...

Lex Fridman (2:26:47.920)

How do you know that?

Joscha Bach (2:26:48.880)

If they have to control the same problems every day. When I walk, there are going to

Joscha Bach (2:26:53.440)

be processes in my brain that model my walking pattern and regulate them and so on. But it's

Lex Fridman (2:26:58.160)

going to be pretty much the same every day.

Lex Fridman (2:26:59.840)

But that could be...

Lex Fridman (2:27:00.400)

Every step.

Lex Fridman (2:27:01.360)

But I'm talking about intellectual reasoning, thinking. So the question, what is the best

Joscha Bach (2:27:04.960)

system of government? So you sit down and start thinking about that. One of the constraints

Joscha Bach (2:27:09.360)

is that you don't have access to a lot of facts, a lot of studies. You always have to

Joscha Bach (2:27:16.560)

interface with something else to learn more, to aid in your reasoning process. If you can

Joscha Bach (2:27:23.600)

directly access all of Wikipedia in trying to understand what is the best form of government,

Joscha Bach (2:27:28.160)

then every thought won't be stuck in a loop. Every thought that requires some extra piece

Joscha Bach (2:27:33.360)

of information will be able to grab it really quickly. That's the possibility of if the

Joscha Bach (2:27:38.640)

bottleneck is literally the information that... The bottleneck of breakthrough ideas is just

Joscha Bach (2:27:47.600)

being able to quickly access huge amounts of information, then the possibility of connecting

Joscha Bach (2:27:51.840)

your brain to the computer could lead to totally new breakthroughs. You can think of mathematicians

Joscha Bach (2:27:59.760)

being able to just up the orders of magnitude of power in their reasoning about

Joscha Bach (2:28:08.240)

mathematical proofs. What if humanity has already discovered the optimal form of

Joscha Bach (2:28:12.400)

government through an evolutionary process? There is an evolution going on. So what we

Joscha Bach (2:28:17.840)

discover is that maybe the problem of government doesn't have stable solutions for us as a species,

Joscha Bach (2:28:23.280)

because we are not designed in such a way that we can make everybody conform to them.

Lex Fridman (2:28:28.720)

But there could be solutions that work under given circumstances or that are the best for

Joscha Bach (2:28:33.360)

certain environment and depends on, for instance, the primary forms of ownership and the means

Joscha Bach (2:28:38.400)

of production. So if the main means of production is land, then the forms of government will be

Joscha Bach (2:28:45.760)

regulated by the landowners and you get a monarchy. If you also want to have a form of

Joscha Bach (2:28:50.320)

government in which you depend on some form of slavery, for instance, where the peasants have

Joscha Bach (2:28:56.240)

to work very long hours for very little gain, so very few people can have plumbing, then maybe

Joscha Bach (2:29:01.680)

you need to promise them to get paid in the afterlife, the overtime. So you need a theocracy.

Lex Fridman (2:29:08.560)

And so for much of human history in the West, we had a combination of monarchy and theocracy

Joscha Bach (2:29:14.880)

that was our form of governance. At the same time, the Catholic Church implemented game theoretic

Joscha Bach (2:29:21.040)

principles. I recently reread Thomas Aquinas. It's very interesting to see this because he was not

Joscha Bach (2:29:27.040)

dualist. He was translating Aristotle in a particular way for designing an operating

Joscha Bach (2:29:32.560)

system for the Catholic society. And he says that basically people are animals in very much the same

Joscha Bach (2:29:39.440)

way as Aristotle envisions, which is basically organisms with cybernetic control. And then he

Joscha Bach (2:29:44.320)

says that there are additional rational principles that humans can discover and everybody can

Joscha Bach (2:29:48.880)

discover them so they are universal. If you are sane, you should understand, you should submit to

Joscha Bach (2:29:53.520)

them because you can rationally deduce them. And these principles are roughly you should be willing

Joscha Bach (2:30:00.240)

to self regulate correctly. You should be willing to do correct social regulation. It's

Joscha Bach (2:30:06.800)

intraorganismic. You should be willing to act on your models so you have skin in the game.

Lex Fridman (2:30:17.040)

And you should have goal rationality. You should be choosing the right

Joscha Bach (2:30:20.400)

goals to work on. So basically these three rational principles, goal rationality he calls

Joscha Bach (2:30:26.320)

prudence or wisdom, social regulation is justice, the correct social one, and the internal regulation

Joscha Bach (2:30:33.280)

is temperance. And this willingness to act on your models is courage. And then he says that

Joscha Bach (2:30:40.400)

there are additionally to these four cardinal virtues, three divine virtues. And these three

Joscha Bach (2:30:45.200)

divine virtues cannot be rationally deduced, but they reveal themselves by the harmony, which means

Joscha Bach (2:30:49.920)

if you assume them and you extrapolate what's going to happen, you will see that they make sense.

Lex Fridman (2:30:55.360)

And it's often been misunderstood as God has to tell you that these are the things. So basically

Joscha Bach (2:31:00.080)

there's something nefarious going on. The Christian conspiracy forces you to believe

Joscha Bach (2:31:05.360)

some guy with a long beard that they discovered this. So these principles are relatively simple.

Joscha Bach (2:31:11.840)

Again, it's for high level organization for the resulting civilization that you form.

Joscha Bach (2:31:16.480)

A commitment to unity. So basically you serve this higher, larger thing,

Joscha Bach (2:31:21.520)

this structural principle on the next level. And he calls that faith. Then there needs to be a

Joscha Bach (2:31:28.480)

commitment to shared purpose. This is basically this global reward that you try to figure out

Lex Fridman (2:31:32.160)

what that should be and how you can facilitate this. And this is love. The commitment to shared

Joscha Bach (2:31:36.000)

purpose is the core of love, right? You see the sacred thing that is more important than your own

Joscha Bach (2:31:40.960)

organismic interests in the other, and you serve this together. And this is how you see the sacred

Joscha Bach (2:31:45.840)

in the other. And the last one is hope, which means you need to be willing to act on that

Joscha Bach (2:31:51.520)

principle without getting rewards in the here and now because it doesn't exist yet.

Joscha Bach (2:31:55.520)

Then you start out building the civilization, right? So you need to be able to do this in the

Joscha Bach (2:31:59.520)

absence of its actual existence yet. So it can come into being. So the way it comes into being

Joscha Bach (2:32:06.640)

is by you accepting those notions and then you see these three divine concepts and you see them

Joscha Bach (2:32:12.960)

realized. Divine is a loaded concept in our world because we are outside of this cult and we are

Joscha Bach (2:32:18.800)

still scarred from breaking free of it. But the idea is basically we need to have a civilization

Joscha Bach (2:32:23.520)

that acts as an intentional agent, like an insect state. And we are not actually a tribal species,

Joscha Bach (2:32:28.960)

we are a state building species. And what enables state building is basically the formation of

Joscha Bach (2:32:35.520)

religious states and other forms of rule based administration in which the individual doesn't

Joscha Bach (2:32:40.240)

matter as much as the rule or the higher goal. We got there by the question, what's the optimal

Joscha Bach (2:32:45.600)

form of governance? So I don't think that Catholicism is the optimal form of governance

Joscha Bach (2:32:50.000)

because it's obviously on the way out, right? So it is for the present type of society that we are

Joscha Bach (2:32:54.400)

in. Religious institutions don't seem to be optimal to organize that. So what we discovered right now

Joscha Bach (2:33:01.280)

that we live in in the West is democracy. And democracy is the rule of oligarchs that are the

Joscha Bach (2:33:06.640)

people that currently own the means of production that is administered not by the oligarchs

Joscha Bach (2:33:11.440)

themselves because there's too much disruption. We have so much innovation that we have in every

Joscha Bach (2:33:17.360)

generation new means of production that we invent. And corporations die usually after 30 years or so

Lex Fridman (2:33:23.520)

and something other takes a leading role in our societies. So it's administered by institutions

Lex Fridman (2:33:29.280)

and these institutions themselves are not elected but they provide continuity and they are led by

Joscha Bach (2:33:35.680)

electable politicians. And this makes it possible that you can adapt to change without having to

Joscha Bach (2:33:40.640)

kill people, right? So you can, for instance, have a change in governments if people think that the

Joscha Bach (2:33:44.960)

current government is too corrupt or is not up to date, you can just elect new people. Or if a

Joscha Bach (2:33:50.160)

journalist finds out something inconvenient about the institution and the institution has no plan B

Joscha Bach (2:33:55.920)

like in Russia, the journalist has to die. This is when you run society by the deep state. So ideally

Joscha Bach (2:34:02.400)

you have an administration layer that you can change if something bad happens, right? So you

Joscha Bach (2:34:09.360)

will have a continuity in the whole thing. And this is the system that we came up in the West.

Lex Fridman (2:34:13.600)

And the way it's set up in the US is largely a result of low level models. So it's mostly just

Joscha Bach (2:34:17.920)

second, third order consequences that people are modeling in the design of these institutions. So

Joscha Bach (2:34:22.880)

it's a relatively young society that doesn't really take care of the downstream effects of

Joscha Bach (2:34:27.120)

many of the decisions that are being made. And I suspect that AI can help us this in a way if you

Joscha Bach (2:34:33.120)

can fix the incentives. The society of the US is a society of cheaters. It's basically cheating is

Lex Fridman (2:34:39.520)

so indistinguishable from innovation and we want to encourage innovation. Can you elaborate on what

Joscha Bach (2:34:44.000)

you mean by cheating? It's basically people do things that they know are wrong. It's acceptable

Joscha Bach (2:34:48.400)

to do things that you know are wrong in this society to a certain degree. You can, for instance,

Joscha Bach (2:34:52.720)

suggest some non sustainable business models and implement them. Right. But you're always pushing

Joscha Bach (2:34:57.920)

the boundaries. I mean, yes, this is seen as a good thing largely. Yes. And this is different

Joscha Bach (2:35:05.280)

from other societies. So for instance, social mobility is an aspect of this. Social mobility

Joscha Bach (2:35:09.520)

is the result of individual innovation that would not be sustainable at scale for everybody else.

Joscha Bach (2:35:14.720)

Right. Normally you should not go up, you should go deep, right? We need bakers and if we are very

Joscha Bach (2:35:18.960)

very good bakers, but in a society that innovates, maybe you can replace all the bakers with a really

Joscha Bach (2:35:23.760)

good machine. Right. And that's not a bad thing. And it's a thing that made the US so successful,

Joscha Bach (2:35:29.280)

right? But it also means that the US is not optimizing for sustainability, but for innovation.

Lex Fridman (2:35:34.880)

And so it's not obvious as the evolutionary process is unrolling, it's not obvious that that

Joscha Bach (2:35:39.600)

long term would be better. It has side effects. So you basically, if you cheat, you will have a

Joscha Bach (2:35:45.920)

certain layer of toxic sludge that covers everything that is a result of cheating.

Lex Fridman (2:35:50.400)

And we have to unroll this evolutionary process to figure out if these side effects are so damaging

Joscha Bach (2:35:55.600)

that the system is horrible, or if the benefits actually outweigh the negative effects.

Lex Fridman (2:36:03.600)

How do we get to which system of government is best? That was from,

Lex Fridman (2:36:07.920)

I'm trying to trace back the last like five minutes.

Joscha Bach (2:36:10.720)

I suspect that we can find a way back to AI by thinking about the way in which our brain has to

Joscha Bach (2:36:16.640)

organize itself. In some sense, our brain is a society of neurons. And our mind is a society

Joscha Bach (2:36:24.480)

of behaviors. And they need to be organizing themselves into a structure that implements

Joscha Bach (2:36:30.720)

regulation and government is social regulation. We often see government as the manifestation of

Joscha Bach (2:36:36.480)

power or local interests, but it's actually a platform for negotiating the conditions of human

Joscha Bach (2:36:40.960)

survival. And this platform emerges over the current needs and possibilities and the trajectory

Joscha Bach (2:36:46.960)

that we have. So given the present state, there are only so many options on how we can move into

Joscha Bach (2:36:52.560)

the next state without completely disrupting everything. And we mostly agree that it's a

Joscha Bach (2:36:56.400)

bad idea to disrupt everything because it will endanger our food supply for a while and the entire

Lex Fridman (2:37:01.200)

infrastructure and fabric of society. So we do try to find natural transitions,

Lex Fridman (2:37:06.960)

and there are not that many natural transitions available at any given point.

Lex Fridman (2:37:10.720)

What do you mean by natural transitions?

Lex Fridman (2:37:12.080)

So we try not to have revolutions if we can have it.

Joscha Bach (2:37:14.880)

Right. So speaking of revolutions and the connection between government systems and the mind,

Joscha Bach (2:37:21.760)

you've also said that in some sense, becoming an adult means you take charge of your emotions.

Joscha Bach (2:37:29.120)

Maybe you never said that. Maybe I just made that up. But in the context of the mind,

Lex Fridman (2:37:35.840)

what's the role of emotion? And what is it? First of all, what is emotion? What's its role?

Joscha Bach (2:37:42.240)

It's several things. So psychologists often distinguish between emotion and feeling,

Lex Fridman (2:37:46.880)

and in common day parlance, we don't. I think that emotion is a configuration of the cognitive

Joscha Bach (2:37:52.800)

system. And that's especially true for the lowest level for the affective state. So when you have

Joscha Bach (2:37:57.840)

an affect, it's the configuration of certain modulation parameters like arousal, valence,

Joscha Bach (2:38:03.920)

your attentional focus, whether it's wide or narrow, inter reception or extra reception,

Lex Fridman (2:38:08.080)

and so on. And all these parameters together put you in a certain way. You relate to the

Joscha Bach (2:38:13.040)

environment and to yourself, and this is in some sense an emotional configuration.

Joscha Bach (2:38:17.200)

In the more narrow sense, an emotion is an affective state. It has an object,

Lex Fridman (2:38:22.240)

and the relevance of that object is given by motivation. And motivation is a bunch of needs

Joscha Bach (2:38:26.640)

that are associated with rewards, things that give you pleasure and pain. And you don't actually act

Joscha Bach (2:38:31.440)

on your needs, you act on models of your needs. Because when the pleasure and pain manifest,

Joscha Bach (2:38:35.520)

it's too late, you've done everything. So you act on expectations that will give you pleasure and

Joscha Bach (2:38:40.480)

pain. And these are your purposes. The needs don't form a hierarchy, they just coexist and compete.

Lex Fridman (2:38:45.680)

And your brain has to find a dynamic homeostasis between them. But the purposes need to be

Joscha Bach (2:38:51.840)

consistent. So you basically can create a story for your life and make plans. And so we organize

Joscha Bach (2:38:57.920)

them all into hierarchies. And there is not a unique solution for this. Some people eat to make

Joscha Bach (2:39:02.160)

art and other people make art to eat. They might end up doing the same things, but they cooperate

Joscha Bach (2:39:07.200)

in very different ways. Because their ultimate goals are different. And we cooperate based on

Joscha Bach (2:39:12.800)

shared purpose. Everything else that is not cooperation on shared purpose is transactional.

Joscha Bach (2:39:16.800)

I don't think I understood that last piece of achieving the homeostasis.

Lex Fridman (2:39:26.640)

Are you distinguishing between the experience of emotion and the expression of emotion?

Joscha Bach (2:39:30.960)

Of course. So the experience of emotion is a feeling. And in this sense, what you feel is

Joscha Bach (2:39:37.520)

an appraisal that your perceptual system has made of the situation at hand. And it makes this based

Joscha Bach (2:39:42.800)

on your motivation and on your estimates, not your but of the subconscious geometric parts of your

Joscha Bach (2:39:50.400)

mind that assess the situation in the world with something like a neural network. And this neural

Joscha Bach (2:39:56.480)

network is making itself known to the symbolic parts of your mind, to your conscious attention

Joscha Bach (2:40:02.000)

by mapping them as features into a space. So what you will feel about your emotion is a projection

Joscha Bach (2:40:08.640)

usually into your body map. So you might feel anxiety in your solar plexus, and you might feel

Joscha Bach (2:40:12.800)

it as a contraction, which is all geometry. Your body map is the space that is always instantiated

Lex Fridman (2:40:18.880)

and always available. So it's a very obvious cheat if your non symbolic parts of your brain

Joscha Bach (2:40:26.960)

try to talk to your symbolic parts of your brain to map the feelings into the body map.

Lex Fridman (2:40:31.120)

And then you perceive them as pleasant and unpleasant, depending on whether the appraisal

Joscha Bach (2:40:35.040)

has a negative or positive valence. And then you have different features of them that give you

Joscha Bach (2:40:40.720)

more knowledge about the nature of what you're feeling. So for instance, when you feel

Joscha Bach (2:40:44.160)

connected to other people, you typically feel this in your chest region around the heart.

Lex Fridman (2:40:48.240)

And you feel this is an expansive feeling in which you're reaching out, right? And it's

Joscha Bach (2:40:53.200)

very intuitive to encode it like this. That's why it's encoded like this. It's a code in which the

Joscha Bach (2:40:59.760)

non symbolic parts of your mind talk to the symbolic ones. And then the expression of emotion

Joscha Bach (2:41:04.080)

is then the final step that could be sort of gestural or visual and so on. That's part of

Joscha Bach (2:41:09.680)

the communication. This probably evolved as part of an adversarial communication. So as soon as

Joscha Bach (2:41:14.480)

you started to observe the facial expression and posture of others to understand what emotional

Joscha Bach (2:41:19.360)

state they're in, others started to use this as signaling and also to subvert your model of their

Joscha Bach (2:41:23.920)

emotional state. So we now look at the inflections, at the difference between the standard face that

Joscha Bach (2:41:29.280)

they're going to make in this situation. When you are at a funeral, everybody expects you to make a

Joscha Bach (2:41:33.600)

solemn face, but the solemn face doesn't express whether you're sad or not. It just expresses that

Joscha Bach (2:41:38.080)

you understand what face you have to make at a funeral. Nobody should know that you are triumphant.

Lex Fridman (2:41:44.080)

So when you try to read the emotion of another person, you try to look at the delta

Joscha Bach (2:41:48.000)

between a truly sad expression and the things that are animating this face behind the curtain.

Lex Fridman (2:41:56.320)

So the interesting thing is, so having done this podcast and the video component, one of the things

Joscha Bach (2:42:03.920)

I've learned is that now I'm Russian and I just don't know how to express emotion on my face.

Joscha Bach (2:42:10.720)

One, I see that as weakness, but whatever. The people look to me after you say something,

Joscha Bach (2:42:16.960)

they look to my face to help them see how they should feel about what you said,

Joscha Bach (2:42:22.960)

which is fascinating because then they'll often comment on why did you look bored or why did you

Joscha Bach (2:42:28.000)

particularly enjoy that part or why did you whatever. It's a kind of interesting, it makes

Joscha Bach (2:42:32.800)

me cognizant of I'm part, like you're basically saying a bunch of brilliant things, but I'm

Joscha Bach (2:42:38.800)

part of the play that you're the key actor in by making my facial expressions and then

Joscha Bach (2:42:45.920)

therefore telling the narrative of what the big, like the big point is, which is fascinating.

Joscha Bach (2:42:51.200)

Makes me cognizant that I'm supposed to be making facial expressions. Even this conversation is hard

Joscha Bach (2:42:56.240)

because my preference would be to wear a mask with sunglasses to where I could just listen.

Joscha Bach (2:43:01.920)

Yes, I understand this because it's intrusive to interact with others this way. And basically

Lex Fridman (2:43:07.680)

Eastern European society have a taboo against that and especially Russia,

Joscha Bach (2:43:11.840)

the further you go to the East and in the US it's the opposite. You're expected to be

Lex Fridman (2:43:17.360)

hyperanimated in your face and you're also expected to show positive affect.

Joscha Bach (2:43:22.160)

Yes.

Lex Fridman (2:43:22.560)

And if you show positive affect without a good reason in Russia,

Joscha Bach (2:43:27.600)

people will think you are a stupid, unsophisticated person.

Joscha Bach (2:43:33.280)

Exactly. And here positive affect without reason goes either appreciated or goes unnoticed.

Lex Fridman (2:43:40.800)

No, it's the default. It's being expected. Everything is amazing. Have you seen these?

Lex Fridman (2:43:45.840)

Lego movie?

Joscha Bach (2:43:47.600)

No, there was a diagram where somebody gave the appraisals that exist in the US and Russia,

Lex Fridman (2:43:52.800)

so you have your bell curve. And the lower 10% in the US, it's a good start. Everything

Joscha Bach (2:44:02.080)

above the lowest 10%, it's amazing.

Lex Fridman (2:44:04.400)

It's amazing.

Lex Fridman (2:44:06.000)

And for Russians, everything below the top 10%, it's terrible. And then everything except the

Lex Fridman (2:44:14.800)

top percent is, I don't like it. And the top percent is even so.

Joscha Bach (2:44:23.680)

It's funny, but it's kind of true.

Joscha Bach (2:44:27.200)

There's a deeper aspect to this. It's also how we construct meaning in the US. Usually you focus on

Joscha Bach (2:44:33.440)

the positive aspects and you just suppress the negative aspects. And in our Eastern European

Joscha Bach (2:44:40.400)

traditions, we emphasize the fact that if you hold something above the waterline, you also need to

Joscha Bach (2:44:46.880)

put something below the waterline because existence by itself is as best neutral.

Joscha Bach (2:44:51.360)

Right. That's the basic intuition, at best neutral. Or it could be just suffering,

Joscha Bach (2:44:56.240)

the default is suffering.

Joscha Bach (2:44:56.800)

There are moments of beauty, but these moments of beauty are inextricably linked to the reality

Joscha Bach (2:45:02.160)

of suffering. And to not acknowledge the reality of suffering means that you are really stupid and

Joscha Bach (2:45:07.280)

unaware of the fact that basically every conscious being spends most of the time suffering.

Joscha Bach (2:45:12.560)

Yeah. You just summarized the ethos of the Eastern Europe. Yeah. Most of life is suffering

Joscha Bach (2:45:19.680)

with an occasional moments of beauty. And if your facial expressions don't acknowledge

Joscha Bach (2:45:24.480)

the abundance of suffering in the world and in existence itself, then you must be an idiot.

Joscha Bach (2:45:30.640)

It's an interesting thing when you raise children in the US and you, in some sense,

Joscha Bach (2:45:36.000)

preserve the identity of the intellectual and cultural traditions that are embedded in your

Lex Fridman (2:45:40.960)

own families. And your daughter asks you about Ariel the mermaid and asks you,

Lex Fridman (2:45:46.800)

why is Ariel not allowed to play with the humans? And you tell her the truth. She's a siren. Sirens

Joscha Bach (2:45:53.840)

eat people. You don't play with your food. It does not end well. And then you tell her the original

Joscha Bach (2:45:58.080)

story, which is not the one by Anderson, which is the romantic one. And there's a much darker one,

Joscha Bach (2:46:02.640)

which is Undine's story. What happened? So Undine is a mermaid or a water woman. She lives on the

Joscha Bach (2:46:11.040)

ground of a river and she meets this prince and they fall in love. And the prince really,

Joscha Bach (2:46:15.440)

really wants to be with her. And she says, okay, but the deal is you cannot have any other woman.

Joscha Bach (2:46:20.800)

If you marry somebody else, even though you cannot be with me, because obviously you cannot breathe

Joscha Bach (2:46:24.240)

underwater and have other things to do than managing your kingdom as you have here, you will

Joscha Bach (2:46:29.520)

die. And eventually after a few years, he falls in love with some princess and marries her. And

Joscha Bach (2:46:35.760)

she shows up and quietly goes into his chamber and nobody is able to stop her or willing to do

Lex Fridman (2:46:41.520)

so because she is fierce. And she comes quietly and sad out of his chamber. And they ask her,

Lex Fridman (2:46:47.440)

what has happened? What did you do? And she said, I kissed him to death.

Joscha Bach (2:46:52.000)

All done.

Lex Fridman (2:46:53.760)

And you know the Anderson story, right? In the Anderson story, the mermaid is playing with

Joscha Bach (2:46:59.280)

this prince that she saves and she falls in love with him and she cannot live out there. So she is

Joscha Bach (2:47:04.240)

giving up her voice and her tale for a human like appearance so she can walk among the humans. But

Joscha Bach (2:47:11.360)

this guy does not recognize that she is the one that you would marry. Instead, he marries somebody

Joscha Bach (2:47:16.880)

who has a kingdom and economical and political relationships to his own kingdom and so on,

Joscha Bach (2:47:22.160)

as he should. And she dies.

Joscha Bach (2:47:25.200)

Yeah. Instead, Disney, the Little Mermaid story has a little bit of a happy ending. That's the

Joscha Bach (2:47:35.040)

Western, that's the American way.

Joscha Bach (2:47:37.040)

My own problem is this, of course, that I read Oscar Wilde before I read the other things. So

Joscha Bach (2:47:41.280)

I'm indoctrinated, inoculated with this romanticism. And I think that the mermaid is right. You

Joscha Bach (2:47:46.960)

sacrifice your life for romantic love. That's what you do. Because if you are confronted with

Joscha Bach (2:47:51.520)

either serving the machine and doing the obviously right thing under the economic and social and

Lex Fridman (2:47:57.680)

other human incentives, that's wrong. You should follow your heart.

Lex Fridman (2:48:04.000)

So do you think suffering is fundamental to happiness along these lines?

Joscha Bach (2:48:09.520)

Suffering is the result of caring about things that you cannot change. And if you are able to

Joscha Bach (2:48:14.640)

change what you care about to those things that you can change, you will not suffer.

Lex Fridman (2:48:17.840)

But would you then be able to experience happiness?

Joscha Bach (2:48:22.160)

Yes. But happiness itself is not important. Happiness is like a cookie. When you are a child,

Joscha Bach (2:48:27.680)

you think cookies are very important and you want to have all the cookies in the world,

Joscha Bach (2:48:30.720)

you look forward to being an adult because then you have as many cookies as you want.

Lex Fridman (2:48:35.200)

But as an adult, you realize a cookie is a tool. It's a tool to make you eat vegetables.

Lex Fridman (2:48:40.000)

And once you eat your vegetables anyway, you stop eating cookies for the most part,

Lex Fridman (2:48:43.280)

because otherwise you will get diabetes and will not be around for your kids.

Joscha Bach (2:48:46.560)

Yes, but then the cookie, the scarcity of a cookie, if scarcity is enforced,

Lex Fridman (2:48:51.760)

nevertheless, so like the pleasure comes from the scarcity.

Joscha Bach (2:48:54.560)

Yes. But the happiness is a cookie that your brain bakes for itself. It's not made by the

Joscha Bach (2:48:59.760)

environment. The environment cannot make you happy. It's your appraisal of the environment

Joscha Bach (2:49:03.600)

that makes you happy. And if you can change the appraisal of the environment, which you can learn

Joscha Bach (2:49:07.840)

to, then you can create arbitrary states of happiness. And some meditators fall into this

Joscha Bach (2:49:11.920)

trap. So they discover the womb, this basement womb in their brain where the cookies are made,

Lex Fridman (2:49:16.560)

and they indulge and stuff themselves. And after a few months, it gets really old and

Joscha Bach (2:49:20.400)

the big crisis of meaning comes. Because they thought before that their unhappiness was the

Joscha Bach (2:49:25.360)

result of not being happy enough. So they fixed this, right? They can release the newer

Joscha Bach (2:49:29.600)

transmitters at will if they train. And then the crisis of meaning pops up in a deeper layer.

Lex Fridman (2:49:36.400)

And the question is, why do I live? How can I make a sustainable civilization that is meaningful to

Joscha Bach (2:49:40.480)

me? How can I insert myself into this? And this was the problem that you couldn't solve in the

Joscha Bach (2:49:44.240)

first place. But at the end of all this, let me then ask that same question. What is the answer

Joscha Bach (2:49:53.360)

to that? What could the possible answer be of the meaning of life? What could an answer be? What is

Joscha Bach (2:49:59.440)

it to you? I think that if you look at the meaning of life, you look at what the cell is. Life is the

Joscha Bach (2:50:06.000)

cell. Or this principle, the cell. It's this self organizing thing that can participate in evolution.

Joscha Bach (2:50:14.160)

In order to make it work, it's a molecular machine. It needs a self replicator and an

Joscha Bach (2:50:18.160)

entropy extractor and a Turing machine. If any of these parts is missing, you don't have a cell

Lex Fridman (2:50:22.560)

and it is not living. And life is basically the emergent complexity over that principle.

Joscha Bach (2:50:27.520)

Once you have this intelligent super molecule, the cell, there is very little that you cannot

Joscha Bach (2:50:32.560)

make it do. It's probably the optimal computronium and especially in terms of resilience. It's very

Joscha Bach (2:50:37.920)

hard to sterilize the planet once it's infected with life. So it's active function of these three

Joscha Bach (2:50:45.200)

components or the supercell cell is present in the cell, it's present in us, and it's just...

Joscha Bach (2:50:51.600)

We are just an expression of the cell. It's a certain layer of complexity in the organization

Joscha Bach (2:50:55.600)

of cells. So in a way, it's tempting to think of the cell as a von Neumann probe. If you want to

Joscha Bach (2:51:02.000)

build intelligence on other planets, the best way to do this is to infect them with cells

Lex Fridman (2:51:07.360)

and wait for long enough and there's a reasonable chance the stuff is going to evolve into an

Joscha Bach (2:51:11.600)

information processing principle that is general enough to become sentient.

Joscha Bach (2:51:16.400)

That idea is very akin to the same dream and beautiful ideas that are expressed to

Joscha Bach (2:51:21.360)

cellular automata in their most simple mathematical form. If you just inject the system with some

Joscha Bach (2:51:26.000)

basic mechanisms of replication and so on, basic rules, amazing things would emerge.

Joscha Bach (2:51:32.160)

The cell is able to do something that James Trardy calls existential design. He points out

Joscha Bach (2:51:38.000)

that in technical design, we go from the outside in. We work in a highly controlled environment in

Joscha Bach (2:51:42.880)

which everything is deterministic, like our computers, our labs, or our engineering workshops.

Lex Fridman (2:51:48.080)

And then we use this determinism to implement a particular kind of function that we dream up and

Joscha Bach (2:51:53.120)

that seamlessly interfaces with all the other deterministic functions that we already have in

Joscha Bach (2:51:57.600)

our world. So it's basically from the outside in. Biological systems designed from the inside out

Joscha Bach (2:52:04.080)

as seed will become a seedling by taking some of the relatively unorganized matter around it and

Joscha Bach (2:52:11.200)

turning it into its own structure and thereby subdue the environment. Cells can cooperate if

Joscha Bach (2:52:16.560)

they can rely on other cells having a similar organization that is already compatible. But

Joscha Bach (2:52:21.040)

unless that's there, the cell needs to divide to create that structure by itself. So it's a

Joscha Bach (2:52:27.280)

self organizing principle that works on a somewhat chaotic environment. And the purpose of life in

Joscha Bach (2:52:32.640)

this sense is to produce complexity. And the complexity allows you to harvest entropy gradients

Joscha Bach (2:52:38.960)

that you couldn't harvest without the complexity. And in this sense, intelligence and life are very

Joscha Bach (2:52:43.920)

strongly connected because the purpose of intelligence is to allow control under conditions

Lex Fridman (2:52:48.800)

and the conditions of complexity. So basically, you shift the boundary between the ordered systems

Joscha Bach (2:52:53.760)

into the realm of chaos. You build bridge heads into chaos with complexity. And this is what we

Joscha Bach (2:53:00.960)

are doing. This is not necessarily a deeper meaning. I think the meaning that we have priors

Joscha Bach (2:53:05.200)

for that we are all for outside of the priors, there is no meaning. Meaning only exists if the

Joscha Bach (2:53:09.280)

mind projects it. That is probably civilization. I think that what feels most meaningful to me is

Joscha Bach (2:53:16.720)

to try to build and maintain a sustainable civilization. And taking a slight step outside

Joscha Bach (2:53:22.880)

of that, we talked about a man with a beard and God, but something, some mechanism, perhaps must

Lex Fridman (2:53:34.400)

have planted the seed, the initial seed of the cell. Do you think there is a God? What is a God?

Lex Fridman (2:53:42.480)

And what would that look like? If there was no spontaneous biogenesis, in the sense that the

Joscha Bach (2:53:48.000)

first cell formed by some happy random accidents where the molecules just happened to be in the

Joscha Bach (2:53:54.160)

right constellation to each other. But there could also be the mechanism that allows for the random.

Joscha Bach (2:53:59.600)

I mean, there's like turtles all the way down. There seems to be, there has to be a head turtle

Joscha Bach (2:54:04.720)

at the bottom. Let's consider something really wild. Imagine, is it possible that a gas giant

Joscha Bach (2:54:10.480)

could become intelligent? What would that involve? So imagine you have vortices that spontaneously

Joscha Bach (2:54:16.240)

emerge on the gas giants, like big storm systems that endure for thousands of years.

Lex Fridman (2:54:21.360)

And some of these storm systems produce electromagnetic fields because some of the

Joscha Bach (2:54:24.880)

clouds are ferromagnetic or something. And as a result, they can change how certain clouds react

Joscha Bach (2:54:30.240)

rather than other clouds and thereby produce some self stabilizing patterns that eventually

Joscha Bach (2:54:34.960)

lead to regulation feedback loops, nested feedback loops and control. So imagine you have such this

Joscha Bach (2:54:40.560)

thing that basically has emergent self sustaining, self organizing complexity. And at some point,

Joscha Bach (2:54:44.960)

this breaks up and realizes and basically lam solaris, I am a thinking planet, but I will not

Joscha Bach (2:54:50.320)

replicate because I can recreate the conditions of my own existence somewhere else. I'm just

Joscha Bach (2:54:55.680)

basically an intelligence that has spontaneously formed because it could. And now it builds a

Joscha Bach (2:55:01.840)

von Neumann probe and the best von Neumann probe for such a thing might be the cell.

Lex Fridman (2:55:05.680)

So maybe it, because it's very, very clever and very enduring, creates cells and sends them out.

Lex Fridman (2:55:10.560)

And one of them has infected our planet. And I'm not suggesting that this is the case,

Lex Fridman (2:55:14.400)

but it would be compatible with the Prince Birmingham hypothesis. And it was my intuition

Joscha Bach (2:55:19.120)

that our biogenesis is very unlikely. It's possible, but you probably need to roll the

Joscha Bach (2:55:24.160)

cosmic dice very often, maybe more often than there are planetary surfaces. I don't know.

Lex Fridman (2:55:28.240)

So God is just a large enough, a system that's large enough that allows randomness.

Joscha Bach (2:55:37.360)

No, I don't think that God has anything to do with creation. I think it's a mistranslation

Joscha Bach (2:55:41.440)

of the Talmud into the Catholic mythology. I think that Genesis is actually the childhood

Joscha Bach (2:55:46.800)

memories of a God. So the, when. Sorry, Genesis is the.

Joscha Bach (2:55:51.600)

The childhood memories of a God. It's basically a mind that is remembering how it came into being.

Joscha Bach (2:55:57.280)

Wow.

Lex Fridman (2:55:57.840)

And we typically interpret Genesis as the creation of a physical universe by a supernatural being.

Joscha Bach (2:56:03.680)

Yes.

Lex Fridman (2:56:04.240)

And I think when you read it, there is light and darkness that is being created. And then you

Joscha Bach (2:56:12.160)

discover sky and ground, create them. You construct the plants and the animals and you

Joscha Bach (2:56:18.880)

give everything their names and so on. That's basically cognitive development. It's a sequence

Joscha Bach (2:56:24.320)

of steps that every mind has to go through when it makes sense of the world. And when you have

Joscha Bach (2:56:28.320)

children, you can see how initially they distinguish light and darkness and then they

Joscha Bach (2:56:33.040)

make out directions in it and they discover sky and ground and they discover the plants and the

Joscha Bach (2:56:37.120)

animals and they give everything their name. And it's a creative process that happens in every mind

Joscha Bach (2:56:41.520)

because it's not given. Your mind has to invent these structures to make sense of the patterns

Joscha Bach (2:56:46.160)

on your retina. Also, if there was some big nerd who set up a server and runs this world on it,

Joscha Bach (2:56:52.560)

this would not create a special relationship between us and the nerd. This nerd would not

Joscha Bach (2:56:57.120)

have the magical power to give meaning to our existence. So this equation of a creator god

Joscha Bach (2:57:03.280)

with the god of meaning is a sleight of hand. You shouldn't do it.

Lex Fridman (2:57:07.840)

The other one that is done in Catholicism is the equation of the first mover,

Joscha Bach (2:57:12.080)

the prime mover of Aristotle, which is basically the automaton that runs the universe. Aristotle

Joscha Bach (2:57:17.440)

says if things are moving and things seem to be moving here, something must move them. If something

Joscha Bach (2:57:23.600)

moves them, something must move the thing that is moving it. So there must be a prime mover.

Joscha Bach (2:57:28.080)

This idea to say that this prime mover is a supernatural being is complete nonsense.

Joscha Bach (2:57:33.280)

It's an automaton in the simplest case. So we have to explain the enormity that this automaton

Joscha Bach (2:57:39.120)

exists at all. But again, we don't have any possibility to infer anything about its properties

Joscha Bach (2:57:45.520)

except that it's able to produce change in information. So there needs to be some kind

Joscha Bach (2:57:51.440)

of computational principle. This is all there is. But to say this automaton is identical again with

Joscha Bach (2:57:56.400)

the creator of the first cause or with the thing that gives meaning to our life is confusion.

Joscha Bach (2:58:02.000)

No, I think that what we perceive is the higher being that we are part of. The higher being that

Joscha Bach (2:58:08.080)

we are part of is the civilization. It's the thing in which we have a similar relationship as the cell

Joscha Bach (2:58:13.200)

has to our body. And we have this prior because we have evolved to organize in these structures.

Lex Fridman (2:58:20.560)

So basically, the Christian God in its natural form without the mythology,

Lex Fridman (2:58:24.720)

if you undress it, is basically the platonic form of the civilization.

Lex Fridman (2:58:30.480)

Is the ideal?

Lex Fridman (2:58:32.480)

Yes, it's this ideal that you try to approximate when you interact with others,

Joscha Bach (2:58:36.240)

not based on your incentives, but on what you think is right.

Joscha Bach (2:58:38.880)

Wow, we covered a lot of ground. And we're left with one of my favorite lines, and there's many,

Joscha Bach (2:58:45.520)

which is happiness is a cookie that the brain bakes itself. It's been a huge honor and a

Lex Fridman (2:58:54.880)

pleasure to talk to you. I'm sure our paths will cross many times again.

Joscha Bach (2:58:59.680)

Joshua, thank you so much for talking today. I really appreciate it.

Lex Fridman (2:59:02.480)

Thank you, Lex. It was so much fun. I enjoyed it.

Joscha Bach (2:59:05.760)

Awesome. Thanks for listening to this conversation with Joshua Bach. And thank you to our sponsors,

Joscha Bach (2:59:12.080)

ExpressVPN and Cash App. Please consider supporting this podcast by getting ExpressVPN at

Joscha Bach (2:59:18.000)

expressvpn.com slash lexpod and downloading Cash App and using code lexpodcast. If you enjoy this

Joscha Bach (2:59:27.360)

thing, subscribe on YouTube, review it with five stars in Apple Podcast, support it on Patreon,

Joscha Bach (2:59:33.440)

or simply connect with me on Twitter at lexfreedman. And yes, try to figure out how to

Joscha Bach (2:59:39.440)

spell it without the E. And now let me leave you with some words of wisdom from Joshua Bach.

Joscha Bach (2:59:46.400)

If you take this as a computer game metaphor, this is the best level for humanity to play.

Lex Fridman (2:59:52.720)

And this best level happens to be the last level, as it happens against the backdrop of a dying

Joscha Bach (2:59:59.760)

world. But it's still the best level. Thank you for listening and hope to see you next time.

Joscha Bach (30:05.040)

When we learn, we at some point discover that everything that we sense is part of the same

Joscha Bach (30:10.960)

object, which means we learn it all into one model and we call this model the universe.

Lex Fridman (30:14.800)

So the experience of the world that we are embedded on is not a secret direct via to physical

Joscha Bach (30:19.680)

reality. Physical reality is a weird quantum graph that we can never experience or get access to.

Lex Fridman (30:24.720)

But it has these properties that it can create certain patterns that are systemic interface to

Joscha Bach (30:29.600)

the world. And we make sense of these patterns and the relationship between the patterns that

Joscha Bach (30:33.520)

we discover is what we call the physical universe. So at some point in our development as a nervous

Joscha Bach (30:40.960)

system, we discover that everything that we relate to in the world can be mapped to a region in the

Joscha Bach (30:47.120)

same three dimensional space, by and large. We now know in physics that this is not quite true.

Joscha Bach (30:52.720)

The world is not actually three dimensional, but the world that we are entangled with at the level

Joscha Bach (30:56.960)

which we are entangled with is largely a flat three dimensional space. And so this is the

Joscha Bach (31:02.720)

model that our brain is intuitively making. And this is, I think, what gave rise to this intuition

Joscha Bach (31:07.840)

of res extensa of this material world, this material domain. It's one of the mental domains,

Lex Fridman (31:12.560)

but it's just the class of all models that relate to this environment, this three dimensional

Joscha Bach (31:17.440)

physics engine in which we are embedded. Physics engine which we're embedded. I love that. Just

Joscha Bach (31:22.800)

slowly pause. So the quantum graph, I think you called it, which is the real world, which you

Joscha Bach (31:32.800)

can never get access to, there's a bunch of questions I want to sort of disentangle that.

Lex Fridman (31:37.280)

But maybe one useful one, one of your recent talks I looked at, can you just describe the basics?

Lex Fridman (31:43.840)

Can you talk about what is dualism? What is idealism? What is materialism? What is functionalism?

Lex Fridman (31:49.920)

And what connects with you most in terms of, because you just mentioned there's a reality

Lex Fridman (31:53.600)

we don't have access to. Okay. What does that even mean? And why don't we get access to it?

Joscha Bach (32:00.160)

Aren't we part of that reality? Why can't we access it? So the particular trajectory that

Joscha Bach (32:05.040)

mostly exists in the West is the result of our indoctrination by a cult for 2000 years.

Joscha Bach (32:11.120)

A cult? Which one? Oh, 2000 years. The Catholic cult mostly. And for better or worse,

Joscha Bach (32:15.600)

it has created or defined many of the modes of interaction that we have that has created

Joscha Bach (32:20.800)

the society. But it has also in some sense scarred our rationality. And the intuition that exists,

Joscha Bach (32:29.920)

if you would translate the mythology of the Catholic church into the modern world is that

Joscha Bach (32:35.520)

the world in which you and me interact is something like a multiplayer role playing adventure. And the

Joscha Bach (32:41.680)

money and the objects that we have in this world, this is all not real. Or as Eastern philosophers

Joscha Bach (32:48.080)

would say, it's Maya. It's just stuff that appears to be meaningful. And this embedding in this

Joscha Bach (32:54.320)

meaning, if you believe in it, is samsara. It's basically the identification with the needs of

Joscha Bach (33:00.160)

the mundane, secular, everyday existence. And the Catholics also introduced the notion of

Joscha Bach (33:06.880)

higher meaning, the sacred. And this existed before, but eventually the natural shape of God

Joscha Bach (33:12.480)

is the Platonic form of the civilization that you're part of. It's basically the superorganism

Joscha Bach (33:17.040)

that is formed by the individuals as an intentional agent. And basically, the Catholics

Joscha Bach (33:22.640)

used a relatively crude mythology to implement software on the minds of people and get the

Joscha Bach (33:28.240)

software synchronized to make them walk on lockstep, to basically get this God online

Lex Fridman (33:34.240)

and to make it efficient and effective. And I think God technically is just a self that

Joscha Bach (33:40.320)

spends multiple brains as opposed to your and my self, which mostly exists just on one brain.

Joscha Bach (33:45.280)

Right? And so in some sense, you can construct a self functionally as a function is implemented

Lex Fridman (33:50.240)

by brains that exists across brains. And this is a God with a small g.

Joscha Bach (33:54.720)

That's one of the, if you, Yuval Harari kind of talking about,

Lex Fridman (33:59.920)

this is one of the nice features of our brains. It seems to that we can

Joscha Bach (34:03.600)

all download the same piece of software like God in this case and kind of share it.

Lex Fridman (34:07.680)

Yeah. So basically you give everybody a spec and the mathematical constraints

Joscha Bach (34:12.240)

that are intrinsic to information processing,

Lex Fridman (34:16.080)

make sure that given the same spec, you come up with a compatible structure.

Joscha Bach (34:20.160)

Okay. So that's, there's the space of ideas that we all share. And we think that's kind

Lex Fridman (34:24.160)

of the mind, but that's separate from the idea is from Christianity for,

Joscha Bach (34:32.160)

from religion is that there's a separate thing between the mind.

Lex Fridman (34:35.200)

There is a real world. And this real world is the world in which God exists.

Joscha Bach (34:39.600)

God is the coder of the multiplayer adventure, so to speak. And we are all players in this game.

Lex Fridman (34:45.840)

And that's dualism.

Joscha Bach (34:47.280)

Yes. But the aspect is because the mental realm exists in a different implementation

Joscha Bach (34:53.680)

than the physical realm. And the mental realm is real. And a lot of people have this intuition

Joscha Bach (34:59.280)

that there is this real room in which you and me talk and speak right now, then comes a layer of

Joscha Bach (35:04.880)

physics and abstract rules and so on. And then comes another real room where our souls are

Lex Fridman (35:10.160)

and our true form isn't a thing that gives us phenomenal experience. And this is, of course,

Joscha Bach (35:14.400)

a very confused notion that you would get. And it's basically, it's the result of connecting

Joscha Bach (35:20.880)

materialism and idealism in the wrong way.

Joscha Bach (35:24.800)

So, okay. I apologize, but I think it's really helpful if we just try to define,

Joscha Bach (35:30.240)

try to define terms. Like what is dualism? What is idealism? What is materialism? For

Lex Fridman (35:34.160)

people that don't know.

Lex Fridman (35:34.960)

So the idea of dualism in our cultural tradition is that there are two substances, a mental

Joscha Bach (35:40.000)

substance and a physical substance. And they interact by different rules. And the physical

Joscha Bach (35:46.000)

world is basically causally closed and is built on a low level causal structure. So

Joscha Bach (35:51.600)

they're basically a bottom level that is causally closed. That's entirely mechanical

Lex Fridman (35:56.160)

and mechanical in the widest sense. So it's computational. There's basically a physical

Joscha Bach (36:00.240)

world in which information flows around and physics describes the laws of how information

Joscha Bach (36:05.200)

flows around in this world.

Lex Fridman (36:06.560)

Would you compare it to like a computer where you have hardware and software?

Joscha Bach (36:10.400)

The computer is a generalization of information flowing around. Basically,

Lex Fridman (36:14.080)

but if you want to discover that there is a universal principle, you can define this

Joscha Bach (36:17.760)

universal machine that is able to perform all the computations. So all these machines

Joscha Bach (36:23.680)

have the same power. This means that you can always define a translation between them,

Joscha Bach (36:27.840)

as long as they have unlimited memory to be able to perform each other's computations.

Lex Fridman (36:33.680)

So would you then say that materialism is this whole world is just the hardware and

Lex Fridman (36:38.480)

idealism is this whole world is just the software?

Joscha Bach (36:40.800)

Not quite. I think that most idealists don't have a notion of software yet because software

Joscha Bach (36:46.640)

also comes down to information processing. So what you notice is the only thing that

Joscha Bach (36:51.920)

is real to you and me is this experiential world in which things matter, in which things

Joscha Bach (36:56.000)

have taste, in which things have color, phenomenal content, and so on.

Lex Fridman (37:00.080)

You are bringing up consciousness. Okay.

Joscha Bach (37:02.160)

This is distinct from the physical world in which things have values only in an abstract

Joscha Bach (37:07.760)

sense. And you only look at cold patterns moving around. So how does anything feel like

Joscha Bach (37:15.040)

something? And this connection between the two things is very puzzling to a lot of people,

Joscha Bach (37:19.200)

of course, too many philosophers. So idealism starts out with the notion that mind is primary,

Joscha Bach (37:23.360)

materialism, things that matter is primary. And so for the idealist, the material patterns that

Joscha Bach (37:30.320)

we see playing out are part of the dream that the mind is dreaming. And we exist in a mind

Joscha Bach (37:37.120)

on a higher plane of existence, if you want. And for the materialist, there is only this

Joscha Bach (37:43.360)

material thing, and that generates some models, and we are the result of these models. And in

Joscha Bach (37:49.600)

some sense, I don't think that we should understand, if we understand it properly,

Joscha Bach (37:53.600)

materialism and idealism as a dichotomy, but as two different aspects of the same thing.

Lex Fridman (37:59.760)

So the weird thing is we don't exist in the physical world. We do exist inside of a story

Lex Fridman (38:04.000)

that the brain tells itself. Okay. Let my information processing take that in.

Joscha Bach (38:15.040)

We don't exist in the physical world. We exist in the narrative.

Joscha Bach (38:18.080)

Basically, a brain cannot feel anything. A neuron cannot feel anything. They're physical

Joscha Bach (38:22.160)

things. Physical systems are unable to experience anything. But it would be very useful for the

Joscha Bach (38:26.720)

brain or for the organism to know what it would be like to be a person and to feel something.

Joscha Bach (38:30.960)

Yeah. So the brain creates a simulacrum of such a person that it uses to model the interactions

Joscha Bach (38:36.720)

of the person. It's the best model of what that brain, this organism thinks it is in relationship

Joscha Bach (38:41.680)

to its environment. So it creates that model. It's a story, a multimedia novel that the brain

Lex Fridman (38:46.000)

is continuously writing and updating. But you also kind of said that

Joscha Bach (38:50.000)

you said that we kind of exist in that story. Yes.

Joscha Bach (38:53.760)

In that story. What is real in any of this? So again, these terms are... You kind of said

Joscha Bach (39:04.800)

there's a quantum graph. I mean, what is this whole thing running on then? Is the story...

Lex Fridman (39:11.040)

And is it completely fundamentally impossible to get access to it? Because isn't the story

Lex Fridman (39:16.640)

supposed to... Isn't the brain in something existing in some kind of context?

Lex Fridman (39:24.560)

So what we can identify as computer scientists, we can engineer systems and test our theories this

Joscha Bach (39:30.640)

way that might have the necessary insufficient properties to produce the phenomena that we are

Joscha Bach (39:36.480)

observing, which is the self in a virtual world that is generated in somebody's neocortex that is

Joscha Bach (39:44.240)

contained in the skull of this primate here. And when I point at this, this indexicality is of

Joscha Bach (39:48.480)

course wrong. But I do create something that is likely to give rise to patterns on your retina

Joscha Bach (39:55.440)

that allow you to interpret what I'm saying. But we both know that the world that you and me are

Joscha Bach (40:00.480)

seeing is not the real physical world. What we are seeing is a virtual reality generated in your

Joscha Bach (40:05.760)

brain to explain the patterns on your retina. How close is it to the real world? That's kind

Joscha Bach (40:10.000)

of the question. Is it when you have people like Donald Hoffman that say that you're really far

Joscha Bach (40:18.400)

away. The thing we're seeing, you and I now, that interface we have is very far away from anything.

Joscha Bach (40:24.960)

We don't even have anything close to the sense of what the real world is. Or is it a very surface

Joscha Bach (40:29.840)

piece of architecture? I imagine you look at the Mandelbrot fractal, this famous thing that

Joscha Bach (40:35.840)

Bernard Mandelbrot discovered. You see an overall shape in there. But if you truly understand it,

Joscha Bach (40:43.040)

you know it's two lines of code. It's basically in a series that is being tested for complex

Joscha Bach (40:50.080)

numbers in the complex number plane for every point. And for those where the series is diverging,

Joscha Bach (40:56.240)

you paint this black. And where it's converging, you don't. And you get the intermediate colors

Joscha Bach (41:04.160)

by taking how far it diverges. This gives you this shape of this fractal. But imagine you live

Joscha Bach (41:13.040)

inside of this fractal and you don't have access to where you are in the fractal. Or you have not

Joscha Bach (41:18.160)

discovered the generator function even. So what you see is, all I can see right now is a spiral.

Lex Fridman (41:23.520)

And this spiral moves a little bit to the right. Is this an accurate model of reality? Yes, it is.

Joscha Bach (41:28.000)

It is an adequate description. You know that there is actually no spiral in the Mandelbrot fractal.

Joscha Bach (41:33.680)

It only appears like this to an observer that is interpreting things as a two dimensional space and

Joscha Bach (41:39.200)

then defines certain regularities in there at a certain scale that it currently observes. Because

Joscha Bach (41:44.000)

if you zoom in, the spiral might disappear and turn out to be something different at a different

Joscha Bach (41:47.680)

resolution. So at this level, you have the spiral. And then you discover the spiral moves to the

Joscha Bach (41:52.240)

right and at some point it disappears. So you have a singularity. At this point, your model is no

Joscha Bach (41:56.960)

longer valid. You cannot predict what happens beyond the singularity. But you can observe again

Lex Fridman (42:01.520)

and you will see it hit another spiral and at this point it disappeared. So we now have a second

Joscha Bach (42:06.320)

order law. And if you make 30 layers of these laws, then you have a description of the world

Joscha Bach (42:11.040)

that is similar to the one that we come up with when we describe the reality around us.

Joscha Bach (42:14.720)

It's reasonably predictive. It does not cut to the core of it. It does not explain how it's

Joscha Bach (42:19.440)

being generated, how it actually works. But it's relatively good to explain the universe that we

Joscha Bach (42:24.560)

are entangled with. But you don't think the tools of computer science, the tools of physics could

Joscha Bach (42:28.560)

get, could step outside, see the whole drawing and get at the basic mechanism of how the pattern,

Joscha Bach (42:35.040)

the spirals are generated. Imagine you would find yourself embedded into a motherboard fractal and

Joscha Bach (42:40.320)

you try to figure out what works and you somehow have a Turing machine with enough memory to think.

Lex Fridman (42:46.000)

And as a result, you come to this idea, it must be some kind of automaton. And maybe you just

Joscha Bach (42:51.920)

enumerate all the possible automata until you get to the one that produces your reality.

Lex Fridman (42:56.320)

So you can identify necessary and sufficient condition. For instance,

Joscha Bach (42:59.920)

we discover that mathematics itself is the domain of all languages. And then we see that most of

Joscha Bach (43:05.440)

the domains of mathematics that we have discovered are in some sense describing the same fractals.

Joscha Bach (43:10.320)

This is what category theory is obsessed about, that you can map these different domains to each

Joscha Bach (43:14.320)

other. So they're not that many fractals. And some of these have interesting structure and

Joscha Bach (43:19.360)

symmetry breaks. And so you can discover what region of this global fractal you might be embedded

Joscha Bach (43:26.480)

in from first principles. But the only way you can get there is from first principles. So basically

Joscha Bach (43:31.200)

your understanding of the universe has to start with automata and then number theory and then

Joscha Bach (43:35.760)

spaces and so on. Yeah. I think like Stephen Wolfram still dreams that he'll be able to arrive

Joscha Bach (43:41.520)

at the fundamental rules of the cellular automata or the generalization of which

Joscha Bach (43:46.560)

is behind our universe. Yeah. You've said on this topic, you said in a recent conversation

Joscha Bach (43:54.160)

that quote, some people think that a simulation can't be conscious and only a physical system can,

Lex Fridman (44:00.560)

but they got it completely backward. A physical system cannot be conscious. Only a simulation can

Joscha Bach (44:05.920)

be conscious. Consciousness is a simulated property that simulates itself. Just like you said,

Joscha Bach (44:11.920)

the mind is kind of the, we'll call it story narrative. There's a simulation. So our mind

Joscha Bach (44:17.920)

is essentially a simulation. Usually I try to use the terminology so that the mind is basically

Joscha Bach (44:24.560)

a principles that produce the simulation. It's the software that is implemented by your brain.

Lex Fridman (44:29.760)

And the mind is creating both the universe that we are in and the self, the idea of a person that

Joscha Bach (44:36.560)

is on the other side of attention and is embedded in this world. Why is that important that

Joscha Bach (44:41.600)

idea of a self, why is that an important feature in the simulation? It's basically a result of

Joscha Bach (44:48.160)

the purpose that the mind has. It's a tool for modeling, right? We are not actually monkeys. We

Joscha Bach (44:53.120)

are side effects of the regulation needs of monkeys. And what the monkey has to regulate is

Joscha Bach (45:00.560)

the relationship of an organism to an outside world that is in large part also consisting of

Joscha Bach (45:06.240)

other organisms. And as a result, it basically has regulation targets that it tries to get to.

Joscha Bach (45:12.400)

These regulation targets start with priors. They're basically like unconditional reflexes

Joscha Bach (45:16.640)

that we are more or less born with. And then we can reverse engineer them to make them more

Joscha Bach (45:20.480)

consistent. And then we get more detailed models about how the world works and how to interact with

Joscha Bach (45:24.720)

it. And so these priors that you commit to are largely target values that our needs should

Joscha Bach (45:31.040)

approach set points. And this deviation to the set point creates some urge, some tension. And we find

Joscha Bach (45:38.160)

ourselves living inside of feedback loops, right? Consciousness emerges over dimensions of

Joscha Bach (45:42.640)

disagreements with the universe, things that you care, things are not the way they should be,

Lex Fridman (45:48.000)

but you need to regulate. And so in some sense, the sense self is the result of all the

Joscha Bach (45:52.560)

identifications that you're having. And that identification is a regulation target that

Joscha Bach (45:56.560)

you're committing to. It's a dimension that you care about, you think is important. And this is

Joscha Bach (46:01.440)

also what locks you in. If you let go of these commitments, of these identifications, you get

Joscha Bach (46:07.040)

free. There's nothing that you have to do anymore. And if you let go of all of them, you're completely

Joscha Bach (46:12.000)

free and you can enter nirvana because you're done. And actually, this is a good time to pause and say

Joscha Bach (46:17.920)

thank you to a friend of mine, Gustav Soderström, who introduced me to your work. I wanted to give

Joscha Bach (46:23.440)

him a shout out. He's a brilliant guy. And I think the AI community is actually quite amazing. And

Joscha Bach (46:29.360)

Gustav is a good representative of that. You are as well. So I'm glad, first of all, I'm glad the

Joscha Bach (46:34.160)

internet exists, YouTube exists, where I can watch your talks and then get to your book and study

Joscha Bach (46:40.640)

your writing and think about, you know, that's amazing. Okay. But you've kind of described

Joscha Bach (46:46.800)

sort of this emergent phenomenon of consciousness from the simulation. So what about the hard

Joscha Bach (46:52.720)

problem of consciousness? Can you just linger on it? Why does it still feel like, I understand

Joscha Bach (47:02.960)

you're kind of, the self is an important part of the simulation, but why does the simulation

Joscha Bach (47:08.800)

feel like something? So if you look at a book by, say, George R. R. Martin, where the characters

Joscha Bach (47:14.880)

have plausible psychology and they stand on a hill because they want to conquer the city below

Joscha Bach (47:19.840)

the hill and they're done in it. And they look at the color of the sky and they are apprehensive

Lex Fridman (47:24.080)

and feel empowered and all these things. Why do they have these emotions? It's because it's

Joscha Bach (47:27.920)

written into the story, right? And it's written to the story because there's an adequate model of the

Joscha Bach (47:32.080)

person that predicts what they're going to do next. And the same thing is true for us. So it's

Joscha Bach (47:37.680)

basically a story that our brain is writing. It's not written in words. It's written in perceptual

Joscha Bach (47:43.040)

content, basically multimedia content. And it's a model of what the person would feel if it existed.

Lex Fridman (47:50.560)

So it's a virtual person. And you and me happen to be this virtual person. So this virtual person

Joscha Bach (47:56.160)

gets access to the language center and talks about the sky being blue. And this is us.

Lex Fridman (48:01.760)

But hold on a second. Do I exist in your simulation?

Joscha Bach (48:05.440)

You do exist in an almost similar way as me. So there are internal states that are less

Joscha Bach (48:13.120)

accessible for me that you have and so on. And my model might not be completely adequate.

Joscha Bach (48:20.640)

There are also things that I might perceive about you that you don't perceive. But in some sense,

Joscha Bach (48:25.120)

both you and me are some puppets, two puppets that enact a play in my mind. And I identify

Joscha Bach (48:31.360)

with one of them because I can control one of the puppets directly. And with the other one,

Joscha Bach (48:36.080)

I can create things in between. So for instance, we can go on an interaction that even leads to

Joscha Bach (48:41.680)

a coupling to a feedback loop. So we can think things together in a certain way or feel things

Joscha Bach (48:46.400)

together. But this coupling is itself not a physical phenomenon. It's entirely a software

Joscha Bach (48:50.880)

phenomenon. It's the result of two different implementations interacting with each other.

Lex Fridman (48:54.960)

So that's interesting. So are you suggesting, like the way you think about it, is the entirety

Joscha Bach (49:02.080)

of existence a simulation and where kind of each mind is a little subsimulation,

Lex Fridman (49:08.640)

that like, why don't you, why doesn't your mind have access to my mind's full state?

Joscha Bach (49:18.640)

Like, for the same reason that my mind doesn't have access to its own full state.

Lex Fridman (49:22.880)

So what, I mean,

Lex Fridman (49:25.760)

There is no trick involved. So basically, when I know something about myself,

Joscha Bach (49:29.280)

it's because I made a model. So one part of your brain is tasked with modeling what other parts of

Lex Fridman (49:33.760)

your brain are doing.

Joscha Bach (49:35.200)

Yes. But there seems to be an incredible consistency about this world in the physical

Joscha Bach (49:40.480)

sense that there's repeatable experiments and so on. How does that fit into our silly,

Joscha Bach (49:46.560)

the center of apes simulation of the world? So why is it so repeatable? Why is everything so

Joscha Bach (49:50.960)

repeatable? And not everything. There's a lot of fundamental physics experiments that are repeatable

Lex Fridman (49:59.520)

for a long time, all over the place and so on. Laws of physics. How does that fit in?

Joscha Bach (50:05.040)

It seems that the parts of the world that are not deterministic are not long lived.

Lex Fridman (50:10.320)

So if you build a system, any kind of automaton, so if you build simulations of something,

Joscha Bach (50:17.040)

you'll notice that the phenomena that endure are those that give rise to stable dynamics.

Lex Fridman (50:23.520)

So basically, if you see anything that is complex in the world, it's the result of usually of some

Joscha Bach (50:28.000)

control of some feedback that keeps it stable around certain attractors. And the things that

Joscha Bach (50:32.640)

are not stable that don't give rise to certain harmonic patterns and so on, they tend to get

Joscha Bach (50:37.200)

weeded out over time. So if we are in a region of the universe that sustains complexity, which is

Joscha Bach (50:44.400)

required to implement minds like ours, this is going to be a region of the universe that is very

Joscha Bach (50:50.160)

tightly controlled and controllable. So it's going to have lots of interesting symmetries and also

Joscha Bach (50:55.680)

symmetry breaks that allow the creation of structure. But they exist where? So there's

Joscha Bach (51:02.800)

such an interesting idea that our mind is simulation that's constructing the narrative.

Lex Fridman (51:06.240)

But my question is, just to try to understand how that fits with this, with the entirety of the

Joscha Bach (51:14.880)

universe, you're saying that there's a region of this universe that allows enough complexity to

Joscha Bach (51:19.360)

create creatures like us. But what's the connection between the brain, the mind, and the broader

Joscha Bach (51:27.280)

universe? Which comes first? Which is more fundamental? Is the mind the starting point,

Lex Fridman (51:32.720)

the universe is emergent? Is the universe the starting point, the minds are emergent?

Joscha Bach (51:37.680)

I think quite clearly the latter. That's at least a much easier explanation because it allows us to

Joscha Bach (51:42.480)

make causal models. And I don't see any way to construct an inverse causality.

Lex Fridman (51:47.440)

So what happens when you die to your mind simulation?

Joscha Bach (51:51.760)

My implementation ceases. So basically the thing that implements myself will no longer be present,

Joscha Bach (51:57.600)

which means if I am not implemented on the minds of other people, the thing that I identify with,

Joscha Bach (52:01.680)

the weird thing is I don't actually have an identity beyond the identity that I construct.

Joscha Bach (52:07.680)

If I was the Dalai Lama, he identifies as a form of government. So basically the Dalai Lama gets

Joscha Bach (52:14.160)

reborn, not because he's confused, but because he is not identifying as a human being. He runs on

Joscha Bach (52:21.760)

a human being. He's basically a governmental software that is instantiated in every new

Joscha Bach (52:27.040)

generation and you. So his advice is to pick someone who does this in the next generation.

Lex Fridman (52:31.680)

So if you identify with this, you are no longer a human and you don't die in the sense that what

Joscha Bach (52:37.280)

dies is only the body of the human that you run on. To kill the Dalai Lama, you would have to kill

Joscha Bach (52:42.320)

his tradition. And if we look at ourselves, we realize that we are to a small part like this,

Joscha Bach (52:48.160)

most of us. So for instance, if you have children, you realize something lives on in them. Or if you

Joscha Bach (52:53.360)

spark an idea in the world, something lives on, or if you identify with the society around you,

Joscha Bach (52:58.400)

because you are in part that you're not just this human being.

Lex Fridman (53:01.280)

Yeah. So in a sense, you are kind of like a Dalai Lama in the sense that you,

Joscha Bach (53:07.040)

Joshua Bach, is just a collection of ideas. So like you have this operating system on which

Joscha Bach (53:12.160)

a bunch of ideas live and interact. And then once you die, they kind of part, some of them

Joscha Bach (53:17.760)

jump off the ship.

Joscha Bach (53:18.880)

You put it put it the other way. Identity is a software state. It's a construction.

Joscha Bach (53:23.360)

It's not physically real. Identity is not a physical concept.

Lex Fridman (53:27.920)

It's basically a representation of different objects on the same world line.

Lex Fridman (53:32.160)

But identity lives and dies. Are you attached? What's the fundamental thing? Is it the ideas

Lex Fridman (53:41.120)

that come together to form identity? Or is each individual identity actually a fundamental thing?

Joscha Bach (53:46.400)

It's a representation that you can get agency over if you care. So basically,

Lex Fridman (53:49.920)

you can choose what you identify with if you want to.

Joscha Bach (53:53.120)

No, but it just seems if the mind is not real, that the birth and death is not a crucial part

Joscha Bach (54:04.400)

of it. Well, maybe I'm silly. Maybe I'm attached to this whole biological organism. But it seems

Joscha Bach (54:16.000)

that being a physical object in this world is an important aspect of birth and death.

Joscha Bach (54:23.840)

Like it feels like it has to be physical to die. It feels like simulations don't have to die.

Joscha Bach (54:30.000)

The physics that we experience is not the real physics. There is no color and sound in the real

Joscha Bach (54:34.640)

world. Color and sound are types of representations that you get if you want to model reality with

Joscha Bach (54:40.560)

oscillators. So colors and sound in some sense have octaves, and it's because they are represented

Joscha Bach (54:45.920)

probably with oscillators. So that's why colors form a circle of use. And colors have harmonics,

Joscha Bach (54:52.320)

sounds have harmonics as a result of synchronizing oscillators in the brain. So the world that we

Joscha Bach (54:58.240)

subjectively interact with is fundamentally the result of the representation mechanisms in our

Joscha Bach (55:03.680)

brain. They are mathematically to some degree universal. There are certain regularities that

Joscha Bach (55:08.000)

you can discover in the patterns and not others. But the patterns that we get, this is not the real

Joscha Bach (55:12.720)

world. The world that we interact with is always made of too many parts to count. So when you look

Joscha Bach (55:18.400)

at this table and so on, it's consisting of so many molecules and atoms that you cannot count

Joscha Bach (55:23.440)

them. So you only look at the aggregate dynamics, at limit dynamics. If you had almost infinitely

Joscha Bach (55:29.120)

many particles, what would be the dynamics of the table? And this is roughly what you get. So

Joscha Bach (55:34.400)

geometry that we are interacting with is the result of discovering those operators that

Joscha Bach (55:39.680)

work in the limit that you get by building an infinite series that converges. For those parts

Joscha Bach (55:44.480)

where it converges, it's geometry. For those parts where it doesn't converge, it's chaos.

Joscha Bach (55:48.960)

Right. And then so all of that is filtered through the consciousness that's emergent in our

Joscha Bach (55:55.200)

narrative. The consciousness gives it color, gives it feeling, gives it flavor.

Lex Fridman (56:00.000)

So I think the feeling, flavor and so on is given by the relationship that a feature has to all the

Joscha Bach (56:06.720)

other features. It's basically a giant relational graph that is our subjective universe. The color

Joscha Bach (56:12.720)

is given by those aspects of the representation or this experiential color where you care about,

Joscha Bach (56:18.720)

where you have identifications, where something means something, where you are the inside of a

Joscha Bach (56:22.880)

feedback loop. And the dimensions of caring are basically dimensions of this motivational system

Lex Fridman (56:28.320)

that we emerge over. The meaning of the relations, the graph. Can you elaborate on that a little bit?

Joscha Bach (56:34.880)

Like where does the, maybe we can even step back and ask the question of what is consciousness to

Lex Fridman (56:41.120)

be sort of more systematic. Like what do you, how do you think about consciousness?

Joscha Bach (56:47.120)

I think that consciousness is largely a model of the contents of your attention. It's a mechanism

Joscha Bach (56:52.240)

that has evolved for a certain type of learning. At the moment, our machine learning systems

Joscha Bach (56:58.000)

largely work by building chains of weighted sums of real numbers with some nonlinearity.

Lex Fridman (57:05.120)

And you learn by piping an error signal through these different chained layers and adjusting the

Joscha Bach (57:13.360)

weights in these weighted sums. And you can approximate most polynomials with this

Joscha Bach (57:19.680)

if you have enough training data. But the price is you need to change a lot of these weights.

Joscha Bach (57:24.640)

Basically, the error is piped backwards into the system until it accumulates at certain junctures

Joscha Bach (57:30.000)

in the network. And everything else evens out statistically. And only at these junctures,

Joscha Bach (57:34.800)

this is where you had the actual error in the network, you make the change there. This is a

Joscha Bach (57:38.320)

very slow process. And our brains don't have enough time for that because we don't get old

Joscha Bach (57:42.720)

enough to play Go the way that our machines learn to play Go. So instead, what we do is

Joscha Bach (57:47.600)

an attention based learning. We pinpoint the probable region in the network where we can

Joscha Bach (57:52.960)

make an improvement. And then we store this binding state together with the expected outcome

Joscha Bach (57:58.800)

in a protocol. And this ability to make index memories for the purpose of learning to revisit

Joscha Bach (58:03.600)

these commitments later, this requires a memory of the contents of our attention.

Joscha Bach (58:10.160)

Another aspect is when I construct my reality, I make mistakes. So I see things that turn out to

Joscha Bach (58:15.120)

be reflections or shadows and so on, which means I have to be able to point out which features of

Joscha Bach (58:20.400)

my perception gave rise to a present construction of reality. So the system needs to pay attention to

Joscha Bach (58:28.000)

the features that are currently in its focus. And it also needs to pay attention to whether

Joscha Bach (58:33.360)

it pays attention itself, in part because the attentional system gets trained with the same

Joscha Bach (58:37.200)

mechanism, so it's reflexive, but also in part because your attention lapses if you don't pay

Joscha Bach (58:41.840)

attention to the attention itself. So it's the thing that I'm currently seeing, just a dream

Joscha Bach (58:46.960)

that my brain has spun off into some kind of daydream, or am I still paying attention to my

Joscha Bach (58:52.000)

percept? So you have to periodically go back and see whether you're still paying attention. And

Joscha Bach (58:56.800)

if you have this loop and you make it tight enough between the system becoming aware of the contents

Joscha Bach (59:01.520)

of its attention and the fact that it's paying attention itself and makes attention the object

Joscha Bach (59:05.680)

of its attention, I think this is the loop over which we wake up. So there's this attentional

Joscha Bach (59:12.720)

mechanism that's somehow self referential that's fundamental to what consciousness is. So just

Joscha Bach (59:19.600)

ask you a question, I don't know how much you're familiar with the recent breakthroughs in natural

Joscha Bach (59:23.760)

language processing, they use attentional mechanism, they use something called transformers to

Joscha Bach (59:31.040)

learn patterns and sentences by allowing the network to focus its attention to particular

Joscha Bach (59:37.920)

parts of the sentence and each individual. So like parameterize and make it learnable

Joscha Bach (59:42.960)

the dynamics of a sentence by having like a little window into the sentence. Do you think

Joscha Bach (59:51.200)

that's like a little step towards that eventually will take us to the attentional mechanisms from

Joscha Bach (59:58.160)

which consciousness can emerge? Not quite. I think it models only one aspect of attention.