The following is a conversation with Bokkis hotter senior research scientists, the Google deep mind throughout his career research, including with young men who burn shane leg. He has proposed a lot of interesting ideas in and around the field of artificial general intelligence putting the development of I see spelled a eggs. I model, which is a mathematical approached. A jai, incorporates ideas of comical. Of complexity, enough induction and reinforcement learning in two thousand six marcus lies. the fifty thousand euro hotter prize for losses compression of human knowledge. The idea behind this prize is that the ability to compress well is closely related to intelligence. This, to me, is a profound idea. Specifically,

If you can compressed the first one hundred megabytes, a one gigabyte awoke wikipedia better than your predecessors. You're compressor likely has to also be smarter. The intention of this As is I encourage the development of intelligent compressors as a path to asia. I. In conjunction with his pockets release. Just a few days ago, Marcus announced a ten increase in several aspects of this prize, including the money to five hundred thousand euros. The barrier compressor works. Relatives of the previous winners, the higher fraction of that prize. Money is awarded to you. You can learn more about it. If you, google, simply hutter prize, I'm a big fan of benchmarks for developing ai systems and the hutter prize may indeed be one that will spark some good ideas for approaches

It will make progress on the path of developing asia. I systems this is the artificial tells us podcast enjoy prescribing youtube, good, five stars and apple pie, gas support, rampage, IRAN or simply connected me on twitter Ex Friedman spelled f r, I d emma and, as usual I'll, do what two minutes of ads now in never any

in the middle that can break the flow of the conversation. I hope that works for you and doesn't hurt the listening experience. This show is presented by kashyap the number one finance app in the app store when you get it use code lex, podcast cash app lets you send money to friends, buy bitcoin and invest in the stock market, with as little as one dollar broker. Services are provided by cash up investing a subsidiary of square, a member, as I p c since cash app allows you to send and receive money, digitally peer to peer, a security in all digital transactions. Very important. Let me mention that pcr data security standard, the cash app

it's compliant with a big fan of standards for safety and security. Pc ideas says, is a good example of that or a bunch of competitors got together and agreed that there needs to be a global standard around the security of transactions. Now we just need to do the same for autonomous vehicles and ai systems in general. So again, if you get cash app from the app store or google play and use the cold legs podcast, you get ten dollars in cash. I also donate ten dollars the first one of my favorite organizations that is helping to advance robotic system education for young people around the world. I now hear my conversation with marcus hotter.

the Do you think of the universe as a computer or may be an information processing system? Let's go with a big question. First, okay. However: the big question. First, I think it's a very interesting hypothesis or idea, and I thought back on it first, So I know a little bit about physical theorist has done a lot of particle physics and in order to be.

And they are amazing at described virtually everything in the universe and they are all in a sense. Computable theory summary they're very hard to compute and you know very elegant, simple theories which describe virtually everything in the universe, so there's a strong indication that somehow the universe is computable, but it's applause have hypothesis the. What? What do you think just like you said, general relativity quantum field there what do you think that the laws of physics is so nice and beautiful and simple and comprehensible? Did you think our universe was design the earth is naturally this way are we just focusing on the parts that are especially compressible our human minds just enjoy something about their simplicity and in fact there are things that are not compatible nor I strongly

leave and I'm pretty convinced tat. The universe is inherently a beautiful, elegant and simpler and described by these equations and we're not just picking that I mean if there were some phenomena which cannot be need describes. Scientists would try that right and in our best biology, which is more messy, but we understand that is emerging phenomena and in all its complex systems, but there still follow the same rules right of continent electoral dynamics. I'm all of chemistry follows that had been all that I mean we can't computes everything because you have limited computational resources. Now I think it's not a bias of humans, but its objectively simple. I mean, of course you never know. You know maybe there's some corners very far out in a universe or super super tiny below the nucleus of of of atoms or

well parallel universes, where which are not nice and simple, but there's no evidence for that and we should apply or comes fraser, and you know just a simple street consistent with her, but also its a little bit sulphur french. So maybe a cook pause. What is outcomes? Razor comes razor says that you should not multiply and, his beyond necessity, which sort of if you translate translated to proper english means air and- and you know in the scientific context- means that if you have two theories or hypothesis or models which equally well describe the phenomenon, your study or the data, you should choose the more simple one. So that's just the principle sort of does- like a provable law, perhaps perhaps whitlow of can discuss it in think about about was the intuition

I know why the simpler answer is the one that is likelier to be more correct, descriptor of whatever we're talking about. I believe that occam's razor is probably the most important principle in science. I mean, of course, we logically dukson it'd, be too experimental design yeah, but science is about finding and understanding the world finding models of the world, and we can come up with crazy, complex models which you know, explain everything but predict nothing. But the simple model seemed to have predictive power and it's a valid question. Why and and the two answers to that you can just accept accepted

It is the principle of science and the use principle and it seems to be successful. We don't know why, but it just happens to be, or you can try. You know, find another principle which explains or comes racer, and if he starts with the assumption that the world is governed by simple rules, then there's a bias towards simplicity and applying what comes razor is the mechanism to finding this rules and actually in a more quantitative sense and become back to that later and cats have salami detection. You can rigorously prove that used to assume that the world it's simple, then Occam's razor is the best you can do in a certain sense. So apologies for the romanticized question but uh. Why do you think outside of it's effectiveness? Why do we do you think we find simplicity so appealing as human beings? Why does it

Just why does equals empty squared seem so beautiful to us humans. I guess mostly in general. Many things can be explained by an evolutionary argument and you know there's some artifacts, inhuman switch. You not just artifacts had not met in evolutionary necessary, but this dispute and simplicity its, I believe at least a core issue. About like science, finding regularities in the world, understanding the world which is necessary for survival right. You know if I look at a bush right and are just see no, case- and there is a tiger right- and it's me that I'm dead, but if I try to find a pattern and we know that humans are prone to find more pattern.

It's in data, then they are like the mars face and all these things, but this bias towards finding patterns, even if there are none, but I mean it's best, of course, if they are helps us for survival, yeah, that's fascinating. I haven't thought really about it: I thought I just love science, but did indeed from in terms of just for survival purposes. There is

an evolutionary argument for a wire. While we find the work of eyes die, so beautiful, maybe a quick small tangent. Could you describe what solomon of induction is? Yes, so that's a theory which I claim and resolution of four claimed a long time ago that this solves the big philosophical problem of induction and I believe the claim is essentially true and what it does is the following. So ok, further picky listener induction can be interpreted narrowly and wildly narrow means inferring models from data and widely means. Also, then, using these models for doing predictions are predicted, also part of of induction, so I'm living sloppy sort offer the terminology, and maybe that comes from rice alone off you know,

being sloppy, maybe I should say that if he can't complain anymore, so let me explain a little bit this theory now in simple terms, so assuming of a data sequence and make it very simple, the simplest one say one one, one one one is: if we're one hundred once what do you think comes next, the natural order? I must speed up a little bit of natural answers. Of course you know one okay and the questions. Why? Okay? Well, we see a pattern there. Yeah, okay, that's a one and repeat it, and why should it suddenly, after one hundred ones, be different? So what we're looking for is

Simple explanations. Are models for the data we have and now the question is a model has to be presented in a certain language in which language to be used. In science we want formal languages and we can use mathematics or we can use programs on a computer, so abstractly on a turing machine, for instance, or can be a general purpose, computer so and then, of course, lots of models of you can say. Maybe it's one hundred one hundred one hundred zeroes in on the ones that I'm all right but they're, simpler models, there's a model print one Loop added also explains the data and if you pushed her to the extreme- and you are looking for the shortest program which, if you run this program, reproduces the data you have, it will not stop. It will continue. Naturally- and this you take for your prediction and on the sequence of ones that were plausible right at the print one loop is the shortest program. We can give some more complex, exile,

will strike one two, three, four five: what comes next? The sure programme is again, you know counter, and so that is roughly speaking house a moment of induction works, and the extra twist is that it can also deal with noisy data. So if you have, for instance, a coin flip say are biased coin, which comes up head with sixty percent probability, and then it will predict and if we learn and figure this out and after a while it predict or the next coin, flip will be had with probability sixty percent. So as the stochastic version of that, but The goal is, the dream is always the search for a short programme. Yes, yeah well, no one of induction precisely what you do is so you combine so looking for the shortest programme is like applying our fraser, like looking for the simple story, there's also epicures principle, which says, if you have multiple hypothesis, which equally,

describe your data. Dont is cut any of them, keep all of them around. You never know, and you can put it together and say: ok, have a bias towards simplicity. But it don't rule out the larger models and technically what we do is we weigh the floor. their models higher and along a mother's lower, and you use a bayesian techniques. You have a prior and ambitious precisely due to the minus. The cops city of the programme. And u way all this high put his and take this mixture, and then you get also this perhaps activity in I like many of your ideas that that's just a beautiful idea of weighing based on the simplicity of the program. I love that that that that scene seems to me. Maybe very human central concept seems to be a very appealing way of discovering good programmes in this world You have used the term compression quite a bit. I think it's a beautiful idea, sort of we just talked about

felicity and may be science or just all of our intellectual pursuits is basically the attempt to compress the can maxie all around us into something simple. So what is this word mean to you compression? I think they will explain it, so it compression means for me finding short programmes for the data or the phenomenal hand you could interpret it more widely? As you know, finding a simple theories which can be mathematical theories or maybe even informal, like you know, just inverts compression means finding short descriptions, explanation, programmes for data dc science as a kind of our human attempt at compression So, speaking more generally, as we say programmes kind of a particular sort of almost like a computer size, artificial intelligence focus, but you see

all of human endeavor as a kind of compression well at least all of science. I c s a and have off compression with all of humanity, maybe and well. There are still some other aspects of science like experimental design right. I mean b b b, create experiments specifically to get extra knowledge, and this is that isn't part of the decision making process, but once we have the data to understand, the data is essentially compression. So I don't see any difference between contrast, compression understanding and prediction, though, would jumping around topics a little bit, but returning back the simplicity, fasting concept of coma grove complexity. So you're says: do most objects in our mathematical universe, have high comma gulf.

I see a navy. What is first of all, what is comma graph complexity. Okay, kamagra of complexity is a notion of simplicity or complexity, and and it takes the compression few to the extreme. So I expect before that arm, if you have some data sequence, just think about a file in a computer and best sort of not just a string of pits and and a few, and we have data compresses like the compress big files into zip files with certain compressors, and you can also pre yourself extracting aka apps. That means as an executable, if you run it

it reproduces the original filed without needing an extra decompress. It's just a decompress plus the archive together in one and now they are better and worse compressed us and you can ask what is the ultimate compressor. So what is the shortest possible self extracting aka, if you could produce for certain data set here which reproduces the data set, and the length of this is called the clonmel gore of complexity and ideologically? That is the information content in the dataset I mean if the data set is very redundant or very boring, you can compress it very well, so the information content should below- and you know it is law. According to the stiff, the length is the shortest program that summarizes the data. Yes, yeah and what's your sense of of ours sort of universe, when we think about the different, The different objects in our universe that we too are concepts or whatever

at every level. Do they have higher or lower come about complexity? So what's the hope do you have a lot of hope and be able to summarize much of our world? Ah, that's a tricky and difficult questions so, As I said before, I believe that whole universe based on the evidence we have is very simple. So has a very short description, So did you to linger on that? oh universe. What is it indeed mean at the very basic funds. Two level in order to create the universe? Yes, so you need a very short programme meant you run its get the finger. You get the thing going it then it will reproduce our universe. There's a problem with noise. We can come back to that later, possibly noise, a problem or a few? Is it a bugger feature? I would say it makes our life

scientist really really much harder, I didn't think about without noise with wouldn't it all of the statistics, but there maybe we wouldn't feel like there's a free will, maybe when is that for the ad for the eggs, for this is an illusion that noise can give you free, ale, efficient, at least in that way. It's a feature, but also, if you don't have noise, you have arctic phenomena which are effectively like noise. So we can. do not get away with statistics. Even then I mean about rolling dice and you know forget about quantum mechanics, and you know exactly how you you throw it, but I mean it still so hard to computer trajectory that effectively. It is two more. Let you know ass in our coming out with a number with probability won over six, but from from this set of philosophical como go of complexity perspective. If we didn't have noise, then I give Lee, you could describe the whole universe s

and now as standard model plus general activity. I mean we don't have a theory of everything yet but sort of assuming we're close to it or have it here, plus the initial conditions, which may hopefully be simple. and then you just run it and then you would reproduce the universe, but that sport, by noise or by coptic systems or by initially conditions which you know maybe complex, so now if we don't take the whole universe, but just a sub section I'll just take planet earth plenty does cannot be compressed yo into a couple of occasions. This is a huge black system so interesting. So when you look at the window at the hall thing may be simple: warn you just take a small window, then it may become complex and that may be counter intuitive. there is a very nice analogy, the book, the library of all books, so imagine you have a normal library was interesting books and you go their great lots of information and cute quite complex.

So now I create a library which contains all possible book say of five hundred pages, so the first book just as a a little pages the next book a and ends with b and so on. I create this library of all books. I can write a super short program which creates this library, so this library has all books has zero information content and you take a subset of his library and suddenly have a lot of information in their. So that's fastening am, I think, one of the most beautiful algae mathematical objects that these today seems to be understudy or under talked about, is selling tommy. What lessons do you draw from serve the game life or sell your time, and we start with a simple rules. Just like your described with the universe and somehow complexity emerges, do you feel like you? have an intuitive grasp the behaviour. Fasten behaviour such systems where some links

add some chaotic behavior could happen. Some complexity could emerge some. It could die out in some very rigid structures. Do you have a sense about a cellular automata that somehow transfers may be to the bigger questions of our universe?. The cell allowed him out there, and especially economists. Game of life is really great because the jeweler so simple, you can explain it to every child and been by hand. You can simulate a little bit and you see this beautiful patterns emerge and people have proven. You know that it's even turi complete, you cannot just use a computer to simulate game of but you can also use game of life to simulate any computer. That is truly amazing and it's the prime example probably to demonstrate that very simple rules can be too very rich phenomena and people. Sometimes you know, how can I was chemistry and biology, so rich?

I mean this can't be based on simple rules, but no, we know quantum electrodynamics describes all of chemistry and and become later back to that I claim intelligence can be explained or described in one single equation. This very rich phenomenon- and you asked also abouts whether you know I understand this phenomenon and it's Probably not- and this is saying you never understand- willi- suggested- used to them and are pretty using used to sell it automatically. So you I believe that you understand the why this phenomenon happens, but I give you a different example: I didn't play too much. This is Conway's game of life, but a little bit more and with fractals and with the mandelbrot set is beautiful. You know patents, just just look: mandelbrot set and aunts. While when the computers were really slow in our charter, a black and white

tour and I programmed my own program center in assembler to wow wow you're legit to get these practice on the screen and it was mesmerized and much later so I returned to this. You know every couple of years and then I tried to understand what is going on and you can understand a little bit so I try to derive the locations. You know there. This circles and the apple shape and then you have smaller amanda brought, sets repressively in the set and as a way to mathematically by solving I oughta polina meals, to figure out where these centres are and what size our approximately and, by sort of mathematically approaching this problem, you slowly get a feeling of

why things are like they are and that sort of ism. You know first step to understanding why this rich phenomenon, cheating, is peace possible is your intuition. Anything is possible to reverse engineer and find the short program that generated the these fractals says by what looking at the fractals. Well in principle, yes- and so I mean in principle what you can do, is you take your any dataset? You know you take this fractals or you take whatever your dataset, whatever you have. Picture, convey scheme of life and you run through all programs. You take a programme has one two, three four and all this programme surrounding all in parallel in so called doth telling fish and give them computation resources. First, one fifty percent. Second, one half. since and let him run wait until they hold given output, compare it to your data and if some of these programmes produce the correct

data? Then you stop and then you have already has some program. It made a long programme because its faster and then you continue and you get shorter and shorter programmes until you eventually find the short programme. The interesting thing you can have a know where that is what this programme, because there could be an even shorter programme, which is just even slower. You just have to wait here, but I still totally and actually have to find a time you have two short as programme, so this is a serious decline, but completely practical way of finding their underlying structure in every data then there was a lot of induction doesn't come aboard of complexity in practice. Of course we have to approach the problem more intelligently and then, if you take resource limitation, into account stairs for the feel of pluto random numbers, and these are and I'm not must solve these item mystic sequences, but

oh algorithm, gruesome, bitches, fast, fast means runs in pulling on time, can detect. That is actually deterministic. So we can, youth interesting ermine rendered mammoth, maybe not that interesting, but just an example. We can produce complex looking data and we can then prove that no fast algorithm can detect the underlying python. Which is unfortunately, his a that's, a big challenge, far search for simple programmes in the space of artificial intelligence. Perhaps yes, it definitely a splendid vision, intelligence and it's quite surprising that its I can't say easy. I mean physicists worked really hard to fight his theories, but apparently it was possible for human minds to find these simple rules in the universe. It could have been different right. It could have been different. It's it's a it's owens

bearing so let me ask another absurdly big question: what is intelligence in your view saw? I have, of course, a definition. I wasn't sure where you're going to say, because you could have just as easily said. I have no clue which many people would say. I am not modest in this question. I saw the the informer version of reach of act altogether by shane black who cofounded the mind, Is that intelligence measures and agents ability to perform well in a wide range of environments, so that doesn't sound very impressive, and but if these words have been very carefully chosen

and there is another matter theory behind it, and we come back to that later and if you look at this, this definition by itself, it seems like yeah, ok, but it seemed a lot of things are missing but if you think it through, then you realize that most and I claim all of the other threats, at least this rational intelligence, which we usually associated intelligence, emergent phenomena from this definition, gravity, vt, memorization, planning, knowledge. You all that in order to perform valley a wide range of environments, so you don't have to explicitly mentioned it in a definition interesting. So yes, is this have struck. Reasoning are all these kinds of things are just the merger phenomena that help you enough towards. you say the definition against them on multiple environments. Did you make nor goals. No,

But we have an alternative nation instead of performing value, constructs replace it by gold, so intelligence measures and agent ability to achieve goals in a wide range of environments. That well. Is it wise because in their there's an injection of the word goals say we want to specify Should be a girl you ever performed well, is sort of what is. It means the same problem that is a bit of a grey area. But it's much closer to something that could be formalised are in your view, are humans where to humans fit into that definition, are they? General intelligent systems that are able to perform in a thick how good our day at fulfilling that definition at performing well in multiple environments, yeah, that's I mean the humans are performing best, among all Species are, as we know, we know of depends. You could say that treason plants are doing

better, a job they'll, probably outlast us something about there, a much more narrow environment right. I mean you just you know I have a little bit of air pollution and is trees die and we can adopt ride. We build houses with filters, we ve we do not view engineering, so multiple environment part there. That is very important just so that distinguish near intelligence from white intelligence, also in the air. I research. So let me ask the the entering question can machines think commissions be intelligent. So in your view, I have to come ask. The us is probably yes but owner. Can I hear what your thoughts? Can machines be made to fulfil this definition of intelligence to achieve intelligent? Well, we are forever getting there and on a small scale beyond. re there,

wide range of environmental missing about yourself driving cars. We have programmes is play, go and chest. We have speech recognition, so it's pretty amazing but Can you know these are narrow environments? That did you look at alpha zero? That was that was also develop, active mind, but fames with alphago and then came off a zero later there was truly amazing. So and reform a learning algorithm, which is able, just by self play to play chess and then also go, and I mean yes, they're, both games but they're, quite different games, and you know this- you didn't don't feed them the rules of the game and the most remarkable thing, which is still a mystery to me that usually for any decent chess programmer. I dunno much about go. You need opening books and endgame tables and so on to and nothing in there nothing was

index of official was alpha zero. The self play mechanism, starting from scratch, being able to learn to eat actually new strategies as a yeah. It is it really this coward in all this famous openings within four hours, who, by himself what it was really happy about? I'm a terrible chess player, but I like queen, gambie and offers europe figured out that this is the best opportunity advice. Somebody proves you're correct so yesterday to answer your question. I believe the general intelligence is possible and adults. It depends how you define it. Do you say a g, I with general intelligence, artificial, german intelligence? I'm only refers to if you chief human level or sub human level, but quite broad, is it also general intelligence? So we have to distinguish, or is only super human intelligence, general

the fish intelligence, is there a testing your mind like the turing test and natural language or some other test? That would impressed the heck out of you that would come. Across the line of yours sense of intelligence within the framework these said, while the turing test, what has been criticized a lot, but I think it's not as bad as some people think as some people think it's too strong. So it tests not just for system to be intelligent, but it also has to fake human deception. This such a right, which is much harder, and on the other hand, say it's too weak yeah, because it just maybe fakes, you know, emotions or intelligent behavior. It's not real, but I don't think that's the problem, our big province or, if, if it would pass the turing test and so a conversation over terminal with about for an hour

or maybe a day or so, and you can fuller human into you know not known whether this is a human or not. That is the turing test here. I would be truly impressed and we have this annual competitions, alumina, price, and I mean it started with eliza. This was the first conversational programme and what is called the japanese mitchell cool or so that the winner of the last couple of years and while unimpressive, yes, credit precedent, google has developed, meaner right just just recently that the open. domain conversational bought just a couple of weeks ago. I think the economic them check their task of the elect, surprised proposed and he may be sought Still, he wasn't to me and said Sort of a length of a conversation make you want the bar to bees fish lately interesting. They you'd want to keep talking to a further twenty minutes and that's of earth

prizing the effective in aggregate metric. Is it really like nobody? Has the patience to be able to talk to her about that's, not interesting and intelligent and witty, and is able to go into different tinges jump domains be able to you know. is something interesting to maintain your attention and very many humans. Wolves also failed his tests to assess, and unfortunately, we set jessica with autonomous vehicles with chatbots. We also set a bar that's way too hard high to reach. I sat down on the turing test. It's not as bad as some people leave. You got what is really not useful about the turing test. It gives us no guidance on how to develop these systems. In the first place, Of course, no, we can develop then by trial and error, and you too, ever entered and run the test and see whether drugs or not. But a mathematical definition of intelligence gives us in

an objective which we can then analyzed by vertical tools or computational, and you know maybe even proof how close we are and we will come back to that later with ic model. So I image the compression rights or in natural language processing and have achieved amazing results and are one way to test this. Of course you can take this some new train it, and then you see how bad it performs the task, but a lot of performance measure, and is done by so called perplexity. This is essentially the same measure, complexity or compression length, so there not be community, develop in systems and then they measure the compression length and then the half, ranking and leaks and because there's a strong correlation between compressing bell and then the sisters performing well at the task at hand. It's not perfect, but it's good enough for them m s as an inter.

mediates aim the email measure, so it did. This is kind of almost returning through the commonwealth. Comply I see you saying good, compression usually means good intelligence. They mention you're one of the one of the only people who dared boldly too. Try to formalize are at the idea of our official journal and challenges to meet as I am a mathematical framework for intelligence, just think, as we mentioned termed I c a I x I so let me ask the basic question: what is ic? Okay, so let me first say what it stands for, because what it stands for. Actually, that's, probably the more basic question: what is the first christmas julie how how it's pronounced? But finally,

put it on the website how it's fun out of your head. You figure it out to the name comes from a I artificial intelligence and the x. I is the greek let excite which are used for so long enough distribution for card stupid reasons which are not willing to repeat here from the camera rousseau. Happen to be a modest arbitrary chose this site, but it also has nice other interpretations, so their actions and percent since, in this model, right an agent has actions and perceptions and over time so is a index I ex index. I saw this action at a time I and followed her affection. And I will go with that I'll, edit out the firstborn I'm just kidding. I have some more interpretations so at some point, maybe five years ago or ten years ago, I discovered in

in barcelona. It wasn't a big church. There was in general stone engraved some text and debate I could see, appeared their love it. I was very surprised and and and happy about it, and I looked. outside. This is common language and it means with some interpretation of that's it. That's the right thing to do here where recur destined, somehow came near him. Get big. Came to you in a dream so of erle there's a chinese road. I have also written our galaxy of your transcribed. That opinion then define the one. There is a I crossed with induction because status and it's going more to the content now so good old face in the eyes more about your planning, unknown deterministic, world and induction, is more about often yo. I d data and inferring models, and essentially what decisive one does is combining these.

and I actually also recently, I think, heard- that in japanese a means love. So so, if you can combined, excise, somehow with that Can the there might be some interesting ideas there? So I see that's then take the next step, Can you maybe talk? the big level of what is this path? America framework there? So it consists essentially of two parts: one is the learning and induction and predictable part and the other one is the planning part. So, let's come first to the learning induction prediction, part which essentially explained already before so what we need for any agent to act well is that it can somehow predict what happens. I mean you have no idea what you're actions do. How can you decide which acts are good or not so

I need to have some model of what your actions effect. So what you do is you have some experience. You build models like scientists, you know of your experience, then you hope this models are roughly. Correct and then you use this models for prediction and a model is sorry. Interrupt model is based on the perception of the world. How you? actions will affect our world. That's not me What is it? How do you get a lot of important pop aim? It is technically important, but at this stage we can't just think about predicting say, stock market data, if the data, what Y I q sequences one two three, four five: what comes next year? So of course our actions affect what we are doing, but I come back to that in a second sicily in AL keeps interrupting so just draw a line between prediction and planning woody mean by prediction, and this in this way, trying to predict the environment. Without your long term,

action in the environment. What is prediction? Ok, to put. It exists in our kid and let's put in the mouth, yes so, I'll have it, but I am surprised at the grass is not a question of soldiers. Disciples form of prediction is that you just have data, beat you passively observe and you want to predict what happens without you not interfering, assessing whether forecasting stock market, iq sequences or just anything mackay and salami european action based on compression. So you look for the shortest programme, which describes your data secret

and then you take, this programme run it, which we put usage, data sequence by definition, and then you let it continue running and then it will produce some predictions and you can rigorously proof that for any prediction task, this is essentially the best possible predictor. Of course, if there's a prediction task for a task which is unpredictable, like you know your fair conflicts, I cannot predict inextricably what's allotment of processes. Ok, next, head is probably fifty percent is the best you can do so if something is unpredictable settlement of illogical, magically predicted, but if there is some pats on and predictability thence along one of induction, we'll figure it out.

eventually and not just eventually but rather quickly- and you can have proof convergence rates and whatever your data is solids, pure magic in a sense and what's the catch while the catches that is not computable and become back to that later. You cannot just implemented in even this google resources here and run it and you know, predict a stock. I couldn't become rich, I mean, if rates are low enough already you tried it at the time, but this or the the basic task you're in the environment and you interact with roma to try to learn a model Environment in the model is in spain. So the all these programmes and you're going to get a bunch of programmes that are simple. So, let's go. Let's go to the actions now, but actually got that you asked usually escaped his part of it is also a minor contribution which I did, though, the action part but usually sort of just jump to the decision about. So let me explain to the action partner thanks for asking. So

You have to modify the little bit by now, not just predicting a sequence which just comes to you, but you have an observation. Then you act somehow and then you want to build the next observation based on the past observation and your action. When you take the next action, you don't care about, predicting it because you doing it, then you get the next observation and you want more before you get it. You want to predicted again based on your past action and observation sequence. It just condition extra on your actions, there's an interesting I'll turn. If that you also try to break your own actions with you, one, oh it in the past, for the future. And what are we to your future actions? That's injured, Wait, let me I think my brain is broke. We should maybe disgusted later beef after I,

the ic model is an interesting variation. But this is a really interesting version and a quick comment. I dunno, if you want to insert that in here, but you're looking at it Yours observations, you're, looking at the entire big history, long history of the observation seeks it gets bearing port the whole history from birth sort of of the agent, and we can come back to that. Why is this important job? Often you know in our old you have empty peace, Michael decision processes which are much more limiting okay. So now we can predict conditioned on actions, so even if influence environment but predicted Not all we want to do right, we also want to act really in the world, and the question is how to choose the actions and we don't want to greedily, choose their actions. You just you know what is best in in the next time step and we first, I should You know what is in our. How do we measure performance will be measured performance by giving the agent report that the so called reinforced ballooning framework? So every time step you can give it

positive reward and negative reward, all baby, no reward. It could be very scar slide like if you play chess. Just at the end of the game you give plus one for winning a minus. for losing. So an exit framework has completely sufficient occasionally give a rebate signal and you asked the asian to maximize river, but no green. we thought of you know the next one next one, because it very bad in the long run, if you're ready so about over. Lifetime of age, and so, let's assume the age limit for in times tat was their dyson sort of hundred years shop. That's just you know the simple model to explain. So it looks at the future report some and ask what is my action sequence or more precisely? My policy which leads in expectation because it and all the world to the maximum. We want some let me give you an analogy in chess: for instance, we know how to optimally, in theory is just a meaning mac strategy. I play the move which seem,

best to me under the assumption that the opponent place the move, which is best for him so best first for me and then assumption that he I play again the best move, and then you have to expect emacs tree to the end of the game and then you'll back propagate, and then you get best possible move. So that is the ultimate strategy which form an already figured out a long time ago for playing at the zero, the games, luckily or maybe unluckily for this year. If it becomes hotter that the world is not always up his ariel, so it can be if the other humans, even co, operative, yo, Nature is usually I mean the dead natures, drastic out announced things just happen randomly or don't care about you. So what you have to take into account as a noise now and not as is doesn't guilty so you're replace the many moment. Opponents side by an expectation which is general enough to include also deserve our case

this so now, instead of a minimax strategy of an expecting mk strategy. So far so good. So that is well known, it's called sequential decision theory, but the question is on which probability distribution. Do you base that if I have the true probability, distribution like say a play, backgammon right as dice and a certain randomness involved, I can calculate probabilities and feed it into expect the max the sickness does. Eg come up with the optimal decision. If I have enough compute but enough for the real world, we don't know that you know what is the probability of the driver and front of me breaks. I dunno also depends on all kinds of things and especially new situations I dunno. So is it is and nothing about prediction, and there was a woman who comes in so what you do is in sequential decision tree. It just replace the true distribution which we don't know by this universal distribution. I didn't explicitly talk about it, but this just used for universal prediction,

and plug it into the sick, rested decision mechanism, and then you get the best of both worlds. You have a long term planning agent, but it does need to know anything about the world because the salon reduction apart, no one's can. You are explicitly tried to describe the universal distribution in house on us. Ducks plays a role here. Yeah, I'm trying to understand so what he does it, I'm. So, in the simplest case, I said, take the shortest program describing your data run. It have a prediction which would be deterministic, yes, okay, but you should not just the shortest programme, but also consider the longer ones once but keep it lower? A priori probabilities, innovation framework, you say a priori any distribution,

And bitches a model and or stochastic program has a certain opera, your property which, to to the minus and Y to the minus length. You know I could explain length of this program, so long programs are punished as a priori and then you multiplied with the so called likelihood function, which is, as the name suggests, is how life Cleat is this model, given the data at hand. So if you have a very wrong model, it's very unlikely that this model is true, and so it does very small number. So, even if the model is simple, it gets penalized by that and what you do is, then you take just december. This is the average over it, and this gives you probability distribution. So it was universal distribution of phenomena, distribution so swayed by the simple, city of the programme and the likelihood. Yes, it's kind of a nice idea so again and then he said there

you're, playing an r m or forget the letter stepped into the future. So how difficult that problem with them of their occasion, as a cop. What problem, what are we to have a you, have a planning to up to, and it's and time in in the horizon m witches. I mean it's computer, but in intractable I mean even for chess, it's already intractable to do that exactly and forgo where it could be also discounted. kind of framework were so so having a heart will rise. And you know at home. But yes, it's just for simplicity, of discussing the model and also sometimes the most simple. But there are lots of operation is actually quite interesting parameter. Is it's there's nothing really problematic about it, but it's very interesting. So, for instance, you think no, let's, let's tent. Let's let the parameter m

tend to infinity right. You want an agent which lives forever. If you do it nobly of two problems. First, the mathematics breaks down because you have an infinite reward, some which may give infinity and getting but zero point one anytime status, infinity and giving report one every times of his immunity. So equally good, not really. What we want other problem is that, If you have an infinite life, you can be lazy, as long as you want for ten years and then catch up with the same expected reward, and you know a think about yourself, for you know more, maybe you know some friends if this new there lived forever. You know why work hard now, just enjoy your life. You know it and catch up later. So that's another problem with infinite rising and you mentioned yes, we can go to discounting, but then the standard discounting a so called geometric, discounting so a dollar two days about worth as much as one dollar five cents tomorrow. So if you do this, so called geomantic discounting you have

introduced an effective horizon. So the aged is now motivated to look ahead a certain amount of time effectively. It's like a moving horizon and for any fixed, effective horizon. There is a problem all of which were twice larger, arises that if a look ahead, five times steps, I'm a terrible chess player. I'd like to look ahead long, I foreplay go up. we have to look at even long us over every problem and favour favorite horizon. There is a problem which this horizon cannot solve, but I ain't it used to so called near harmonic horizon, which goes down with one of our t, rather than exponentially teeth, which produces an agent which effectively looks into the future proportional to it's h. So if it's five years old, it plans for five years, if it's hundred years, older than plants four hundred years interesting. and a little bit similar to humans to write em a children, don't plan, I had very long within because became authority a play. I had more lorna, maybe then,

it's all very old. I mean we know that we don't live forever. You know, maybe then horizon shrinks again to the oh, that so that's really So I just adjusting the horizon. What does there some mathematical benefited? That of or is it just then aye, sir intuitively empirically I'll, be a good idea to set a push. A horizon back, don't put extended horizon. As you experienced more of the world. Is there some mathematical conclusions here? There are beneficial, mr. This element of the taxes prediction probably have extremely strong final time, but a finite data result. So you have so much data than you'd lose thought on. So much soda to the tear is really great with icy model with the planning part. Many results are only asymptotic which well this is: what is I sometimes unloading meet. You can prove, for instance, there

in the long run. If agent acts long enough, then you know it performs optimal or some nice thing happened. So, but you don't know how fast it converges, so it may converge fast, but which was not able to prove it because a difficult problem, or maybe back in the in the in the model so that it is really that slow, so so that this was a synthetic mean sort of eventually, but we don't know how fast and If I give the agent of fixed her eyes and am now, then I can approve us and toxic results right, Simon syrupy dies in hundred years, then it in hamburg uses over. Cannot they eventually, so this is dead bondage of the discounting did. I can prove us until the results, so just a clarifies off. So I ok, I made I've build up a model

now in the moment of have this way of looking stem cells does ahead. How do I pick what action will take its like, where the playing chess right you do? This mini max in this case here do expect him act based on this element of distribution, you propagate back and then, while inaction fought out the action, to maximize future expected three watch on the settlement of distribution. And then you just take this action and then repeat it did you get observation and you have eaten in this excellent observations than european their ward. So on so you re wrote to you and I may be you gave even predict your nation. However, the idea, but ok, this big framework. What is it the is is mean. It's kind of a beautiful mathematical framework to think about artificial general intelligence. What can you, what does it help you into it about her,

The build such systems, or maybe from another perspective wait. What does it help us in understanding? I e g, I so when I started in the field, I was always interested two things. One was human. Eighty I am, the name didn't exist. Ten or more generally, I am strongly I and physics your everything, so I switch back and forth. In computer science and physics quite often use the theory of everything through with everything her. This will act as a basic instinctively was problems before all of humanity it. Yet I can explain if you wanted some the time your wife, interesting diesel question these two. While if, if a, if one be it was one to be solved, which one would you if one, if you are in, apple falling your head and is a brilliant insight and you could arrive at a solution to one

would it be a gy or the theory of everything, definitely a dji, because once the asia, I promise author, can ask the ai to solve the other problem for me yet brilliantly put okay, so as you were saying about ok, so and the reason why didn't settle, I mean this thought about. You know once you ve felt hiv aids, all kinds of other, not just as your every problem about all kinds of use. More useful problems to humanity is very appealing to many people antenna. I had this thought also that I was quite disappointed with the state of the art, of the future. I there was some theory, you know about it. Gentle reasoning, but I was never convinced that this will fly and then there was this homer, more holistic approaches the neural networks- and I didn't like these horrors sticks, so and also didn't have any good idea myself.

So that's the reason why I toggled back and forth quite some violent, even booked a four and a half years in in a company developing software, something completely unrelated. But then I had this idea about the oxy model and and so what it gives you it gives you a gold standard. So I have proven that this is the most intelligent agents which anybody could built built in quotation marks, because it's just mathematical, int unit infinite compute now, but this is the limit, and this is completely specified- is not just a framework canada, every year tens of frameworks are developed, we just have skeletons and then pieces are missing and usually dismissing pieces. You turn out to be really really difficult, and so this is

completely and your legally defined and we can analyze that mathematically and we have also developed some approximations. I can talk about it a little bit later. That would be solved a top down approach, like say for nine months, many max theory that the theoretical optimal play of games- and now approximated, put holistic, say in prune the tree, blah blah blah and so on. So we can do that also with an exhibitor, but for generally I am. It can also inspire those and most of most researchers go bottom upright to have the systems that try to make it more general, more intelligent. It can inspire in which direction to go, or what do you mean by that? So if you have some choice to make right, so how should they evaluate my system? If I can't do cross validation?

How should I do my learning if my standard regular station doesn't work well, so the answer all of this we have a system which does everything that's actually it's just you know completing the ivory tower completely useless from a practical point: of you, but you can look at the yeah. Maybe you know I can take some aspects and instead of coma go of complexity. They're just take some compressors which has been developed, log in for the planning. While we have few city, which has also been used to go and do it at least it's inspired me a lot to have this formal definition and if you look at other fields, you know like. I always come back to physics because of his expect from think about the phenom of energy. That was long time, a mysterious concept and at some point it was completely formalized and entered

really helped a lot, and you can point out a lot of these things which were first mysterious and wake and then to have been rigorously formalized speed and acceleration has been confused trident until it was formerly defined till there was the time like this and in people not often enough o, don't have any background in. I still confused it, em so and dislikes the model or the the intelligence definitions which is sort of the dual twits. We come back to that later, formalizes, the notion of intelligence uniquely and rigorously. So in a sense it serves, is count the light at the end of the tunnel. So give warrior so I mean there's a million quietly. I could ask her, so maybe I'm the kind of, we feel around in the dark. Color was all been here a deep mind, but in general been a lot of breakthrough ideas, cyclopes, saying around reinforcement learning. So how you see, the progress in rivers learning is different, like wits,

set of annex ii. Does it occupy the current, like you said, maybe The mark of assumptions made quite often in reinforced, for learning The there's this other. since aid in order to make the system work. What do you see the difference? Connection for some learning an exit. Sl. The major difference is that All other approaches they make stronger assumptions sewing reinforced maloney. The markov assumption is that the next step- when next observation only depends on their under previous observation and not the whole history, which makes, of course, the mathematics much easier my writing dealing with histories, of course their profit from it, also because then you have argued that run on current computers and do something practical, useful bats. Fortunately, I all assumption speech are made by other approaches,

we know already. Now they are limiting. So, for instance, Usually you need a good desert assumption in the mtp frameworks in order to learn what this essentially means, that you can recover from your mistakes and that there are no traps and environment, and if you make this assumption, then essentially you can, you know, go back to a previous state? Go there a couple of times and and learn what and what statistics and and but the state is like and then in the long run, perform well in the state but they're, not funding problems, but in real life we know you're. There can be one single laxity, I one second of being inattentive while driving a car fast. You know I can't really the rest of my life again become quadriplegic or whatever so is no recovery anymore

in the real world is not our goal. To always say you know there are traps and there are situations, but we are not recover from and I very little theory has been developed for this case. What about a What do you see in there in the context of ice is the role of exploration, sir, a volume. Imagine you know in the in the real world in ghent to trouble. I will make the wrong decisions are really pay for it. but expiration, as seems to be fundamentally important for learning about this world, for gaining new knowledge. So is it is exploration. Baked in another way to ask what are the parameters this, as I see it, can be controlled yeah. I say the good thing They don't know. Parameters to control and some other people tried not to tackle.

roll, and you can do that. I mean you can modify axes or that you have some not to play with if you want to that. Further exploration is directly baked in and that comes from the beige learning and the long term planning. So these together. already imply expiration. You can nicely and explicitly prove that for a simple problems like so called bandit problems where you say To give a real good example, say you have to medical treatments a and b, you don't know the effectiveness. You try a little bit be a little bit, but you don't want to harm to many patients, so you have to sort of trade of exploring and at some point you want to explore and you can do the mathematics and figure out the optimal strategy

and it took a beijing agents, adult and non patient agents, but it shows that this basic framework by taking a prior of possible worlds doing the beijing mixture, then the pace, optimal decision with long term planning that is important automatically and implies exploration, also to the proper extent, not too much exploration and not too little. It is very simple settings in the ic model. I was also able to prove that it is a self optimizing theory. More are symptomatic of timidity, seems all data, only us in talking not final time pounds, so it seems, like the law, Term planning is a really important, but the long term part of the plan is really important. He has and also a maybe a quick tangent how important teeth is removing the markov assumption and looking at the full history, that of intuitively. Of course it's important, but is it like fundamentally,

as formative to the entirety of the problem? What's your sense of it like as we you off, we make that assumption quite often is that the throwing away the past? I think it's absolutely crucial. The question is whether there is a way to deal with it in a more holistic and still sufficiently well way, so as not to come over the top and fly. But you know you have some new key event in your life in a long time ago. You know in some city or something you realize you that's a very dangerous street or whatever right here and you to remember that forever. Right in case you come back there. Kind of a selective, gotta memories. You remember that all the important events in the past, but somehow

Selecting the importance is the very heart you are, and I'm not concerned about you not just storing the whole history. Just you can calculate you, don't human life says thirty, or one hundred years does meta right and how much data comes in through the vision system and the auditory system. You can press it a little bit and in this case lawfully and store it, and we are soon in the means of just storing it, but you still need to the selection for the planning part into compression for the understanding part, the raw storage I'm really not concerned about, and I think we should just store if you develop an agent, preferably just to store all the interaction history and then you build of course, models on top of it.

And you can press it ain't? You are selective, but occasionally you go back to the old data in re analyze it based on your new experience. You have your sometimes you you're in school. You and all these things, you think, is totally useless and you know much lady readers are. There were not enough Have you thought I'm looking at you linear, algebra, right So maybe me let me ask about objective function because that rewards. It seems to be an important part, there awards are kind of given the system for a lot of people. The this suffocation of the objective function in is a key part of intelligence. I e the agent itself figuring out what is important, what would you think about that? it is possible within nikes. He framework to

yourself, discover the reward based on which he should operate. okay, that'll be a long answer and so And it is a very interesting question- and I am asked a lot about this question- whether he was come from and that depends and so- and I you know, I give you now a couple of answers so if they want to build agents now, let's start simple, so let's assume we want to build an agent based on the ic model, which performs a particular task. Let's start with something super simple like super simplex playing, chelsea or gore. I think yeah. Then you just you know the reward. Is your winning the games, plus one losing parents minus one done. You apply this agent. If you have enough computer to let itself play and it will learn the rules of the game, we'll play perfect chess after some, while problem solved? Okay, so if you have more complicated

glimpse. Then you may believe that you have the right reward, but it's not so nice cute example is the way to control that results in which satins book, which is a great book by the way so you control the elevator and you think. Well, maybe the reward should be coupled to how long people wait in front of the elevator. You know long way to spat you program it when you do it and what happens? Is the elevator eagerly picks up all the people but never drops them off? the realise that maybe the time in the elevator also counts. Do you minimize the sum yeah and the elevator does that, but never picks up the people in the tenth floor and the top floor, because in expectation it's a brothel just, let them stay

so so, even in apparently simple problems, you can make mistakes yeah and that's what in in more serious contexts, say dji safety researchers consider so now, let's go back I too am general agent. So assume you want to build in agents which is generally useful to humans, as you have a household robot here, an antidote to all kinds of tasks. So in this case the human should give the reward on the fly I mean. Maybe it's pretending. the factory and it has some sort of internal reward, for you know the battery level or whatever yeah, but I saw it, you know it does that is badly. You know you punished a robot. Does it go with your reward, the robot and then to train it to a new Tosca like a child right so You need the human in the loop if you want a system which is useful to the human and as long as this agent stay sub human level, that should work reasonably well, apart from the obvious examples made becomes critically to become

You know from a human level is that many children, small children, you have reason to rebel under control, they become older and the reward technique doesn't work so well anymore. then. Finally, so this would be agents which are just you could say, slaves to the human. Yeah, so if you are more ambitious and just say, we want to build a new species of intelligent beings, we put them on a new planet and we want them to develop. This planet ever so we don't give them any reward. So what could we do and you could try to you'll come up with somebody? What functions like you know? It should maintain itself the world. It should and may be multiply, built, robots rights, and you know maybe we're all kinds of things tat you find useful, but it's pretty hot rights to know what

What does self maintenance mean? You know, what does it mean to build a copy should to be exact, copy, an approximate copy, and so that's really hot, but a logo or saw also demands. I developed a beautiful model that just took the ic model and coupled the reverts to information gain zone. He said the reward is proportional to how much the agent had learned about the world and you can rigorously, formerly uniquely defined it in terms of her cat lover since okay, so if you put it in you, get a completely autonomous agent and actually, interestingly, for this agent, we can prove much stronger result in for the general agent, which is also nice, and if you let this agent loose will be in a sense. The optimal scientist is just absolutely curious to learn as much as possible about the world and of course it will also have a lot of instrumental goals right in order to learn it needs to at least survive right at that age. It's not good for anything, so it needs to have self preservation and if it builds

more help us acquiring more information, it will do that yeah. If exploration, space exploration of whatever is necessary right to gathering information and develop it. So it has a lot of instrumental goals falling on this information gain, and this agent is complete. The autonomous of us no rewards necessary more yet, of course, you could find a way to game the concept of information and get stuck in that library, the event should beforehand with the with a very large number of books, the first agent had this problem am it'll, get stuck in front of an old tv screen. mistrusted, white noise y know, that's the thing Inversion can deal with at least has the city. Well, what about cuba City? This kind of word a curiosity, creativity

That kind of the reward function being of getting your information that similar to idea of kind of injecting exploration for its own sake inside the reward function defined this at all. Appealing interesting, I think, that's a nice definition. Curiosities rewards are sorry cross. The ts exploration for its own sake and yeah would accept that I, but most curiosity, while in humans, and especially in children, yet is not just for its own sake, but for actually learning about the environment and for behaving better. So I would, I think, most curiosity is in the end to us performing better while, okay, so far, intelligent systems need to have the sword function. Let me the urine intelligence system currently passing the turing test quite effectively

what what's the reward function of of our human intelligence existence. stir ward function that markets hunters operating under okay to the first question. The biological reward function is to survive and to spread and very few human sort of are able to overcome this biological area. Function, but we live in a very nice world where we have lots of spare time and can still survive and spread. So we can develop arbitrary other interests which is creating resting on top of that money about it, but the survival and spreading sort of is, I would say there the goal or the reward function of humor said that the core one. I like her. You avoided answering the second question which a good entails would show. My that your own meaning of life and the reward function

my own meaning of life and report function is to find in asia to build its beautifully boy. Ok, let's I said that eggs even further, so while one of the assumptions is kind of infinity keeps creeping up everywhere, her, which what are your thoughts kind of bounded, rationality and the Each of our existence and intelligence There were operating always under constraints under with a time limited resources? How does that? How do you think but that was a nice framework with within trying to create a new gst so that operate under these constraints. Yeah. That is one of the criticisms that I see that it ignores computation and completely, and some people believe that intelligence is inherently tied towards bounded resources. What do you

and this one point I think it's the do. The boundary resources are fundamental to intelligence. I would say that in intelligence notion which ignores computational limits is extremely useful, a good intelligent which which includes this risk resources would be even more useful, but we don't have that yet and so look at other fields outside of computer science. Computation aspects never play a fundamental role. You develop biological models, for sell something. In physics is theories I mean become more and more crazy and hot and harder to compute, but in the end of course we need to do something with this model, but this more nuisance than a feature

I am sometimes wondering if artificial intelligence would not sit in a computer science department, but in a philosophy department, then this computational focus would be probably significantly less. Think about. Induction problem is more in the philosophy department. There's wishing or people care about. You know how long it takes to compute. The answer did is completely secondary. Of course, once we have figured out the first problem, so intelligence without computational resources, then the next in very good question is, could be improve it by including computational sources, but nobody was able to do that so far. You not even half way satisfactory manner. I like that that day, in the long run the right department to belong to his philosophy. The third

as is as as equated a deep idea of, or even to early to think about big picture, philosophical questions, big picture questions were even in the computer size apartment where you ve, measured approach nation does a lot of infinity. Alot of of huge resources needed are approximations approximations to see that within NATO framework that are useful, you ever have developed a couple of approximation. And what we do There is that the Salama induction part, which was you'll, find the shortest program, describe your data. We just replaced by standard data, compresses right and the better compress gets. You know the better. This part will become be focused on a particular compressor called context. Tree weighting. Bitches are pretty amazing, multiple well known and has beautiful theoretic

Poverty is also works reasonably valley practice, so we use that for the approximation of the induction into learning, into public support and for the planning part be essentially just took the ideas from a computer go from two thousand and six. It was cheaper very also narrative mind will develop the so called you sit here: algorithm upper confidence about forthwith egoism, on top of the monti college research. So the approximate is planning part by sampling and its successful on some small toy problems. We don't want to lose the generality right and that sort of the handicap right. If you want to be general, you have to give up something so, but this single age and was able to play your small.

games like kuhn, poker and tic, tac, toe and and and and even pac man, I and and the same architecture, no change the agent doesn't know the rules of the game, really nothing at all myself. By player with this environments. So here again, schmidt who were proposed, something called on machines, which is a self improving programme. Very rights, its own code, whoa so mathematically philosophically was relationship in your eyes If you're familiar with it between aches in the girl machines, you're familiar with it, he developed it. While I was in his lap and saw the girl machine explain briefly and you give it a task, he could be a simple task. As you know, finding prime factor in numbers fight. You can formerly write it down, there's a very slow. I wouldn't do that. Just all told a fact: russia, or play chess right

Emily about idols and too many months till the end of the games feel right down what the girdle machine should do. Then it'll take part of its resources to run this programme and other parts of the sources to improve this programme and when it finds an improved version which prove obliquely computes the same answer. So that's the key part here. It needs to prove by itself that this change of programme still satisfies the original specification, and if it does so, then it replaces the result,

program by the improve program and, by definition, does the same job but just foster train, and then you write proofs over it and over it, and it's it's. It's developed in a way that 'em all parts of this Google machine can self improve, but it stays provably consistent with the original specification, so 'em. From this perspective, it has nothing to do with icy, but if you put now, could I see you as the starting axioms in it would run oxy, but you know that takes forever but if it finds approvable speed up off. I see it will replace it by this and that this- and this may be eventually comes up with a model which is still the icy model cannot be, I mean just for their

knowledgeable rita axes incompatible under can prove that. Therefore there cannot be a computer ble exact. I got the computers there needs to be some approximations and is not dealt with the girdle machine, so you have to do something about it, but this activity, l model which is finally computer but which we could put in which part of access and non compatible. This element of induction part the inaction oak. I saw but there's ways off getting computable approximations of the ic model, and so then it's at least computable. It is still way beyond any resources anybody will ever have, but then the girdle machine could sort of improve it further and further in an exact way. So what this says is it's theoretically possible that the the the girl machine process could improve? Isn't?

Isn't, or is you are ready? Optimal is optimal in terms of the river collected over its interaction cycles, but it takes infinite time to produce one action and the world. You know continuous, whether you wanted on Malta, so the model is assumed had an oracle which you know solve this problem and ended the next summit, milliseconds or direction time you need gives the answer. Then axes optimal, I saw the optimal incense of dates are also from learning efficiency and data efficiency, but not in terms of the patient time ago, mission in theory, but probably not probably, could make it go faster yet and interesting. Those to compose the supply adjusting the the perfect intelligence combined with self improvement. Surprise.

While south improvers, as he always improved like it, yeah I was getting the correct answer in your proving rebeautified is ok see. You also mentioned that differ kinds of things in the chase of solving this, a reward survived for the goal, interesting things could emerge so is there a place for consciousness with an icy? where, where it is, maybe you can comment, because, I suppose, humans are just another sensation, wakes agents and we seem to have consciousness. You save humans, in sensation of max agent. Yes, for that would be amazing, but I think this is for the smartest and most rational humans. I think maybe they are very crude approximations interesting. I am here

I tend to believe again, I'm russian, so I tend to believe our flaws are part of the optimal, so the the wizards tend to laugh, often criticize our flaws, and I tend to think that that's actually close to optimal behavior, but some flaws. If you think more carefully about, are actually not flaws here, but I think still enough flaws. I don't know, as is unclear as a student of history, I think all the suffering that women in do It is the civilization as possible that that's the option. A modest suffering. We need to endure To minimize long term suffering, that is that your russian background over the threshold whether for humans are not associations of annex iii agent. Do you think there is cautious? This is something that could emerge

In a computational former framework like eggs, he let me also ask your question: do you think I'm conscious as your question you your journey? that that ties confusing me, but I think I do think it makes me unconscious because it strangle smear if, if an agent would solve the imitation game posed by touring. I think that would be dressed similarly to you, because there is a kind of flamboyant interesting complex, ex behaviour patterns cells that year, human, your cautious but thou. Why do you ask Was it a yes or a? No? Yes, I think you have eighty you're conscious yes here so I knew explained sort of somehow why? But you infer that from my behavior right, you can never be sure about that, and I think the same thing will happen:

With any intelligent weight can develop if it behaves in a way sufficiently close to humans. or maybe you ve, not humans. I mean you know. Maybe talk is also sometimes a little bit self conscious right. So so, if it began in a way where we attribute typically consciousness. We would attribute consciousness to this intelligent systems, and I support in particular that, of course does not answer the question where they really conscious and that's the you know the big hard problem of consciousness. You know, maybe I'm Zombie I mean not the movie someday, but the philosophical zombie is to you. The display of consciousness close enough to caution this from a perspective, rage, I'd it at the distinction the hard problem. Cautious is not an interesting one. I think we don't have to worry about the consciousness problem, especially hard problem,

and for developing a g. I think you'll be progress at some point. We have solved all the technical problems and this system will behave, intelligent and and soup intelligent, and these consciousness will emerge. I mean definitely will display. Behaviour which we will interpret is conscious. and and then it's a philosophical question: did this consciousness really emerge or is it a zombie which, just you know, fakes everything and we still don't have to figure that out, although it may be interesting and at least from a philosophical point of his very interesting, but it may also be sort of practically. Interesting know that some people say you know if it's just faking consciousness and feelings, you know, then we don't need to be concerned about. You know rights, but if it's real conscious and has feelings, then we need to be concerned. I can't wait till the day where ass systems exhibit cautiousness gazelle truly be some the hardest ethical questions about.

do that it is rather easy to build systems which people ascribed consciousness and I give you an analogy. I may remember: maybe it was before you were born the tamagotchi frequent board, how sir Why this the yeah but you're young right? Yes, it's good thing. Thank you. Thank you very much, but I was also in the soviet union. We didn't have that wouldn't have any of those fun things, but you have heard about this tamagotchi. You know really really primitive. Actually, for the time it was, and you know you could race, you know this and and and and kids got so attached to it. and you know I didn't want to let it died and would have probably before would have asked you know the children? No, do you think the stomach it is conscious and they will the assets of us. I was just, I think, that's kind of a beautiful thing actually no cause that cautiousness ascribing cautiousness, as seems to create a deeper connection, which is a powerful thing, but you have to be careful.

excited that will. Let me ask about the asia community. Broadly, you gonna represent some of the most his work on a jar of at least earlier in deep mind represent. serious work on idea these days, but damn Why, in your senses, the asia midi, so smaller has been so small until may do you mind came along like why, I am more people seriously working on human level. Superhuman level, intelligence from a former perspective, are capable of former perspective, that sort of yarn an extra point the cup of reason: semi, nay. I came in waves writing winter or summer, send another big promises which were not fulfilled and people got disappointed and

that's no way. I saw the particular problems which seem to require intelligence was all this. To some extent successful endeavor improvements, small steps and if you build something which is you know and useful for society or industrial useful, then there's a lot of funding. So I guess it was in past the money and which drives people to develop specific. System solving specific costs. But you would think that rightly stun university you should be able to do ivory tower research ends. It's probably better long time ago, but even nowadays, there's quite some pressure off of doing applied, research or translational research, and you know it's harder to get grants as a theorist, so that also drives people away. It's maybe also harder attacking the general intelligence problem. So I think

enough. People I mean. Maybe a small number were still interested in and formalizing intelligence and and and thinking of general intelligence. But I know not much came up right or not. Not much great stuff came up. So what do you think? We talked about the formal big, a light at the end of the tunnel, but from an engineering perspective, what do you think it takes to build new Xy system? Is it in I dunno? If that's a stupid question or distinct question from everything we'll be talking about, I exit What do you see as the steps that are necessary to take two started? Try to build something for you when the loop and now and then you go ahead and do it I have pointed this conversation. Try squeeze out in their nicer What's your intuition is. It is in Robotic space is something it has a body interest exploit world is in. Learning space like the efforts of the physician offices,

are there any kind of exploring how you can solve it do in assimilation and gaming, world is their stuff instead of the other traps Our work, unnatural, english processing, certainly be attacked. Open, a dialogue like what? What what do you see a promising pathways. Let me pick the embodiment: maybe I'm so embodiment is important. Yes, and no, I don't believe that we need a physical robots walking all rolling around interacting with the real world in order to achieve a joy, and I think it's more of a distraction probably than hell.

for this sort of confusing the body with a mind, ample industrial applications or new term applications. Of course we need robot us for all kinds of things here, but for solving the big problem, at least at this stage I think it's not necessary, but dances also. Yes, that I think the most promising approaches that you have an ape didn't, and you know that can be a ritual agent. You know in a computer, into acting within environment, possibly are three simulated environment like in many computer games and and you train and learn the agent, even if you don't intend to later put it sort of you notice. I grew up in a robot brain and leave it forever. Individuality getting experience in a low voice. Assimilated treaty world is possibly, as I possibly important.

To understand things on a similar level as humans do, especially if the agent or primarily the aged one needs to interact with a human right and if you talk about objects on top of each other, in space and flying a cars and so on, and the aid and has no experience with even virtual three levels is probably heart to grasp. So if we develop an abstract agent, savvy take them at america path, and we just want to build an agent which can prove theorems and becomes a I'd better mathematician, then this agent needs to be able to to reasoning very abstract spaces, and there may be a sort of putting it into three d in immense immolate about is even harmful, it should serve you put it in. I dunno environment budget creates itself or so

it seems that give an interesting, rich, complex trajectory through life in terms of your journey of ideas. So it's interesting to ask what books technical fiction, philosophical and books ideas, people had a transformative effect books are most interesting cause. Maybe people could do also read those books and see if they could be inspired as well. You're, lucky last books and not singular book, it's very hard and try to pin down one book that I can do that at the end. So the most the books, which were most transformative for me or which again most high, recommend to people interested in I, both perhaps the day on top of the area. I would always start with ruslan orbic, artificial intelligence and modern approach? Debts

I bible? It's an amazing book. It's very broad covers you know all, approaches to eye, and even if you focused on one approach, I think that is the minimum. You should know about the other approaches out there, so that should be our first book forth. This should be coming out. Some power k interesting depot there, deep learning chap another must be written in good, fella, ok and then the next book. I would recommend the reinforcement any book. I suddenly bottle that abuse the full book? If there's any problem with a book, it makes r l feel and look much easier than it actually is its very gentle book very nice to read the exercise. You can very quickly you now get somewhere else systems to run you not very toy problems, but it's a lot of fun and you very in a couple of days

You feel you know, you know what I like about. That's it's much harder than the book. Come on. Now is an awesome book that idea and mm a b. I mean they're, so many books out there. If you liked information, theoretic, approached and eskimo go of complexity by and the tawny parts are. Probably you know some. Some short article is enough. You don't need to read a whole book, but the answer is it's a great book and if you have to mentioned one all time, favorite book so different flavor, that's a book which is used in the international baccalaureate for high school students in several countries. That is from Nicholas alkynes theory of knowledge, second edition or first not to third place the third

they put, they took out all the fun so that this us all the interesting or to me interesting philosophical questions about how we acquire knowledge from our perspective from mouth, from from physics, am and ask. How can we know from anything? I book is called theory of knowledge from which person is this almost like a philosophical exploration of how we get knowledge from I think yes, yeah I mean. Can religion tell us, you know about something about the world. Can science tell us something about the book? Can mathematics or is it just playing with symbols and and and you're just open ended questions, and I mean it's for highschool students, so they have the resources from hitchhiker's guide to the galaxy from star wars and the chicken cross, the road and it's it's it's fun to read and but is also quite deep. If you could live one day of your life over again as it you truly happy, or maybe, like we said with a box, was truly transformative. What

day one moment: would you choose the something pop into your mind? Does need to be a day in the past or can it be attained a future was space. Time is an emerging phenomena, so it's all the same anyway. Ok, ok from the past, you really good safe from the future. nobody value from the future. From the serpent past, I would say when I discovered my similar I mean it was not in one day, but it was one moment. May I realized come ago of complexity? I didn't even know that it existed, but Eric discovered solve this compression you myself, but immediately. I knew I could be the first one, but I had this idea and in a new about sequentially, this and I knew, if I put it together. This is the right thing and yeah are still when think back about this moment, I'm I'm super excited about was there.

Is there any more details and context that moment in apple fall in your head, were so like if you look at the end, talk about ganz, there's beer involved there is there some more context of what sir What's your thought? It was just that Nord was much more mundane, so I worked in this company. So in this sense the four and a half years was not completely wasted and so and I worked on an image into population problem and I develop a quite need new interpretation techniques and they patented, and then I go in it, which happens. Quite often I got of overboard and thought about you know, yet it's pretty good, but it's not the best. So what is the best.

It's a good way of putting into interpolation and then I thought yeah, you you, you want a simple picture, which is if you course credit recovers the original picture, and then I thought about the simplicity concept more in quantitative terms and then everything developed and somehow of the full view for mix of also being physicists and think about the big picture of it then led you to probably and the big with idea. So, as a physicist, I was probably trained. Not to always thinking computational terms, just ignore that and think about the other to the fundamental properties it you want half so what about, if you could relive one day in the future or death or or would there be when a solve the gy problem and the practice impact Besides, syria softly the dikes about albania, factors and then ass, the first question: what would be a first question? What's the meaning of life

I don't think there's a better way to end it. Thank you without it is a hugely to finally meet you there did you? I was a pleasure. My set you thanks for listening to this conversation, marcus hotter and thank you for presenting sponsor cash, app download. It is cold legs, podcast you'll get ten dollars and ten dollars or go to first, an organization than spires and educate young minds to become science and technology innovators of tomorrow, if enjoys by gas subscribe, I knew tube good five stars, an apple pie, guess support pay. John, are simply connected me on twitter lex. friedmann and now let me leave you some words of wisdom from albert einstein. Measure of intelligence is the ability to change. Thank you for listening and hope to see you next,

time the.