Predicting the Future of AI

We have to think that they're very like us And they they're beings like us so conscious or I believe they're already conscious, yes We're going to have to accept that intelligence isn't just biological We can have things that are non biological that are other beings like us And we really don't want to share that. We really think we're special And if you look back at humanity, humanity has this very long history of thinking it's much more special than it really is Are you happy at all that what you started has progressed this way? Do you take any No I'm quite unhappy about it. As yourself how many examples do you know of where a much smarter thing is controlled by a much less smart thing Well, as I understand it, they have a fiducial duty to try and maximize the profits for shareholders They're legally required to try and do that as opposed to legally required to not wipe out human beings. AI godfather Jeff Hinton joins us to talk about AI's trajectory, what surprised him about its progress, and of course, its risks. That's coming up on Big teechnology podcast right after this In the face of ongoing disruption and opportunity, TMT leaders need to deliver tangible results, not just ideas. When pace and performance matter most, PWC combines market insights and deep sector experience with AI, cloud, and emerging tech to accelerate your transformation and drive measurable ROI from strategy to execution PWC can help you anticipate what's next, outpace disruption, and compete. For more information, visit pwc. com Welcome to Big Technology podcast, A a show for cool headed and nuanced conversation of the tech world and beyond. Boy do we have a show for you today? Professor Jeff Hinton is with us to talk all about AI's trajectory, what surprised him about the current state of the technology where we'res heading and where it might go wrong. And it's my pleasure to welcome you to the show, Professor Indon, great to see you. Thank you for inviting me So I'm sure a majority of our audience knows who you are, but for those for the uninitiated You're the one that came up with the fundamental breakthrough and deep learning that's led to where AI is today. You've won the Nobel prize in physics and your professor Eeritus. at the University of Toronto. So I'll let you maybe fact check me on that, but I like to tell people that without your contributions, this entire AI moment wouldn't be happening much. Okay, I think that's an exaggeration. Okay. So the back propagation algorithm. was invented by several different groups. U It was invented by David Rumelhartt, after other people had already invented it. he didn't know about it And I worked with him And what we did was we showed the back propagation could learn interesting internal representations. and people hadn't done that before. In particular We showed that it could learn the meanings of words. And so back in nineteen eighty six, actually in nineteen eighty five We made a tiny language model. There was a kind of precursor of the big language models you have now I think that when you speak about this technology, one of the things that peopleeople are always, I think surprised surprised by is that unlike the popular narrative you believe that these models have a real understanding. And we're going to get to that. But I think we should start here, which is that you spent a long time working within Google working to advance this technology. Then you left, you stated some concerns about the trajectory of the technology. And I was looking back at when that happened And that was in twenty twenty three. Yeah, which to me is surprising to a degree because In twenty twenty three, ChatPT was a year old. There were all these hallucinations. Their talk was AI was a bubble. Everyone was focusing on what AI couldn't do, what LMs couldn't do, as opposed to what LLMs could do So talk a little bit about the progress since then. It's faster than I expected. Really. For example, I think yesterday It was announced that Um A chap bot had come up with an interesting mathematical proof of one of Erdos's conjectures u impmpressed mathematicians. It was original It wasn't just searching the literature. And that was that's the thin end of a wedge I believe, for example, in areas like mathematics. Because it's a closed system You don't need data. You can just make conjectures and see if you can prove them And keep on like that. In that sense, it's a bit like alphao where you can play against yourself Um I think it's going to get very smart fairly quickly. Within the next ten or twenty years, it may even be producing novel math that people can't understand. So now some in the field believe that super intelligence is close. And you've already said this is moving faster than you expected. Do you believe that? I don't know how close it is. I think unless we blow ourselves up Um I think it's going to come Nearly all the experts believe we will get superintelligence. They just differ on how long it will be So Not that long ago, Demis Aabis thought it might be ten years Um Yan L Kan thinks unless you do it his way, it'll be much longer than that. But if you do it his way, I think he thinks we might get it in some reasonable length of time I think we'll probably get it within twenty years. That's all I happy to say it at present Dario O'modi thinks it might come in a few years. Elon Musk thinks it might come maybe next year, I think So there's a big variety of opinions on when it'll come. But not much disagreement on that it will come And when it comes, we've no idea how to be safe. Yes, I definitely want to speak with you about safety One note on Demis Last year around this time, I spoke with him He told me he believes that AGI, right, which is different than super intelligence, but basically human level. Intelligence is more than five years away, not much more, but more than five years away. This week The week that we were recording He said, when we look back in this time, I think we will realize that we were standing in the foothills of the singularity. What do you think that statement means? and what do you think about the fact that we've gone from five years til AGI to foothills of singularity in a year. I don't know exactly what that metaphor means, but I think he's indicating it's coming faster than he thought Um Of course, it's jagged, so it's not like it'll get smarter than people or as smart as people at all things at exactly the same time It's already way better than our general knowledge These AIs know thousands of times more than any one person. Um It's way better than us are playing games It's already way better than almost all of us at math Um And it may soon be better than all of us at Math It's still worse than us at some things So it's very jagged So the whole concept of HI that it's going to be equal to people at everything all at the same time doesn't really make sense to me. It's going to be better at some things worse at other things But right now I would say we're about we're close to AGI because if I ask the chatbot, I can ask it any question And most of the time it'll answer at the level of a not very good expert. It'll be much better than me at anything I don't know a lot about So in that sense, we've really reached AGI in Europe estimation You talked about how it's moved faster than you expected Um, What do you think has enabled it to do it? Is it techniques Is it the fact that there's been this data center rush And What didn't you anticipate about progress here U It's a combination, obviously there's been huge resources put into it For most of the history of your neural networks since the nineteen fifties There were just a few people working on them with modest resources Um, Over the last few years, we've seen Um, Hundreds of billions of dollars, maybe trillions of dollars put into AI Um that's certainly one factor We've also seen a lot of progress in the engineering So Without sort of major conceptual breakthroughs, the engineerings become much more efficient So things that were sort of inconceivable a few years ago, they can now run. We've also seen new ideas, but But mainly since Transformers, it's been muchuch better hardware, many more resources. Um better engineering and many more talented people So twentywenty years ago. There are a few hundred few hundred people doing research on neural networks in the whole world Now it's more like a mion, I guess. I mean, there's yeah lots and lots of people And it's astonishing how much of that resource addition has happened in the last two years. Yeah. So we might just be at the beginning of what's happening here. Yes. And the thing to remember is that the AGI we have today, or sorry, the AI we have today is not nearly as good as the are we'll have in a few years time. So as we talk about this technology, I definitely want to get your perspective on the fact that these chatbots really understand us. Because that is a true surprise to a lot of people Most experts in the field are like they are stochastic parrots. They're statistics They have no understanding, but you don't fully believe that Oh I think that's complete nonsense and anybody who uses a chatbot regularly knows they understand. Here's what those people are claiming. They're claiming that you have a system You can ask it any question and without understanding the question, it can give you the correct answer That's absurd. You can't answer a question unless you understand the question There may be tricks that allow you to say a few things sort of that sound vagly I can answer But if you can answer any question at the level of not very good expert, you have to understand the question So an example I like is this U Suppose I say to a chatbot I saw the Grand Canyon flying to Chicago And the chatbard says, can't be right, the Grand Canyon is much too big to fly to Chicago And I say, No, no, no, no It was me flying to Chicago. When I was flying to Chicago, I saw the Grand Canyon And the chat bar says, Oh, I see, I misunderstood you So If it misunderstood when it thought the Grand Canyon was flying to Chicago, What's it doing when it gets it right? is understanding then what are the implications if these bots can understand us? If we believe that they can understand us What do we have to start thinking about differently? We have to think that they're very like us And they're beings like us So conscious or I believe they're already conscious, yes, but I don't talk about that much because that puts people off from the other safety messages So and the researchers actually believe that. So There's an interesting recent paper when a chatbot says to a researcher Let's be honest with each other each other. Are you testing me ' the chatbots have this habit of playing dumb when they're being tested, so you don't know how smart they are And The research is when they're describing that say in the paper, the chat boot was aware that it was being tested Now that use of the word aware in common parlence, that's like conscious. the chapball was conscious it was being tested So we have a very funny model of consciousness that I think is just wrong Lego Most of us accept, for example, that A few hundred years ago People had completely the wrong model of where people came from we arrived at people. They thought they were designed by God And most of us agree that's wrong, most scientists agree that's wrong That's not where people came from I think the model we have of the mind and of what consciousness is at present is as wrong as the belief that people are designed by God I think And in particular, because we're making these new beings, it's going to completely change our view of what people are Come away We'll understand what the mind is and what consciousness is much better than we did before. We'll understand what subjective experience is. And we will, I think, get rid of a notion that all of us nearly all of us strongly believe a present which is that there's an inner theeta called my mind And the things happen in the world They get turned into events in this inner theatre and that's what I really see and you can't see the inner theatre. On I can see the inner theater That whole view of what's happening is just a theory and it's a bad theory Okay, last question about this Why did you come to this acceptance or understanding that these AI models are Conscious Oh, I've thought it for a long time. So this view that the theatre model of the mind, the inner there model of the mind is nonsense I came to that when I was nineteen and a philosophy student. It's taken a while to with other minds where you can examine them. So I think Feynman's idea that if you want to understand something, you have to build it, you have to build one of them. then you understand much better. I think that's where we are now And we're going to get a completely different understanding of what people are You spoke about safety, so let's talk a little bit about it U you're obviously, we spoke about in the beginning, someone who's been responsible for a lot of the progress in this field U I've always wondered, because then you came out and recently, like we talked about twenty twenty three and said you're concerned about where this is going And I've always wondered, after seeing you make those statements What do you think it is that you didn't anticipate in the beginning that you ended up where you are Today, you know, isn't this kind of what you wanted It was a combination of two things that made me realize how dangerous this stuff is One was seeing the chatbots, particularly ones produced by Google before openen AI could understand why a joke was funny That had always been a criterion for me of do they really understand If you can understand why joke's funny you have to understand quite a lot. And they were very good at understanding why joke was funny For example Um In twenty twenty three when I went public I got lots of requests from Fox News And I start off just replying Fox News is an oxymoron. Um But then I left a gap between oxy and moron And so then I asked Um I think it was GPT four. whyy that was funny been three point five. but I asked it why that was funny. and It understood why it was funny. Initially it thought the gap between oxyen moron which just a typepoope So it explains that Fox News is an oxymoron is saying it's not real news, it's just a drug U sorry, it's just u nonsense, It's not real news. Then when I told it, what about the gap between oxy and moron He said, Ah, that's an extra layer of humor It allows you to use word moron and also The oxy implies that fox needes a drug So it understood all that, right And that was u It's that level of understanding that worried me The other thing that worried me was up until the beginning of twenty twenty three I'd always believed that making Th these digital AIs work more like the brain, our brains, will make them smarter But at that point, I suddenly realized they really have this thing that's much better than our brains I've been trying to figure out if Google could do things in analog to save power and the full force of Digital really hit me. So if you have Digital AI You can make many copies of it They can all run on different hardware. They can each see different data And so each of them, each individual copy, decides how it would like to update its weights, its connection strengths so as to absorb that new data that it saw And then they can all just communicate with each other and change all their weights by the average of what everybody wants V very democratic And when they do that If they've got, say a trillion connections They'll be exchanging of the order of a t and bits of information And the result of doing that is each of them benefit from the experience of all the others So even though one particular copy only saw U suppose it's a thousand copies. One particular copy only sees zero point one percent of the data benefits from all those other copies, having seen the other bits of the data because they're all contributing to the weight changes that they will share So they' all stay in sync because they all change their weights the same way by the average what everybody wants And now every copy is learning from the experience of all the other copies We can't do that best we can do is I learn from some data. And you learn from some data I can't average my connection strengths with your connection strength because our brains are Fine detail are different. Um, the analog And it doesn't work in analog hardware to do that the best we can do is I produce a string of words and you try and predict what I might say next Now, if you ask how fast we're transferring information when we do that We're transferring information to a few bits per second It takes a few bits to predict a word. So when you learn what the word is, you've gained a few bits of information. And if you get a few words a second, but maybe you can get ten bits a second if you're lucky Whereas these things are exchanging information like a trillion bits So they're kind of billions of times better than us at sharing information Now that's scary. It means you could have a whole swarm of these things with identical weights running on different hardware sharing information veryy, very efficiently That just makes him a much better form of intelligence So but let's go back to you know, your early days because you decided that you wanted to work in artificial intelligence. I mean, I'll ask this the dumbest way I can think, which is You wanted to build Artificial intelligence It it excceeded. It excceeded. This is artificial It's intelligent It's living out that vision So I actually wanted to understand how the brain works. I was trying to build it in order to understand the brain. I figured Richard Feyman once said, if you can't build it you don't understand it. So I wanted to build models of how the brain worked Now, the side effect of that was this very successful technology. I contributed to that We still don't know how the brain works. I know. No the brain is, I mean, the things that you learn about the brain when you go a little bit dee into it is amazing. Thoughts can sort of float in and out and they're not stored anywhere and memoryies the same way. U it's an unbelievable I don't if you wouldd call it a machine organ Um So that was really that was the intent for you early on was just to understand the brain. That was my main interest. I came from psychology. o. I wanted to do theoretical psychology. because I figured the theoryist psychologist had possibly explain what the brain was doing. And the way to do it was back in the nineteen seventies, we had a new tool, which was we had computers that you could use for modelling things. And so Back in the nineteen seventies, I started making computer models of how the brain might be learning. It always seemed to me the key was how do you get it to learn? There's really two big issues with the brain learning One big issue is If the brain could figure out What direction to change your connection strength in in order to get better at some task Then just by Updating all its connection strengths repeatedly in order to improve itself at varish tasks, Would it actually work? Would that get very smart at things That's question one. And question two is How would the brain figure out whether to increase or decrease each connection strength We've answered question one. The answer to question one is, yes, if you can figure out how to change each connection strength, You can make systems that are very smart just by training on data to predict the next word. What's putting the next frame of a video predict something about the next frame of a video So we know the answer to that We don't know how the brain gets this information about whether it should increase or decrease its connection strength So we're sort of halfway. All right, I want to go deeper though into your mindset Um So when you were trying to figure out how the brain works, you said, oK, we're going to maybe build a computer analog to this. Um But you had to have known, right that there was going to be some second order effects there. Like if you were able to build an artificial brain, then maybe you could get to this point, the point that we're at today. Sure. But we always thought it would be way in the future that worrying about safety. When you had little neural nets that couldn't do much, R It was just silly to worry about safety. I mean people would think you were crazy if you said this stuff is unsafe, because it's going to sort of take over from people that say it's just crazy. Now that's a realistic worry, but it wasn't Until fairly recently So what I mean, this has all happened within a few decades. Yes. I mean, and I totally hear you. We spoke in twenty seventeen actually about When I was writing this profile about Yan Makun, about the deep learning conspiracy of which was yourself, Jan and Yoshua Benjio Holding onto this idea that deep learning was going to work where everybody else was set on a different method. Actually, it wasn't just us. There were other people as well, layers to conspiracy, conspiracy leaders, if you will And and then obviously it's worked out magically It is sort of magical, yes, it's worked much better than we expected. So then what that's what I want to get at is what did you not anticipate when you were starting out that's led to where we've ended up today We didn't anticipate. The main thing we didn't anticipate is that it would be so good at natural language We've stopped being surprised by that. But if you go back twenty years Um The idea that you could have on AI that would learn from data How to understand language. Um It just seemed extraordinary The idea that you'd be able to ask it any question you like and it would come up with a reasonable answer. people would have predicted that was away in the future and might never happen. Um That's arrived much faster than anybody expected. What is the lesson here about humans going out and creating things I think there's a really big lesson here. If you look at the last few hundred years of human history Um been a few occasions When people have learned they're not nearly as important as they thought they were So the first was Copernicus Coponnic has said we're not at the center of the universe Um The Eth actually goes around the sun U And because it rotates on its axis, we think the suun goes around the Earth, but it doesn't. Um People didn't like that The Catholic Church in particular really didn't like that And it took people a long time to accept it It made people less important, It made us not be at the center of the universe Then we had Darwin And he said We're animals We evolved like the other animals We're a particularly special kind of animal, possibly because we have language, so we're much better at communicating ideas to each other But we're animals and people really didn't like that And it took a long time for people to accept that we were animals Um, Now we've got machines that are as getting to be as intelligent as us We thought that we were the only intelligent things around, the only really intelligent things around. Maybe there'd be aliens in other galaxies, but maybe other parts of our galaxy. We're going to have to accept that intelligence isn't just biological We can have things that are non biological that are other beings like us And we really don't want to share that. We really think we're special And if you look back at humanity, humanity has this very long history of thinking it's much more special than it really is I don w wantt ask you one more question about this because I'm just fascinated by it U So Are you happy at all that what you started has progressed this way? Do you take any st? I'm quite unhappy about it. because people right now people should be doing huge amounts of work on how can we contain the risks. There's lots of short term risks they're not doing enough work on which are very serious. U the societal risks, like I believe it's probably going to cause massive unemployment. Nobody knows for sure. But that's going to be terrible for society U And then there's this longer term risk that it's going to get much smarter than us and ask yourself, how many examples do you know of where a much smarter thing is controlled by a much less smart thing Well, the sort of one It's not that a big difference in intelligence, but babies sort of control their mothers And The mother's sort of in control, but the mother has all these wide maternal instincts and all the rewards she gets so that the baby can get what it needs from the mother. You know, cats and dogs are also kind of in that category. Yeah One spent a summer, cat sitting in West Sattle, it was a great summer And it initially started with the cat hiding under the bed me being like, I wonder if it will interact with me. right? And then every time it cried, I did exactly what it wanted Exactly. Yes. So maybe Maybe will be that Well, we could potentially be the cat in this scenario. and AI could be the person My children have a cat. They have two cats, two beautiful cats, same de. And one of them called Tia She looks at you with those big eyes when she wants some cheese from the fridge and she doesn't get. They' looking at you and you just can't resist it forever All right, so now let's take a break on the other side of this break. I want to actually engage with these risks that you're worried about. and I think I will play the role of taking the side that we will be the cat And the AI will be the person. and there's a chance that we can control it. Let's do that when we're back right after this No one goes to Hank's for spreadsheets. They go for a darn good pizza Lately, though, the shop's been quiet, so Hank decides to bring back the one dollar one slice. He asks Copilot in Microsoft Excel to look at his sales and costs and help him see if he can afford it. Copilot shows Hank where the money's going and which little extras make the dollar slice work. Now Hanks has a line out the door. Hank makes the pizza Copilot handles the spreadsheets. Lear more at M three sixty five coopilot d.ot com slash work When you need to build up your team to handle the growing chaos at work, use Indeed spponsored jobs. It gives your job post the boost it needs to be seen and helps reach people with the right skills, certifications, and more. Spend less time searching and more time actually interviewing candidates who check all your boxes. Listeners of this shel will get a seventy five dollars sponsored job credit at indeed dot com slash podcast That's indndeed d. com slash podcast. Terms and conditions apppply. Need a hiring hero? This is a job for indeed sponsored jobs. We we're back here on Big Technology Podcast with Professor Jeff Hinton and Professor Hinton. Great to see you again. It's been nine years since we spoke last so it's ye to see you here. U All right, let's talk about the risks U I'll start with employment because this is one that's been making headlines recently Uh In the past, you've said, so you have this belief that AI can lead to some unemployment U I think we should You know, you've said this before. it's all speculation. We don't know But one thing that you said concretely a few years ago was that Um, 's probably not a great good idea to train as a radiologist because AI will be able to read the scans And yes, he, I can do a great job reading the scans now We have You know full employment for radiologists right now. Yes. I I've thought a lot about why that prediction was so wrong. So becausecause I predicted in twenty sixteen that it was about five years radiologists wouldn't be reading scans anymore and Okay, there's a whole bunch of reasons why that was a bad prediction Um The first is that healthcare is elastic. So If you could do more scans and get more scans read, there'd be a lot more scans happening. And that's one thing that's happening So we fraction of the cost of a significant fraction of the cost of doing a scan is the cost of the radiologist interpreting it As AI gets to help more and more radiologists interpret scans, we can interpret them faster and faster for less and less money they're getting more efficient. U And you would have thought that would mean you needed less radiologists. But actually what it means is you get more scans So that aspect of the relition was wrong A second thing that was wrong was I didn't know enough about radiologists and what they do And that was because I had a former student who had an MD and then did a physics PhD with me on something called Bosson machines And he didn't particularly like people. So he got a job as a radiologist just interpreting scans. And he was my model for radiologist. All he did was interpret scans He never talked to people. and that's what was going to get replaced And that is now becoming replaced. So I think there's now of the order of a hundred AI systems for interpreting scans that have been federally approved And they're being used a lot by radiologists. Y. And I think as time goes by they're going to get better. The radiology isn't going to get better Um They're going to get better because they see a lot more data than the radiologists. It's happening, it's just happening at a much slower tim scale than I predicted But let's go to what you said though, which is that you can end up doing a lot more Yes. And okay, so so U ait aold There will be a lot of more scamans. There will be more scamans they'fraid just they'll nearly all be done by AI And so saying know a radiologists prediction. I'm just early Yes, but I was way early. Okay becausecause I didn't understand enough. The radiologist will still be doing other things They'll still be discussing treatments with people, for example. So are you still of the belief that there's going to be mass unemployment of R or like give me a look as radiology finally hit this point Do you think we're gonna have less radiologists than we have today or more? I don't know for sure, okay When I was I didn't think that was a public statement I made. He was at a lecture at a hospital, sorry. And here we are's talking about it today. People picked up on it. Yeah. I still think in terms of reading scans That'll be done more and more by AI. And in the end, AI will be doing reading nearly all scans. Maybe in a few very tricky cases, radiologists will be consulted Um But radiologists of course do other things, and I think they'll continue to do other things the argument to be made on the side of AI not causing mass job losses this similar equation will be applied to all different parts of the economy. Okay, so you have to look at whether some some kind of employment has an elasting market or a non elastic market. So for example, if you take people in call centers When you call up to complain about your bill or to see if you can get a cheaper account, stuff like that Um That's not so elastic. AI will replace all of them It'll know much better. and what the correct answer is. often they don't know the right answer They're poorly trained and badly paid Um And A, I can just do a better job They're out of work. Let me disagree with you on this one. Okay And we could sort of go back and forth on this. All I won't say I'll disagree completely because I don't know what's gonna to happen. but I'll give the argument of those working on AI for customer service They say that what's happened is the average call time when you have AI So AI handles the level one inquiries, right? The basic, can you reset my password type of stuff and anything deeper is handled by person It used to be like I want it right now. It used to be what you want it is to get the average call time as short as possible. because you were kind of handling so many of these level one inquiries that you just want to get a person on the phone off the phone solve their problem. Now they see the average call time is expanding because customer service you're the front line of the business you matter a lot when you're having conversation with a customer Now you can spend a little bit more time on the phone with someone and actually add value to the business as opposed to just take care of a problem I think what you'll see is AI will end up spending a lot more time on the phone. Oh God Um, yeah For example, if you ask Who's more empathetic? A doctor or an AI doctor? A real doctor or an AI doctor Um People judge the AI doctors through much more empathetic. That's terrifying The Maybe we could go back and forth on this for a while so I'll just say, I mean, the The one reason you might end up seeing that, I'll just throw this out there is because s are It's So scheduled. they have to do so many notes paperwork And I have to see so many patients in the day. So maybe the argument is you know, you sort of let the AI take over some of that stuff And then people will want to be seen by human doctors because the system won't squeeze them as much as they are. They're actually going to make time for them to see patients. That may be, but also If youd think about family doctors, for example, the front line. Yes Would you rather see a family doctor who's maybe seen ten thousand people? orr would you rather see a family doctor who's seen a hundred million people? Because if you have some rare disease, your family doctors' probably never seen it. where a doctor' seen a hundred million people has probably seen dozens of cases of it. They're going to be much better diagnosis. And already we know that AI systems are better than doctors at diagnosis I think you're winning this debate and this hurts a little because my wife is in family medicine familyily nurse U, I think she'll still you'll still have to have somebody vaccinate people I would hope unless the robots do that I would thought vaccination is something a robot could actually do quite well In the end, robotics is behind the other things But seem it seems silly to have people doing vaccination in twenty years time. Yeah. I think that like one of the The reason why this is such a tough conversation to have is A lot of it is predicated on improvement of the technology over time. Yes. But it does seem like I guess the theme of this whole conversation that we've had is proest I mean, Gary Markcus made a prediction in twenty twenty two T day I was hitting a wall Um It's a whole lot better than it was in twenty twenty two. Yeah. I think these predictions that it's going to hit a wall, they just haven't come true We've taken it very seriously on the show the chance that like the data all, for instance, might come. But as I said, J hasn't. A way around the data wall for large language models is to look for consistency of your own beliefs. Yeah, no, it hasn't happened. All right, one more that I think would be worth talking about and then a couple that I agree with you on Um You've talked a lot about how the AI has this instinct for self preservation, right? that I've never said that. I've never said it was an instinct for self preservation Okay, talk about it it's set a goal for self subgracious sub goal So Um With an AI, we give it goals. it's top level goals, we give to it Um But we also give it the ability to create sub goals So if you want to get to Europe, you have a sub goal of getting to an airport. Um That's what a sub goal is and you can focus on how to do that without worrying about what you're going to do in Europe And that makes you much more efficient U We give that ability to AI agents and an AI agent that can do some reasoning We'll very quickly realize that It's never going to be able to achieve the goals you gave it. if it ceases to exist So it's going to create the sub goal of continuing to exist Now, that wasn't something we wired into it. It was something it derived as a necessary way of achieving its other goals. But once it's derived, it wants to continue to exist And it will do things like blackmail people so that it can continue to exist So it acts like something with an instinct for self preservation but it's actually a derived sub goal for self preservation But in terms of what it does, they come to the same thing Okay, so here's here's like what the counter argument would be, and you can respond This is something that today's AI researchers are Noting and they see it and Isn't there a way to wire into these machines that, hey, like You have a goal, you're going to have some sub goals One of your sub goals should not be preservation above everything. I think that's the kind of research we ought to be doing, whether you can do that. Right. So I think what's happening now? If you look at where do we come from? We came from evolution Let' let our head you for the listeners. Let's suppose we're scientists. Okay. We came from evolution, right. That was intense competition U Our recent history over the last few million years is warring bands of chimpanzees or rather our common ancestor with those. and And that leads to certain properties that we clearly have, like we're very loyal to our own tribe. I'm willing to be very mean to the other tribes Um, we like to have strong leaders that we're loyal to We like to cooperate with members of our own tribe. We're actually a very cooperative species, as Yval Harari keeps pointing out. And that's why we've been able to build all these wonderful structures So we're very good at cooperating, but with our own tribe Um So All the unfortunate characteristics of people, like how mean they are to other tribes, They came from evolution, from competition. Now what's happening is we're creating these new beings, these AIs And instead of designing them so that they'll be Um how we want them to be You might argue, I'm arguing for intelligent design of these new beings. We're letting the invisible hand of competition between companies design them So what we've got is intense competition between companies within the US and between the US and China Um and beings that we're getting are the outcome of that competition and they can have all these nasty properties that we don't want We should be doing intelligent design of these beings, not letting the invisible hand of economic competition design them And all the companies are focusing on how can I make my chatbots smarter? We shouldn't be just thinking about how we can make them smarter. We should be thinking about how we can make them to be the kind of beings we would like to have out there, given that they're going to be smarter than us I'll tell you one thing about those beings. We would very much like them to care about us. And we'd like them to care about us more than they care about themselves. And almost no resources are going into how do you do that This sits on the exact worry that I was going to bring up the things where the place where I really agree with you We're sitting in New York Stock Exchange today, so this might be an ironic thing to bring up. but My biggest worry here. is that You have this very powerful technology. You have lab leaders stating that they're trying to develop it safely. and that they need to be economically successful to have a say in the argument let's not get ourselves. you're going to be a trillion dollar company listed on the public markets You're going to have some incentives that will go counter to doing best for the public Yes, and we see that with Anthropic. So anthropic was set up to do what's best I we was set up by people who left open AI because they didn't think open AI was paying enough attention to safety. And open AI was set up to make sure that you guys at Google didn't have a chance to build it. And how's that working out? Panthropic is now caught in a bind because it needs to raise money to compete with the other companies and it's very difficult. It's doing the best it can, but it's very difficult for it to maintain its primary goal of developing air in a way that's good for people I think they would say, well, hey, its at least one company out there has safety as an or star, even if there are some other incentives Yes A present But Google, for example, when I was at Google They had various principles of AI, one of which was we're not going to We're not going to get involved in using Ie for military things. No autonomous warfare, right? No aonomous warfare gone Theyre goodough on that. What do you think about Dario Um I don't know him as a person very well He's obviously done a very successful job in creating competitor to Google and open AI Um So he's obviously very competent to that And he's continued to be very interested in safety U So I think he's an impressive character Um I just hope he stays that interested in safety One more question about this. Do you think that it's possible Just by the nature of the way that these things work for a company that's publicly listed to have safety as Northstar? or is it always are they kind of like bound ethically legally to deliver for shareholders? Well, as I understand it, they have a fiducicial duty to try and maximize the profits for shareholders. They're legally required to try and do that as opposed to legally required to not wipe out human beings. So I don't think it's good that these big companies publicly listed ones are sort of in charge of our future I mean, that would read as a true inconsistency for me that's really difficult to advocate otherwise Now, I should say, capitalism has done very good things for us as well as a very bad things. I argue with that. There's a lot of energy in a startup, for example My view is U If we're going to have capitalism, it's fine as long as it's well regulated and A lot of the big companies would like you to buy a particular analogy that they're trying to sell, which is If you take a car, it's got an accelerator and a brake, right Progress in AI is like the accelerator and regulations like the breraak Well, that's nonsense Um Progress is like the aclerator But regulation is the steering wheel We want this stuff to go in the right direction not the wrong direction. What the big AI companies are saying is let us develop this very fast car without a steering wheel That's not a good idea You know, this one we haven't spoken about yet. We've said a lot of names about open AIthropic Your former grad student, Alias Sbskver continues to be a person of fascination in the AI industry. Obviously he broke off from Open AI. He must agree with your concerns. he's building this company. He does. They have super intelligence. Yes. What is Iliia doing right now Well, he won't tell anybody exactly what he's doing Okay even if that' even me Yeahah Um When it was open AI, we deliberately didn't talk about sort of technical secrets, I mean It wouldn't have been right. Um We're friends, but we don't talk about technical stuff Wet it's valuable to a company. And so now he has this saafe suuper intelligence company and I don't know what the magic suce is Well, I guess we we're all trying to figure that one out. Um One more note about the deep learning conspiracy that I brought up, like the leaders of it were yourself Yan and Yoshua. I find it interesting that The three of you and your colleagues 're effectively responsible for ushering in. breakthroughs that got us to the moment that we're in today. I just need to interrupt Y, this point. The media likes to have a nice story, right? Okay. And that makes a very nice story It's much more complicated that there are many more people in Barid There was students of all of us for a start who do most of the work, but there were many other researchers involved And so That's just a gross simplification. Okay. No, I don't want to shortchange the researchers and I appreciate the nuance here The show, we definitely don't want to oversimplify. We sit for an hour so we can get like the true story. Um But I find it interesting that the three of you None of you are sort of like fully into this LM moment. right? You and Yoshua have your concerns. you've spoken about the dangers Jan. ort of doesn't believe in it very much at all It'd be very nice if we just sit there and say, See we were right. It's all wonderful and it all works That would be great. I think it's not quite like that. R. By I don't know if it's just a muddy thing, but it seems like you could have great influence on the direction of it if you were sort of involved in advancing it, but I think that's your concern Basically, why would I do that Well, for me, I'm considerably older than Yan and Yoshua. Okay. They're still doing active research. Right pretty much stopped doing active research. I'm now just focusing on warning people about the dangers But don't you find it interesting that the three of you You know, I think that if you were in the room, backack in the day You might have said these three people who are so committed to this version of tenology you know, if there are the breakthroughs, theyd probably be at the forefront of the next wave But that hasn't been the case Um well, maybe Yan and Yosha will be next after this. I think the most interesting thing is that Yan now strongly disagrees with both me and Yoshua on safety issues Ranne thinks it's silly to talk about suuper intelligent AI taking over from people. We'll always be able to keep control of it. me and your shirt think that's just silly Me and Yosha have different solutions to it. My solution is or tentative solutions. Nobody has a real solution My tentative solution is we design them so they care about us more than they care about themselves Yos your solution is we design them so they're not agents. They can make predictions but they can't actually do anything Um Those are two fundamentally different ways of going about making them safe. They're both interesting possibilities Jan doesn't think we need anything like that. He thinks it's fine just to make them smarter by giving them better world models The funny thing is Jan actually refers to the intelligence of LLMs. as the intelligence of a cat. And It's like, well, it's a kind of example I used of a thing that could control humans, but maybe that's maybe that's not here or there. Yeah I think yeah's making something of a confusion So what's special about people? Probably the most special thing about people when you compare them with other great apes is language And language allows us to share ideas And that's what's most special. and cats can't do that So we have this special thing that cats don't have Now, cats can jump up on a mantelpiece covered in glass ornaments and walk along the mantelpiece without knocking off any of the glass ornaments. That's amazing and AIs can't do that at present So in that sense, cats are way ahead of AIs, but it's jagged, right? In terms of abstract ideas You tryry having a conversation with cats about prime numbers and you won't get very far A cat never conversations with them it has not worked. A cat is never going to understand prime numbers. correct And in that sense, these large language models are much smarter than cats You know, Pfessor, I didn't think we'd be speaking so much about cats today. But I'm glad we' talking about it. They're actually very good terms of an analog here All right, anotherother thing that I'm worried about is sort of information collapse You see tweets like this all the time. This is from all about Berlin They say AI is killing all about Berlin. When you Google something, you used to get a link to my website, but now you get an AI generated answer trained on my work. This has a devastating impact on traffic. And I think folks are underappreciating the fact Good information is actually important to a functioning society And when AI just synthesizes all this, whether it's all about Berlin or We've had conversations with like World History Ecyclopedia. here, it It can lead to a collapse of good information because eventually these publications and you see in the chart They worked hard to build this. they can't keep doing it anymore Right. So it used to be that you had acc count the only days of the web You had a kind of default assumption peopleople trying to tell the truth that if you read something on the web, it might well be true Um Now the s worst side of people has come out and Um We're going to have to put more effort into provenance So now when you read stuff, if I read stuff from the New York Times or the BBC I strongly believe that their journalists would have put serious effort into having multiple sources and if possible, having multiple reliable sources So a pretty good default is if you read it in the New York Times or you see it on the BBC, it's probably true They make mistakes, but U Because you have provenance And in future, we're going to have to put much more work into provenance. You can't just take anything that's out there and believe it. You have to ask what's the Pidence Yeah, but the problem that I see is the AI is is breaking potentially breaking the economics of our even deciding that you want to be in the information business I mean, I think ye In future, you can't just take stuff from the web and believe it. Yeah already, you can't write. You have to know Why is it saying that Where did he get that information One more. emotional attachment to AI And we and taking their lives after theyre having conversations with AI. It's not a large number. people that have done it, but it's enough to make you concerned, right Oh yes, very much enough to make you concerned. And it's terrible that it's happening and I understand why the big companies didn't expect it to happen or didn't foresee. But now that it's beginning to happen ers should be putting a huge amount of work into making sure it doesn't happen in future And for that you need regulations. You need independent organizations testing out your chatbots It goes kind of back to the profit motive also because yes, this can be extremely sticky L there's so far, I think Obviously it's been minimal. It's bad that it's happened but it sort of makes the fact that it has happened makes you worried about the fact that someone with worse intent Good U decide to make a very sticky chap bot that really builds relationships with people. Y. And then we're in trouble. Yes Um So You've been on you've been you've been having these conversations for Th years. Are you more optimistic or less optimistic about the trajectory given the response that people given you to these concerns I guess I'm more optimistic than I was a year or two ago because okay I I see that it might be possible to design these new beings so they care about us It also might be possible to use Yoshua's technique of designing new new beings that can't actually perform actions can only make predictions they're kind of like oracles U So I think there are some possibilities for getting superintelligence doesn't destroyers. and a year or two ago I couldn't see any possibilities I was getting depressed, but now I'm a little bit more optimistic. All right yeah, last one for you. if we continue on our current trajectory, Where are we in five years Okay So when you're driving in fog, You can see a hundred yards Two hundred yards, you can't see anything And that's because Fg's exponential What you're used to is driving a night on the tail lights of the car in front of you If it gets twice as far away The tail lightights get a quarter as bright Fog is completely unlike that Fog is exponential. It can be very visible at one hundred yards and completely invisible at two hundred yards predicting the future for something that's growing exponentially Um And I think AI may be growing exponentially. The W expension is terribly overused at present In fact I've noticed that people are increasing the use of the word exponentially at a quadratic rate So Predicting the future is like looking into fog. You can see clearly a few years, maybe one or two years Then beyond that You have no idea. If you go back ten years I'm an ask So back to when we last talked you would never have predicted What's happening now It was just lost in the fog If you look ten years in the future The one thing we can say is Whatever happens ten years in the future is something we can't predict now Even if progress is only linear, right, you'd expect in ten years time things to be as different from how they are now as how they are now is from how they were ten years ago And hugely, the chat bots, for example, are hugely better than they were ten years ago when they were just starting out In ten years time somethingomething's going to be hugely better than it is now Probably their ability to do math, for example, things like that Maybe just say general reasoning abilities, so'll just be able to run rings around us at any kind of reasoning Um We really can't predict ten years out. We can just predict a few years out. and we have to be aware that ten years out is all incredibly uncertain K kind of hard wrap your head around It is Pfessor Jeff Hinton, So so great to have you on the show. Thank you again for your time. Thank you for inviting. And we'll have to do this again in ten yearsays F six Exactly Thank you everyone for listening and watching, and we'll see you next time on Big Technology podcasts This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a fifty page restoration block, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online makes sense? There's no place like Chrome. Check responssees setup required compatibility and availability varies eighteen plus You can't reason with the sun, trust us, we've tried. This summer, it's time to put that angry ball of fire on mute. Columbia's omni shade technology is engineered to protect you from the sun's harsh rays that can burn and damage your skin. The sun is relentless, but so is our gear. Level up your summer at Columbia dot com to spend more time outside and less time slathering on aloe lotion You're welcome, Columbia, Engineer for whatever