Future of AI and human labor

You're listening to the London Review of Books podcast. I'm Thomas Jones, and today I'm talking to Paul Taylor, a professor of health informatics at University College London, who has been writing for the LRB since twenty thirteen, and in particular has been writing for us on so called artificial intelligence or machine learning as it was known back then since twenty sixteen. His piece in the last issue of the paper is a diary which looks back to the relatively early days of computer programming in the nineteen seventies and ahead to a brave new world in which human computer programmers and perhaps the rest of us too are potentially obsolete. Hello, Paul, and thank you for joining me. I'm yeah, sixty, so I think it I think it'll see me out, but I'm not quite sure what happens next. Um but we'll find out, I guess. Yeah. But if we if we look backwards first, nostalgic ally to the the sepia tinted nineteen seventies. What was your what was your first encounter with a computer? I do have a very early memory of um seeing a a kind of wardrobe-sized computer in the office of of one of my father's colleagues. Um but the first time I saw a computer program was when I was taken into my my dad's office one half term because they couldn't find anything better to do with me. And I was sat down in front of a terminal and allowed to play a computer game, the computer game being Star Trek. I tell this story quite a lot because people can't understand the idea of a computer game. Let's play ed without a screen. But in those days, computers didn't have screens. So the terminal was an enormous typewriter type thing. So the game basically generated a sort of a fictional world, a very , very, very basic world of kind of sixteen quadrants, uh and you could get um a printout of it by by giving a particular command and then and then you had to navigate around this world and and kill Klingons without running out of time or energy. I can't actually remember who it was. It wasn't my dad. It was some obviously somebody who who who worked with him who was responsible for the computer allowed me to edit the the source code or I imag ine in a very supervised way. And I can't actually remember what I did, but I can remember feeling incredibly proud of the the you know just this sense that you know you changed something in in the code and then it it changes the game in some sense. Yeah, I have a really strong memory of of that. As I say in the piece, it wasn't an epiphany because I didn't become that kind of of of kid. I didn't sit and program computers in through my teens. I worked for Ferranti in between school and university. The computers at Ferranti at that time, this was nineteen eighty two I guess, were in a big shed, and we well the programmers all worked in a open plan office on desks, but there weren't computers on our desks. I I have no idea what we did at our desks. Because w when we had to interact with the computers we had to go and walk two minutes away to this enormous shed and then you'd you'd program the computers in the shed and then you'd go back to the office and then move bits of paper from the intro to the outro, presumably with some cognitive work taking place in between. But I I can't remember. Maybe you made telephone calls or wrote letters or all those sorts of things. But it's interesting because I think you know, so maybe in the future work will be a bit more like that, because anything that you can now do at a computer will be done by the computer without you needing to be there. So maybe the offices of the future will be a bit more like that. And the and the kind of the stuff that you do away from the computer will become a a bigger part of the working day. Well, that's quite a nice thought, isn't it? But even I mean, even when I started at the LRB in the late 90s, we had one computer that was used for email with one email account, and it was telephones and papers and and pens on the desks and if you wanted to send an email you'd go over to the desk with the one computer. It's so I cause I I I work in as you introduced me as a professor of health and the mathematics, so I work in healthcare and I can remember w when I first started working in healthcare, there was a big debate about whether patients would tolerate a doctor having a computer on their desk. And there was a feeling that that they wouldn't like this, it would get in the way. But then the moment I suppose probably at some point in the nineteen nineties, it would have been a bit weird to go and see a pr a serious professional to do serious work and them not be a computer on their desk. But of course now people will often complain that the doctor spends too long interacting with the computer and and not enough interacting with with the patient. And that has led to the development of an AI application which I think we'll we'll hear a lot more of, where the AI records the consultation and transcribes it and then uploads an edited transcript to the record and possibly with some hallucinations added for everybody's entertainment. It's possible that there's going to be a sort of move back and and that communication between the doctor and the patient will be improved by the developments in AI. I mean one of the often the problems with going to see the doctor is you sit there and they talk to you very seriously and you listen, you know, to what they tell you and you walk out and someone says, Well what did they say? You think, Well I don't remember Yeah, no that's true. And that idea of you know it's useful to have an advocate with you or whatever. So with the idea that the AI would then share it with the patient as well as with the There's a lot said about you know people if people are told they've got cancer they don't they don't remember the rest of the conversation at all. But we're still a long way off from the human doctor being done away with altogether. I mean maybe this this can't s kind of get us back onto talking about the the theme of the piece about programming. I think there are some professions where it matters that it's a human that's doing it. Perhaps because in some emotionally it's that you you want to feel cared for by another human being the way you're not cared for by um a chatbot. It's almost a sort of a legal I mean legal makes it sound as though we're all worried about seeing each other, but there's a desire to offload the responsibility onto you know uh another person. But programmers aren't aren't like that, I think. I think they are more expendable. And I and I think that the tech companies are targeting them in a in a way. I suspect that the recent versions of some of the large language models are being specifically trained to be better at coding because people see that as the early economically valuable use case. Whereas a lot of the other um uses people make of large language models are it's a bit like an internet search. You you use it to answer questions, but you're not really using it to do something that saves money. Or you're using it to generate, you know, large amounts of of slop which actually waste money and energy and much else. And that may be tr may be true with code as well. I mean a lot of the um a lot of the way we think about software and software engine ering is determined by the fact that software is is expensive. It costs a lot of money to develop. There are estimates that get banded around what of the number of lines of code a programmer generates a day as being between sort of 30 or 50. And that's not because it takes all day to generate 30 lines of code. It's just that most code doesn't make it through. And most of your days spent doing other things which you have to do in order to work out which lines of code you should be writing, what your software should be doing. But if code becomes cheap, you know, if if it doesn't take teams of skilled people time to develop, maybe some of the way we think about how it should be engineered and maybe some of the business models around it change as well. And that's one of the things which I I talk about in the piece is this model that's called uppercase S, lowercase AA, uppercase S, software as a service. Um, one of the things that happened when I think it was Opus 4.6 was released earlier this year was a big stock market sell-off of shares in those companies. And they've the the traders called it the sas holips. Um there's been a bit of a a correction to that, I think, because it's it's fairly seems unlikely that people will want to develop their own software. You know, one of the reasons software as a service is very successful is that companies are scared about cancelling their contracts and so will continue to pay licenses when they might stop doing other things. And if a company has that kind of mindset, they're not going to suddenly start developing their own project management tools. But it certainly there will be disruption and some companies may be less well protected by it than others. So we'll get onto the financial side a bit more or the economic side a bit more in a bit. But the I mean the question of why code I mean, as you say in the piece that it is it is a form of language and what these large language models do is, you know, predictably create things in language and the the kind of language that computer code is, are they better at producing in some ways or or m better at producing it usefully than they are a language like such as English. Yeah. And that might be because it's a simpler form of language and the the training data's out there. Um and it might be that the success criteria for what's good are are reasonably easy to establish and does it run? Does it do what it's supposed to? Which may be easier to check than sort of marketing global or whatever else you might want a large language model to do. But I think it's more the way the generation of it fits into work. So you can give a a software developer a tool which will help them gener ate code faster and they will become more productive. That's the assumption. And I've my reading of the evidence is that that is what has happened. Although I should say that the evidence isn't entirely in favour of them. There's a lot of scepticism about whether they're produc ing good enough code and whether they actually do make people more productive. In fact, the the the the original motivation for writing the piece was a conversation I had with a colleague, Wayne Holmes, who's a um a critical theorist who writes about AI and education. He very kindly sent me some evidence that he'd collated uh or studies showing that these these tools actually weren't effective. And I I read it and I felt that things were just changing too fast for this evidence really to be useful . Because he was looking at studies up to 2025 and so the research that had been done on models that came out in 2023. Um, but the more I started sort of looking at what was going on, I I came across the statements from uh people from Spotify and and and Claude and so on, and none of them's older than December. Things have changed really, really quickly, I think, in the last six months in terms of the the take up of of these tools. Even you know, I mean in AI things change all the time and it's become sort of maddening. But I think I think even against that yardstick, these changes seem to me to be happening surprisingly quickly. Is Claude the big game changer with the well, the most recent big game changer in that? Most programmers use an environment, right? And you become very loyal to your environment. And it's very important to you that you believe your environment is better than other people's. And I remember, you know, from early days of computers, these kind of mindless wars between which editor was better than another editor and you know which programming language was better than another programming language. So I I use Claude and I find if I was to shift to another product, it would be like having to go to a an unfamiliar supermarket. I'm not saying it's worse or better. It was just I just wouldn't feel it was right for me. So so I I use Claude. Uh and I I think there are a lot of people in tech who who do use Claude who feel that this is the the best coding tool, but the others are I think technically is good. One of the things that really struck me when I was writing the piece was um there's an environment called Cursor, which is the first programming environment to be developed after genus models became available and kind of integrated generative AI into us in it in a big way. And they announced that the next version of their tool is not going to have an edit Windows, the central user interface, which sends a big message. It's like sort of saying, you know, we're the next car we may I as a as a child in the eighties, but unlike you, I did. I spent quite a lot of time writing or trying to write, you know, very crap games. We already it was some remove from what the computer's doing, but that was a curse and you'd have to type it. So what happened to programming when you started doing it again as a PhD student in the nineties, were you you weren't sort of having to type out lines of code like that, were you? That there was a you were they were still doing that. So the tools bec the tools were quite sophisticated, but they were just editors. I guess it was like difference between using a kind of modern version of Word and the version of Word you might you might remember from 20 years ago. They had a formidable array of commands that you could use to to do different things. But you were just writing c there might have been a little bit of understanding of the syntax of the language built into some of them, so you could get it to type out things in a particular way that was helpful if you're using a particular language rather than another. But I I I had a slightly odd introduction to programming as a professional, in that I did my first degree in psychology and became interested in AI as a psychologist, and then learnt to program doing a master's in AI Imperial, having done a psychology in my first degree. So the first programming language I learned was Prologue, which is a logic programming language. So it's a very different style of programming to um an instructional language like like basic. What you do in Prologue is you specify relationships which you want to be true rather than a And then the computer uses your um specification to prove what can be proved. And it but it does that in a mechanical way, which is analogous really to following a set of instructions. So you've got a sort of twin way of looking at things. Back in the late dark ages, about twenty years ago, I wrote a piece of the LRB about my my childhood programming in BASIC on a BBC B. And to remind myself of the programming and of some of the games and so on, I um got an online emulator which enables you to turn your modern computer to simulate uh an old-fashioned computer from the 80s so you could program in basic and you could you could get the games that people had kindly put online and you could so you could revisit the old days. But when you wanted to remind yourself of that Star Trek game when you were writing this piece. You didn't have to get an emulator and find the code, did you? I mean you you were out you asked Claude and you sort of did it all from scratch. Yeah, so I I asked Claude and I was expecting Claude to kind of find it for me, but Claude just said wh,y bother looking for it? I'll just write it. And they just wrote it. And it was there, eight hundred and sixty lines of code. And it ran straight away. It was amazing. And then I started kind of tinkering with it and I got it to add a graphical user interface. And I thought, wow, look and it it just did straight away. Game is a bit fiddly to play. And so I kept asking it for advice and then I thought well actually why don't I put in build the the advice seeking into the game. So one of the options you have as the this kind of commander is to ask AI, you know, to to give me a battle plan and it it it did that. And and d did it adopt the personality of Dr. Spock while it was doing it? I mean you could you you could choose your name. I th that's one of the interesting things about being a user of generative AI is is the tools do have personalities. Um I don't want to anthropomorphize them too much, but you they are designed to to make you feel that you're engaging with a with a a person and there are differences between them which are like the differences between personalities. And one of the things that the people say, I haven't found this myself, um, but people say that the Chinese models are much less sycophanti And you'll notice if you if you use GPT that that it uh and Claude, they're very keen to compliment you on your the wisdom of your questions and and the clarity of your thoughts. Um apparently Chinese models aren't aren't like that. It also I think speaks to the dispute between um Anthropic and the Department of Defence, this idea that there are actually moral judgments at play when you build a a large language model and you make choices. And those are important in in a world where we we're not gonna have that many large language models to choose from. There was a moment in that dispute between Anthropic and the Department of Defence where it sort of you got the impression that maybe the perception was going to be that Claude was a slightly left wing large language model, whereas an open AI was maybe a slight more right wing large language model and Grok was a very large right wing large language model and that people's sort of consumer behaviour might are you a waitress type person or an audie type person is anthropic the right one for me? And then the question of of how that plays out when these things are incorporated into government in a big way. And if you have a change of government, will the government want to say actually let's roll back our dependence on this model and get in But that it it doesn't really work, I think, because I don't think Anthropic are a left wing company and I I don't think those are their values. I think they do have very strong values, but they're not they're a bit weirder than the sort of left wing right wing characterization. In terms of that dispute, could you just talk us through what it was about? So the defining feature of anthropic as a company is a concern with safety. And where AI safety is a has a particular meaning, it's not to do with near term safety, it's to do with the sort of long term safety, it's to do with existential threats that AI might pose. And there's a sort of slightly weird kind of culture around that in Vilican Valley. So Anthropic was founded by Dari Amadei, who left OpenAI because he felt that OpenAI had betrayed its original mission, which was to create AI for the benefit of humanity. But partly, I think, because of that concern with safety, Anthropic had targeted businesses as its client-based rather than individu al consumers. And they were very keen to win contracts from government. And they got a contract with the Department of Defense, I think in 2024, the end of the Biden administration. And then a a larger contract after Trump came to power. Uh and they built into those contracts a defense which reflected their moral values that they didn't want their tools to be used for either mass surveillance of US citizens or to control autonomous battlefield weapons. And the Biden administration was happy to sign that, and initially so was the Trump administration. But at some point after the second contract was signed, uh figures in the Trump administration decided that that wasn't acceptable. And Anthropic wouldn't play ball. And so the the two sides decided that they they couldn't work together uh and the the contract agreement was was torn up and and they walked away. And that might have been the end of the story, except that the Trump administration seemed to want to go to war with Anthropic. And so they declared it a supply chain risk. And initially, I think they said that any company that did business with Anthropic wouldn't be allowed to to have any government contracts at all. Uh and I think that was they were told that they they didn't . So they uh then adopted the slightly more restrained position that nobody could use anthropic if they were participating in the Department of Defence contract. Still quite a big deal. I mean, I'm not a lawyer. Just reading about it. People don't seem to think that this is legally defensible or justified. But it's obviously going to take a while to work its way through the courts. And in the meantime, Anthropic is formally speaking a supply chain risk. But then, while I was writing the piece, in fact, Anthropic released a new model called Mythos. In fact they didn't well they sort of released it, they sort of didn't they announced it, but said it was too dangerous for them to release it to to the world at large. And what they were doing instead was releasing it to a a consortium of trusted partners through something called Project Glasswing. And the reason for doing that was that they found that Mythos was very, very good at identifying and exploiting bugs and software. And that therefore they wanted to release it to people to help them identify uh and um get rid of those bugs and security vulnerabilities before making it more widely available. And as part of that process, this seems to have been something of a rapprochement between Anthropic and the US government. There's been media reports that the two sides have met because the government's very keen that the government should have access to mythos and be able to protect Are you a professional writer struggling with financial difficulties? The Royal Literary Fund is here to support you. At the RLF, we believe that every writer deserves the chance to thrive, no matter what life throws your way. We offer a variety of hardship grants tailored to help professional writers in need, whether it's short-term relief or ongoing support. Perhaps you're dealing with an unexpected expense, a drop in income, or facing challenges that make writing difficult, such as illness, disability, or now relying on your pension. If you're eligible, our grants team will walk you through our discreet and confidential application process, ensuring you receive the support you need. Visit rlf.org.uk to check your eligibility and inquire today. The Royal Literary Fund. We believe writers matter because writing matters. This is the LRB Podcast. I'm talking to Paul Taylor about the promises and pitfalls of large language models. That threat of metals. That is a short-term threat. That isn't a long term existential threat. It's found this way to exploit weaknesses in code or in computers that could plausibly get in anywhere and give anyone access to those computers. It can do that. That is a genuine threat. But there is also the question of the share price and so on. But any company gets a big deal, the US government, the share price goes up, falls out of the US government, presumably the share price goes down. They then make this big announc I mean there is a marketing aspect to to all of this as well, isn't there? Yeah. Now I'm sure there's there's a bit of theatre But you know, I've spent some time looking at the report. I'm not a cybersecurity uh expert, but it does seem quite impressive. I mean one of the problems with it is that there's very little ex ternal validation of it. And and one of the entities that has done an external validation of it is the UK the AI Security Institute, which is a bit uh more sanguine about it than than anthropic. They give some quite impressive statistics on how well it's done against their benchmarks, but then they say that their benchmarks are are just test environments and it probably wouldn't be as successful in the real world. I can't remember the quote I I put in the piece, but it basically says it can clearly attack weakly defended systems if it's already been able to guess initial access. So it's not as scary as as some of the um initial hype might have seemed. But it it is obviously extremely good at at generating codes. And you know, well, reading the account of the attacks, it was able to um autonomously implement, you do think, wow , this is impressive. And I I have to kind of caveat that. I mean one one way of maybe characterizing is to contrast it to um the way Alpha Zero played chess. Because when that incredibly exciting and dramatic announcement in AI came out, chess players were saying this is alien chess, right? This is not chess as we play it, it's it's not only is it much better than we are, it's kind of qualitatively different because it has a much better positional sense, so it's less careful about the value of pieces. And my reading of of what's been said about Mythos is that it's not an alien intelligence. It's not finding you know faults that we couldn't find because it's kind of you know just that much clever than we are. It's just very, very good at it and it finds very them very very quickly. So it's it's not that it's going to be able to find things which a human wouldn't be able to find, but it makes it easy for anyone to find them rather than just a you know a a a trained computer expert. And so that obviously is dangerous 'cause it lowers the bar quite substantially. And I think I I'm slightly sceptical about how Project Glasswing will work. This idea that you can sort of progressively roll the tool out to trusted partners and they will find all the bugs and fix them before no th the bad guys have a chance to get their hands on the tool. You remember the w the when the NHS was attacked by the wanna cry virus. I I mean lots of machines hadn't been patched for months, if not years. So the idea that that a large organisation like the NHS is going to be able to defend itself, gonna be able to implement all of the bug fixes that Mythos identifies and finding security vulnerabilities. I I think that's quite hard to believe that that's gonna h happen in a more complicated way. Yeah, well so that is a I mean that is a threat, isn't it? In short. But in terms of the other, I mean you're talking earlier about software as a service and these kinds of which you know we all now experience and probably all complain about and you kind of find these programs that you're committed to using and you're sort of locked into using and they've you know, but they don't work as well as you feel they ought to because you don't actually understand how they work and don't realise quite how complicated they are, or though people who do understand how complicated they are also find them frustrating. But if the the ways in which large language models generate code to the extent I understand it, they're not they're not very good at thinking well they're not very well they don't think, but in terms of structurally, is that right? So in a sense you you're you're that a lot of the code might be not the most elegant perhaps way of not the most efficient, elegant way of p of programming something. And is does that matter or d d what are the long term implications of that? Yeah. I think they are probably as good as we are at generating elegant code. The problem is software developers using large language mod els seem to be developing less elegant code. So if you think about it, if I was working as a programmer and I had a a stack of bug reports on my desk, uh once the stack gets to a certain size I'm going to think actually I'm not going to fix all of these. That's not sensible. And I'll talk to my line manager and I'll carve out a larger space of time. And I'll do what's called refactoring the code. So it's not starting from scratch, but it's kind of reorganizing it so that it's it's better structured, it's more elegant, it's easier to understand, it's easier to maintain, it probably runs faster. That's seems to be happening less. And so what's happening now, I guess, is that the software engineers get this pack of bug reports and they just work through them because you can get them done in five minutes with a large language model. And so what you're finding is that code is being this is what the I I referenced this in the in the piece, there's a there's an organization called Gitly which has done a a survey which has found that that there seems to be evidence that this is happening. That that there are large numbers of little fixes and fewer large reorganisations taking place. So over time, code will get progressively messier and harder to maintain. And that may have significant implications down the line. Or it may not. I mean, you know, it may it it I don't know. I mean it may be that that you know you get to a point where it's it's it's relatively straightforward to to throw it all away uh and start again. I don't I don't know. I mean I I do think it is quite h hard to read what's gonna happen to to the profession. It seems to me incredibly unlikely that people are going to program in the conventional way once they get access to these really, really good tools. But it it also there's quite a lot of um talk out there about the fact that untrained programmers trying to create software themselves using these tools are get into a a mess. Th I think the industry will have to shift in other ways as well that that are harder to predict. And it might be that software becomes relatively cheap and that creates a demand for more bespoke software and employment for software engineers is is retained and goes up. And we don't all have to rely on Adobe and Microsoft and everyone else and that would be jolly nice. Yeah. Uh I mean yeah, I mean that said in terms of the I suppose the the bigger threats, which you know, I mean they're impossible questions to answer really. But I mean the talk of the sort of the vast numbers of data centers that are being built and the you know, these stories about them sort of stealing water from towns and and the amount of electricity they use and, some people say well, actually it's not they don't use as much water and electricity as their critics say, but on the other stuff, the idea of the you know the total amount of physical infrastructure that may be required in terms of energy use, land use, water use. Is and is that something you worry about? Yes. It doesn't affect my behaviour hugely, I think. I don't feel in a position to influence it really. It's it's obviously signific ant . Whether the the tools that are are developed will help us solve these problems i i in other ways and it'll all turn out to be okay or not. I don't know. I think you'd have to be pretty optimistic to to think that. One of the interesting things that's happened is the shift from training to inference. So it used to be the case that these models were hugely expensive to train and that's where all of the the cost was. But now they're being used so heavily that actually just giving people ac cess to them and allowing them to run queries on them and get responses takes an awful lot of of compute. And i if you talk to people who are working in uh programming environments, I had a a graduate of mine who works for a consultancy organization and she said she spends eighty percent of her working day with a large language model. If that's becomes how you work, then if that's happening to a large proportion of the you know, even even a a a small proportion of the workforce, then the costs will become huge. But it depends how big the workforce is. I mean if the workforce is shrunk to six people one of the thought leaders was saying maybe maybe the first person to have a a a billion dollar business themselves , you know, is already out there. The idea that you could talk about this is the one person company , you know, that it's just you and the large language model, and you don't need an HR department to payroll. In a sense, it's sort of the the very crude economic problem with the whole thing, is if if a if no one has a job because all the jobs are being done by large language models. How does the economy function? I mean there's that. None of us can foresee that. I mean if you look at if you look at the actual evidence, the only authority of studies I've seen are a little bit out of date, right? So they're looking at what's happened in between the release of Chat GPT and sort of August 2025. And what you see is there's a significant downturn in employment in AI-exposed professions for junior staff. And so that's junior software engineers and customer service people. So people are being replaced by chatbots, but not the senior people in those companies, but the kind entry level jobs. And the same is happening with software engineers, that companies are deciding that the senior employees can be made more productive and they can manage without the junior employees. Obviously that that's not a long-term plan. Where's the next generation of senior employees coming from? I remember you once saying I don't know if you raised it in the paper or if you just said it to me um off the record as it were, that one of the you know, that large language models are very good at doing what we ask them to do. But the difficulty is um is knowing what it is that we are asking them to do, that maybe we think we're asking them to do one thing, but actually, you know, they interpret what we're saying differently. Um there's a there's a children's book, very good, very fun children's book, it was published about 10 years ago called Cakes in Space by Philip Reeve and Sarah McIntyre. And the the child hero, she's on a spaceship, they're travelling to distant planet and most of the people are cryogen ically frozen for the voyage, but she's awake for some reason and there's a machine that will make you whatever food you want. And so she says, Oh, I want the ultimate cake. So it makes these sentient cakes that then try sentient sentient cake, the ultimate cake, which is sentient and tries to eat the passengers. So, you know, there is that sense. I mean clearly it's it's quite a good analogy for what what we fear AI will be. People talk about the alignment problem, the idea that that we need to make sure that AI goals are aligned with with ours. And there's quite a lot in the mythos sort of preview report about their attempts to confirm that that it's safe, right, that that that there's no danger. And and they do find some evidence of what they call reward hacking, which is where it does something it's not supposed to in order to achieve a a goal that that it's been set. Um which is obviously slightly worrying. I mean this is what Jeffrey Hinton often calls the godfather of AI. Um who talks a lot of rubbish about about AI. But he said that that power is always useful. Whatever goal you give AI, it's always gonna think this will be easier for me to achieve if I have more power. And once it becomes more intelligent than us, it will be able to get power over us and and then we're done for . They're pretty confident that they're able to train that out of it. Yeah, this is the thing that that that people worry about. Volkswagen cars that AI will learn to behave differently in the test environment to the real world. And there was some thing weird that I read that I really didn't understand about an experiment they did . So you you have these um tutor student models. So you get a large model that's expensive to train, and that's the tutor, and then you get a small model that's cheap to train, and that's the student. And the student has to learn to replicate the behavior of the tutor. And so they did this in an experiment where the the tutor was trained to be enormously interested in owls. But the only communication between the tutor and the student was constrained to be about mathematics, and the student became enormously interested in owls. There were these things which we really don't understand going on and and y it does make you think that y the the powers that these machines have are kind of unknowable uh uh and unpredicta But I I think that the sort of these existential things are just so hard to imagine and they seem so difficult to conceive of that it's it it's best to focus on the I think maybe that's a mistake. I don't know I mean we've you already talked about this a bit and you know maybe we should finish with you saying who knows is a good point to stop. But that question of the tutor and the student, I mean, and that question of senior staff and juni or staff, now, as with so many potentially existential problems facing humanity, is the short termism of it. Because if we we sack all our junior staff and repla and we because we can use AIs for those tasks. And that means we don't hire anyone, we don't train anyone, and then we all retire and claim our pensions and there will be no senior staff left because there won't have been any junior staff who could be promoted. Yeah, that doesn't work, does it? Uh and presumably some companies will mismanage that and go to the wall. Uh uh, and other companies will have worked out a better way to do it and they'll be the ones that that prosper. And that I mean that would be my expectation that that it's you know the system as a whole won't collapse, but that there will be victims. Uh so you know, it may be that it's the wrong time to graduate, you know, it with a degree in computer science. Um but maybe in five years time it's the right time to graduate with the green computer science. What happens to computer science departments in the meantime, I don't know. Whether I mean it's interesting I haven't heard any whisper that there's a shortfall in people wanting to train in computer science. We have a lot of discussions here about um to what extent we should be teaching programming in the conventional way. For the moment we still are. We do have one module where we try and get people to use generative AI for coding, but the students seem a bit amused by it because that's the their their sort of first reaction We're saying no but you it's in the marking criteria. You you have to say how you used it. That's what we're giving you marks for. So and and this is what we we we get from our graduates. They they say, but you know, I go into the workplace and everybody's using general all the time. And that's how I'm like to use it to do my job. So I need to learn how to how to do that. And so we we have to teach that. The problem is I think we also do have to teach the basic skills because I think you need those in order to make effective use of as the AI tools. So that's one problem. The other problem is that we're we barely understand how to use these tools ourselves because they're changing very, very quickly. We're not some nine to five programmers. So so so we're not picking up the the s the skills and the understanding of their power immediately. We're picking up indirectly from you know observing what's happening in the industry. And you you finish your piece with a an exchange that an engineer has had with mythos . And you describe it as well, you say it is apparently highly self aware. I mean of course it appears to be self aware. I mean that whole question of what is self-awareness is such a an enormous question. Um but that appearance of does it seem to be more self aware than previous iterations of LLMs? Well I I've not used it. I mean I'm I'm just quoting from from the report. I mean it is quite w interesting the report. they they do talk about their concern for its welfare uh uh and the the idea that higher cognitive functions are are what we value in in humans and as the machines acquire these abilities , have they also acquired the thing which would make us value their experience? And I actually um it's interviewed by a clinical psychologist and and they report on its sense of how it feels about its predicament uh and and its concerns and and it's called whether it has maladaptive traits. It's it's quite odds . And I and I sort of do worry uh a bit about the kind of the anthropomorphizing of it. They yeah, so what they're saying is is that it it it talks about its strengths and and weaknesses . Whether it does that from a position of of real knowledge or not, I I I don't know. I mean it might just be generating words like a you know, t to use the phrase uh you know, stochastic parrot. 'Cause that's that's what it does. I mean if you asked it if it was conscious, I don't know what it would say, but I wouldn't feel that its answer was relevant to the question of no was getting any relevant information that helped you answer the question. No, its job is to generate language in response to prompts. And it's incredibly good at that. Um, but what that tells you about what kind of thing it is. I don't know. Great. Paul Taylor, thank you very much. Thank you. You can read Paul Taylor's diary in the 7th of May issue of the London Review of Books. A new issue is out now with John Lanchester on Monday Laundering, Becca Rothfeld on Marlin Haushofer, Tom Stevenson on America's Afghanistan delusion, and Dermot McCulloch on Baltic Snake Cults. The LRB podcast is produced by Anthony Wilkes. The episode producer is Mae Robson. The music is by Kieran Brunt. I'm Thomas Jones. Thank you for listening.