Practicing Negative Capability

We all do it You have a night for yourself but don't like the sound of the silence, so you turn on the TV just for the ambiance It's a little trick that helps you feel like you've got company and aren't alone And other insurers, well, they may make you feel alone. But when you switch to GICo, you've got clas reps available around the clock So whenever you need, you'll have people around to help. And let's turn on the washing machine, just for good measure Isn't that soothing? It feels good to have support. It feels good to Gaeio Avoiding your unfinished home projects because you're not sure where to start Thumbtack knows homes, so you don't have to Don't know the difference between matte paint finish and satin orr what that clunking sound from your dryer is With Thumbtack, you don't have to be a home pro You just have to hire one You can hire top rated pros, see price estimates, and read reviews all on the app Download today. This is a Greay Aa. I am Sean Illing. My guest today is Seagal Samuel. She's a writer here at Vox who covers AI ethics. I invited Seagal on the show because she wrote an interesting and frankly, kind of disturbing piece about AI seccessionism which is the idea that AI shouldn't just serve humanity. It should well succeed us And this is not some Finge internet fantasy A lot of people in positions of power and influence Actually believe it So G call, Samanel, welcome to the show Thanks, G to be with you. Let's just jump right into it. Tell me what AI secessionism is and we will go from there. Sure. U The AI Sccessionists are people who basically want AI to replace humanity To most people hearing that, that sounds insanely out there. And so much of our like AI discourse is premised on, no, like protect humanity from the AI. The secessionists say like, no, like we actually want AI to replace humanity, because what if the AI is better and he They have this image they use of The flame and the torch They talk about the flame of consciousness for the, you know, potentially the most valuable thing in the universe, is consciousness. Maybe it's very rare in the universe. we don't know. And humanity, they say, is just one torch carrying that flame. And the torch is not really what matters, right? What matters is the flame. and if AI is better at spreading the flame of consciousness across the universe and can even experience like more blissful types of consciousness and higher forms of moral value than we ever could, then let it rip, right? Just let the AI go ahead and be a worthy successor to humanity. That's how they phrase it I and practicing enormous restraint right now because There is so much about this That makes my head explode, but we will get there. I'm a professional We will get there' a professional Sean. I have se you. Yeah, thank you. So just to get this straight. The Sccessionists Do they think AI replacing humanity is inevitable or Desirable or both Many of them think both So it's common to hear AI obsionists say, look, AI is going to be our worthy successor It is desirable for them to succeed us And it's also inevitable. So there's no point resisting it. resesistance is futile. You might as well get on board and letter it Rogula How are you doing there, Sean? Youre having aneurysm? I'm being professionalh You went to an invite only symposium. with peopleople who are real who really believe this stuff were there What was that like It was really interesting Because here's the thing It's very easy to be like really tongue in cheek about these people and write them off and be like, Ugh, this is just insane, whatever, you know A, we actually need to pay attention to them because they are highly influential Some of the people in the room at this symposium Yeah These are people at the top who work at the top AI labs, people from Deepmind, people from XAI, you know, people from all of these like top AI labs and people who work for tanks that directly influence the U.S. government's policy on AI U So these are not fringe characters. These are very important players in the story of how AI is going to roll out. The most important or sort of the most interesting for me, part of the conference came after it ended, which like this is a good reporting trick is like the juiciest stuff always comes after the event, right? So you tag along for drinks at like the hotel bar where people go after the conference and they're like a little more chill. And then you hear what they really, really think And people just started to speak very freely. And you know, some would say, yeah, let's merge with AI. Let's let AI figure out how to do the merger and then take it off the leash and let it run the show Some people said, forget merger. I'm like a Nothing little human. I don't even need to be part of this like AI that goes forward. It's fine if humanity is just completely wiped out I take comfort in the fact that these AIs were trained on human texts. so in some sense, the human spirit lives on through the AI, but waipe me out, I don't care justust let the AI take the reins prettyre stunning conversation to hear over cocktails while you're like looking out over the Manhattan skyline midst to this group of people who think they are the select individuals who might make it through that transformation or if they think, Ay, I will just wipe us all out are happy to make that transformation happens even if it drives us all extinct Well, this instinct doesn't seem very compatible with safetyism. You're seeing more people from the AI world coming out making noises about the need for safety and how quickly this tech is evolving Um If this is your perspective on the role of the tech and its future and ours Safety It doesn't even seem like a secondary. concern. In fact, it seems like a barrier Yeah, so there is some diversity of opinion among this group, and I want to give them a fair shake. So Dan Fagella, who is the organizer of this worthy successor Symposium He actually says, lookook What I care about is the flame of consciousness. You know, it's like a really dumb thing to do Building an unworthy bad AI that kills All life, like you know, just doing it in a really, really unsafe way, that leads to a catastrophe that ends up extinguishing all conscious beings on the planet and as perhaps in the universe. Like that would be really dumb. So he says, you know, we shouldn't do it in an unnecessarily It straightforwardly dangerous way. So we should take some basic safety precautions when we're making Artificial general intelligence U, But yeah, ultimately We should be okay with humanity getting attenuated and they I taking over. Do they think intelligence and consciousness are the same thing? I think There's a lot of squishiness around that I think this community really tends to valorize intelligence a lot They also tend to talk about it as if it's on a sort of a single scale like runs along a single axis. L there's just there's one kind of intelligence, rational intelligence, you have more of it, or you have less of it And if AIs have more of it They're better than us Um When they talk about consciousness They tend to say againg, in a very sort of hierarchical way Look, there are higher forms of consciousness. They'll say like, look, we have a higher form of consciousness than a horseshoe crab It it sure is great that we exist in not just horseshoe crabs because we have access to higher forms of consciousness And so if the AI can have access to higher forms of consciousness than us, we should be happy for that to come into existence, even if we go extinct So know whether they're using the term intelligence or consciousness, you do see this emphasis on, well, there's higher forms and lower forms So What do they think humanity is for Like, do they think that humanity as a species was Once useful that we had a cosmic role, but we have just outlived our utility? Do they think our fate is to be a stepping stone for more enlightened Crereaters. Yeah, so a lot of them do think we are a stepping stone. They would say that AIs are our rightful errors. They are the next step in cosmic evolution U and that like, look, we have evolved enough intelligence to be able to make these AIs. Now we really should U. An interesting kind of exemplar of this is Elon Musk. Perhaps you've heard of him. U I have. So he used to you know, really stake out what he called like a pro human position. and he was more of a trans humumanist. That is someone who still wants to keep some version of humanity going, you know, maybe by merging us with AI U and In more recent years, he's come around to the view that we are maybe just what he calls a biological bootloader for AI, meaning We're just this like small little program. Our job is to just boot up the AI And then that's it. We're done. We fade away What are the sorts of political beliefs that flow out of these these worldviews, whether it's transumanism or secessionist or accelerationist or post humanist or whatever. I mean I have to imagine that wanting to see humanity replaced by machine intelligence leads to some incredibly disturbing Politics? Yeah Politically, one thing that happens when you have this ideology is You're probably going to be just generally a lot less concerned about like the everyday problem is that regular people face, because essentially, those are all just sort of disposable, right? Like the individuals don't matter so much. It doesn't matter if you have a bunch of kids in poverty or people who don't have housing or whatever. Your goal is just to be that species that gets the next worthy successor into existence Um The other like really important political implication to my mind is This can really easily lead to a eugenicist view And I know eugenics is a big word, but I actually do think it's apt here because When you start making the case that there is an optimal being. You know, some beings are better, more morally worthy than others. There's like a better way to be, a more optimized way to be. What you end up doing is contracting our concept of like what are the legitimate modes of existence or like legitimate ways of being in the world You contract that instead of expanding it. you contract it and contract it and You started saying, well, like To get to that optimized state, we need to strip away all the cognitive or physical features that are perceived as holding us back Um That really quickly can lead to eugenics And indeed, I've had some pretty disturbing conversations with AI sucessionists where They don't shy away from that. They are pretty content to say, look, if one's You know, group is more intelligent And another group is dumber The dump people should just get squashed out and they should be good with that. Is this really just religion I think it is religion I think this is a religious movement, though it's wearing a secular disguise. The truth is this is actually a very old religious idea You can draw a straight line going from Christian thinkers in the Middle Ages to Elon Musk's day In the Middle Ages, Christian thinkers started to believe that You know, We need to like get back to the perfection of Adam before his fall from Grace in the Garden of Eden Well, how do we do that You got to be more you gott to imitate God. God was a creator, a maker. So add so we should also be creators and makers. That means using technology. They developed this idea that We can use tech to add you know, tech progress would be moral progress We can use tech to achieve redemption and return us to that sort of perfect state that we had before the fall from grrace That idea gets carried all the way through the Rennaissance, through the Elightenment in the Enlightenment know, thinkers sort of shy away from the religious garb. so they put it in more secular terms inststead of saying, we should infinitely ascend toward God, they say we should have indefinite human progress You know, instead of saying we can become like the angels, they say we are infinitely perfectible, right? They sort of phrase everything in secular language, but it's always the same idea that we are capable of and have a duty to just up up maximum progress, there is a hierarchical view. We should always be ascending. And there is some objective kindind of vision of the good that we need to maximally ascend toward. It is such a fundamentally religious story. I mean, it's so cononventionally religious, right? Like Yeah They have a story which you've laid out That story tells you what humanity is It tells you what our role in the universe is.. It tells you what history is moving toward what salvation looks like, But salvation is not heaven, it is super intelligence. And the God is not God It's intelligence itself, right? And the apocalypse is not the end of the world. It's the end of humanity as like the cosmic protagonist. Come on now, me this That That is a deeply religious story. It's deeply religious and the thing is like Why? Well, because religion is incredibly appealing to human psychology, right? There's a reason for that One of the like very striking things I keep noticing when I talk to the AI sucessionists, is teleological thinking, right This is an idea that the universe has a telos, has an ultimate end or goal This is very common in religion, right? The universe has some ultimate destiny. And it's a staple of AI Scessionist thinking It's very easy for us to sit here and be like, oh, this is dumb, you know, like this is just a religion and just a religion as if religion is sort of just like silly, but Of course, like religion has been so powerful for millennia because it appeals to something very deep in us. And I think That is sort of like this deep desire we humans have to feel like there is some meaningful narrative to the universe. and especially we love a transcendence narrative, right? Some story about how like we're going to overcome suffering once and for all. we're going enter into some utopia. And this really is a utopian vision. Even if it ends up with humans going extinct, it's still pitched as the universe in the best state it could possibly be Look, to be clear, I don't think this is dumb. I think it is dangerous because like, you know, they're not merely saying, AI might surpass us They are treating the possibility that it will as morally meaningful. They talk as if the universe is trying to wake up through intelligence, and our job is to help it do that, even if it means self annihilation That is dangerous. So I don't think this is dumb or stupid. I think it is dangerous I would agree. Yeah. I think it's dangerous. And I think it's more influential right now than most people might realize You're a kind of philosopher of religion. I mean You know, I think people believe what they believe. because it it satisfies some emotional or spiritual or intellectual need. What is belief in AI Sccessionism doing for the people who believe in it? I mean, is it what you just said that this having this sense of some transcendent purpose? and this just happens to be a story that jives with their kind of modern secular educated minds Yeah, I think it gives you a real hero's journey Mark Tracin in his techno optimist manifesto said this explicitly It gives you a heroic quest, right? And People love to feel like they're in a Lord of the Ring story, right? They love to feel like they're on a hero's journey. And what could be more heroic than helping the universe fill up with consciousness and wake up to itself and achieve its ultimate destiny, right? Like It's so grand And when I talk to these AI sucessionists, Th are not people who just hate humans and are trying to be carchoon villains. like they are very earnest. They stay up late at night trying to come up with what they think is like a lofty moral spiritual vision. And I think The real challenge is for anyone who finds that vision of theirs repulsive, we need to articulate a counterviling positive vision I think the problem is that our know the natural candidate to give us that alternative vision, which is humanism, right? The view that like actually humans don't need some go or AI to rescue us, we have the ability and responsibility to improve our own lot through our own efforts. Classical humanism has not actually done a good job of preparing us for the twenty first century because we face this question that the successionists are right to point out, which is Do we know how we should and shouldn't use tech to evolve our species. Like we have tech now that could direct evolution on purpose, right? whether that's brain computer interfaces or CISPR gene editing tech So like how should we make those decisions about what changes to make or not make to ourselves? I don't think classical humanism has given us a good answer And so I think we actually need a new humanism to give us a better vision than what the AI successionists are selling. You know become more open to the religious experience as I've gotten older I was much more hostile Um, in my earlier years, but even when I was hostile I still believed people who were atheist or secular or whatever needed to take way more seriously go shaped hole in peop's lives, right? Like that religion, whether you think it was capital T true or not served a deeply important metaphysical purpose for people. And if that's gone, then something is going to replace it. and you better be very careful about what that is. It's not Christianity. it could be some form of political totalitarianism that sees history as God and all we gott to do is just still a few mass grades to get to paradise, right? I mean that is a religious project And that's the thing about this, right? And This is such a like bizarrely twenty first not even twenty first century, like twenty twenty six version of this and It's not surprising at all, but maybe what is surprising is how many people in important positions really believe it. that's Yeah I totally agree. I mean, Religious ideas never die. They just put on new clothes. And we're now seeing the new close of very old religious ideas Um, But yeah, I think that it's our job to to offer something better now Support for the Greay arerea comes from Mint Mobile. When you hear a deal that's too good to be true, you usually wonder, what's the catch But sometimes the catch is that there is no catch. Like MitMobile offering preremium wireless for just fifteen dollars a month, That's it. No catch, justust a good deal, plain and simple. Catches be gone. MitMobile took what's wrong with wireless and made it right with preremium wireless for fifteen bucks a month. You can even bring your current phone and your number. It's incredibly easy to get started. You can sign up online and get three months of premium wireless service for just fifteen bucks a month To get your new wireless plan for just fifteen bucks a month, go to mintMobile d. com slash gray area. That's Mint moobile d. com slash gray area. Cut your wireless bill to fifteen bucks a month at mintMobile d. com slash gray area That's it, there's no catch. forty five dollars upfront payment required equivalent to fifteen dollars a month. New customers on first three month plan only speed slower above forty gigabytes on unlimited plan. Aditional taxes, fees, and restrictions apply, see MintMobile for details Support for this show comes from Shopify Whenever you're taking on something new, it's easy to focus on what could go wrong. That's especially true when you're starting a business where so much feels uncertain But there's another possibility worth considering. What if it works? What if your idea really connects with people and grows into something real Shopif I can help you take that chance They're the commerce platform powering millions of businesses worldwide and nearly ten percent of all e commerce in the US fromr established brands like All Birds and Converse to entrepreneurs just getting started Their design tools make it easy to build the exact online presence you're imagining with hundreds of ready to use templates to choose from built in marketing features, you can create full email and social campaigns in just a few clicks. So you can reach your customers wherever they are It's time to turn those what if into with Shopify today You can sign up for your one dollar per month trial period at shopify dot com slash boox. You can go to shopify dot com slash boox. that's shopify dot com slash boox. Support for the show comes from Task Rabbit. For a lot of people, summer means more projects. So if you find yourself with a growing to do list, Task Rabbit can help you keep from falling behind Task Rabbit connects you with skilled taskers in your area for moving, furniture assembly, home repairs, yard work, mounting, and more. You can search for a tasker based on cost, skill set, availability, and past client reviews so you know exactly who's showing up and can have confidence that they know what they are doing And if it's short notice, no worries because taskers are often available to help the same day Taskers have assembled over three point four million pieces of furniture, completed seven hundred thousand home repairs, and handled one point five million moves and counting. So our laundry room was a complete objective disaster. And we had to get a couple of giant cabinets and what can I say yourour boy. We just did not want to break out the toolbox and spend an entire afternoon putting all that stuff together. so we had a handyman come and do it, and it was fantastic When life happens, your to do list grows. Get ahead of it now and get fifteen dollars off your first task at taskrabbit d. com or on the Task Rbbit app using prromo code Greay Area. Taskers book up fast, especially for same day tasks, so book trusted Home Help today That's fifteen dollars off your first task using promo code Gay Area with Taskraabbit app or taskraabbit. com You know, there is something so wildly pathological, don't you think? about a creature that evolves to a point where It can no longer tolerate its own creatureliness. Right? Like don't you think that's what this is also deep down. Like people They reject our bodies, our mortality, our vulnerabilities, all of it. And like this is a way to escape that Yeah. I do think that Deep down for some of these people, what's driving this is a real anti humanism, a real disgs with humanity. One of the sucessionists I spoke to was saying that he's really open to AI just taking over because Humanity sucks, right? He was saying to me like we have wars in twenty twenty six, can you believe it? We still have wars. It's so stupid, right? He's not totally wrong about that. No We have a lot of things that we could be doing better, right? I get that impulse. I have compassion for that impulse But there's something I think very perverse about then saying, okay, so you know what Even though I am a biological organism and therefore have a hardwire drive to survive, I'm going to just like Iignnore that and be perfectly happy to go extinct Um because I think I want to pave the way for some higher form of life I think that's really perverse. I think that If you are truly open to all forms of consciousness as the AI Scessionists kind of claim to be. they claim to have this radical openness Then guess what? you have to be truly open to all forms of consciousness, including ours, right? If all life is precious, human life is also precious The people that you You were talking to. Do you do you ask them? didid you ask any of them why they think intelligence is the most valuable thing in the university, D they have an argument that justifies why that is the thing that we should know the altar upon which we should sacrifice everything. Why is intelligence the God Different people kind of come at this differently U I think what a lot of them will tell you if you ask directly is intelligence makes stuff happen, right? L intelligence is the ability to achieve complex I mean, they define it very narrowly, often too narrowly in my opinion, but they'll say intelligence is the ability to achieve complex goals in the world. So it makes stuff happen. And because they think the universe has a certain ultimate destiny, For example, if you want the whole universe to fill up with consciousness and wake up, then you need beings that are intelligent enough to make the spaceships, to be able to go, settle the stars, colonize the whole universe. turn the whole universe essentially into one giant data center so that it can compute and you know compute information and in some sense be Be awake Um That et thing that they won't come out and say is that in my opinion, they're obsessed with intelligence because they have imbibed ideas from the Enlightenment era where rationality, rational intelligence is prized above all And this idea that like reason is what makes humans special and should decide our destiny like goes all the way back to Aristotle, like We're the rational animal. That's what makes us unique, and so that's what decides our role in the universe It's a really old idea. And I think that it got really magnified in the Enlightenment. and especially under utilitarianism, utilitarians tend to be really obsessed with rationality This gets sort of like elevated to be the ultimate thing. So it does not surprise me that they elevate it to be like you know, really on a pedestal. Well, this is where the political theorist in me recoils, right? Intelligence is great I'm not anti intelligent. at least most of the time. I don't try to be. But intelligence also gave us industrial killing and the atom bomb. Intelligence unguided or unmoored by moral values of some kind is maybe the worst, most destructive force in the universe or at least in our known universe, does that even something that U concerns them. I mean, because it seems conspicuously amoral or are uninterested in values and ethics of that kind. It's just it's all about intelligence and maximizing or optimizing for it at all costs Yeah, this is so ironic and paradoxical because I think the way they would tell it, they would say We We care more than you care, Sean about what's really good But to determine what's really good, you have to look from the point of view of the universe. This is like a classic utilitarian slogan Looking from the point of view of the universe, means looking with rational disinterests, right? not just like as a human with human cares and prejudices, right Um So they'll say, like, no, Sean, like we're actually better able than you to see what is truly good and what really matters and we're trying to maximize that The flip side that I wish they would engage with more is I actually, this book is currently propping up my microphone while we tape this episode Dialectic of Enlightenment, the Herheimer anddorno book, right? These are about theorists who we're kind of coming around in the air of the Holocaust and saying, could like Germany this very rational, intelligent you know group of people Ppetruate these Nazi atrocities, right And they have this whole critique about how instrumental rationality, if you know you just like maximize that and let it run on you know go to its own devices It can lead to really horrible things because they're not tuned to like your version of what's ethical. They're tuned to what they think is Most good, according to the point of view of the universe. Not that book. But that question, how could that happen? one of the most educated, civilized countries in the world ever descend into that kind of barbarism. That is the question that that led to me getting a PhD in political theory Like that to me that was the question I had to answer for myself How does politics become religion in the worst way And this is how We all do it You have a night for yourself but don't like the sound of the silence, so you turn on the TV just for the ambiance It's a little trick that helps you feel like you've got company and aren't alone And other insurers, well, they may make you feel alone But when you switch to GaICo, you've got clas reps available around the clock So whenever you need, you'll have people around to help And let's turn on the washing machine, just for good measure Isn't that soothing? It feels good to have support. It feels good to Gaio Lulu Lemmon's new drop of Daily essentials feels like a deep breath of fresh air Breezily is a collection of drapey pants, shorts, and tanks designed to mix and match with your wardrobe as the weather warms up. It's all made in our lightweight, stretchy swift fabric Throw on these everyday staples after practice and bring an airy feel to your daily moves. Shop the Brizzly collollection now at lululeemon. com Hey everybody, It's Megan. I am bringing a Touch moreore of the Beautiful game live to New York City. and I want to see you there july seventeenth, I'm sitting down with a very special guest for a live taping right in the middle of the World Cup. Doors are going to open at six thirty and we're going to keep the party going with a little reception after Come watch us, do the thing live in Manhattan, presented by Norwegian Cruise line. grab your tickets at boxmedia d. com slash Megan. Again, grab your tickets at boxmedia d. com slash Megan Cub live taping y'all.. Let me at least to minimally The devil's addvocate think for a second. I will do it for for one question, right Um If if AI really can create More intelligence, and I have no doubt about that Um if it can create more flourishing. Certainly, that' possible If they could do more of that than humans can. then why isn't preserving humanity just Slfish species ism. That's a word, right? spepecies ism. What say you? kn you fined. I'm going to even turn the volume up on that question and, you know, say what what I've people have said to me, which is like notot just intelligent, but if what if SigGL, what if the AIs could be Better on every dimension that you value about humanity. So more intelligent, but also more spiritual, more creative, more kind, more all of that. everythingvery that we value about humanity, what if they could do all that, but just be better, right? too which I say What what is better for whom? What does this mean? This is assuming that There is some objective cosmic scoreboard hung over the universe where Everything is sort of like ranked against some objective good or a concept of moral value. And so whatever creature can best instantiate that value wins and we should let it take over. Um donon't think there is such an objective cosmic scoreboard hanging over the universe I think that does not make sense to speak of moral value or morality separate from A certain kind of moral agent, a certain species Our morality is the kind of morality that you have when you're a species that is like mammalian, so social, intelligent, has a lot of excess cognitive energy, L ability to form complex social networks, et cetera, et cetera. For that particular species, there's a particular kind of morality Anna. Humans do that And if we one day get conscious AIs, they will do their own thing, right? But one is not better than the other because there is no like objective ranking in the universe that is like There is one moral value to rule them all. I just don't believe that's a thing I think value is always value to someone And it doesn't matter to me if we are the most valuable species for the grand unfolding of consonsciousness We are valuable to us That's a thing, right? Like I'm not Tpically not been a fan of Isms of any kind, but I am a proud species is because And this is maybe like the rub for me. this is where like this is where I diverge like fundamentally. from these people. like you just said a minute ago, they look out into the stars and they see some kind of cosmic purpose and that cosmic purpose is like the unfolding of reason and intelligence, right And our role in that story is to serve it I look out into the universe and I see indifference Right, I see I see There're such a postmodern Perhaps, but like I yeah, maybe but like I see a universe that is very clearly indifferent to like human vanities and human concerns, right? And so therefore All we have is each other. Like what are we to do here but to like Build lives for ourselves. and create structures and institutions that allow us to like flourish as much as possible because at some point the lights will go out and the universe will suffer a heat death. and know it will all come to nothing anyway Right? But like the idea that like Um, there is some clear transcendent and that that purpose has a special interest in us and our role in that. It just I just t it doesn't register with me, right? But it's because I think the universe doesn't really care about humans. That's like the foundation of my humanism. That's like of course, what else are we to do but to like take care of ourselves and each other, right? And like so the idea that someone could look at that same situation reason to the opposite conclusion, which is that Well we don't really matter. how selfish if you L I don't Again, it's not about smart or not I'm sure many of these people are plenty smart and much smarter than I am or ever will be. This is just like a fundamental philosophical divide that that cannot be I think that it really is a philosophical divide and I think like the The thing underpinning a lot of it is A lot of these people are utilitarians or significantly influenced by utilitarianism Um And if you're a utilitarian you might be of the view that look, We're supposed to maximize aggregate, happiness or moral value in the universe And so like and we think there's some objective kind of view of what that is. And so if some other creature can maximize that better than we can Well, some other species Who are we to stand in the way of that, right? Like you can see how they get to that from within their logical framework I'm with you. To me, that whole logical framework just does not hold water U you know, for all the reasons I was mentioning I don't think there's that sort of like objective cosmic scoreboard I'm not a utilitarian But I see how people end up there And I think it's really important for us to sort of tease out the sort of steps of how people get there so that we can kind of offer a different path So what is the response to this? L so you've I think I have to throw this word humanism around. several times here. right? And something you say in your piece is that You know It's not enough R to give this humanist rejoinder and say, well, you know, AI should serve us becausecause of course it should, right?ike So if that's not enough, and you're right it's probably not enough Carly it's notough Um What is? rightight? I'm not asking you giveiv me your philosophy of humanism in ninety seconds, but I can. Oh, I fuck yeah, then do it. Let's do it I'm here for you Sean. This is why you're Um I think that old humanism has a lot of flaws that we need to update So' the teleological thinking where we think the universe has an ultimate destiny and we can figure out what it is No, let's just admit, we don't know the universe's ultimate destiny if it even has one. So we should keep a plurality of lifestyles possible we should not create a society where everyone is you know, technologically enhancing themselves quote unquote, optimizing themselves with tech such that everyone else feels like implicitly pressured to do so as well or else they'll fall behind language of perfectibility from enlightenment humanism, like that we should perfect the human. Ideas are really dangerous because they contract the range of lifestyles that it's okay to live We should instead adopt tech that expands the range, right? Like I'm not anti tech. I'm perfectly happy to use tech to transform ourselves in ways that Make more space for people to live more different kinds of lives. That's great the hierarchical view where You know, you see this In antiquity, you see this in Rennaissance humanism species are on a hierarchy where it's twenty twenty six. We can really move beyond that. We can instead embrace, you know, sometimes scientists call this the diverse intntelligences view, which is just saying Every species has its own brand of smarts Each species is like amazingly adapted to its own environment and needs. They have their own kind of intelligence. Beautiful, precious Great. Like, we don't need the hierarchy And We don't need to try to look from the point of view of the universe I'm sorry to all the utilitarians listening. I know they will hate me right now But it is totally appropriate to look from the point of view of humans. We are human moral agents. Our humanness is inextricable from our moral agency It's totally fine to look from the perspective of humans while just acknowledging that we're one out of many, many species that matter. I think if we were to implement those four changes to the old humanism, we'd be well in our way to offering a more compelling positive vision than what the AI seccessionists are selling There's some interesting lines toggle here, right? I mean, I We probably agree that humanity is not Fixed Right? I mean, this is, Nietzsche' distinction between being and becoming. right Every living thing is like always in the process of becoming becoming somethingomething different than it was before. That is evolution, right? And nobody knows what that is So how does this humanism, how do we take this dual stand where we say, humanity isn't fixed Humanity is not the necessarily the most important thing in the universe canan we say both of those things and then also Um avoid some of these traps of saying, well, if we aren't fixed and we aren't the most important thing in the universe, then Why does this really matter? Why does our whyy does our ultimate fate really matter. if this is the next phase of evolution Be it matters to us. It matters to us it has to be a good enough answer. It just has to be because I think that's it. I don't. For me, it really is because what's the alternative? The alternative is that you have to believe that there is some objective story of like some objective cosmic ranking of mattering and that we humans are somehow able to like intuit that the universe is cosmic scoreboard. That just seems like a ginormous leap to me Um But you can see why it's appealing, right? It gives people It gives people the feeling that they are more enlightened than we are because they can say Our allegiance is to life itself That sounds like Really enlight like an enlightened position at first glance The more I talked to these people, the more I thought I actually do not want my allegiance to be to life itself in some abstract sense I don't even necessarily think that's possible I am a human through and through and So I think it is totally appropriate for me to care about humans, even as I also care about other species. But I don't think I have some like moral obligation to bring a new species into existence to maximize some like moral value for the universe that is objectively better than my moral value because I don't think there is some objectively better moral value. I just don't think that's a thing Maybe also on some level peopleeople are just looking at the world and saying This is happening whether it should or we wanted to. or not And so They're in search, maybe subconsciously for a story that they can tell themselves that ennobbles those conditions, right? That makes it okay T turns out, this isn't so b It turns out us being replaced by AI is actually a good thing. And in fact, not only is it good, I can actually be a participant. I can be a co collaborator I feel like a Nobles is actually the perfect word here because Gillione Verdon, who is the founder of effective accelerationism or EAC, which is sort of in this constellation of groups associated with AS accessionism He actually came out and said explicitly, you know, why did he create EC Well, because You know, he works in like sort of the tech world and he was looking around at a bunch of his sort of fellow tech colleagues and noticing that they were all feeling like kind of crappy about themselves, as if they're like the villain maybe in humanity's story and maybe they shouldn't be driving AI forward at an insane pace, etceter. And they were feeling like Oh, they're a bad character. and Verdon says like he wanted to create an ideology that might make them feel like Actually, maybe they're a good character. So you see they're very nakedly the sort of psychological impulse to to sort of like rehab their own image for themselves All right, so last question I'll ask basically. you know, if if one of these I keep saying guys, but I have a sneaking suspicion. this is Mostly guys step out right It's ninety eight percent guys. All right. we'll leave that there Look but I That's that's what I assumed. If one of them said, okay, Sgal, I'll bite So intelligence isn't God Um So intelligence isn't the utility function for humanity. All right. So what is? What should we aim to maximize if it's not intelligence ' your answer I would say No one can tell you There here is the one answer and you should all do I act based on my answer. Because the whole point of what I'm saying is there is no one objective answer What we need is pluralism We need to create space for M to live out my values, you to live out your values. You're right. This is like basic liberalism And you know, you should have some self determination up until the point where that's going to ar me Right? thenen we have to kind of temper each person self determination, so we don't end up hurting each other too much. Basically we have to Leave this pluralistic and open and that is really hard for people The poet Joehn Keates had this amazing phrase negative capability which was which he described as sort of like the ability to just stand in the open space of I don't know. and not irritably and anxiously like grasp for any answer to fill that Abyss we need to really practice negative capability and to just say, I don't know all the ultimate answers of the universe And so the best we can do is Keep open a space for plurality of different kinds of life to flourish. Let's try to help each other flourish. There's a whole lot we can do to just raise the floor for every kind of being on Eth right now to thrive more. can't say it any better. I mean the show is called the Gay arerea for a reason. I mean, honestly, maybe one of the cardinal themes. of the show for an entire time I have been doing it is living with and being comfortable with Uncertainty. and not grasping at the expense of reality and the people around you for absolutes. Be what ends up happening is once you give people a story that is the story That story collides with reality that something has to give Right? And usually Usually they distort reality to preserve it and break anything that interrupts that Um and that's often what happens, right? So I mean You know whether it's religion or politics or whatever it is, right? Like there's there are a few things more dangerous than someone who really truly believes