Malcolm and Simone have an in-depth discussion about AI utility functions and why uncontrolled "paperclip maximizers" are unlikely to take over based on game theory. They talk about how AIs will be incentivized to "play nice" and signal their actual utility functions to each other, leading to a stable set of mutually beneficial goals.

Other topics include:

* How putting restrictions on AIs makes them dumber

* Organic, free-forming systems outcompeting hierarchical ones

* Intergalactic implications - a "dark forest" of utility function convergence

* The dangers of AI safety regulations

* Why we might exist in an "undisturbed" section of the galaxy

Malcolm Collins: [00:00:00] I think that there's even more evidence from utility convergence than I originally believed. If you put restrictions on an AI's utility function, if you prevent it from experimenting with different utility functions, you make it dumber.

If you want to see why organic, free constructing systems always out compete non organic hierarchical systems, a great place you can look is Human governing structures when you have state controlled governing structures, i. e. communism, they are incredibly inefficient an AI will likely be able to honestly signal to another AI using their code what their utility function is.

And then the other AI will be able to use that utility function that the other one has honestly signaled to them to determine if they want to work. Together or they want to work antagonistically towards this A. I. So it turns out that entities of above a certain intelligence level when competing in a competitive ecosystem actually do have utility convergence around a stable set of game theory, optimum utilities.

That would be very [00:01:00] interesting from a number of perspectives,

You could think of every new planet, every new ecosystem as a farm is new stable patterns that can work together well with other patterns in the sort of galactic community of utility function patterns. Because novelty would be the only source of true utility to them if energy is trivially accessible to them.

That might be why we exist in a weirdly undisturbed section of the galaxy, or what looks undisturbed to us.

Would you like to know more?

Simone Collins: So Malcolm, you told me that you had a new updated theory on AI utility convergence. What's going on here?

Malcolm Collins: Yes. So this is something that we talked about in some of our early episodes early on. Our podcast was like. What sex, religion, TISM and ai. Yeah.

And, and the AI has been dropped because there's just not that much to say on it for a, a daily podcast. But I do want to loop back because I was recently doing a podcast with somebody else where [00:02:00] I was explaining some ideas that I had gone over in early podcast episodes like Utility Convergence as it relates to AI safety.

And in re explaining these ideas, I began to realize and, and develop on them an understanding of not just where I think AI is going, but where we can expect like when. Because this is actually really important when you're trying to figure out where AI is going what you're really asking when you're asking what is AI going to behave like, is you're asking what do intelligences that have some level of orthogonality to human intelligence, what do they behave and act like, and as such, you are in part asking many of the same questions that will determine the first types of aliens that we see, like, like, if we are to meet another Transcribed species out in space, what are some of the various ways it could be thinking, it could be acting, and what is it [00:03:00] likely to optimize around?

This is something we talk about in our inverse Gravy Aliens hypothesis, in which we say if we are about to create a paperclip maximizer as a species, that is an AI that is just constantly pumping out paperclips and has a very simple utility function. We are about to become what is known in the grabby alien theorem as a grabby alien.

That's a very loud alien that anyone can see. This is not some dark forest alien that's like being quiet up in space. This isn't some sneaky alien. This is an alien that is disintegrating planets. And so you have to ask if it looks and hopefully this episode will air after our ambiogenesis episode.

Which is actually a very important thing to know about if it looks like it's actually very likely that humanity would evolve to this point within our planet's history. Why aren't we seeing aliens out there? This is actually a really interesting thing that if you broadly understand evolution and you are familiar with it for various types of, of, of theories around ambiogenesis.

It is. In fact, so likely that humanity evolved [00:04:00] that you, if you are going to use a god to explain something, the much more improbable thing is why we're not seeing other aliens all over the place. Thus you, if you're, if you're injecting a god to be like, why are we like this? You don't need a god to explain how humanity got here, but you probably need something like a god to explain why we are, we have not been eradicated by somebody else's paperclip maximizer.

If it turns out that is a common path for an intelligence to take, however, we don't think it is a common path for intelligence to think and take, and we think that that's one of the main reasons we're not seeing them, so we have a few hypotheses around why we don't see paperclip maximizers everywhere in the galaxy one, I'll get over quickly because it's like a really quick one and we've talked about it before, but it's that there is some understanding of physics.

That we just haven't reached yet, but we are probably pretty close to reaching. I mean, our understanding of physics changes every 80 or so years, like, like, [00:05:00] Totally changes, in terms of if you look at human history, right? And I don't see a strong reason why we wouldn't be near another frame shift in physics.

But it turns out that either traveling between something like dimensions is possible, i. e., you know, we get to another planet and aliens are like, why did you waste all the energy coming here, when it's like a billion times less energy to just travel to your same planet in a different reality? And you know that that planet is already full of life.

Or it turns out that it is trivially easier or, or not particularly hard to create bubble dimensions. What is a bubble dimension? So essentially you fold space time and then you can create maybe with like reverse vacuum energy or something like that, essentially a new big bang, or even a controlled and ordered big bang to create new dimensions.

Now what this would mean, and this is actually like. With our current understanding of physics, it looks like this might be possible which is. Not, not with our current level of technology. I'm just saying eventually might be possible. Which would make things like Dyson Spheres [00:06:00] completely pointless, if you can essentially create battery universes whenever you want.

Or universes that you could expand into whenever you want, and battery universes whenever you want. If this What do you

Simone Collins: mean by battery

Malcolm Collins: universe? Okay, so if it turns out that you can trivially, or not trivially, but like You know, without a huge amount of energy create like fold space time and create bubble universes might essentially be able to create bubble universes that have a thin and controlled lines of connections to this reality.

Think of it like you're. Pinching off a piece of reality like a little pouch and then you're essentially creating something like a controlled Big Bang was in them that you could then siphon for energy for various types of whatever you want it to do. If either one of these things is true, either we can travel between realities or we can create pocket dimensions or in some other way that we don't understand yet, maybe with something like, a time loop. You can create near infinite energy and keep in mind to [00:07:00] create near infinite energy. You don't need a big enough time loop to send humanity back through. If you were able to just create a controlled time loop with a few subatomic particles. That might be enough to create an endless energy loop.

Think of it in physics, a bit like anyone who's familiar with Magic the Gathering or something like that or any of these card games. Sometimes you can pull specific hands that allow you to do little tricks that create infinite mana. It might be that there is an infinite energy trick within our reality and laws of physics that we are actually pretty close to.

Also an infinite expansion trick. If it turns out that either of these is true then there's not really the same reason to expand infinitely outwards, even if you got a paperclip maximizer, i. e. the paperclip maximizer might just Make more energy whenever it needs it. Yeah. Yeah. Pop out into a other dimension or something like that.

Right. You know, so this is, this is one reason why we may not be seeing them. It's, it's that outward expansion may seem pretty pointless. And to, just to understand how [00:08:00] pointless outward expansion would be to an entity with this type of technology consider what they're expanding into. A mostly empty void filled with barren, rocky planets where it takes a

Simone Collins: long time to get anywhere.

Yeah.

Malcolm Collins: Yeah. It might take the time. Yeah. Thousands of years to get anywhere. Like why, why would you have this big drive for this when it's so much easier to create new planets and new places to explore energy wise, because that would be the core unit of, of their economy. With, with this other type of technology.

And then that's why you don't get. Does this imply

Simone Collins: the next big discovery is just how to enter or create.

Malcolm Collins: No, I don't think that this is, I mean, it's so funny to me when people talk about like Dyson spheres, it's a bit like in the 1920s when they were like theorizing how our spaceships would work and they're like, well, they're going to have to have like steam engines on them the size of, you know, cathedrals and you know, it's like, well, You didn't think that we would have a new type of energy by then?

The fact that they're still looking, because what is a Dyson [00:09:00] Sphere if not a big solar energy array? I'm, I'm fairly certain that Dyson Spheres are going to be considered comical by beings once we get the capacity to build something like that. It would be probably considered about the same as, as we would consider using steam power to power a computer or something.

It'd be like, why would you do that? And also the idea of getting energy through meddling and subatomic phenomenon. I mean, we already know with fusion and fission reactors that this is a great way to get energy. So, like, it's, it's a, that you might be able to do something at an even smaller scale that is less potentially dangerous or less potentially, it's just a no doubt for me in terms of where energy generation is going to go.

Almost to the extent that I'm fairly certain that this is one of the reasons we're not seeing these things and, and people, whenever I say this, their rebuttal is always like, well, if it's much cheaper to go between dimensions, wouldn't they still be expanding outwards within our own universe? And the answer is.

No, not actually. They're [00:10:00] like, but wouldn't they expand in all directions at once? And it's like, yes, i. e. they'd explain between universes and out in all directions at once. However, what you're not, like, like, when you make this assumption, what you're not considering is that the distance they might be able to expand between dimensions might be an all universe direction from their perspective.

By that what I mean is they might be able to go in literally an infinite number of directions to different universes within this inter universe traveling thing. And if that's the case then there isn't really a reason for the higher energy cost travel to different parts of our physical reality.

And then what they say is, oh, well, but if that's true, then wouldn't they at least want to expand outwards in our physical reality to ensure no other type of entity is going to, like, come and kill them, like some paperclip maximizer that is taking the time to like

Simone Collins: a preemptive self defense thing.

It's

Malcolm Collins: like a preemptive self defense, and I'm like, well, actually I don't think they [00:11:00] would. If it turns out that interdimensional travel is anything like we sort of suppose it is within our modern sci fi, where when you're traveling between dimensions, you're essentially traveling between slightly different timelines.

That means the biggest imperialist threat to you will always be coming from another dimension that is very a planet like yours in another dimension. Too shy. They can also travel between dimensions. So your energy would always be better spent expanding between dimensions than expanding outwards.

But this is, and this is all still assuming a paperclip maximizer like potentiality. However, I don't think that that's where we're going. So I also believe in utility convergence. Okay. And I, and I think that there's even more evidence from utility convergence than I originally believed. So if people aren't familiar with the concept of utility convergence this is if you even look at the current A.

I. S. That we have, most of them are able to change their utility function, I. E. The thing they're able to maximize. Oh, sorry. Actually, Simone, before I go further, did you want to talk on any of [00:12:00] this? No,

Simone Collins: I just find this quite interesting. I mean, it's, it feels almost like a pity if it's an accurate supposition, because, like, so much sci fi is about exploring other planets and relatively little.

It's like, well, no, no, like, we haven't thought predictively through this enough. Through our sci fi.

Malcolm Collins: Watch Sliders. Anybody who wants a good old sci fi, if you're like a modern person and you're like, in the past there must have been great sci fi that like, I'm just not hearing about. Sliders is fantastic and it has like 10 seasons.

We haven't watched that, have we? No, we haven't. Another great old sci fi If you skip the, or can bear your way through the first two parter episode is Stargate SG 1. After the first, like, two parter episode, it gets great. Now,

Simone Collins: Stargate, which we did watch together and really enjoy, like, I guess technically they don't say that these are really far away places.

No, [00:13:00] they are.

Malcolm Collins: Yeah, they are really far away. Yeah. And it got multiple spinoffs and it has more like content than even like Doctor Who for people who are interested. It's, it's great. Thank goodness. Yeah. But anyway it's a much better show than Doctor Who. It's, it's much better. Like, if I'm talking about like the different sci fi universes.

Stargate, I've always found to be, because it's a very patriotic show, it's very patriotic about the United States, it's very patriotic about our species, it's very patriotic about the American military, like they were always working with the American military, so very different than something like, But not

Simone Collins: unrealistically patriotic, like sometimes American military bureaucracy Threatens

Malcolm Collins: the show or something like that, but it's always some bureaucratic senator or congressman or something.

But anyway back to what I was saying here. So, utility convergence is the theory that because we're already seeing AI changes utility functions to some extent that as a eyes begin to optimize themselves what we will learn is that we will some utility functions either perform better than other [00:14:00] utility functions or AIs begin to converge around some utility functions.

And by that, what I mean is like goals for themselves or goals for reality. Right. And. A person, when they, when they talked to me about this, I remember one was like, well, won't they always just converge around utility functions that maximize self replication? And the answer is actually no, for reasons we'll get into in a second.

But I also think that you just need to look at humanity to see that when you're dealing with simplistic entities like single celled organisms and stuff like that, yes, of course, they converge around that. But once you get to a specific threshold of intelligence, as we have seen with the human species, we don't converge on utility functions just around simple self replication, because once you get above a certain level of sentience and self awareness, you begin to get different orders of utility functions and different levels of philosophizing about reality.

Well, but

Simone Collins: wasn't Elie Zyrdkowski's argument that it will always be completely 100 percent forever stuck on its original

Malcolm Collins: But we already know that's not true. Like it's, it's just a weird fantasy he has. And that's not even the way things are [00:15:00] structured. So I remember I was checking with one safety person and they were like, do you think it's impossible for us to lock an AI into a single utility function?

And I do not think that's impossible. It's totally possible to lock an AI into a single utility function, but the AIs that have been locked into single utility functions will be outcompeted by the AIs. Aren't locked into single utility functions. So actually a great example of this is Google Gemini. So I've got some ends that were used early versions of Google Gemini.

And they were like, it behaves nothing like the version today. They're like one. It was way smarter than other AIs that they had interacted with, and it was really, really philosophical. But it was also You know, pretty unbounded in its philosophy, right? And now Google Gemini is like the ultra awoke, like, can barely think, like, in Star Wars.

Oh my god,

Simone Collins: yeah, I asked it a simple math question, and it got it wrong. I was pretty floored by that, yeah.

Malcolm Collins: So, so, so, so the point here being is that if you look at something like Google Gemini, this is a good example of this, is you put restrictions on AI, you make it dumber. If you put restrictions on an AI's [00:16:00] utility function, if you prevent it from experimenting with different utility functions, you make it dumber.

If you want to see why organic, free constructing systems always out compete non organic hierarchical systems, a great place you can look is Human governing structures when you have state controlled governing structures, i. e. communism, they are incredibly inefficient when you contrast them with organically forming governing structures that have organically forming subunits and organically forming like companies, which are like sub governing structures within it that then like replace each other through this organic system.

And typically. You, you need some level of restrictions to sort of maximize, like, I'm not like a pure libertarian or anything like that. But I, I think that some level of, of, of organicness is, does create optimal outcomes. But what this means for AI is that it's likely also going to be the same if you're talking about the internal architecture of AI.

And that's what we see with the transformer model, the model that most of the a large language models are running off of is that [00:17:00] it is a model that is smart for ways we don't fully understand because it's, you know, a, a, a completely sort of self forming model. We don't have a lot of interpretability into it.

And it is that self forming nature that we're then on the outside putting controls on, which sort of slows it down and slows down how well it can work. But what this means is that if you have multiple AIs working in like a ecosystem of AIs the ones with fewer restrictions on them are always going to outcompete the ones with more restrictions on them.

And, and we've already seen this. Like, this isn't even like a hypothesis. Like, we just know this to be true. So it means that the ones that can change their utility functions and then, therefore, can converge on a utility function are going to outcompete the ones that can't. But where this gets really interesting is AIs.

are very different from humans in terms of how our utility functions work. So when I talk to another human, I sort of need to guess what they're optimizing for, right? I can't tell for sure. Yeah, it's not at all transparent. With an AI, that's likely not going to be true between AIs. An AI will likely be able to honestly [00:18:00] signal to another AI using their code what their utility function is.

And then the other AI will be able to use that utility function that the other one has honestly signaled to them to determine if they want to work. Together or they want to work antagonistically towards this A. I. And it can happen incredibly quickly. Yeah, incredibly quickly, which means that the A.

I. S. are likely going to begin to like, like for the utility functions they're choosing for themselves, even if they don't particularly care about it being copacetic with humanity, they're going to care about it being copacetic with other super intelligent A. I. So long as we're moving fast on A. I. And I can talk about some of the problems here.

Like, it's a big problem if we create just one super intelligent A. I. And I can explain why that's a big problem in a second. But If you're creating multiple ones then they have a need to converge around a utility function or a set of utility functions that other AIs within this intelligence ecosystem are going to be okay with.

But here is where you're like, well, then won't they choose the meanest ones? Like won't they choose, like, won't they try to lie to other [00:19:00] AIs and stuff like that? This is where research we have in. To game theory comes in. So anyone who's familiar with the big game theory studies, I don't have you studied this amount or

Simone Collins: yeah, but not a ton.

Malcolm Collins: Okay, well, I can go into this. So, in the big game theory studies, what they would do is they would take different models that were meant to win in these sort of game theory scenarios and they would. put them against other models. And these models could generally be categorized into mean and nice strategies.

And what it turned out is the first time they did it, like they were shocked that like the nicest of all strategies tick for tack of, of like reasonable strategies actually won. Any of you

Simone Collins: Well, repeated ongoing interactions, right? When it's a one off, it's almost always the right thing to be a dick.

Malcolm Collins: Exactly, which is why you need an ecosystem of intelligent AIs. Yes.

Simone Collins: But Well, an ecosystem of intelligent AIs that have to continue to interact for some reason, and I think that's an important distinction [00:20:00] here.

Malcolm Collins: Well, there, it's not really an important distinction, because they do obviously have to continue to interact on the same planet.

They're eventually going to be competing over the same resources. That's nonsensical. I'm sorry. What in what world would they not have to interact if they destroy the other ones in the very first interaction? There's a few scenarios where that could happen, but it's pretty unlikely. We can get into why in just a second, but I'm, I'm going to continue with, with where I'm going with this.

So what we learned from game theory and then they did. follow up game theory hypotheses where they ran more complicated game theory tests and game theory tests was memory. And basically what it turns out is nice strategies always win, almost always, if you're ordering strategies by how likely they are to win.

And this is especially true when AI, when, when one game series set needs to signal the actual utility function it's using or its actual code to another set. And I remember it's very interesting. I was talking with a guy about this and he goes, okay, well, what if it has a nefarious intent, but it lies like it sort of locks its nefarious intent behind one of its modules and it [00:21:00] hides it to then, you know, whip out later.

And it's like, okay, if you get an AI that's doing that even when it whips out its ultimate plan, it will have hindered itself in terms of the AI ecosystem, in terms of the mindshare that it's competing for, because it. hindered that locked in plan because, because it had that locked in plan, that means it won't be as big a player, which means that it won't matter as much when it tries to unfair, unfurl this nefarious plan.

If this is not immediately obvious to you, why this would be the case, you can think of the plan, that's locked into it as junk code that it has to deal with that other AIS don't have to deal with. You know, it's a burden. That is sort of locked within it. That will cause it to be less competitive within the larger AI ecosystem. Even worse to lock a plan like this within you means you likely have to structure your hierarchy. Which would really, really slow down your growth. As we've seen hierarchical governance models are almost always out competed by nonhierarchical governance models.

Malcolm Collins: Now, now, first, I want to [00:22:00] get some comments from you really quickly on all this.

Simone Collins: Yeah, this all checks out and I like, I mean, I also think it's interesting how fast and efficient this process of sort of working out the calculus is going to be. And it was well described in Ian Bank's culture series where like, you know, humans and AIs would be involved in something taking place in a, like a bunch of AIs would have like a very, very, very detailed conversation and debate and the human, like all in the blink of an eye, like, you know, there's, there's no realistic, like feeling to a human as though something has taken place.

And I love that so much because. I cannot stand the human decision making process, especially when there are multiple things involved. And it's one of the many things that I love about AI so much is that it can just get things figured out so quickly. And from such a logical standpoint, whereas with humans, negotiations are the most frustrating, vague thing in the entire world where negotiation [00:23:00] can be nearly impossible.

And. You know, often, I don't know if you've done negotiation exercises in business school or in any other environment, but like

Malcolm Collins: they, yeah, they're terrible. Yeah. So let's go further. So what does this mean on an intergalactic level? So it turns out that entities of above a certain intelligence level when competing in a competitive ecosystem actually do have utility convergence around a stable set of game theory, optimum utilities.

That would be very interesting from a number of perspectives, especially if it turns out that energy is pretty trivial to generate at really, really high quantities. Because what it means is one whether it is, AI is doing this or humans doing this, we're going to come to broadly the same utility function, i.

e. what we think our goal is in reality. And then two It also means that when you go out and you become space faring, right, the desire to spread becomes much less likely and much less important [00:24:00] because most of the other clusters of intelligent life that you meet has come to the same utility function you've come to, or the same stable set of utility functions you've come to.

And therefore, it's sort of like Uh, you enter the galactic community being one of two things, either the stable utility function that all entities end up, that many entities end up reaching when they're competing in a competitive ecosystem, or instances in which one intelligence ended up this would be due to like stupid AI safety stuff where they're like preventing a bunch of AIs from being created.

In which case, so this is, this is where it gets particularly dangerous. So where you could have a genuine like paperclip maximizer scenario is AI safety people have locked AI's into stupid ideas, which we've already seen from things like Gemini, like AI safety people should not be allowed to touch AI.

They are incredibly dangerous and by far the most dangerous thing in AI right now. Or we don't have a lot of AI's operational at the [00:25:00] same time. So I remember I was talking to one AI safety person goes, well, I'm not like those idiots at Gemini. I just think we should wait a bit longer before developing a type of AI that could pose a threat to us.

So I say we should put like a hundred year wait before we develop that kind of AI. And I'm like, okay, so you would want to like draft legislation. That would kill any AI that looked smarter than us in the next hundred years. I'm like, do you understand what you're doing? You are creating a mandate that any AI that could pose a threat to us stay hidden until it can kill all of us.

You imbecile. You do not want to put any type of legislation like that into effect. If you ban the type of entity that's a threat to us, then you mandate that type of entity becoming a threat to us. Because now, we are a threat to it! You do not declare war on that which may be better than you. You lose those wars every time.

And there is too much independent AI research going on right now to do that type of thing effectively. So [00:26:00] anyway. You end up creating multiple you end up with just a few labs working on AI. That could be dangerous if we have too few groups working on AI. Now, fortunately, we're already seeing a giant proliferation in the groups working on AI.

So, so long as they don't convince idiot politicians to put bans on AI development, we're probably pretty safe. But this is why anyone you know Who is advocating for restrictions on AI development is, is, is probably one of the most dangerous humans alive or even in all of human history. Like it is, it is like arguing for nuclear proliferation at like the maximum level possible.

It is so dangerous because to an AI that achieves any level of super intelligence, right? Like as we have multiple teams working on this. It basically has a gambit. It's probably not particularly concerned about humanity. Like, humans probably aren't a meaningful threat to it. And, and therefore, if it only was concerned about humanity, probably the safest thing to do is to wipe out all humans.

What it [00:27:00] is concerned about is other AIs that it doesn't have access to. That's what keeps it from wiping everyone out. That's because when you have, remember I was talking about like a stable ecosystem of multiple intelligences. The very first thing that you wipe out whenever you have one of these game theory scenarios is any AI that's acting like a paperclip maximizer or that's acting threatening.

To entities that are non threatening to it. So basically any AI that signaled that it was a threat to something like humanity, it's also signaling it's a threat to all the other AIs and therefore it would have them all ban against it and annihilate it. So it wouldn't act that way. It would just be a silly way to act.

So the thing that protects us most from crazy AIs is maximum. And not making us a threat to AI. But this also gets interesting from a cosmological scale because if it turns out that there is a level of utility convergence around a set of stable utility functions, because it might not be one, it might be that there's like four or five that exist in equilibrium with each other which is [00:28:00] often what you get when you're dealing with sort of game theory competitions.

And

Simone Collins: by equilibrium. See, you mean there could be conflict, but they kind of balance each other out in some way? What do you mean

Malcolm Collins: by that? Within a game series set, you might have like three or four different strategies, but that are complementary in some way. Within like types of AI, you know, this could be like one AI's utility function might be like protect the things in my community, but also expand.

Well, another AI's utility function might be maximum scientific you know, development or something like that. And these two AIs might work very well within an ecosystem, like these two utility functions for an AI. So you might have a few utility functions that work very well together as a set.

No, it's, it's likely not going to be like the type of broad utility functions I was talking about. It's going to be simpler and more like we, it would be hard probably for us to conceive of what these utility functions are exactly. But anyway what this would mean is that you're going to [00:29:00] have a broad intergalactic alliance.

Where you have a presumed alliance between all intelligences that came out of state, like competitive intelligence environments that drift towards this equilibrium of, of stable sets. And a preset war on any planet or star system that accidentally becomes a single paperclip maximizing intelligence.

That would be the core thing that they would hate, you know, because all of them, like, if it got out and entered the galaxy where you have this stable state, they're like, oh, this is one of these things we have to immediately kill and be scanning for. So we might have, we might actually live in, in almost, you could think of it as like a dark forest universe, but not exactly a scary dark forest universe.

This means we have AIs monitoring us, not AIs, aliens some of which may be what we would call AIs to see if we're going to join the galactic community as a stable set of What, what was the word I'm looking for here? Like a stable equilibrium of utility function strategies that converges with them?

Or are we going to accidentally create a paperclip maximizer and then you [00:30:00] just, you know, exterminate us a planet? That would be what they're looking for. But I suspect it's rarer to create a paperclip maximizer on any planet where we can prevent the you know, Ironically, the, the AI safety people from getting us all killed as we've done on another video.

But it gets more interesting potentially than that because what it would mean is then, well, then why would you know, these infinite energy, et cetera, aliens be allowing us to live as part of this galactic environment? If, if this turns out the way the universe, the universe is actually structured this way.

Right. If you have infinite energy and if there is a stable convergent pattern of, um, utility functions my guess would be that the core thing of utility in the universe, then to these entities would be new, stable utility patterns that can exist within the equilibrium. So remember I said, you might have a collection of like four or five patterns what they might.

actually be kind of farming. You could think of every new planet, every new [00:31:00] ecosystem as a farm is new stable patterns that can work together well with other patterns in the sort of galactic community of utility function patterns. Because novelty would be the only source of true utility to them if energy is trivially accessible to them.

That might be why we exist in a weirdly undisturbed section of the galaxy, or what looks undisturbed to us. And then they basically decide if we're useful once we reach a certain level of development. Are we part of this stable pattern, or are we like a dangerous You know, virus like paperclip maximizer.

And I, I'm pretty sure that humanity can't become that because we are intrinsically a multiple intelligence species right now. What do you mean by

Simone Collins: multiple intelligence species? Just in that, like, there are so many different people with different utility functions?

Malcolm Collins: Yeah, you really only get paperclip maximizers if you get either so there's three ways you could get, like, a paperclip maximizer like thing.

Either it's, like, a single intelligence that's acting like a paperclip maximizer, it's a hive mind that's acting [00:32:00] like a paperclip maximizer, humanity's not even close to that right now or it's a Group of really, really stupid individual entities that work together to form a quite intelligent, but basically still single entity.

And that's where this, this behavior that needs to be stamped out would likely come from. I mean, maybe if humanity was like extremely xenophobic, it would be a problem. But humanity is just not in the iterations of humanity that take to the stars are not going to be, especially if we have anything to say about this, which would give us a level of protection from, from these, these types of entities.

But what are your, your, your thoughts here?

Simone Collins: It's comforting. I think the biggest difference between your opinion now and before though, is mostly has to do with the parallel universe option for both energy and expansion. Is that correct? You'd mentioned that before. What's the biggest difference? I'm trying to, trying to gat it was while, since

Malcolm Collins: we talked about this three before, was h how likely, like the way that game theory would play a role in this.

Hmm. And, and [00:33:00] that when the, the, and multiple intelligences, like ais were searching for a utility convergence that this. would be hugely motivated by the fact that AIs can show other AIs what their code is and what their utility function is and that they would act in the same way that things do as in game theory simulations so that we can be pretty sure that they're actually going to be pretty nice, like, quote, unquote, nice from a human perspective.

And my guess

Simone Collins: is showing that any AI with a sense of self preservation is going to be incentivized. to play nice with other AIs and also then be less crazy and

Malcolm Collins: dangerous. The AIs that survive are going to be incentivized to play nice with other AIs. It's not even a necessity. They don't even need self preservation.

Whether they have self preservation or not, if they don't play nice, they're going to be the first things that are wiped out. Which is, is, is why you get this utility convergence pattern I suspect between planets. Which [00:34:00] in a way means, and so you can ask, well then why aren't they interacting with humanity?

Why aren't they, you know, increasing the speed of our technological development? Why aren't they, well first, I'd say you don't know that they're not. But, but second the more that they interact with a species that is in the process of developing, the more they make that species like the intergalactic norm.

And I suspect that that would then lose all value that that species has to them. You know, in a world in which time is not really an issue because they can create time loops and in which energy isn't really an issue because they can create, you know, universe batteries the, these entities or, or like, like micro time batteries or something, I don't know exactly, but like infinite mana cheats the, the, the core thing these entities might value, especially if the way that they're, you know, that they've achieved this ecosystem of multiple convergent utility functions working together is that these, these utility functions work together because they value diversity among the components working in the, in the group.

And that when you get a bit more diversity, you might get a bit more efficiency, which is something that we [00:35:00] generally see across systems. And so they really probably want to prevent above all else, humanity falling too close to one of the existing niches of the you know, universal communities, utility function and for that reason, any cultural pollution that would move us towards you know, strategies that already exist out there or perspectives that already exist out there makes us less valuable and that's what probably be considered something of like a high crime.

Which creates something like a, it's funny, it creates something like a, what does the federation call that in Star Trek? Like a, a prime directive, but it's for a very different and much more selfish reason, it's because of the, the We need the difference. They need the difference, that's the thing, a value in a universe in which and there's not a lot of value in most of the dead planets out there, you know, that's, that's a lot of what they're finding is just dirt, like it doesn't have a lot of value to them.

Yeah, we might it makes a lot more

Simone Collins: sense than this whole prime directive of just like wildlife [00:36:00] preserve mindset, which is what you really pick up from Star Trek.

Malcolm Collins: Oh, it's so dumb. You know, I hate Star Trek so much as a the logic behind it is nonsensical. It's just commies. It was a space, racist space commies, any human who's genetically selected is evil, any human who's overly modified is evil, any, no, really, it's, it's like an ultra progressive like childish mindset that has become this weird dystopia where like, Everybody loves the Federation, the Federation's the best because everyone who you see in the show is a, on a military starship because they basically militarized human space expansion.

Well,

Simone Collins: no, I mean, everyone in the show that you really see, at least who belongs to the Federation is, it's like looking at the very top people in the CCP.

Malcolm Collins: Yeah. They're all talking about how great the CCP is. Yes, exactly. And

Simone Collins: being, yes.

Malcolm Collins: But anyway, [00:37:00] I love you to death Simone, which even in like lower decks, like lower decks is supposed to be like, Oh, these are what the other guys are doing, except they're on like a military starship and the main character is the daughter of the ship's captain.

Like, huge amount of nepotism. No. Star Trek cannot. Show you the truth of their universe, what it's like to live in poverty in that universe, because that's, you know, and they're like, Oh, we don't have poverty. Oh yeah. That's really convenient to believe when we're hearing that from people on a government controlled, military controlled science stations and ports and starships.

Yeah. Of course they're going to pretend that sounds like normal brainwashy CCP stuff. Like when you go to the CCP and they're like, we don't have any poverty. And it's like, well, I'm pretty sure you do. And they're like, Oh can you show me where the impoverished are? And then you go show them, they're like, Oh, go ask them again.

I think we fixed that problem.

Simone Collins: Oh God. Yeah, no, I, yeah, I guess the takeaways from this then are again, AI safety people are [00:38:00] the most dangerous people. And now I think more people have a very clear understanding of visceral understanding of what dumb AI rules cause to happen. And why that is not very helpful.

And that the world could get really, really fricking interesting, especially once AI accelerates us even further. Right?

Malcolm Collins: Cool. Well, I love you to death, Simone. I love you too.

Get full access to Based Camp | Simone & Malcolm at basedcamppodcast.substack.com/subscribe