Yeah, this is Yud’s argument, but I just don’t get it. Does the technology to end the world by sending a couple emails around already exist? If so, why hasn’t the world ended?
People generally don't want to end the world. Those with the power to do so already are living generally good lives so they see little reason to potentially sacrifice the world for more power. AI could have completely different utility functions than people though, so an AI might have less qualms about ending the world.
