> Or did you think that if strong AI showed up the Governments of the whole world would not rush to have their own?
Governments can't "rush" on the same scale an AI can, and a strong AI with an appropriate value function wouldn't permit the existence of any other strong AI that doesn't share its value function (since such an AI would fall somewhere between "counterproductive" and "potential existential threat"), nor would "of their own" have any meaning unless that value function was hopelessly broken.
There shouldn't be a "second" strong AI, only a first.
> It's not clear that an emulation in a computer would be anywhere near as fast as the actual thing
Humans operate relatively slowly on the scale of computers; we're just massively parallel and have a fundamentally different architecture. That's not at all insurmountable even for human-built technology, let alone technology-built technology.
> All your assumptions are predicated on the ability to emulate a human in software
Several of my assumptions are, true, though I can imagine other possibilities that would have similar effect.
> An even larger assumption is that it's even possible to do that.
That's not a larger assumption, that's the same assumption I stated.
> Can an intelligence program something smarter than itself? I don't think so.
On what basis?
> I don't think humans can.
Humans don't have to build a system smarter than ourselves, just build one as smart as ourselves and run it a lot faster with access to a lot more data.
Governments can't "rush" on the same scale an AI can, and a strong AI with an appropriate value function wouldn't permit the existence of any other strong AI that doesn't share its value function (since such an AI would fall somewhere between "counterproductive" and "potential existential threat"), nor would "of their own" have any meaning unless that value function was hopelessly broken.
There shouldn't be a "second" strong AI, only a first.
> It's not clear that an emulation in a computer would be anywhere near as fast as the actual thing
Humans operate relatively slowly on the scale of computers; we're just massively parallel and have a fundamentally different architecture. That's not at all insurmountable even for human-built technology, let alone technology-built technology.
> All your assumptions are predicated on the ability to emulate a human in software
Several of my assumptions are, true, though I can imagine other possibilities that would have similar effect.
> An even larger assumption is that it's even possible to do that.
That's not a larger assumption, that's the same assumption I stated.
> Can an intelligence program something smarter than itself? I don't think so.
On what basis?
> I don't think humans can.
Humans don't have to build a system smarter than ourselves, just build one as smart as ourselves and run it a lot faster with access to a lot more data.