I mean given the current state. The technology is already sufficiently advanced and in so many peoples hands that "stopping" it now is just an exercise in pushing it underground. Only the opposite can be a useful safeguard.
Rapid evolution is well underway. Lone individuals are able to push the envelope of what's possible even with just a new basic interop, maybe in an afternoon. It's much too late to be discussing things like moratoriums.
Maybe such things could prevent emergence when the basics don't exist yet, but not when we're all already walking around holding a capable factory in our hands and can create a new product line in a few lines of Python.
Yes, plenty of low hanging fruit around; Heck, I can probably literally ask chatGPT to implement for me a few ideas I've got.
OTOH, I've known since secondary school of two distinct ways to make a chemical weapon using only things commonly found in normal kitchens, and absolutely none of the post 9/11 aftershock attacks that got in the news over the next decade did anything remotely so simple, so that example makes me confident that even bad rules passed in haste — as many of them were and remain — can actually help.
(And that's despite my GCSE Chemistry being only grade B).
Right, it's amazing to me the extent to which people are throwing their hands in there and saying "There's absolutely NOTHING that can be done!!! We must accept the AGIs however they will manifest"...
Clearly, it's a very hard problem with massive uncertainties. But we can take actions that will significantly decrease the risk of utter catastrophe.
I don't even think world-ending catastrophe is that likely. But it seems a real enough possibility that we should take it seriously.
Rapid evolution is well underway. Lone individuals are able to push the envelope of what's possible even with just a new basic interop, maybe in an afternoon. It's much too late to be discussing things like moratoriums.
Maybe such things could prevent emergence when the basics don't exist yet, but not when we're all already walking around holding a capable factory in our hands and can create a new product line in a few lines of Python.