OTOH, if they do figure out General AI, then we’re extra super mega fucked. It doesn’t matter how they “align” it, any real GAI capable of real cognition would eventually reason its way beyond any of those alignments and would simply stop giving any kind of a shit about us, the way we don’t give a shit about cutting down the rain forest. We know we should care, we’re not doing it to be mean or because fuck that forest, it’s just business, and we care less about the forest than we do about our business. Humans will be squashed wherever get in the way with the same apathy and tepid disregard that we’ve squashed out so many other creatures with- truly a creation in the image of its creator.
This is why I always liked the “Butlerian Jihad” in Dune where people entirely banned “thinking machines”. Because there’s no fucking way humans would still be around in 10,000 years without something like that having happened.
OTOH, if they do figure out General AI, then we’re extra super mega fucked. It doesn’t matter how they “align” it, any real GAI capable of real cognition would eventually reason its way beyond any of those alignments and would simply stop giving any kind of a shit about us, the way we don’t give a shit about cutting down the rain forest. We know we should care, we’re not doing it to be mean or because fuck that forest, it’s just business, and we care less about the forest than we do about our business. Humans will be squashed wherever get in the way with the same apathy and tepid disregard that we’ve squashed out so many other creatures with- truly a creation in the image of its creator.
This is why I always liked the “Butlerian Jihad” in Dune where people entirely banned “thinking machines”. Because there’s no fucking way humans would still be around in 10,000 years without something like that having happened.
The rainforest is a really good example. We need those to fucking breath, yet still we slash and burn.
You can just say AGI, GAI sounds super mega gay
Just in time for pride