29. AI Creation and the Cosmic Host (2024)

By Nick Bostrom.

Abstract:
There may well exist a normative structure, based on the preferences or concordats of a cosmic host, and which has high relevance to the development of A-eye. In particular, we may have both moral and prudential reason to create superintelligence that becomes a good cosmic citizen—that is conforms to cosmic norms and contributes positively to the cosmopolis. An exclusive focus on promoting the welfare of the human species and other terrestrial beings, or an insistence that our own norms must at all cost prevail, may be objectionable and unwise. Such attitudes might be analogized to the selfishness of one who exclusively pursues their own personal interest, or the arrogance of one who acts as if their own convictions entitle them to run roughshod over social norms—though arguably they would be worse, given our present inferior status relative to the membership of the cosmic host. An attitude of humility may be more appropriate.

Read the full paper:
https://nickbostrom.com/papers/ai-creation-and-the-cosmic-host.pdf

More episodes at:
https://radiobostrom.com