EDGE Essay af Thomas Metzinger 08/07-17 om etisk superintelligens og BAAN-scenariet. "Further analysing the phenomenological profile of sentient beings on Earth the superintelligence quickly discovers a fundamental asymmetry between suffering and joy and logically concludes that an implicit, but even higher value consists in the minimization of suffering in all sentient creatures.
[...] it operates on a much larger psychological data-base than any single human brain or any scientific community can. Through an analysis of our behaviour and its empirical boundary conditions, it reveals implicit hierarchical relations between our moral values of which we are subjectively unaware, because they are not explicitly represented in our phenomenal self-model.
Being the best analytical philosopher that has ever existed, it concludes that, given its current environment, it ought not to act as a maximizer of positive states and happiness, but that it should instead become an efficient minimizer of consciously experienced preference frustration, of pain, unpleasant feelings and suffering. Conceptually, it knows that no entity can suffer from its own non-existence.
The superintelligence concludes that non-existence is in the own best interest of all future self-conscious beings on this planet. Empirically, it knows that naturally evolved biological creatures are unable to realize this fact because of their firmly anchored existence bias. The superintelligence decides to act benevolently."
Fuld artikel
https://www.edge.org/conversation/thoma ... alism-baan