Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.

There was conversation on Facebook over an argument that any sufficiently complex system, whether a human, a society, or an AGI, will be unable to pursue a unified goal due to internal conflict among its parts, and that this should make us less worried about "one paperclipper"-style AI FOOM scenarios. Here's a somewhat edited and expanded version of my response:

1) yes this is a very real issue

2) yet as others pointed out, humans and organizations are still able to largely act as if they had unified goals, even if they often also act contrary to those goals

3) there's a lot of variance in how unified any given human is; practices such as therapy and certain flavors of meditation can make a person significantly more unified than they used to be; if you were intentionally [...]

---

First published:
December 11th, 2023

Source:
https://www.lesswrong.com/posts/hHnpn3mEPbJMFLj4g/quick-thoughts-on-the-implications-of-multi-agent-views-of

---

Narrated by TYPE III AUDIO.

“Quick thoughts on the implications of multi-agent views of mind on AI takeover” by Kaj_Sotala

LessWrong (30+ Karma)