In seeking the root cause for concern about AI safety, the last chapter recognised AIs as complex systems subject to rules. Seen from this perspective, the structural rules[Def] of current AIs were observed to be incompatible with the principles securing life’s survival on Earth. This is a huge concern.

In this chapter I look at a second issue – AI’s “pre-training”. This, within the same framework, instils what Cilliers referred to as regulative rules.1[Def] They are the focus for much present interest, although acknowledged, widely, to be little understood.2,3,4 This chapter offers a way to understand them.

Recently, in a much publicised post, Evan Hubinger, an Anthropic safety researcher, aired his concern that there is a greater than 10% risk of AIs killing all humans within the next decade. Many in this field have voiced similar concerns for many years. Now, however, they are more specific, the timescales are getting shorter, and fewer are disagreeing.1,2

The proximate causes are: AIs’ growing powers, the approach of their “recursive self-improvement” [Def],3 the antagonism they can display towards humans, and the well-documented failure of AI developers to align AI values with human values.4,5 The root cause of their troubling behaviour is, however, underexplored. This is the issue I address here.