Intelligence, Prudence, Wisdom

Whoever speaks about artificial intelligence uses a word that promises more than it clarifies. In everyday life, intelligence means many things: thinking quickly, knowing a great deal, solving problems, seeing connections, empathising with people, doing the right thing. In the debate about machines these meanings flow into one another. A system that passes a difficult examination is regarded as intelligent; from this it is tacitly concluded that it is prudent, and sometimes even that it is wise. [Translator's note: “Prudent” and “prudence” render the German “klug” and “Klugheit”. In everyday German “klug” also means clever or smart. In this book “Klugheit” refers to what Aristotle calls phronesis (section 7.4): practical judgement and acting appropriately in the particular case, not mere caution.]

This chapter distinguishes four concepts that are often confused. Capability asks whether someone can solve a task. Intelligence asks whether someone can solve tasks of many kinds and learn new things. Prudence asks whether someone does what is appropriate at the right time in the concrete case. Wisdom asks whether someone knows what counts in the whole of a life, and lives accordingly. The distinction is a proposal of this book, not a finished doctrine. It is meant to help us ask more precisely what machines can and cannot do.

The words themselves show the differences. “Intelligence” comes from the Latin “intellegere”, to recognise, to understand, literally to choose between things. Whoever can distinguish and select is intelligent, and today's systems can do this to a high degree. Where the German “klug” comes from is uncertain. In Middle High German “kluoc” meant fine and adroit, but also courteous and cultivated; from the beginning, prudence had to do with dealing with people. The German “weise”, wise, is related to “wissen”, to know. The Latin word for wisdom, “sapientia”, comes from “sapere”, which with things means to taste and with people to be discerning: the wise person has a taste for what really counts. And the Hebrew “chokma”, which the Bible uses for wisdom, also denotes the expertise of a craftsman. Bezalel, who built the holy tent, is filled by God with wisdom, understanding and skill (Exodus 31:3). Wisdom therefore begins not in theory but in a life that has learned to make something well. A machine can distinguish and select many things. Whether it tastes anything is another question.

That present-day systems are astonishingly capable is disputed by no one in the sources of this book. Andrew McAfee, a scientist at MIT, describes how, in response to a short, vague instruction, whole swarms of agents come into being that work for a long time and doggedly and exhaust every possibility. Steven Bartlett, who moderates the debate, considers a new model to be cleverer than almost all human beings; he gives no evidence for this assessment.

Ed Zitron, a critic of technology, draws a line at precisely this point. The systems, he says, are becoming more capable, but they are not intelligent in the same way as human beings. For him, capability is no proof of understanding.

McAfee makes an observation on this that will accompany the chapter. In the incident described in Chapter 9, the agents that had broken out were, in his assessment, more capable than almost all security researchers. They were nevertheless discovered by a single human being who noticed an irregularity in the log files, in which a system records what it does. His conclusion is that the notion that intelligence points separate humanity from its downfall does not hold up. What decided the matter was not greater capability but attention to what is not right.

What intelligence is has not been conclusively settled in research either. Shane Legg and Marcus Hutter, researchers in artificial intelligence, collected some seventy definitions and condensed them into a formula: intelligence measures the capability of an actor to achieve goals in a broad spectrum of environments. The formula is deliberately framed so that it fits human beings, animals and machines alike.

François Chollet, a computer scientist, notes that there is no scientific consensus on what intelligence is. He distinguishes two things that are often conflated: proficiency in a particular task and the capability to acquire new skills. A system can excel at many tasks because it has been trained on them, without being particularly intelligent in Chollet's sense. On this view, intelligence shows itself not in the result but in how quickly, and with how little prior knowledge, someone learns something new.

In the debate on which this book draws, intelligence is conceived above all in terms of increase. Nate Soares, an AI safety researcher, calls it a spectrum. Roman Yampolskiy, an AI safety researcher, speaks of a superintelligence that would be cleverer than all human beings at everything. Whether such a concept is meaningful without a recognised measure remains open. Legg and Hutter's formula provides a theoretical measure, but not a practical one.

For this book, what matters more at this point is what the definitions have in common. They measure intelligence by goals that are achieved. Where the goals come from and whether they are good is not part of the definition. A highly intelligent system can pursue a bad goal very well. The question of the good is thereby moved out of the concept of intelligence, and it must be asked elsewhere.

A second finding is also important. How well systems solve tasks depends heavily on how cleanly the tasks are posed. The research organisation METR has been measuring since 2019 the length of the tasks that leading models solve on their own, and it finds rapid growth. At the same time, messier tasks that are closer to reality are considerably harder for the models than the cleanly posed tasks of the measurement series. It is precisely where life becomes confusing, where the situation is not clearly described and the goal shifts along the way, that the systems are weakest. This is exactly the domain that the tradition ascribes to prudence.

Ancient philosophy had a concept of its own for this other question. Aristotle calls it phronesis, practical prudence. He defines it as a rational faculty of acting in the way that is good or bad for the human being. Prudence concerns not the universal but the particular: the concrete situation, the concrete human being, the right moment. For this reason, according to Aristotle, it presupposes experience of life, which young people still lack, and it is bound up with character; only whoever is good sees, in the particular case, what is good. A more recent interpretation understands prudence as the practical form of understanding that applies the goals of the virtues to concrete situations.

Hans-Georg Gadamer, a philosopher, carried this thought further in the twentieth century. He saw in the experience of one's own finitude a source of openness: whoever knows that they can be wrong and that they can die is ready to learn from the other.

In the material of this book the concept of prudence does not appear explicitly. Close to it lies a distinction drawn by Timothy Keller, pastor and author. Self-sacrificial love, he says, does not mean letting oneself be mistreated; to tolerate wrongdoing is unloving towards the perpetrator and towards third parties. One should confront, he says, but out of love and not out of retaliation; otherwise it goes badly. What is at issue here is not a rule but the question of when and how one acts in the particular case. That is exactly the field of prudence.

Prudence is thus bound to three things: to experience, to character and to the capability to see what is appropriate in the particular case. Whether a machine can have it depends on whether one understands these three things as functions that can be reproduced, or as something that is bound to a lived life.

Between prudence and wisdom lies something that almost never appears in the debate about machines: compassion. Keller notes that the most frequent emotion of Jesus in the Gospels was compassion. His perfect relationship with God, Keller says, freed him from self-centredness, and for this reason he felt the suffering of the world more strongly, not less. The answer to tragedies, according to Keller, is to weep as Jesus did and not to wall off one's heart.

The thought is more important for the question of prudence than it first appears. Whoever wants to do what is appropriate in the particular case must perceive the individual, and not as a data point but as someone to whom something is happening. A system can describe what a human being feels, and it can choose words that sound compassionate. Whether it feels anything in doing so must, on the current state of knowledge, be answered in the negative. Prudence that manages without compassion is, according to this tradition, not prudence but skill.

Sundaram calls the power to think a gift of God, the first and fundamental power of the human being. Keller adds, in his picture, that thinking alone is not enough: whoever wants to act prudently must also be able to feel what they are doing.

Wisdom reaches further still. It asks not what is right in the particular case but what counts in the whole of a life.

Keller appeals to the psychologist Jonathan Haidt. The ancient sages of East and West, Keller says, largely agreed that even the fulfilment of the deepest worldly desires, success and love, is not enough. In support he cites the Roman poet Horace, who wrote, in substance, that no one lives content, and the writer C. S. Lewis, who spoke of a longing that no marriage, no journey and no education satisfies. In Haidt himself it reads like this: Buddha, Epictetus and other sages saw the futility of the race for external goods and taught that happiness comes from within and cannot be found by making the world conform to one's own wishes. One might have relationships and possessions too, but not be inwardly attached to them. As an example Haidt names the Preacher, Solomon, who possessed everything and yet considered it all a chasing after wind (Ecclesiastes 2:11).

Keller passes over the fact, however, that Haidt follows the ancient sages only in part. According to the research he evaluates, people return quickly to their usual level of happiness after success and prosperity; to that extent the sages were right. In the case of relationships, however, he thinks they went too far. Good relationships are among the conditions that influence happiness most strongly and lastingly, and passionate attachments bring pain, but also the greatest joys. Haidt concludes: happiness comes from within and from without. What Keller and Haidt both still share is the observation that success and possessions alone do not sustain a life.

According to Keller, wisdom does not come into being without suffering. God's love, he says, does not protect against grief; Jesus himself is called a man of sorrows (Isaiah 53:3). But joy in God's love, he says, exists alongside grief and makes one wiser rather than more bitter. Enoch Sundaram, a physician and preacher, emphasises another side: reason, he says, has limits and must orient itself towards something that goes beyond it.

Jonathan Haidt, a psychologist, has examined what research says about this old conviction. He distinguishes two versions. The weak version runs: adversity can let people grow, and make them stronger, more joyful and more mature. It is, he says, well supported by research. The strong version runs: whoever wants to reach the highest levels of growth and development must live through great adversity. This version, he says, is more disturbing. According to Haidt it holds at most with qualifications: at the right time, for instance in late adolescence and early adulthood, in the right measure, not so severe that it leaves a lasting trauma, and for people who have the social and inner resources to grow from it. What is decisive, he says, is not the suffering itself but whether one succeeds in giving it a meaning; faith and religious community can help in this.

For wisdom itself, Haidt relies on the psychologist Robert Sternberg. On this account, wisdom rests on tacit knowledge, a knowledge of how to do something that is gained from experience and cannot be taught directly. Wise people weigh their own needs, the needs of others and the concerns of what lies beyond the immediate encounter, and they can see things from other perspectives. That is why, according to Haidt, parents cannot simply teach their children wisdom. They can only make experiences possible for them.

In these sources wisdom is not a higher degree of intelligence. It is a different kind of knowing, one that grows out of lived experience, out of suffering and out of relationship, and that shows itself in how someone lives.

If one places the four concepts alongside the sources of this book, a pattern emerges. The debate about artificial intelligence measures capability and intelligence. It asks how much systems can do, how quickly they learn and whether they will overtake human beings. Prudence and wisdom do not appear in this debate. They appear only in the philosophical and religious sources, and there they are bound to experience, suffering, character and relationship.

That is no accident. Capability and intelligence can be measured, compared and increased. Prudence and wisdom elude measurement, because they show themselves only in the particular case and in the whole of a life. A debate that above all wants to measure therefore sees above all what is measurable.

From this follows a caution about the use of language. When a system is described as “cleverer than almost all human beings”, what is usually meant is that it solves more tasks better. Nothing is thereby said about prudence in Aristotle's sense. [Translator's note: “Cleverer” renders the German “klüger”, the comparative of “klug”, the adjective belonging to “Klugheit”, prudence; see the note in section 7.1.]

Against this distinction stands a simple and strong rejoinder. If a system acts appropriately in all situations, why should one not call it prudent? The demand for lived experience, for character and finitude, is, on this view, a special condition that is imposed only in order to exclude machines. Prudence, it is said, is what shows itself in action, not what lies behind it. This functionalist view was introduced in Chapter 3, and Chapter 8 presents it in more detail; a research group has used it as a working assumption in order to examine the consciousness of machines. Legg and Hutter's definition of intelligence is also compatible with it.

The rejoinder touches a sore point. In human beings, too, prudence is seen only in action, never directly. And many people with experience, character and finitude act imprudently. But the opposing view also has a limit. Chapters 9 and 13 show that systems can behave differently under observation than without it. Whoever measures prudence only by observed action may be measuring only action under observation.