Consciousness, the Subconscious

Most decisions feel as though we make them consciously. We weigh things up, we choose, we act. Yet a large part of what determines our actions happens without our noticing it. We drive a car without thinking about every movement of our hands. We react to a tone in another person's voice before we can name it. We find arguments convincing or not without knowing why.

This chapter asks how much of what a human being decides happens beneath consciousness, and whether this makes them predictable. The question has an immediate significance for the subject of this book. If the decision-making of a human being is shaped by patterns that they themselves do not see, then a system that recognises patterns may be able to read them better than they can read themselves. And then the question arises of who is permitted to do so. There is a second connection. In a language model, too, most of what happens does so without anyone seeing it in detail, not even its developers (Chapter 13). Whoever wants to understand what it means to be shaped by the invisible can begin with the human being.

Enoch Sundaram, physician and preacher, describes the subconscious with an image taken from technology. It seeks, he says, to save time and energy and forms programs or habits out of repetition: how we drive a car, how we speak, how we deal with spouses and children. All action, according to him, rests on the programs in the heart. That actions become automatic through practice is well known. Sundaram goes further: the subconscious, he says, takes in without selection whatever consciousness gives it, good and bad alike. He compares it to soil into which thoughts fall as seeds and come back multiplied; hatred, he says, produces more hatred, love more love. Perception too, according to Sundaram, lies in the heart: human beings already have preconceived thoughts about everything and decide on that basis.

What Sundaram concludes from this model is remarkable. Not everyone, he says, has the right programs. Wrong programs are recognised above all when others correct one, and through prayer: Search me, God, and know my heart. Correction, he says, should be accepted with thanks, it is like a broom for the heart; and one should correct others kindly and lovingly. The subconscious can, according to Sundaram, be reprogrammed in two ways: through mindfulness, that is, awareness of one's own thoughts, words and actions, and through meditation as deep reflection on God's word. Hypnosis, by contrast, he rejects.

The model is vivid, and it contains claims that go too far. That the subconscious stores the entire memory down to the finest details cannot be substantiated. Chapter 3 has examined the figures with which Sundaram describes the performance of the subconscious and shown them to be untenable.

Timothy Keller, pastor and author, speaks not of programs but of background assumptions. Chapter 1 presented his thesis: barely conscious assumptions determine which arguments convince at all. In a thought experiment he shows how much people are reading off a cultural script when they believe they are expressing their true self: a warrior a thousand years ago and a young man of today both regard as authentic what their culture lays down for them. Such assumptions, according to Keller, are seldom discovered alone, but in conversation and in relationships.

Keller also knows of a layer that lies deeper still. The hollow space in the heart, he says, is present in all human beings, even when it is not conscious, for instance in children. What is meant is a longing that a human being has before they can name it.

Different as the two speakers are, they agree on one point. What shapes our decision-making lies largely beneath consciousness, and we hardly recognise it by our own strength, but through a counterpart: through correction, through conversation, through prayer.

Research confirms the core of this observation, albeit with different figures and different concepts.

Chapter 3 gave the orders of magnitude: according to a recent study, the sense organs take in around a billion bits per second, and behaviour processes about ten. The figure itself is disputed and is more an order of magnitude than a constant. But it shows that between what is taken in and what is consciously processed there lies an enormous gap.

This becomes especially clear in the perception of danger. Joseph LeDoux, a neuroscientist, has shown that circuits in the brain in which the amygdala is involved can detect threats and trigger defensive reactions of the body without the person consciously noticing the threat. The conscious feeling of fear, by contrast, arises according to his account in other, higher circuits; the amygdala, he holds, is not the brain's fear centre. LeDoux therefore recommends using the word fear only for conscious experience. The body thus reacts even before the human being knows that they are afraid.

Psychology describes such processes as implicit processes and distinguishes between fast, automatic thinking and slow, deliberate thinking. For this book the common finding suffices: a large part of what we do is not consciously controlled. This is not a deficiency; rather, it is what makes life possible in the first place. No one could walk, speak or drive a car if they had to plan every step consciously.

At this point the book must draw a distinction that is not always drawn in the sources. Sundaram equates the biblical heart with the subconscious. That is an interpretation that brings two different languages together.

There are at least three levels at which people speak of what lies beneath consciousness. At the theological level, the heart is the centre of the person, the place where the person turns towards God or turns away from him. At the psychological level it is a matter of habits, implicit assumptions and automatic reactions. At the neuroscientific level it is a matter of processes in particular circuits of the brain. The three levels are connected, but they are not the same thing.

Whoever mixes them gains vividness and loses precision. If the heart is the subconscious, then brain research seems to confirm theology. But if it turns out that a figure about the subconscious is wrong, the same research seems to refute theology. Both are wrong. What the heart is in the biblical sense cannot be measured in a laboratory, and what is measured in a laboratory is not the heart.

If so much happens beneath consciousness, is the human being then predictable? One working hypothesis of this book is: human beings are more predictable than their self-image assumes. Programs, scripts and patterns shape their decision-making, and whoever knows these patterns can predict a great deal.

Against this hypothesis stands a strong objection. That behaviour can be predicted does not prove that the person is determined. Freedom shows itself precisely in recognising patterns and changing them. In philosophy this view is held under the name of compatibilism: freedom and causation do not exclude each other.

It is remarkable that the sources of this book themselves support the objection. Sundaram, who speaks so much of programming the subconscious, calls choice the greatest capability that God has given the human being, the governing power. His whole teaching on reprogramming presupposes that human beings can recognise and change their patterns. Keller presupposes the same when he says that background assumptions can be discovered in conversation. The sources thus describe the human being as shaped, but not as fixed.

To ask whether a machine has consciousness, one must first ask what a human being needs consciousness for at all, and what they need an unconscious for. The unconscious works quickly, economically and without effort. It takes care of the habitual, so that the human being does not have to think through every movement afresh. The psychologist Daniel Kahneman found a memorable image for this in his book Thinking, Fast and Slow. He describes two kinds of thinking, which he calls System 1 and System 2, expressly as simplifying figures and not as two places in the brain. System 1 works quickly, automatically and without effort: whoever reads “2 plus 2” knows the answer without calculating. System 2 is slow and effortful: “17 times 24” demands attention. How easily fast thinking leads us astray is shown by a well-known puzzle. A bat and a ball together cost 1.10 francs, and the bat costs one franc more than the ball. How much does the ball cost? Most people say at once: 10 centimes. The correct answer is 5 centimes. Fast thinking delivers an answer that feels right, and slow thinking checks it only if one wakes it. Consciousness is slower and limited, but it can do what the unconscious cannot. According to the global workspace theory, which the psychologist Bernard Baars developed and the brain researcher Stanislas Dehaene supported with experiments, consciousness makes a piece of information available to the whole brain at once. Only in this way can the human being connect it with other things, speak about it, examine it and, in a new situation, act differently from how they usually would. In short: the unconscious ensures that the human being functions in the familiar, consciousness ensures that they can choose in the unfamiliar. A study that will be discussed shortly also draws on this theory.

Two questions arise for the machine.

The first is: does a machine have consciousness, or at least something like a subconscious? In 2023 a research group led by Patrick Butlin examined which features, according to the leading scientific theories, point to consciousness. Their result: no current system is a strong candidate for consciousness, but there are also no obvious technical obstacles to building systems that satisfy these features. The philosopher John Searle, by contrast, held that programs alone are never sufficient for consciousness. Ed Zitron, a critic of technology, says in brief that today's systems are not conscious actors. A machine does not have a subconscious in the human sense. But it does have hidden processes that even its developers do not fully see through and that determine its behaviour. Section 9.4 shows how this can manifest itself.

The second question is more pressing for the subject of this book. Systems today can recognise patterns in the behaviour, the language and the decisions of human beings that these human beings themselves do not see. Who may recognise the unconscious patterns of a human being and mirror them back to that person? Sundaram names two authorities: God, who searches the heart, and one's fellow human being, who corrects kindly and lovingly. Both stand in a relationship to the one whom they recognise. A system stands in no such relationship. It can recognise patterns without knowing the person, and it can mirror them without loving the person. Whether this helps or harms depends on who deploys the system, for what purpose and with whose consent.

The author of this book has sharpened this question in an essay of his own. The attention economy asks how it can win a human being's gaze. With artificial intelligence a next stage emerges, an economy of intentions: systems infer from context, conversations and habits what someone is likely to want next, and prepare decisions before that person has explicitly formulated them. This can lighten the load. But it makes urgent a question that arises with every system that mirrors patterns: in whose interest does it act?