Consciousness, the Subconscious

Most decisions feel as though we make them consciously. We weigh things up, we choose, we act. Yet a large part of what determines our actions happens without our noticing it. We drive a car without thinking about every movement of our hands. We react to a tone in another person's voice before we can name it. We find arguments convincing or not without knowing why.

This chapter first asks which habits and background assumptions shape our actions without our consciously noticing them. To this end it presents the interpretations of Enoch Sundaram and Timothy Keller and then asks what research can actually confirm. Not every statement about the heart is a psychological statement, and not every comparison with a computer is already an explanation of the brain.

Then follows the question that is particularly important for this book: if human beings show recurring patterns, how well can their decisions be predicted? Artificial intelligence can recognise such patterns in language, behaviour and data. It can thereby help to understand a human being better, but it can also help to influence them. The more information someone has about us, the more pressing the questions become of whom this information belongs to and whose interests its use serves. In a language model, too, many technical processes take place that even its developers cannot fully trace (Chapter 13). That is not yet a human subconscious, but it is a reason to ask more precisely about the difference.

Enoch Sundaram, physician and preacher, describes the subconscious with an image taken from technology. It seeks, he says, to save time and energy and forms programs or habits out of repetition: how we drive a car, how we speak, how we deal with spouses and children. All action, according to him, rests on the programs in the heart. That actions become automatic through practice is well known. Sundaram goes further: the subconscious, he says, takes in without selection whatever consciousness gives it, good and bad alike. He compares it to soil into which thoughts fall as seeds and come back multiplied; hatred, he says, produces more hatred, love more love. Perception too, according to Sundaram, lies in the heart: human beings already have preconceived thoughts about everything and decide on that basis.

What Sundaram concludes from this model is remarkable. Not everyone, he says, has the right programs. Wrong programs are recognised above all when others correct one, and through prayer: Search me, God, and know my heart. Correction, he says, should be accepted with thanks, it is like a broom for the heart; and one should correct others kindly and lovingly. The subconscious can, according to Sundaram, be reprogrammed in two ways: through mindfulness, that is, awareness of one's own thoughts, words and actions, and through meditation as deep reflection on God's word. Hypnosis, by contrast, he rejects.

Sundaram's image is vivid. It connects habits, the way we deal with correction and the spiritual question of how a human being can change. In doing so he goes as far as the notion that the subconscious preserves the entire memory down to the finest details. First, this image should be made understandable in its own logic. Whether all the factual claims associated with it hold up is examined in the section on research.

Timothy Keller, pastor and author, speaks not of programs but of background assumptions. Chapter 1 presented his thesis: barely conscious assumptions determine which arguments convince at all. In a thought experiment he shows how much people are reading off a cultural script when they believe they are expressing their true self: a warrior a thousand years ago and a young man of today both regard as authentic what their culture lays down for them. Such assumptions, according to Keller, are seldom discovered alone, but in conversation and in relationships.

Keller also knows of a layer that lies deeper still. The hollow space in the heart, he says, is present in all human beings, even when it is not conscious, for instance in children. What is meant is a longing that a human being has before they can name it.

Sundaram and Keller use different languages: the one a theological program model, the other the image of cultural and personal background assumptions. In doing so they do not claim the same thing about the brain. One shared insight remains: our decisions are shaped by presuppositions that we do not always consciously see. Both consider a counterpart important for recognising such imprints, be it through correction, conversation or prayer.

Research can examine part of what both authors describe: how do habits form, and how do perception and reactions unfold before we consciously notice them? In doing so it does not confirm the theological image of the heart as such; it investigates processes with other methods and concepts.

Chapter 3 gave the orders of magnitude: according to a recent study, the sense organs take in around a billion bits per second, and behaviour processes about ten. The figure itself is disputed and is more an order of magnitude than a constant. But it shows that between what is taken in and what is consciously processed there lies an enormous gap.

This becomes especially clear in the perception of danger. Joseph LeDoux, a neuroscientist, has shown that circuits in the brain in which the amygdala is involved can detect threats and trigger defensive reactions of the body without the person consciously noticing the threat. The conscious feeling of fear, by contrast, arises according to his account in other, higher circuits; the amygdala, he holds, is not the brain's fear centre. LeDoux therefore recommends using the word fear only for conscious experience. The body thus reacts even before the human being knows that they are afraid.

Psychology describes such processes as implicit processes and distinguishes between fast, automatic thinking and slow, deliberate thinking. For this book the common finding suffices: a large part of what we do is not consciously controlled. This is not a deficiency; rather, it is what makes life possible in the first place. No one could walk, speak or drive a car if they had to plan every step consciously.

But it follows neither that the subconscious stores the entire memory completely, nor that the specific figures from Sundaram's talks are correct. For these claims the necessary evidence is lacking, as Chapter 3 has shown. A sound finding and an interpretation that goes beyond it can stand side by side without being confused.

At this point the book must draw a distinction that is not always drawn in the sources. Sundaram equates the biblical heart with the subconscious. That is an interpretation that brings two different languages together.

There are at least three levels at which people speak of what lies beneath consciousness. At the theological level, the heart is the centre of the person, the place where the person turns towards God or turns away from him. At the psychological level it is a matter of habits, implicit assumptions and automatic reactions. At the neuroscientific level it is a matter of processes in particular circuits of the brain. The three levels are connected, but they are not the same thing.

Whoever mixes them gains vividness and loses precision. If the heart is the subconscious, then brain research seems to confirm theology. But if it turns out that a figure about the subconscious is wrong, the same research seems to refute theology. Both are wrong. What the heart is in the biblical sense cannot be measured in a laboratory, and what is measured in a laboratory is not the heart. I will need this distinction also when I ask about the consciousness of a machine.

If so much happens beneath consciousness, is the human being then predictable? I think: to a certain degree, yes. Habits, preferences, experiences and reactions leave patterns behind. Whoever knows such patterns in a concrete situation can predict behaviour statistically better. That is not the same as the complete calculation of a human being or of a single decision. But it is enough to tailor recommendations, advertising and increasingly also the personal answers of an AI system.

Against this hypothesis stands a strong objection. That behaviour can be predicted does not prove that the person is determined. Freedom shows itself precisely in recognising patterns and changing them. In philosophy this view is held under the name of compatibilism: freedom and causation do not exclude each other.

It is remarkable that the sources of this book themselves support the objection. Sundaram, who speaks so much of programming the subconscious, calls choice the greatest capability that God has given the human being, the governing power. His whole teaching on reprogramming presupposes that human beings can recognise and change their patterns. Keller presupposes the same when he says that background assumptions can be discovered in conversation. The sources thus describe the human being as shaped, but not as fixed. Precisely this combination makes them interesting for a learning system: predictable patterns are useful without the freedom of the individual being abolished thereby.

Why human beings examine consciously.

To ask whether a machine has consciousness, one must first ask what a human being needs consciousness for at all, and what they need an unconscious for. The unconscious works quickly, economically and without effort. It takes care of the habitual, so that the human being does not have to think through every movement afresh. The psychologist Daniel Kahneman found a memorable image for this in his book Thinking, Fast and Slow. He describes two kinds of thinking, which he calls System 1 and System 2, expressly as simplifying figures and not as two places in the brain. System 1 works quickly, automatically and without effort: whoever reads “2 plus 2” knows the answer without calculating. System 2 is slow and effortful: “17 times 24” demands attention. How easily fast thinking leads us astray is shown by a well-known puzzle. A bat and a ball together cost 1.10 francs, and the bat costs one franc more than the ball. How much does the ball cost? Most people say at once: 10 centimes. The correct answer is 5 centimes. Fast thinking delivers an answer that feels right, and slow thinking checks it only if one wakes it. Consciousness is slower and limited, but it can do what the unconscious cannot. According to the global workspace theory, which the psychologist Bernard Baars developed and the brain researcher Stanislas Dehaene supported with experiments, consciousness makes a piece of information available to the whole brain at once. Only in this way can the human being connect it with other things, speak about it, examine it and, in a new situation, act differently from how they usually would. In short: the unconscious ensures that the human being functions in the familiar, consciousness ensures that they can choose in the unfamiliar. A study that will be discussed shortly also draws on this theory.

Does a machine have a consciousness of its own?

Two questions arise for the machine.

The first is: does a machine have consciousness, or at least something like a subconscious? In 2023 a research group led by Patrick Butlin examined which features, according to the leading scientific theories, point to consciousness. Their result: no current system is a strong candidate for consciousness, but there are also no obvious technical obstacles to building systems that satisfy these features. The philosopher John Searle, by contrast, held that programs alone are never sufficient for consciousness. Ed Zitron, a critic of technology, says in brief that today's systems are not conscious actors. Subjective experience has not been scientifically demonstrated for today's AI systems. I myself go further and hold the conviction that these machines have no consciousness of their own. Whether other kinds of artificial systems might one day satisfy the features of consciousness remains a research question. In any case, today's machine does not have a subconscious in the human sense. But it does have hidden technical processes that even its developers do not fully see through and that determine its behaviour. Section 9.4 shows how this can manifest itself.

To whom does the knowledge about us belong?

The second question is more pressing for the subject of this book. Systems today can recognise patterns in the behaviour, the language and the decisions of human beings that these human beings themselves do not see. Who may recognise the unconscious patterns of a human being and mirror them back to that person? Sundaram names two authorities: God, who searches the heart, and one's fellow human being, who corrects kindly and lovingly. Both stand in a relationship to the one whom they recognise. A system stands in no such relationship. It can recognise patterns without knowing the person, and it can mirror them without loving the person. Whether this helps or harms depends on who deploys the system, for what purpose and with whose consent.

The author of this book has sharpened this question in an essay of his own. The attention economy asks how it can win a human being's gaze. With artificial intelligence a next stage emerges, an economy of intentions: systems infer from context, conversations and habits what someone is likely to want next, and prepare decisions before that person has explicitly formulated them. This can lighten the load. But it makes urgent a question that arises with every system that mirrors patterns: in whose interest does it act?

That human data can also be a commodity is not a theoretical idea. In February 2024 Reddit stated that Google would receive structured access to public posts and comments via the Reddit Data API, also in order to improve products and the training of models. In May 2024 Reddit and OpenAI announced a partnership with access to Reddit content. The company Reddit itself was not sold in the process. What is at stake is the economic value of access to what people have written.

Three levels must be distinguished here. Publicly posted texts can serve as material for search and model training. What I tell an AI assistant about my concrete situation helps it give more fitting answers; that is not automatically public training data. And platforms can draw inferences from usage behaviour and content that are valuable for advertising and other business models. Not every AI system uses or sells personal information in the same way. But the more a provider knows about people, the better it can recognise patterns and the greater its ability to influence decisions. The question of power is: who may use this knowledge, for what purpose and with what consent?