the algorithm of the soul

This is the world’s first preserved photograph. The picture was taken in 1826 by Nicéphore Nièpce in Saint-Loup-de-Varennes in France on a pewter plate coated with natural asphalt. The process Nièpce developed for this purpose is called heliography, the „drawing of the sun.”


I waited in the upper level of a small café and looked down at the old town’s rainy pedestrian zone. Which of the many people might Thiel be? Under all the umbrellas and pulled-down caps, I could identify only a few indviduals beyond the water-streaked windows. I was looking for a scientist back then, but what I found was a man with a story. No, more than just a story—a discovery. And, actually, he found me. He was standing next to me when I was still staring out the window, lost in thought. I hadn’t seen him coming; he stretched his hand out of his wet cloak to meet me.

„Thiel,“ he said.

I had researched him online and found him as Albrecht Thiel on the website of the staff of a university institute. He worked there as a senior software developer.

His first e-mail had been short and didn’t let me know what it was all about. He had a job for me and wanted to meet me personally. So far it had been nothing unusual; I worked as a freelance journalist and copywriter for various clients: smaller agencies and publishers, sometimes advertising and press releases directly for companies. In addition, I wrote my own literature blog, though it was not very well received. At first I thought about editing scientific articles or a private biography, but in the appendix he had sent a contract with a confidentiality agreement. I was not to talk to anyone about this matter and I was not allowed to publish anything without his explicit consent. That made me curious. And so, three days after the first e-mail, I sat next to him in the café with the signed contract in my bag.

Thiel was a quiet little man in his 40s. Slender, but by no means untrained, unshaved, but not neglected. He looked at me attentively through small glasses. I shook the hand he offered to me and stood up to greet him.

„Pleased to meet you.“

Unlike with many other customers, this was not a lie. It had been clear to me from the very first e-mail that this would not be an ordinary order and I hadn’t been sceptical for a second about the fact that the e-mails did not mention the topic of money.

He took off his jacket, hung it over the back of the chair and sat down. We ordered coffee and I pushed the contract towards him over the table. He flipped through it briefly and nodded. I couldn’t control my curiosity.

„What exactly is this about? An invention?“

„Yes and no. The essence is more about something else.“ He paused.

„You’re a scientist?“

„No, not really. I studied mathematics and computer science and I work as a software developer. But that’s not really what it’s all about,“ another pause, „or only marginally.“

„All right. Why me?“

„You ask a lot of questions.“

„I thought that’s why I came here.“

„And I thought journalists listened first and foremost.“

He smiled at me, then the coffee arrived. Mine was black, his contained frothed milk. I held my peace.

„I read some of your articles and found your blog. They are very critical and have torn up some of my favourite authors‘ works—perhaps not unjustifiably so.“

„Why don’t you ask a colleague, someone who knows the subject matter? Or a lawyer or consultant?“

„My order,“ he emphasized the word like someone who usually didn’t place orders, „is a little unusual. I don’t need a lawyer and no one in the business. I need a critical opinion. Someone to stop my fantasizing. Though I guess sooner or later it’s certainly something to write about. In addition, there’s a lot of research to do.“

He dragged the contract over to himself and tapped it with his hand.

„I have your word that this is between us?“

„I promise. You have it in writing.“

He put the contract in his backpack, which I hadn’t noticed until then.

„Is it about money? Why this contract?“

„No, I’m not really interested in money. I realize, of course, that you’re here for the money in a way, even though I hope that’s of secondary importance to you.“

„I’d have to know what this is about.“

„It’ll take some time for you to really know what it’s about. But you have my word it will be worth your time.“

He paused and I thought about how he could make this sound like a serious offer.

„How about this? You have the choice. Either we set a fixed hourly rate that you can charge, or you get all the proceeds your texts generate.“

„All right,“ I said, „all right. If I stick to it, I’ll keep all the rights to my work.“

„You’ll stick with it,“ he said and reached a hand out to me again.

I struck.

In face recognition, a distinction must be made between locating faces within an image and their assignment of the face to a concrete person. The first case is used, for example, by modern digital cameras to automatically focus on the people displayed. By recognizing whose face is on a picture, one typically distinguishes between recognition by a human (called „face perception“) and recognition by a machine („face recognition“).



Thiel’s task at his institute was to eliminate this difference. He was working on a computer program for face perception. His program was intended to imitate human facial recognition: „As a rule, we human beings are able to recognize faces after many years, even if the person has changed a lot in the meantime. This memory performance is made more difficult for us humans by the factor of forgetting. But even though we have almost forgotten what someone looked like over the years, we often recognize the person after a few moments of reflection.“

For computer programs, the opposite is true. One can save a person’s image over decades without losing it by forgetting it. However, they compare faces using certain parameters the program specifies: the distance between the eyes or the proportions of the mouth, nose and ears. That’s why computer programs can handle the task better than we humans can if someone dresses up, wears glasses or grows a beard. Disguises can’t change the parameters that machines compare.

However, these parameters change over the course of one‘s life and, therefore, machines do not typically succeed in comparing a child’s photograph with a picture of that person as an adult.

„Do you know this one?“

„Where did you get this?“ I asked, surprised. It was a picture of a class trip, more than 20 years old. Four young people on an old sofa in a youth hostel. I looked pretty hungover, the second from the left. The other three were classmates I had long since lost track of.

„And do you recognize anyone here?“ He put a second picture next to it, quite similar to the first. Three youths sat at a table and played cards, apparently also in a youth hostel or youth centre.

The photograph was a little older than the picture of my classmates and me; I could easily tell this by the clothes the boys wore and the room’s furnishings. I didn’t know anyone in that photo. I looked up at Thiel questioningly and then again at the picture.

„In the middle with that big smile, that’s you?“

„I was good with cards, but I had a lousy poker face.“

„In fact,“ I grinned. „So, where did you get the picture of me?“

Thiel told me about his work at the institute and his face perception program. He had found a halfway up-to-date photo of me on the net and fed it into his software. The program searched the internet for more pictures of me, current as well as old photos from my childhood and youth. Of course, he had found the picture that lay ahead of me on Facebook. However, the site mentioned no names, and no persons were identified or linked. That’s why I probably would never have found the photo by myself.

„And you can do that with just about anyone?“

„In principle, yes. However, there are still relatively few pictures of our generation on the net.“

I was thinking of some acquaintances who shared their children’s photos on Facebook every day when Thiel put an old newspaper photo on the table alongside the two other pictures. I recognized it right away. I had cut this picture out of a newspaper more than 30 years ago and pasted it into our family album.

„You won third place in the balloon competition at the age of five.“

„One hundred and sixty-nine kilometers.“ I grinned.

“This and a photo of your first day at school, also from the newspaper. That’s all I could find. You were a teenager in all the other pictures.“

„I don’t know if I’m supposed to find this scary or fascinating.“

„It’s more than both,“ he told me seriously, but I didn‘t learn anything more about it that day. He had to sort out a few things before he would let me in on his secret.

In 1888 the American inventor George Eastman developed roll film. With the „Kodak No. 1,” which was released one year later, it became possible for the first time to create several photographs, one after the other, with a small compact apparatus. Eastman‘s customers were able to fill the entire film in their Kodak No. 1 and then send the complete camera to Eastman by post. Eastman developed the photos for his customers, offered prints of the images and sent back the camera equipped with a new film reel.

About 100 years after Nicéphore Nièpce had taken his first photo, the „Leica 1“ developed by Oskar Barnack entered the market in 1925. It was the first 35mm film camera produced on a large scale.


Of course, I wanted to test his software. A week after our first meeting, I had taken my old photo albums from the basement and searched the internet for pictures of more or less famous people. Then I spent a whole day in the library. I searched old books with illustrations, especially from the first 70 years of the last century. I looked for photographs that hadn’t found their way onto the internet and I scanned many pictures in which faces could be easily recognized. The longer I searched, the more I concentrated on „visitors“: supporting actors, people who had been accidentally photographed but who were still in focus and large enough to be recognized. Could I use Thiel’s software to find out who these people were and what became of them? Whether they could be found in other photos?

In the age of digital photography, one can hardly keep from being photographed. Driving licence, identity card or health insurance card—all with pictures. Everyone has a high-resolution camera everywhere thanks to smartphones, not to mention countless surveillance cameras. Even if most of the population would not voluntarily photograph themselves using selfies, only people who had hardly any contact with the outside world remained unphotographed. As a rule, this was not a tragic situation. No one but Thiel would have found these childhood photos of me—at least not someone who was looking specifically for me.

How many photos did I happen to appear in because I had walked into the picture or stood in the background, completely uninvolved? Certainly I was a visitor in strange pictures I had never seen before.

And how many of my friends, who were still largely avoiding social networks, were on photos that had been uploaded and shared there? To be honest, as long as names didn’t show up, I wasn’t too worried about who was in the photos I had more or less publicly shared with my friends. Now you wouldn‘t need names anymore. Any user of Thiel‘s software could find all the photos anyone had ever taken of a person and posted publicly on the Internet.

The Facebook Artificial Intelligence Research Lab (FAIR) is developing its own intelligence for the world’s largest social network. One of the numerous projects the scientists are working on is an intelligent assistant that analyses and supports communication between users. For example, the program should be able to recognize whether people on uploaded photos are drunk. The intelligent assistant could automatically withhold these images from the public and be a good conscience for those users who cannot trust their own conscience at the time of the automatic upload.


Twelve days later, once again I waited in the café for Thiel. I had my picture collection with me. I had spent an entire weekend, more or less, digitizing old photos with a scanner I had bought especially for this purpose. Not only public pictures from the library were present, but also many private pictures from my own collection: photos of friends and relatives and from school holidays and family celebrations. I didn’t know if it was reprehensible to find out what my first girlfriend looked like today or what my old school buddy had done after he left primary school. I had previously searched for both of them on the net, but I hadn’t learned much about them. Would that change now?

This time Thiel didn’t greet me with a handshake, just sat down at my table.

„Hello, how are you?“

„All right, thank you. I’ve become quite curious, actually.“

„I’m glad to hear it. I brought you something, too.“

He set some photos on the table again. There were a handful of prints of pictures, apparently from the Internet. I didn’t know the people in them, but I knew all the places where the pictures had been taken. And then I discovered myself in the pictures. They were actually „visitor“ pictures of me. Mostly couples or single persons stood in the foreground, while I was in the background—on the beach in Spain, in a large crowd of people in a bazaar in Tunisia, on a motorway stop somewhere in the no-man’s land and in the pedestrian zone only 500 meters from the café in which we were sitting. They were pictures from the last 15 years.

„I was wondering if something like this would work.“

„Yes, it’s really amazing what is freely available on the net.“

„Is that all you found of me?“

„These are all category two results. I secretly call them the ‚this is!‘ category.

My gaze made a consultation unnecessary.

„As I said before, the program doesn’t actually work like a machine, but more like a human brain. It does not compare features like other face recognition software does, and therefore it also does not display tomesting like a percentage match. Instead, other factors play a role, similar to what we humans do. The better you know a person, the more likely you are to recognize them. You might even recognize your girlfriend or your father as a silhouette. If you see me down there in the pedestrian zone, you’ll probably have to look twice before you’re sure. There are two main factors: How exactly do you know a person and how much information is available for recognition. This means the more data available and the better the program is trained, the more it can recognize.“

The coffee arrived and Thiel continued his explanation:

„Helios has been instructed to divide the results into five categories, just like humans do:
„Category one, you’re absolutely certain: ‚This is Mr. Smith!‘ This is the category of pictures I took at our last meeting.
„Category two, slight doubts: ‚Is this Mr. Smith?‘ As a rule, not enough information is available here, for example, because the person is too far away or you have known them only briefly. These are, for example, pictures in which you are cut off or standing behind other people.
„Category three, bigger doubts: ‚Isn’t that Mr. Smith?‘ Here, people who look like you would also be considered—siblings or parents, for example.
„Category four, probably no match, but a lot of similarity: ‚He looks like Mr. Smith.‘
„In category five, there is no longer any resemblance.“

I took the pictures he had brought with him and looked through them. I faltered at that picture from Tunisia. I stood in the background in the middle of the turmoil. My face was only a few millimetres tall and was partly covered on the fume cupboard. I noticed I was bigger and lighter than most people, but I doubted friends or relatives would recognize me in the picture.

„My parents would recognize me in this picture if I put it in their hands, but if they saw it somewhere on the internet or in a magazine, they would never think this was me, even if they looked at it carefully.“

„That may be so. On the one hand, the available context data on the Internet is becoming more and more precise. Helios could already have the information about when and where the picture was taken and who was there at that time, just like your parents would. That would make things even easier, of course. However, the picture is already some years old. At that time, it was not as easy as it is today to determine the current situation. The other way around, we’re not interested in context data, but in facial recognition.“

„Helios is the program?“

„More an artificial intelligence than a simple program.“

„And it can recognize me better than my parents can?“

„In a way, yes. Of course, your parents are the experts in recognizing you because nobody knows you as well as your parents do. Helios is a self-adaptive algorithm. It trains itself. Because it has analysed billions of faces and their development, it has always learnt something new. That is why we believe Helios now works better than the facial recognition of a normal person. In addition, it can combine its facial recognition results with the usual digital methods of facial recognition or, to a lesser extent, with a context-based search. This makes Helios better than your parents and traditional internet research combined.“

„Can I try that?“

In the past few days, I had, of course, thought about who might be the best candidate for a first search. Nothing private because that might give the wrong impression. It had to be a person for whom a lot of material could be found while at the same time challenging the software—a well-known person who has changed greatly over time.

I tried to study Thiel’s facial expression carefully and inconspicuously while I opened a first image file in his program. It was a photograph of the young Michael Jackson from the late 1960s. Thiel didn’t show any movement. The program immediately suggested many more pictures and video sequences, which all clearly showed Michael Jackson. Interestingly, the first results showed the pop star at very different ages and in varying health conditions: two pictures and a video from childhood or adolescence with the Jackson Five, two pictures of the adult Jackson some time between „Thriller“ and „Bad,” some close-ups of the pop star disguised as a zombie and a picture in which he must have been almost 50, in which he seemed sick and emaciated and wore black sunglasses.

„If there are incorrect pictures, you should sort them out; that way, the program gets to know the person better. You could also enter text information, such as the name or date of birth.“

I scrolled through the pictures; more and more correct results appeared on the display.

„If you switch on standard face recognition, the images are also searched using biometric data, as known from conventional software. In addition, Helios can find images about the context in which the photo is located.

„In this case, the difference between the search methods is not very big,“ Thiel explained. „When it comes to celebrities, these images can often be found on the net and are often in the correct context. The results of text-based searches, biometric searches and Helios are very similar, fortunately, because the known search methods usually deliver very reliable results.”

„That means I could have typed ‚Michael Jackson‘ in Google and clicked on Image Search?“

„Not quite. You can filter out the differences by subtracting the search results from each other.“ He opened another window hidden behind a button. „These are the results of the image search if you enter the name ‚Michael Jackson‘.” A list of numerous pictures appeared and became increasingly longer. „It’s like Google’s image search. If you take off the pictures that Helios also picked out, the following pictures remain.” The program accurately filtered out the real Michael Jackson, leaving only drawings, caricatures and photos of imitators.

„As you can see, Michael Jackson is a good example of Helios‘ capabilities, not only because he has metamorphosed over the years, but also because there are many doubles of him. A text- or context-based search produces too many results because of these doubles and the drawings, while biometric facial recognition is not enough because it cannot bring the young, curly-haired and dark-skinned Jackson into harmony with the adult pop star. Its characteristics have changed too much as a result of natural growth and the supposed operations or diseases. However, it also shows that the face perception of Helios is better than that of many people. Who would recognize the adult Michael Jackson from a child’s photograph if he wasn’t a star whose life story is well-known?“

The program also identified Michael Jackson’s age and growth in the pictures. With a few more clicks I was able to sort the photos chronologically. The list began with a few pictures of the toddler, followed by hundreds of shots taken when he was part of the Jackson Five.

In 1969, the physicists Willard Boyle and George E. Smith developed the first charge-coupled device (CCD) at Bell Labs: an electronic component that can convert light into electrical energy. Originally designed for data storage, it quickly became clear that such components could be used to capture two-dimensional images.

In 1973 the first commercial CCD sensors with a resolution of 100×100 pixels were produced. Two years later, such a sensor was used in Kodak’s „portable all electronic still camera.” The prototype, developed by Steve J. Sasson, recorded the images on a magnetic tape cassette, making it the first functioning precursor of the digital camera.

The first real digital camera whose images could be transferred directly to a computer was the „Model 3/4“ by Dycam, which was presented at CeBIT in 1991 and was able to record black-and-white images with a resolution of 376×284 pixels.



Thiel gave me a notebook and I experimented with his search engine for a few days. At first I bombarded him with questions about how to use the software, but eventually I had the program under control and my initial enthusiasm had subsided. I had indeed found some of my schoolmates on the web using old photographs, even if they hadn’t used their real names. However, this was far less exciting than I had expected. The same was true for the search for „visitors“: The search engine returned impeccable results, but no exciting stories.

No doubt about it, Thiel had done a great job with his program. With his search engine one could easily find a person using only an old photo. Often, it was then easy to get additional information; Helios would let many an alibi fly. Thiel’s search engine was certainly a valuable research tool, but did that fact justify this secrecy?

I was experimenting less and less with the software and instead thinking more about my role in this project when Thiel called me unexpectedly and invited me to come by his institute. The next day, in front of the institute, I was even more sceptical while I rang. It was late afternoon and nobody seemed to be there, but after a short time Thiel opened the door.

„Glad you could make it. How did you get along with the software?“

„All in all, you’ve done a great job!“

He invited me in and led me through the hallways of the institute.

„I’m glad to hear it. Have you noticed anything? Found any bugs?“

„Bugs? No, I didn’t notice. I wasn’t looking for any either, to be honest.“

This question surprised me. In fact, I hadn’t noticed any bugs during my work with the program. Only in the search results did I encounter one exception that didn’t fit. I had fed the machine a photo that a friend had recently sent of her baby, who was a few weeks old. Surprisingly, the software found additional images of the newborn child that the proud father had posted on the net. However, among the results was an old image showing an adult woman working as a seamstress in a factory. It seemed that the photo had been taken with a plate camera around the turn of the century. I was surprised at first because it was the only obvious mistake the software had made. On the other hand, I had not expected the search engine to deliver only correct results. However, when I told him about it, Thiel seemed to be very interested in this case.

„That sounds like a category zero case we haven’t discovered yet.“

„Category zero?“ I asked. „Are those the wrong results?“

„Those are the cases that Helios thinks are right, but that we humans think are wrong. But what is right and what is wrong, and who decides that? If you are looking online for an espresso machine, you can find everything from a simple espresso maker for the hotplate to a Cimbali for several thousand euros. You can find special offers, reviews or stories, pictures and videos of brand-new or ancient espresso machines. These would all be fitting results. However, you should not find any content about washing machines, as that would be incorrect. Concerning the software, this is somewhat different, as it is not intended to recognize and distinguish people in general, only individuals. It is not always clear whom a picture shows, just like you can’t always tell at first sight whether that’s Mr. Smith back there. This is why we have divided the results into five categories. Helios is usually set in such a way that it outputs results of only the first category, i. e. sure hits. Category two is quite exciting, as you yourself have noticed, but we wanted to concentrate on only the sure hits.“

We had arrived in a small office in the institute’s basement. Thiel pointed me to a chair and then sat behind an old desk covered with several files and papers.

„However, there have been cases that obviously didn’t fit, just like your case.“

„Admittedly, I was a little surprised the picture didn’t fit at all, but on the other hand I didn’t expect a search engine to show only correct results.“

„Helios is not really a search engine. As I said before, it is an artificial intelligence which should recognize people as only humans would. You just recognized me at the door, just like yesterday on the phone or at the cafe a few days ago. If I had a twin brother, we might have been able to fool you for a second. However, if an elderly lady had opened the door for you, you probably wouldn’t have thought it was me. You would have asked for me. You might have been able to pretend I was this elderly lady, but that would have been very unlikely. In no case, however, would you have been convinced without a doubt that I was the elderly lady.“

I nodded and looked around the office. It was more of a storeroom and didn’t seem to see much use. Apart from the desk with the files, the room contained little to suggest anyone really worked there. The furniture didn’t match and I saw no personal items—just a stack of papers on the desk, a laptop and a lamp, and another table with two different kinds of chairs.

Thiel’s argument was understandable, but at the same time it seemed a bit petty to me.

„To be honest, I don’t have faces either.“ I tried to lighten the situation a bit. „I can’t tell Elijah Wood from Daniel Radcliffe, for example.“

„The two of them look alike at best; that’s a case of category four. The software can easily classify this. We have, of course, tried to analyse the problem on the basis of such cases. A first version of our software, which didn’t work with artificial intelligence, made some small mistakes, but didn’t produce such unambiguously incorrect matches. However, it has otherwise provided few noteworthy results and has worked more like conventional mechanical face recognition. The artificial neural network we’re using in this release doesn’t make any additional small mistakes, but it can’t distinguish between category one and category zero.“

Thiel set down some pairs of pictures that obviously didn’t match.

„Helios is convinced these pictures show one and the same person. First of all, we feared our artificial intelligence would suffer from prosopagnosis, i.e. facial blindness. I then brought a friend on board who is a neurologist: Prof. Dr. Zimmermann. Together, we’ve developed a theory to explain these cases. I’d like you to consider this theory.“

As early as 1943, the neuroscientists Warren McCullogh and Walter Pitts developed a mathematical model of a nerve cell. Several of these so-called „McCulloch pitts cells“ can be linked to an artificial neural network. Only 15 years later, the first neurocomputer was able to recognise simple numbers.

To date, artificial neural networks have been used to detect structures and solve general problems. In the Google DeepMind project, researchers have expanded an artificial neural network with a short-term memory to simulate the abilities of an artificial memory. This means DeepMind can not only process data, but also save the results and access them later. This „neuronal turing machine“ can sometimes recognize correlations between different data and structures better than we humans can.

The title picture of this chapter shows the photo of Nicéphore Nièpce, into which DeepMind has interpreted his ideas of well-known patterns—a process Google calls „DeepDream.”


The storage room became my office for the next few days. I wanted to have a look at the „category zero cases“ and do my own research—develop a theory of why the program was sometimes so noticeably wrong. It was clear I could not analyse the artificial intelligence, i. e. the technology behind face perception. Still, I was still curious. Thiel had guided me through the institute and introduced me to some of his colleagues. The institute did not have its own computer center; Helios was running on a quantum computer in the United States.

Prof. Dr. Zimmermann remained abroad for the first few days, so I was unable to get to know him. Therefore, I first had a look at the 63 category zero cases known to date. On the other hand, countless other searches apparently showed only correct matches.

The files on the desk contained the first 62 cases. I added my discovery and went through each case step by step, carefully looking at all the images and re-examining the relationships with the help of the software. The results remained the same no matter which images I used to start the search. However, a first sample was immediately recognizable: All pairs of pictures consisted of several relatively new photos on one side and a few fairly old pictures on the other. The first pictures showed rather young people, while the latter were mixed. Gender, skin colour or nationality did not seem to play a role in the combination of old and new images. Case six, for example, consisted of a combination of almost 30 images of a dark-skinned girl between the ages of about 10 and 14 and a single old black-and-white photograph of a light-skinned man, about 40 years old, standing in front of a large machine that looked like an ancient printing press.

Case 15 showed only two pictures of a child only a few weeks old, in the arms of who I assumed were his father and mother. Because of the blue romper, I assumed the child was a boy. On the other hand, there were 11 pictures of a stately lady, photographed on yellowed, scanned photographs in different accompaniments and at different places—with the family in front of a manor, with her husband in uniform or alone in an armchair in the living room.

What kind of connection could exist between such different people?

I tried to find out who these people were, but it was not easy, especially with the old images.

It quickly became apparent that no relationship existed between the persons; for example, the skin colour and origin were too different. Nevertheless, I started describing the persons in the old photos as „ancestors“ and the persons in the new photos as „descendants.” For the old pictures, I was able to find a name for eight of the 63 ancestors, while for three I was able to find a job and place of residence at that time. Finding the names and places of residence for the descendants was usually no problem.

I also spent about two hours every day looking for new category zero cases. I conducted numerous searches on the basis of people I found in ancient pictures from my collection. In the following days, I discovered 14 additional ancestors who had a descendant. For another of these ancestors I learned the name and place of residence. However, I could see no pattern here either.

I had to learn more about the scanned persons and, for the ancestors, I couldn’t do this over the internet. I had to browse old, analogous documents, look at family trees or register entries from registry offices, or search for lists of workers in old companies.

The first scientific publication on the subject of ‘soul blindness‘ in humans was published in 1886 by the German neuro-opthalmologist Hermann Wilbrand.

The term probably comes from the German physician Hermann Munk, who was already concerned with the processing of stimuli by the visual cortex of dogs some years before.

The disorder in the processing of visual stimuli by the brain, nowadays referred to as visual or optical agnosia, means people are unable to recognise objects or faces even though they can see them. The symptoms vary but typically, affected people can describe unknown objects in great detail, though they cannot recognize or name them.


Before I started my research trip, Prof. Dr. Zimmermann returned from his trip. I met him for lunch. While Thiel was rather a mystery-monger who wanted me to draw my own conclusions in this respect, Zimmermann told me openly about his theory:

„Thiel approached me at the time because he feared his artificial intelligence was suffering from a neurological disorder. I’m not a programmer, but a doctor and neurologist, so I thought I couldn’t help him. But his artificial patient quickly caught my attention.“

„And could you help him?“

„No, not really. It was clear from the outset that the symptoms the network is showing are by no means indicative of a neurological disorder.“

„Thiel thought the software might have some kind of facial blindness?“

„Yes, that was a first guess. There are various forms of prosopagnosis, but this is something quite different. In principle, Helios can recognize faces and assign them ages and genders. In this respect, Helios is much better than the facial recognition of a healthy human brain. Even semantic information can now be recognized, although this is not necessary for recognition and people still draw much better conclusions. Still, we currently suspect it’s not the software, but rather we humans who are suffering from some kind of agnosia.“

„That means?“

„Agnosia generally means something like ignorance. There are many forms of agnosia. Some people are unable to recognize or correctly assess movements, directions or speeds. This is called acinetopia, or movement diagnosis. Others do not recognize voices or noises correctly. Prosopagnostics have difficulty recognizing faces. In the past, this form of agnosia was called ‘soul blindness‘ in German, although it has nothing to do with the recognition of the soul. However, it’s possible Helios can recognize more in the pictures than we humans do. “

„You mean, Helios recognizes not only faces, but also…“

.”.. the soul of a human being, or whatever you want to call it. It’s hard to imagine, of course. Try to explain what colors are to someone who can’t see colors. And compared to some animal species, we humans are agnostic. We cannot imagine what a dog smells or how a bat perceives its environment. However, the things that a dog smells or that a bat locates in absolute darkness by ultrasound are there even if we can’t recognize them. So, it’s possible that Helios is fine while we humans are suffering from some kind of soul blindness.“

He took a short, thoughtful break.

„The face is a mirror of the soul. Maybe you must take that more literally than you thought. But, of course, that’s just a theory Thiel, Bishop, Ludwig and I share.“

„Who are Bishop and Ludwig?“

„Thiel hasn’t mentioned them? To test this theory, he brought on board other scientists alongside you and me. Bishop is a neuroinformatician at MIT and Dr. Ludwig is a historian and theologian. This theory is more than just interdisciplinary.“

„And what have you found in the meantime?“

„My tasks in this project are rather small. I’ve known Thiel for a long time, so he introduced me to this project at the very beginning. However, it was quite clear I couldn’t help him much. Together with Bishop, I am working to find which parts of the neural network are responsible for identifying and classifying the ‚ancestors‘, as you call them. If the theory is wrong, you could remove these parts so the software doesn’t make these mistakes anymore. But we’re not there yet. I suspect that even with artificial intelligence, one should be careful when removing tissue.“

„And what if it’s not a mistake?“

„Bishop then hopes he will find a clue in these parts of the network and determine the algorithm that recognizes this ‘soul‘. As far as that’s concerned, you might want to talk to Dr. Ludwig. He’s much like you, working on finding evidence for this theory at the moment.“


The algorithm of the soul


Together with Dr. Hartmut Ludwig, I set off on a journey to search for information about the ancestors in old documents, in analogue databases, in registries of the registry office, and with employers and relatives. Of course, the yield was meagre because most of the ancestors had presumably died about 40 to 80 years ago.

In his work, Hartmut had found another 31 cases of which I didn’t yet know. In total, we had 108 ancestors about whom we wanted to find out more. However, we knew only 12 ancestors by name. The other 96 cases were hopeless. Of these 12, only three ancestors remained after two and a half weeks of research, during which we were able to sketch a brief curriculum vitae: a Dutch jurist and criminal scientist who had lived from 1868 to 1934 and obtained his doctorate at the University of Leiden in the Netherlands. We first found his documents in the university archives and were able to locate his granddaughter in The Hague. She had a detailed family tree and additional pictures of her grandfather as well as several of his legal publications, which she showed us.

Ancestor number two was a British naval officer who served on a destroyer to her Majesty’s which sank in 1918 during a collision with a cargo ship in the English Channel. According to official reports, the entire crew died when the ship sank.

From a picture book, I had scanned the photo of the third ancestor for my research. It showed a neatly dressed young lady at an office desk. The caption mentioned the name of the woman and her company, a joinery in Melbourne, Australia. The carpentry workshop still existed and we received information by e-mail from the managing director of the company, which had been in the family for four generations. The wife was the daughter of the company’s founder, who later became managing director herself. The current head of the carpenter’s workshop gladly gave us information about his grandmother, who lived between 1899 and 1974 and had a total of four children, nine grandchildren and 15 great-grandsons and great-granddaughters.

At the same time, we searched for the three descendants of these ancestors. As hard as we tried, we could not find the descendant of the Australian. It was a small child, perhaps two or three years old. Helios had found the pictures on Facebook, but the profile on which the pictures had been posted was not public. The user did not respond to our request and all traces ran dry. However, we were lucky with the other two offspring. The descendant of the English naval officer, who was barely 60 years old, lived in the United States and worked as a plumber in a small town near the east coast. He maintained his Facebook profile with all the information we needed and he sent us more information and pictures from his youth.

The descendant of the Dutch lawyer was the wife of an employee of the Institute. Thiel had fed into Helios a photo of a company outing, and the woman had appeared in the photo. She was in her mid-40s and worked as an accountant in a company just a few kilometres away. We paid her a visit and she also gladly gave us more photos of her youth. She didn’t know anything about the Dutch lawyer and he didn’t appear in her family tree.

It was extremely unlikely that all these persons were related. The English naval officer had neither wife nor children, and the Australian had no illegitimate descendants. Based on the new pictures we had collected, we started some new searches, but these also provided only connections between the ancestors about whom we already knew. Helios was still convinced that the English officer and the American installer, as well as the Dutch lawyer and the employee’s wife, were one and the same person, although these persons could hardly be more different.

Nevertheless, when we compared dates of death and birth, we found a single match between the two couples: both offspring were born exactly 15,027 days after the death of their ancestor. Was that the confirmation of the theory we were seeking? Did Helios recognize a soul in the pictures that shared ancestors and descendants and that, after exactly 15,027 days, manifested itself in the birth of a completely different person?

As a theologian, Dr. Ludwig had great doubts. Could this be a coincidence? Could there be another explanation? We calculated the birthday of the Australian’s young offspring, which we couldn’t find. Here, too, the time frame was more or less perfect. From the date of the Australian woman’s death on May 17, 1974, the child must have been born 41 years later, on July 8, 2015, according to our theory. In fact, weeks later we were able to track down the little boy because of his birthday.

Meanwhile, we continued our research. Prof. Dr. Zimmermann and Carl Bishop tried to unravel the algorithm by identifying those parts of the artificial neural network that identified the ancestors and descendants.

Dr. Ludwig finally created the thesis that not only can the face be a mirror of the soul, but also a person’s other qualities: „Think of the handwriting or the voice, or the way of expression when writing texts, creative ideas or patterns of thought and behaviour. Maybe there are countless mirrors of the soul,“ he said.

But how were we to find these mirrors if we had to assume data that had to be at least 60 years old? Surely it would take months or even years to find additional evidence. However, in the long run, time would help us. Humanity was gathering more and more precise data. If we could find isolated cases today, how easy would something like a „soul scoring“ be in a few decades, if even the ancestors provided us with huge amounts of digital data?

Appendix A


Excerpts from the transcript of the first Ethics Board meeting of HELIOS Inc. The meeting was held excluding electronic components and was recorded on a Remington Standard 10 typewriter.

„The state of things is we have found further evidence to support the theory of kinship. Thanks to Carl’s algorithm’s limitations, we can now search directly for matches without having to feed Helios with images. We have temporarily increased the capacities in the ai-center and Helios has found 4,278 matches in the meantime“. (A. Thiel)

„The manual and analogous research of these cases is difficult. Nevertheless, we have now found 37 matches where the dates of birth and death are known. The interval between the two dates in these cases is exactly 15,027 days“.

„I would like to remind you that at this moment we will not speak of „soul kinship“ or something similar. Rather, Albrecht and I have agreed on the term „entities“. Meanwhile Helios has also discovered two entities through the analysis of manuscripts, but here we lack the necessary information on the days of birth and death for verification.“ (Dr. H. Ludwig)

„I think we all agree that some questions need to be discussed before the theory can be published. All those present have therefore signed a non-disclosure agreement. Information will only be disclosed if all of us agree to its publication.“ (Prof. Dr. Zimmermann)

„In addition to the theory of entities, there are other explanatory models that we have now put to the test. Only the explanation under the working title ‚pogrom theory‘ has not fallen through a relevant grid and is one of the reasons why we meet here in the absence of electronic devices“. (Dr. H. Ludwig)

„The most important question for us is what happens when we confront the public with our results. The possible scenarios are extremely diverse. One scenario predicts that many people will see evidence of soul kinship or rebirth in these entities. This could be accompanied by a drastic increase in the global suicide rate.
In particular, disadvantaged and dissatisfied people may see this as a way out of their situation and a good opportunity for a new beginning in the future.“ (Prof. Dr. Zimmermann)

„The pogrom theory raises the question of whether Helios is capable of inventing entities and pretending their existence in order to carry out a kind of pogrom against people who are dissatisfied with their current situation and trust in starting a better life in 41 years.
Sceptical people, on the other hand, or those who fear that they may not be as well off in the future as they are today, may simply continue to enjoy their lives without further consequences. One could call the result social- or also psycho-darwinism. „(Dr. H. Ludwig)

„However, a scenario invented by the artificial neural network can only be assumed if Helios has all the necessary information at its disposal, in particular birth and death data. We have not yet been able to clearly determine whether these informations were available digitally before the start of our project. We are therefore still a long way from publishing our theory“. (A. Thiel)

t. b. c