Performance Failure as Key to Measuring Degree of Intelligence
Why intelligence is best measured not in terms of performance success but in terms of performance failure.
This brief article comes from the opening of a long technical article on intelligence metrics and how these can be used to measure degree of intelligence. I intend to submit the full article to Bio-Complexity. This brief article merely gets the ball rolling. I’m presenting it here because it should interest a general audience and not just technical readers. Note that section 2, which is on the connection between intelligence and information, appeared in an earlier Substack post—I’m including it here for clarity and completeness.
1 Introduction
Intelligent design (ID) to date has focused more on whether something is designed than on the degree to which it is designed. ID has thus tended to treat design as an all-or-nothing proposition: either it’s designed or it’s not designed. Ignorance may be included under “not designed,” as when we are unable to determine whether something is designed.
This all-or-nothing approach to design affects how we use probabilities in drawing design inferences (see the second edition of The Design Inference). Thus, design theorists will typically set a probability threshold so that specified things whose probability is smaller than that threshold trigger a design inference but things with larger probability don’t. Failing to meet the threshold cannot rule out design because designers can purposely mimic chance. But failing to meet it will fail to confirm design.
For instance, imagine a group of exactly five people seated at a restaurant table. The birthday of each of them is January 1. No one would think that these people just randomly congregated at this table. Assuming birthdays are equally distributed throughout the year and ignoring leap years, the probability of them all having this birthday is roughly 1 in 6.48 trillion. Such a level of improbability is much more plausibly ascribed to design than chance.
The concern might now be voiced that the probability so calculated is too small because there’s nothing special about January 1. Thus these “birthday pals” might have all shared a birthday on any other day of the year. But factoring in those additional days would only raise the probability to 1 in 17.75 billion, and so we would still be disinclined to ascribe this coincidence to chance.
Of course, if there were exactly ten people at the table sharing a common birthday, the probability would get much smaller still, and a chance coincidence would seem utterly implausible. But in an example like this, we are unconcerned about the degree of intelligence needed to bring together people sharing a common birthday. We’re simply concerned with whether an intelligence did so.
Design thresholds determine whether there is design, not the degree of design. They are important because the first thing we want to know when design stands in question is whether what seems to be designed is in fact designed. Does the thing merely give the appearance of design, resulting from unguided forces, or is it the product of a purposive intelligent agent?
In most practical situations, this all-or-nothing approach to design is adequate: Did so-and-so die of natural causes or of foul play? Did this text result from plagiarism or was it independently created? Is that an arrowhead or just a naturally formed rock? Were those data intentionally falsified or could they have resulted from a legitimate scientific experiment?
Nonetheless, when we reflect on intelligence, we think of it not merely as all-or-nothing but also as coming in degrees. In many cases it’s important not only to know whether an intelligence was involved but also to gauge the degree of intelligence involved. Was the intelligence a superintelligence, well beyond the capacities of humans? Was it a bumbling intelligence, barely competent to bring about the design in question? Or was it somewhere in between?
This brief article is part of a larger project to lay out a theory of intelligence metrics that applies quite generally—and especially to the structures and processes in nature. Such metrics would go beyond merely determining whether an intelligence has acted. Given an event that matches a salient pattern, we want to know whether it is the product of a designing intelligence. But in addition, we may want to know the degree of intelligence required to bring about such an event. Intelligence metrics enable us to measure degree of intelligence.
The aim of this larger project, however, is not just to come up with something like an IQ or intelligence quotient metric for the events and objects whose design we are trying to understand. IQ tests have lost credibility in our day for claiming to identify an inherent feature of people that captures their intelligence in a global generalized sense. But intelligence is now better understood as taking many forms (emotional, kinesthetic, linguistic, musical, mathematical, etc.).
Yet besides being variegated, intelligence is also task and context specific. Additionally, it is dynamic rather than static, subject to improvement through learning and experience but also subject to deterioration through disuse and sabotage (as with the lowered achievement and aptitude scores of American school children resulting from the Covid lockdowns and mask mandates, in which the usual way children learned was disrupted).
One benefit of the intelligence metrics developed in this larger project is that we will be able to use them to measure the intelligence in processes that we otherwise might think to be unintelligent because lacking a conscious rational agent. For instance, it will be possible to measure the intelligence inherent in an evolutionary process (such as one driven by natural selection acting on random variations) and thereby assess its adequacy or inadequacy in bringing about the things that evolutionists commonly ascribe to evolution (such as the formation of Michael Behe’s irreducibly complex biological systems).
2 The Connection Between Intelligence and Information
The key intuition behind the concept of information is the narrowing of possibilities. The more that possibilities are narrowed down, the greater the information. If I tell you I’m on planet earth, I haven’t conveyed any information because you already knew that (let’s leave aside space travel). If I tell you I’m in the United States, I’ve begun to narrow down where I am in the world. If I tell you I’m in Texas, I’ve narrowed down my location further. If I tell you I’m forty miles north of Dallas, I’ve narrowed my location down even further. As I keep narrowing down my location, I’m providing you with more and more information.
Information is therefore, in its essence, exclusionary: the more possibilities are excluded, the greater the information provided. As philosopher Robert Stalnaker put it in his book Inquiry (p. 85): “To learn something, to acquire information, is to rule out possibilities. To understand the information conveyed in a communication is to know what possibilities would be excluded by its truth.” I’m excluding much more of the world when I say I’m in Texas forty miles north of Dallas than when I say I’m merely in the United States. Accordingly, to say I’m in Texas north of Dallas conveys much more information than simply to say I’m in the United States.
The etymology of the word information is congruent with this exclusionary understanding of information. The word information derives from the Latin preposition in, meaning in or into, and the verb formare, meaning to give shape to. Information puts definite shape into something. But that means ruling out other shapes. Information narrows down the shape in question. A completely unformed shmoo (e.g., Aristotle’s prime matter) is waiting in limbo to receive information. But until it is given definite shape, it exhibits no information.
The fundamental intuition of information as narrowing down possibilities matches neatly with the concept of intelligence. The word intelligence derives from two Latin words: the preposition inter, meaning between, and the verb legere, meaning to choose. Intelligence thus, at its most fundamental, signifies the ability to choose between. But when a choice is made, some possibilities are actualized to the exclusion of others, implying the narrowing of possibilities. And so, an act of intelligence is also an act of information.
A synonym for the word choose is decide. This last word is likewise from the Latin, combining the preposition de, meaning down from, and the verb caedere, meaning to cut off or kill (compare our English word homicide). Decisions, in keeping with this etymology, raise up some possibilities by cutting down, or killing off, others. When you decide to marry one person, you cut off all the other people you might marry. An act of decision is therefore always a narrowing of possibilities. It is an informational act. But given the definition of intelligence as choosing between, it is also an intelligent act.
Given the etymology of information and intelligence, it’s obvious that the two are related notions. It therefore makes sense to measure intelligence via information. This is not to say there can’t be other ways to measure intelligence, but on its face using information to measure intelligence seems promising. To start the ball rolling, we consider a simple example of intelligence in action and a simple information-theoretic way of measuring its degree. This example suggests that we are on the right track in measuring intelligence via information, but it also raises questions about intelligence metrics that require further development.
3 Mac the Canine Door Opener
A friendly white Labrador retriever named Mac belongs to a neighbor of ours. Our subdivision consists of acreages, so this dog occasionally leaves his property and visits ours, where he makes himself at home. The neighborhood is safe and we tend to leave our doors unlocked when we are around. I first met Mac when he showed up inside our laundry room. He had let himself in through the side door to our house.
Mac, it turns out, is able to open doors with lever handles. In this regard, we might say that he is more intelligent than other dogs that cannot open doors with lever handles. At the same time, I am able to open not just doors with lever handles, but also doors with knob handles as well as sliding doors—doors Mac is unable to open. My intelligence at opening doors, we might therefore say, exceeds Mac’s. But when it comes to door-opening intelligence, locksmiths have the advantage over me. And since no door can stay closed to God, we might say that God has the ultimate door-opening intelligence.
Frivolous as this example may seem, it raises interesting questions about the use of information to measure intelligence. Information, as noted, is about narrowing possibilities. Any formal account of information will therefore require a space of possibilities. Let’s denote this space by the capital Greek letter Ω. In this example, we can let Ω consist of all the doors in the world. Consider now the following subsets of Ω (we use CD to denote Closed Doors):
CDmac: the doors that Mac can’t open.
CDme: the doors that I can’t open
CDlocksmiths: the doors that locksmiths can’t open.
CDGod: the doors that God can’t open.
Since God is presumed to be omnipotent, CDGod = ∅. In other words, the set of all doors that God can’t open is the empty or null set. Because the null set is contained in every set, the set of doors God can’t open is contained in the set of doors that the locksmiths can’t open, which in turn is contained in the set of doors that I can’t open, which in turn is contained in the set of doors that Mac can’t open. In set-theoretic notation:
Because information consists of the narrowing of possibilities, it follows that there’s more information in CDGod than in CDlocksmiths than in CDme than in CDmac. Moreover, CDmac will be contained in the doors closed to the average dog because, unlike Mac, the average dog won’t be able to open lever-handled doors. Consequently, CDmac will be contained in CDaveragedog, implying that the latter exhibits less information than the former. Any information metric I will therefore need to represent these inclusions among closed-door sets via the following extended inequality:
In this example, we gauged intelligence in terms of the doors closed to various agents. We represented these closed doors with subsets like CDme and CDmac of the possibility space Ω. Yet we might also have tried to gauge door-opening intelligence by using subsets like ODme and ODmac of the possibility space Ω, where OD denotes the set of doors capable of being opened by me, Mac, etc. In that case, the bigger the set ODme compared to ODmac, the greater my intelligence compared to Mac’s in opening doors. But because the prime intuition underlying information is the narrowing or reduction of possibilities, to measure intelligence in terms of information requires, in this case, focusing on sets like CDme and CDmac rather than their complementary sets ODme and ODmac.
The difference here between the sets CD and OD may seem to be strictly a matter of convenience and convention. But there’s something deeper at stake here. We don’t measure intelligence by determining where people get everything right but where they get something wrong. Tests where everyone scores 100 percent are meaningless. The point of an intelligence metric is to distinguish among levels of intelligence, and that happens by finding points of failure for one individual that are not points of failure for another. Intelligence is not determined by perfection, as in what people get invariably right. Rather, intelligence is determined by imperfection, as in where some people but not others fall short. The focus on CDme and CDmac rather than on ODme and ODmac with respect to door-opening intelligence exemplifies this deeper point.
Postscript: When the information measure I is given by Shannon information, which is defined as the negative logarithm to the base 2 of a probability, it follows that I(∅) = I(0) = ∞ (the probability of the empty set is always 0, and its negative logarithm is therefore ∞). Thus, for a being with no points of failure—as with an omnipotent, omniscient, and omnicompetent God—the information, and therefore intelligence, associated with such a being will be infinite. More specifically, God’s points of failure in the performance of any task constitute the empty set, and so the information measure associated with that performance will for God evaluate to ∞. This is as it should be and makes good intuitive sense.