Introduction
I am currently co-authoring a popular-level book with two colleagues on conservation of information. Following Steve Meyer and Doug Axe, whose books deftly incorporate autobiographical details about their path to discovery, I thought it might help to do the same with our book. Information has been a major theme in my research over the years, and I have a personal history exploring the concept that I can share. To date, my writings on information have focused on exposition. But there can be value in seeing a scientist’s process of discovery, in addition to explaining those discoveries. Hence this history. The history I share here is longer than will go into the book we are doing. I want to be thorough—we can prune later.
Context for my story with information goes back to 1979, when I converted from a New-Age eastern pantheism to Christianity. In coming to Christ, science-faith conflicts were the last thing on my mind. Any presumed conflicts between Genesis and modern science were simply of no concern to me. My dad was a PhD biologist who had taught at the University of Illinois (Navy Pier campus for those with long memories) and the City Colleges of Chicago. He even taught a course on evolution, and I still remember as a teen or pre-teen seeing the textbook for it around the house.
My newfound Christian faith occurred without any fundamental reexamination on my part of the relation between science and faith. Nonetheless, a new worldview raises new questions and prompts the search for new answers. I wanted to make sense of my Christian faith, especially in a world often hostile to it. So I started reading widely, shifting from New Age to Christian literature.
On the apologetics front, I enjoyed Francis Schaeffer and John Warwick Montgomery, and later G.K. Chesterton and C.S. Lewis. With regard to faith and science, I read the creationist literature, both young-earth and old-earth. I quickly became a skeptic of naturalistic accounts of life’s origin and its subsequent evolution. The fossil record simply seemed to me too incomplete. The mechanisms of evolution such as natural selection seemed to me too inadequate.
One thing in particular that I read convinced me a naturalistic origin of life was all washed up. It was short. It was the appendix of a book that’s still somewhere on my shelf. I read it in 1980. I was a junior at the University of Illinois Chicago Circle (subsequently renamed University of Illinois at Chicago) doing a psychology undergrad. A student I met in class and at a weekly Campus Crusade meeting recommended a book titled Close Encounters: A Better Explanation (by Clifford Wilson and John Weldon).
As in our day, UFOs and extraterrestrials had back then captured the public imagination (through such films as Steven Spielberg’s blockbuster Close Encounters of the Third Kind). Yet in the appendix to that book, for whatever reason, there was a mathematical argument against life originating by undirected means, which is to say apart from intelligence. The odds were just too much against it. The argument in that book was probabilistic and it irrevocably shifted my thinking, leading me ever after to think it absurd that the chemical constituents of life could organize themselves spontaneously into life.
Once I had that conviction, the naturalistic blinders were off and the question then arose where else naturalism failed. Naturalism is like a Pandora’s box in that all of scientific explanation must be kept inside it. Once even one area of science comes to seem implausible to naturalistic explanation, all the contents of the box come rushing out and naturalism is no longer a rule that science is obligated to obey.
Naturalism at heart is the claim that nature is complete—Carl Sagan’s “the cosmos is all that is or ever was or ever will be.” So anything that happens in nature must be the result of things and causes in nature. But nature is under no obligation to follow that directive. It’s easy to come up with counterexamples in which natural causes provide inadequate explanations—where things happen in nature that are beyond the power of nature. Spontaneous remissions, near-death experiences, and psi phenomena—if happening as reported—can fall into this category. That’s why naturalists since antiquity have always argued for fraud or mistaken observation in such cases, and not that the phenomena in question happened as described.
In any case, I continued to read about biological and cosmological origins. In reading the young-earth creationist literature of the time, I found two main approaches. On the one hand, there was a move to treat the Bible as a scientific text and thus to shoehorn the science of the day into its teaching as interpreted within a Genesis literalism. On the other hand, there was the effort to critique naturalistic accounts of biological origins on their own terms and from there to offer a scientifically defensible non-naturalistic alternative without invoking faith or Scripture.
It was the latter approach that intrigued me. Figures at the Institute for Creation Research such as Henry Morris and Duane Gish made some effort in their writings to take this second approach. But it seemed to me their heart was more with taking the first approach (hence their focus more on reinterpreting geology to square with a young earth than in making sense of the design inherent in molecular biology).
A.E. Wilder-Smith
The key figure in their circle taking the second approach was organic chemist A.E. Wilder-Smith (1915–1995). Wilder-Smith was a young-earth creationist. Yet I never got the sense from him that his young earth views did the heavy lifting in his arguments against naturalism or for intelligence in nature. (As an aside, Wilder-Smith was a professor from 1964–70 at the same school—the University of Illinois’s Chicago branch—where I was doing my undergraduate in 1980–81, he at the medical campus, me at the university campus.)
Wilder-Smith wrote a number of books, some on general apologetics, but his most significant books, in my view, were about the role of information in biology and how the type of information found in biology undercuts naturalistic attempts to explain biological complexity and function. The most important of these books was The Creation of Life: A Cybernetic Approach to Evolution (1970), in which he treated genetic systems as cybernetic and information-processing systems and argued that their programming originated through intelligence.
The issue for Wilder-Smith was not the mere existence of biomacromolecules but rather accounting for the information processing required for such molecules to arise and interact, their very arrangements and functions resulting from the execution of encoded instructions. Biology was giving us computer programming and symbolic codes which, by mapping functional structures to functional structures, exhibited a tacit semantics.
The point was similar to John Searle’s Chinese Room thought experiment, where the physical substrate could not explain the semantics atop it. Wilder-Smith was pointing to a mystery for naturalism, but a mystery in principle resolvable through the guidance of a real teleological intelligence. As he recounted in one of the recorded lectures I heard by him (I never saw him in person though we did correspond by mail and had one conversation by phone), it was a mystery and one he discussed with the linguist Noam Chomsky, who agreed with him that this was indeed a deep mystery.
Wilder-Smith’s 1970 book The Creation of Life: A Cybernetic Approach to Evolution extensively cites the 1967 proceedings of the famous Wistar symposium: Paul S. Moorhead and Martin M. Kaplan, eds., Mathematical Challenges to the Neo-Darwinian Interpretation of Evolution: A Symposium Held at the Wistar Institute of Anatomy and Biology, April 25 and 26, 1966, Wistar Institute Symposium Monograph, no. 5 (Philadelphia: Wistar Institute Press, 1967).
At this symposium, mathematicians Stanislaw Ulam and Marcel Schützenberger, as well as engineer Murray Eden, offered a mathematical critique of neo-Darwinism. Wilder-Smith’s 1970 book very extensively quotes from that symposium, especially remarks by Schützenberger and Eden. Schützenberger pointed to the coordination problem between a computer code and what it does, analogizing this to the coordination between genotype and phenotype. Coordination is a problem because small changes at the level of the computer code/genotype can very easily disrupt computer function/phenotype, making randomness at the level of code or genotype in the main toxic.
Random genetic changes, even with natural selection at work, thus seem like a recipe for evolution to fail. But where the Wistar mathematicians, though critical of neo-Darwinism, were engaged in pure critique (“this doesn’t work”), Wilder-Smith was looking for an intelligent design alternative (“this is what replaces it”). He didn’t call it “intelligent design” at the time, but throughout the concluding portions of his 1970 book, he refers extensively to design and intelligence, and there he makes an explicit design argument. He even invokes William Paley’s design argument, though he cashes it out not in terms of a watch but in terms of information.
Wilder-Smith’s Wider Influence
If I’m belaboring Wilder-Smith’s ideas about information and his efforts to advance a design argument based on information, it’s not just because he deeply influenced me. He also deeply influenced others in my circle (though at the time I didn’t know they would be in my circle). Take Dean Kenyon, who went on to become a prominent intelligent design proponent and even co-authored an intelligent design textbook, Of Pandas and People. Kenyon along with Gary Steinman had written Biochemical Predestination (1969), in its day regarded as the most significant attempt to account for the origin of life in purely naturalistic terms.
In the 1970s, a student gave Kenyon Wilder-Smith’s The Creation of Life, which sharply criticized Kenyon and Steinman’s argument in Biochemical Predestination (both authors as well as the book itself are extensively cited in Wilder-Smith’s 1970 book). Finding himself unable to refute Wilder-Smith’s criticisms, Kenyon abandoned his purely naturalistic account of the origin of life and from there went on to embrace intelligent design.
By strange coincidence, in just the last few weeks I was introduced to someone who as an undergrad took Kenyon’s course on the origin of life at San Francisco State University in the mid 1970s—in fact the very class where he announced that he no longer accepted the naturalistic views he had defended in Biochemical Predestination. At the same time, Kenyon announced that he now accepted a design alternative. Instantly, according to my new friend who was there, the class size dropped from 200 to 50.
Charles Thaxton, along with Walter Bradley and Roger Olsen, wrote what arguably was the first official book of the intelligent design movement. Titled The Mystery of Life’s Origin, it was published in 1984 with the secular publisher Philosophical Library through the sponsorship of Jon Buell’s Foundation for Thought and Ethics. Wilder-Smith’s fingerprints are all over that book. Thaxton et al. argue that chemistry and energy flows cannot account for the origin of the information content of living systems. The book, in its selected readings (bibliography) at the end, explicitly references Wilder-Smith’s The Creation of Life. Though not making a full-fledged argument for intelligent design based on information, The Mystery of Life’s Origin does in the epilogue raise intelligence as a legitimate hypothesis for explaining life’s origin, along with other hypotheses. Note that an updated and expanded edition of this book appeared in 2020.
Last, I’ll mention Matti Leisola, the Finnish chemist and intelligent design proponent. Influenced in his own thinking about design by Wilder-Smith, he assumed the role of a disseminator of Wilder-Smith’s ideas. As he writes in his autobiography just published by Inkwell Press: “We met Professor Wilder-Smith. I had read his book, Man’s Origin—Man’s Destiny, and it had significantly shaped my thinking. Before travelling to Switzerland, I wrote to him, and he invited our group to his home on the shore of Lake Thun… He wanted to spend a vacation in Finland with his family, and just over a month later, this was arranged. I introduced him to the Professor of Biochemistry, and we met with the rector of the University of Helsinki, who invited Wilder-Smith to give guest lectures.” (Recently I announced the publication of Matti’s autobiography, for which I did the foreword, on this Substack.)
For me personally, Wilder-Smith made the biggest impression on me through his lectures, which back in the early and mid 1980s were mainly available to me on cassette tapes, a technology long since supplanted. I remember listening to one or two of the lectures on information repeatedly in my room at International House on the campus of the University of Chicago as a graduate student in mathematics. Looking back, I would say Wilder-Smith’s intuitions were spot on. Fleshing out his arguments, however, would need to wait until the intelligent design movement came into its own and could bring more powerful methods to bear on the questions he was raising.
My Mathematical Development
In the early and mid 1980s, Wilder-Smith’s ideas about information excited me but I could think of no way to contribute to them as a researcher pushing back the frontiers of knowledge. I was a graduate student in mathematics, focused on probability theory. I got my PhD in 1988 with joint doctoral supervisors in math under Patrick Billingsley (1925–2011) and in physics under Leo Kadanoff (1937–2015). Both were well-known in their fields, Billingsley more as an expositor, Kadanoff as a star researcher (he received the Wolf Prize and was Nobel caliber). Work on nonlinear dynamics in Kadanoff’s research group inspired my work on probability.
As it is, probabilities are everywhere in information theory, and yet probability theory and information theory have a very different feel. Interestingly my math supervisor Pat Billingsley wrote widely used probability textbooks, such as Probability and Measure (3rd edition 1995); but he also wrote a book titled Ergodic Theory and Information (1965). So he saw the connection. And yet the fields are distinct.
To vastly oversimplify, probability theory is about calculating probabilities whereas information theory is about moving bit strings across communication channels. Probability theory feels like pure and applied math. Information theory feels like engineering. Engineers are concerned with moving signals reliably, which means their attention is on average performance of communication systems. Hence the ubiquitous “entropy,” a probabilistic average of information measures. As I became interested in the late 1980s in radically improbable events and how they might point to the activity of intelligent causes, I didn’t see much use for information theory as such. I was focused on particular outcomes, less so on ensembles of outcomes, as in information theory.
I had read Thaxton et al.’s The Mystery of Life’s Origin in the late 1980s (I recall loaning it out from the Evanston Public Library). Information played a prominent role in that book. Even the term specified complexity, with which my name later became closely associated, appeared in the book. But I didn’t see in specified complexity a direct connection to mainstream information-theoretic concepts. As with Wilder-Smith, the emerging design community kept talking about information. We were gesturing at it. There clearly was something to it. And yet there was work to be done connecting our emerging understanding of information with the standard formalism of information theory.
I defended my math dissertation in the spring of 1988. One of the problems I solved in it was providing a general account of uniform probability. I was trying to understand the nature of randomness, and it seemed to me that the most random of all things was sampling from a uniform probability distribution. I wrote up that account of uniform probability and published it in 1990 in the Journal of Theoretical Probability. This work was done at the height of the excitement over nonlinear dynamics and chaos, which commanded a lot of my physics advisor’s (Leo Kadanoff’s) attention back then.
Randomness by Design
As it is, I shared my dissertation with Stanford statistician Persi Diaconis. He and I had gotten to know each other in 1986. When I sent him my dissertation in 1988, he urged me to attend a conference on randomness that he was putting together at Ohio State University, with logician Harvey Friedman and himself as the conveners (fun fact: Friedman was in 1967 listed in the Guinness Book of World Records as the world’s youngest professor, teaching at Stanford at age 18). New York Times science reporter James Gleick, who had written the fantastically popular book at the time simply titled Chaos, covered the conference and on April 19, 1988 reported on it in a NYT article titled “The Quest for True Randomness Finally Appears Successful.”
But that title of the article oversold the reality. When the conference ended, all the speakers and participants sat around in a circle and Persi Diaconis delivered what he took to be the upshot of the conference. He put it this way: “We know what randomness isn’t, we don’t know what it is.” His point was that in talk after talk at this interdisciplinary conference (disciplines represented included mathematics, statistics, computer science, economics, psychology, and philosophy), the point was to identify where randomness was defeated, not to make clear what it actually was.
Thus with random number generation, for instance, we could never be sure just how good any such generator was at mimicking chance because it proceeded by a purely deterministic algorithm and so was inherently hiding nonrandomness. Perhaps some pattern would be discovered in its output that would make clear it was not truly random. Randomness was always a provisional designation until such time as a pattern would be found that made clear it was not random. Thus, we always got at randomness by what revealed itself to be nonrandom.
We all have had the experience of looking at some image that seems devoid of any salient pattern (i.e., it seems random) only suddenly to see something clearly nonrandom in it. In my public talks on intelligent design, I often used the following image (I got it from Harold Booher, who has since passed on). It looks like a random inkblot until you see what is actually there. Moreover, once you see the pattern, you’ll never unsee it (short of amnesia). There is thus a fundamental asymmetry between randomness and nonrandomness, with randomness flipping to nonrandomness but not vice versa.
To sum up then, this 1988 Ohio State randomness conference ended on a thud: We know what randomness isn’t, not what it is. But in my intellectual journey, this unpromising conclusion was the catalyst for much fruitful thinking on my part. In the process of discovery, my Christian faith was indispensable here. From the vantage of naturalism or materialism, randomness is fundamental. Material entities have no mind to oversee them, so there is nothing intentional about their arrangement. Their arrangement may be regarded as resulting from a combination of randomness and self-organization, and any mind or intelligence then becomes a byproduct of such arrangements. Alternatively, on materialistic grounds these arrangements of matter result from some combination of chance and necessity for which mind or intelligence is then causally downstream.
Within my Christian worldview, however, the mind of God is real and the creation of the world is upstream of any causal powers (secondary causation) expressing themselves in the creation and thus characterized by chance and necessity. Given my Christian worldview, it therefore made perfect sense that nonrandomness should have priority (conceptually and causally) over randomness. Accordingly, patterns would come to define nonrandomness (this being how randomness was always defeated in practice, namely, by identifying a salient pattern), in consequence of which randomness would be defined as violating a set of patterns. I therefore flipped the problem of randomness. Nonrandomness became the basic concept, randomness the derivative concept.
I wrote this idea up in a paper titled “Randomness by Design,” which was published in 1991 with the philosophy journal Noûs. It was the easiest peer-reviewed paper I ever got published. The Noûs editor, Héctor-Neri Castañeda, accepted it as is without the need to change a thing. The very title, and the article itself, stressed that design is the fundamental idea, and randomness the derivative idea. I wrote this paper the fall of 1989, my first term as a philosophy graduate student at the University of Illinois at Chicago. It was for a seminar in which I was pursuing a second doctorate, this time in philosophy. The article got me a lot of good attention.
In consequence, Arthur Fine at Northwestern University, who was at the time helping to edit the big 8-volume Routledge Encyclopedia of Philosophy, published in 1998, invited me to contribute the entry on randomness. David Malament, a well-known philosopher of physics loved it and thought I deserved a position in a good philosophy department because of it. Bill Wimsatt, a philosopher of biology at the University of Chicago, endorsing my later book The Design Inference, alluded to it by writing in that endorsement, “Dembski’s analysis of randomness is the most sophisticated to be found in the literature and his discussions are an important contribution to the theory of explanation and a timely discussion of a neglected and unanticipatedly important topic.”
Precursors to the Design Inference
The focus on patterns in understanding randomness put this work in the same ballpark as information, information at some level being a study of patterns. But there was still a lot to be done before the connections between my work and information theory would become explicit. Given that we use patterns to defeat randomness, the next question for me became what sorts of patterns do we use in practice to defeat randomness or chance. I wrote up my thoughts in response to this question in the spring term of my first year as a grad student in philosophy at the University of Illinois at Chicago (first half of 1990). This was in a paper I titled “When Chance Precludes Chance,” and I submitted it to Synthese. As I recall, it was Jaakko Hintikka who got back with me. He rejected the paper but indicated there was a good paper embedded in what I had sent them, which I needed to try to bring to light.
In retrospect, the problem was that the patterns that defeat randomness and chance are specifications, and it would not be until I published The Design Inference: Eliminating Chance Through Small Probabilities with Cambridge University Press in 1998 that I would have an ironclad definition of specification and thus of specified complexity. Interestingly, the academic editor for Cambridge who shepherded the review process of this book was Brian Skyrms, a philosopher who spoke at the 1988 randomness conference at Ohio State University. It was at that conference that I met him, and on account of that meeting that I approached him to try to get The Design Inference published with Cambridge as opposed to elsewhere.
I should mention that in my seeing the importance of specification to design inferences, Richard Dawkins’ The Blind Watchmaker played a crucial role. I read that book in the late 1980s. There Dawkins wrote, “Complicated things have some quality, specifiable in advance, that is highly unlikely to have been acquired by random chance alone. In the case of living things, the quality that is specified in advance is … the ability to propagate genes in reproduction.” Obviously, we have here specified complexity, though the actual terms used are cognates—specifiable for specified, complicated for complex.
For Dawkins, specified complexity required some special effort of explanation, but that effort could be performed by Darwinian natural selection. For me, on the other hand, natural selection, by acting as a probability amplifier over random chance, could not purchase actual specified complexity (actual complexity qua improbability would then be removed precisely because, acting as a probability amplifier, natural selection made the probabilities to be no longer small).
Yet specified complexity, in the absence of removable improbability, was invariably associated with intelligence. So the very thing that for Dawkins confirmed natural selection for me confirmed a designing intelligence (my view of natural selection being nowhere as high as his). I’m condensing here a more extensive argument that I’ve developed at length elsewhere (such as in the second edition of The Design Inference, co-authored with Winston Ewert). But the point I want to make is that in my process of discovery, Dawkins played a crucial role even though I regard his ideas and arguments as deeply flawed. Yet they helped me to formulate my own arguments by putting them in stark relief with his.
Joining Steve Meyer and Paul Nelson
Perhaps the most important thing that the “Randomness by Design” paper did for me in practical terms was put me on the radar of Steve Meyer and Paul Nelson. Before that, I had been flying entirely solo. I had a deep sense of calling that I needed to work on bringing design back into science. But I had no colleagues to talk with about this.
Steve and Paul in 1990–91 were wanting to write a book to rehabilitate the design argument in light of modern science and information theory. They had hoped to enlist Charles Thaxton as a third collaborator. But Thaxton was on his way to Prague to help students and faculty at the university there to integrate science and faith. And in any case, Steve and Paul really needed a mathematician.
I think it was Paul who read “Randomness by Design” and Steve who independently read a critique of strong artificial intelligence by me titled “Converting Matter into Mind,” which appeared in December 1990 in the American Scientific Affiliation’s journal Perspectives on Science and Christian Faith. As part of this article, I needed to include a biosketch. In it, I described my desire to reinstate design in science. I was running a flag up a flagpole and hoping someone would see it. And it was seen.
Given these two articles that I had written, Steve and Paul realized I was to be their third musketeer. To get the ball rolling on our joint book project, Steve arranged for us to meet for several weeks in the summer of 1992 in Cambridge England. This was to get us up to speed on each other’s work and to chart a path forward for a joint book. Steve had finished his philosophy dissertation in 1990 at Cambridge. Paul was still working on his at the University of Chicago. And I was almost finished working on mine at the University of Illinois at Chicago (though really I wasn’t—more on this momentarily).
It was a great blessing for me to rub shoulders with Steve and Paul that summer. As an added bonus, the three of us also got to meet Doug Axe, who was in Cambridge at the time and as skeptical of Darwinian naturalism as we were. Doug gave us a seminar on genes and proteins and how Darwinian processes were inadequate to account for them. Steve and Paul were seasoned and well acquainted with the key people trying to understand the role of information in biology and how it could make clear the design in nature. They were also well-read in the literature of the time. I was at the time much more an autodidact, though with the needed expertise in mathematics. After that summer, I transitioned from autodidact to professional design theorist.
Lessons of Stalnaker and Dretske
Two insights about information that I gained as a philosophy graduate student proved hugely influential on my subsequent thinking about information. They came from two books by epistemologists, MIT’s Robert Stalnaker (Inquiry) and Stanford’s Fred Dretske (Logic and the Flow of Information). I read the Stalnaker book in a philosophy seminar; I read the Dretske book on my own. Whenever I need to give an elevator pitch about what information is, I quote Stalnaker. And when I need to go slightly deeper, I cite Dretske.
In his book Inquiry, Stalnaker writes about information, “To learn something, to acquire information, is to rule out possibilities. To understand the information conveyed in a communication is to know what possibilities would be excluded by its truth.” This remains for me to this day as good a definition of information as exists. It avoids the reduction of information to linguistic symbol conventions (Shannon information). It also avoids tying information entirely to the meaning of communications (semantic information). Both of these are in fact special cases of Stalnaker’s definition as the excluding or ruling out of possibilities. A string of symbols sent across a communication channel rules out the other strings that could have been sent. A communication with a certain semantic content rules out other communications with different semantic content.
The conception of information as ruling out possibilities allows for teleological as well as non-teleological processes to generate information. Thus information does not automatically bias discussions in favor of intelligent design, as though talk about information automatically presupposes teleology or mind. Natural processes, to the extent that they rule out possibilities—which they can and do—thus generate information. This puts teleology and non-teleology on an equal footing vis-a-vis information, and thus allows information to fairly adjudicate between them.
Problems arise when people go with a narrower definition of information than the general one by Stalnaker. Take science writer John Horgan, who was attracted to the idea of information being the basis of reality (an idea that every now and again captures the imagination of some big-name scientist, such as John Archibald Wheeler). Yet in a 2011 Scientific American article, Horgan rejects this view for the following reason:
The concept of information makes no sense in the absence of something to be informed—that is, a conscious observer capable of choice, or free will (sorry, I can't help it, free will is an obsession). If all the humans in the world vanished tomorrow, all the information would vanish, too. Lacking minds to surprise and change, books and televisions and computers would be as dumb as stumps and stones. This fact may seem crushingly obvious, but it seems to be overlooked by many information enthusiasts.
Horgan would be right if information were inherently semantic and presupposed minds. But by Stalnaker’s account of information, it is about contingency—ruling out some possibilities and realizing others. And this can happen without minds thinking about those possibilities or meanings assigned to them.
Fred Dretske’s Knowledge and the Flow of Information confirmed Stalnaker’s definition of information. As Dretske wrote in the opening of his book, “Information theory identifies the amount of information associated with, or generated by, the occurrence of an event (or the realization of a state of affairs) with the reduction in uncertainty, the elimination of possibilities, represented by that event or state of affairs.” By the time I read this, I had already internalized Stalnaker, so I learned nothing new here. But later in Dretske’s book, I read the following, which made a deep impression on me:
It may seem as though the transmission of information … is a process that depends on the causal inter-relatedness of source and receiver. The way one gets a message from s [source] to r [receiver] is by initiating a sequence of events at s that culminates in a corresponding sequence at r. In abstract terms, the message is borne from s to r by a causal process which determines what happens at r in terms of what happens at s. The flow of information may, and in most familiar instances obviously does, depend on underlying causal processes. Nevertheless, the information relationships between s and r must be distinguished from the system of [material] causal relationships existing between these points.
The crucial insight for me, as a proponent of intelligent design, was that the information relationships made sense in the absence of any known physical causal process connecting the two ends of the communication channel. This meant that information relationships were more fundamental than the physical-causal relationships and could obtain and be knowable even in the absence of our knowledge of a physical-causal relationship and, more radically still, even in the absence of any such physical-causal relationship existing at all. A naturalist would, of course, be committed to there being such a relationship. But intelligent design, as it was coming to understand information, saw it as nonreducible to physical causality. Dretske’s distinction here was thus crucial to intelligent design’s emerging conceptual infrastructure.
From Philosophy to Theology
Regarding my doctoral studies in philosophy, I had gone to the University of Illinois at Chicago to work with the philosopher of science Paul Teller, the son of Ed Teller. In applying to this philosophy grad program, I didn’t know that Teller was in the midst of a divorce and would be leaving after my first year (spring 1990) for UC Irvine. After his departure, I took a dissertation on the logic of conditionals that I had already begun and moved it over to another philosopher there. But he left after my second year (1991) for the University of Pittsburgh. I then switched the dissertation, which was largely completed in 1992, to a third supervisor. He stayed put, but fell ill with depression, and so was unable to continue supervising me.
That left still one doctoral supervisor to go. Soon after my third supervisor fell ill with depression, I was doing a postdoctoral fellowship in philosophy at Northwestern University just north of Chicago in the philosophy of science with Arthur Fine and David Hull (for the academic year 1992–93). This was a prestigious fellowship and I got it on the promise that my dissertation was essentially finished. But the fourth person who took over my dissertation on the logic of conditionals never got on board with the project. Everybody else on the committee was fine with it. But as the academic year 1992–93 came to a close, I was without a dissertation and without job prospects except teaching at smaller institutions with crushing teaching loads as I was trying to complete a dissertation. Indeed, the dissertation on the logic of conditionals came to seems like the rock of Sisyphus.
I decided therefore to put the dissertation on hold for a year and clear my head by attending Princeton Theological Seminary. I had done a postdoc in computer science in the fall of 1990 at Princeton University. The university and the seminary are separate institutions but across the street from each other. As it was, I still had a year on an NSF math postdoc, which I had put on hold and resumed after my first year in grad school in philosophy. At Princeton in 1990, I got to know the seminary. I had a deep interest in theology, and so, rather than try jump year-by-year from one temporary philosophy teaching position to another until I could finish my philosophy doctorate, I decided to pursue an M.Div., a professional ministerial degree, at Princeton Theological Seminary, while also trying to jump start my dissertation.
I thought I would have more time to work on the dissertation as a full-time student than as a full-time faculty member, and I was proved right. The M.Div. also proved crucial as my mainstream academic job prospects in subsequent years dried up on account of my work advancing intelligent design. The M.Div. allowed me to teach at seminaries, and I ended up teaching at three of them.
At Princeton Seminary, I was still in touch with Steve and Paul as well as the ever widening circle of ID movement, including such figures as Phil Johnson, Mike Behe, Jonathan Wells, and Doug Axe. Steve, Paul, and I had a grant from the Pascal Centre in Canada to research and write the book on design that we had been planning, one that would go beyond the natural theology of William Paley and bring design squarely into modern science where it could challenge naturalistic forms of evolution, notably Darwinism. I was initially going to write a few chapters for it on design detection, with Steve and Paul then filling out the rest of the book. But as I kept thinking and writing on this topic, it became clear that there was a whole book here of its own, and not just a portion of a larger book.
The Design Inference
In the summer after my first year in seminary (the M.Div. is a three year program), I therefore decided to scrap my former dissertation on the logic of conditionals and write up my work on design detection and inferences as my philosophy dissertation. I worked intensively for 13 months on this project while also taking a full seminary course load. (Tidbit: I wrote the dissertation entirely in J. Gresham Machen’s old room in Alexander Hall.) That fall of 1995, I submitted the new dissertation, titled The Design Inference: Eliminating Chance Through Small Probabilities to my supervisor Dorothy Grover, the philosopher who had shot down my previous dissertation on the logic of conditionals. This time she got behind the project.
I defended in December of 1995. Besides my committee, Paul Nelson showed up at the dissertation defense, along with University of Chicago philosophy professors David Malament and Bill Wimsatt. We all went to a Greek restaurant afterward. I officially graduated the following year, and the dissertation won the “1996 Outstanding Dissertation Award in Fine Arts and Humanities.” One of the philosophers, as I recall, remarked that this was the best dissertation they had seen in 15 or 20 years.
In the dissertation, and then in its revised form for publication with Cambridge University Press in 1998, the terms “specification” and “small probability” appear, but not the term “specified complexity.” The connection between the two was clear to me, with greater complexity indicating greater number of bits to characterize an item in question and thus smaller probability for the item in question to result by chance. And specification was then the type of pattern that in the presence of improbability triggered a design inference.
Specified complexity was therefore getting at something information-theoretic. And yet in The Design Inference, I was not thinking primarily in terms of information. Rather, I saw the design inference as an extension of Fisherian significance testing where the rejection region to eliminate chance was not necessarily given in advance (as with Fisher) but could also be identified after the fact provided that the rejection region constituted a specification, which I characterized in terms of what I called a tractability condition, and which technically was a Kolmogorov-type minimum description condition.
(A brief digression is in order here. Fisherian significance testing provides a widely accepted model for statistical rationality. Some early critics of The Design Inference rejected my work in this book because they adopted an alternative model of statistical rationality, known as Bayesianism. As it is, there is no fundamental conflict between the two. They can be reconciled, and in the second edition of The Design Inference, Winston Ewert and I show how this can be done—see section 5.7).
The design inference was after all an inference. It hinged on a convergence between some low probability event and a match between that event and an independently reconstructible pattern, its independent reconstructibility being that it was easy to state or think up, and thus from a mathematical vantage it required a quantifiably short description.
Specified Complexity
As initially proposed, specified complexity, though clearly overlapping my work, focused more on information as such than on statistical inference. Leslie Orgel, the origin-of-life researcher, introduced the term in 1973 in his book The Origins of Life. There he considered three types of situations:
It is possible to make a more fundamental distinction between living and nonliving things by examining their molecular structure and molecular behavior. In brief, living organisms are distinguished by their specified complexity. Crystals are usually taken as the prototypes of simple, well-specified structures because they consist of a very large number of identical molecules packed together in a uniform way. Lumps of granite or random mixtures of polymers are examples of structures which are complex but not specified. The crystals fail to qualify as living because they lack complexity; the mixtures of polymers fail to qualify because they lack specificity… These vague ideas can be made more precise by introducing the idea of information. (pp. 189–190)
Thaxton, Bradley, and Olsen had picked up on this definition in The Mystery of Life’s Origin (1985), likewise treating it in information-theoretic terms.
In 1996–97, I was still going through the review process for The Design Inference with Cambridge University Press trying to get it accepted there for publication. During that time, I started to make a closer connection between specified improbability as a statistical inference and specified complexity as a form of information, which at the time and subsequently I came also to call complex specified information.
One thing that helped make that connection was the following thought experiment: what would happen if you saw, as the output at the receiver end of a communication channel, something that was improbable and specified? Any coding function responsible for the output would presumably be simple and straightforward—you wouldn’t want it to require a lot of computation to go from input at one end of the channel to output at the other end. In general, whether for algorithmic coding functions or functions characterizing laws of nature, such as evolution equations for dynamical systems, the functions would be well-defined and simply describable.
Now the interesting thing that happens with functions in general is that if a function f maps a possibility space X to a possibility space Y, probabilities are preserved as they are pushed backward. So suppose some subset B of Y is improbable. Then any subset A of X that maps onto B, i.e., f(A) = B, is going to be at least as improbable. Moreover, because f is simple, if B is specified, then so is A in the sense that both will have short descriptions (there are some mathematical details here that I am eliding, such as that A needs to be the full inverse image of B).
To make this concrete, consider the following symbol string:
TOBEORNOTTOBETHATISTHEQUESTION
It is, at least intuitively, both complex (in the sense of improbable) and specified (in the sense of conforming to a simply described pattern—in this case a sentence from the Shakespeare play Hamlet). Now suppose this symbol string is encrypted as a Caesar cipher:
UPCFPSOPUUPCFUIBUJTUIFRVFTUJPO
In this case, each letter is moved up one notch, T to U, O to P, B to C, etc. The coding function that moves each letter down one notch, call if f, thus maps the second of these symbol strings to the first. The specified complexity of the first string is therefore transferred back to the second string. So, if the output exhibits specified complexity, so does the input.
But that’s not all: the amount of specified complexity inputted and outputted is the same provided that all inputs that map to the output are accounted for (easy in this case because f is a one-to-one correspondence) and provided the descriptive complexity added by the function f doing the mapping is accounted for. With this accounting, specified complexity is conserved under functional transformation, and it is this that I initially called “conservation of information.” It is conservation of information for specified complexity.
I wrote up some preliminary thoughts in this vein for a paper titled “Intelligent Design as a Theory of Information,” which the American Scientific Affiliation published in its September 1997 issue of Perspectives on Science and Christian Faith. The paper arose out of a presentation I gave earlier that year at a conference in Austin, Texas (February 23–23, 1997) organized by University of Texas philosopher Rob Koons titled “Naturalism, Theism, and the Scientific Enterprise.”
Looking at a report of this event just now for the American Scientific Affiliation, I found this prescient note: “A growing number of young scientists, scholars and philosophers of science are staking their careers on the prospects of an emerging design paradigm, including Dembski at Notre Dame, Nelson at Chicago, Meyer at Whitworth, …, to name a few.” Tidbit: I sent Noam Chomsky a copy of this paper, and somewhere in my files is a kind reply from him on MIT stationery.
Conservation of Complex Specified Information
I subsequently expanded the argument in that 1997 paper in my 2002 book No Free Lunch: Why Specified Complexity Cannot Be Purchased Without Intelligence. The result holds up to this day. At least one scholar thought it was a big deal, mathematician, physicist, and philosopher of science Wolfgang Smith (1930–2024), calling this conservation of specified complexity result “Dembski’s Theorem”:
Dembski’s theorem is epochal in its significance: it suffices, after all, to invalidate—in a single mathematical stroke—the mechanistic worldview that has dominated Western civilization since the Enlightenment. But in so doing, it raises a crucial question of its own: if horizontal causation cannot produce CSI, what is it, then, that can? Dembski and his colleagues seem to have opted for the notion of “intelligent design.” Yet one sees, on the basis of (*) [i.e., the claim that horizontal causation cannot give rise to irreducible wholeness], that this does not get to the heart of the matter. The fact is that Dembski’s theorem demonstrates the existence of a causation hitherto unsurmised by the scientific establishment, which in fact does not fit into our “flat” cosmology. The fact is that this hitherto unsurmised causation proves to be none other than what I had termed vertical causality in the context of the quantum measurement problem. [Click here for source.]
Smith was a friend and Discovery Institute fellow who passed away in 2024 at the age of 94. I always liked him immensely. I recall reading his book Cosmos and Transcendence with great interest early in my philosophy graduate student days, and contacting him about his book. He was helpful and friendly. When Steve Meyer was still teaching at Whitworth College in 1997, Paul Nelson and I visited him there in the fall. Retired from teaching, Smith was living Idaho at the time, right across the border from Spokane, where Whitworth was. The four of us all had dinner one night. If smart phones had existed then, I’d have a picture of the four of us at the dinner table.
Wolfgang Smith’s excitement about conservation of complex specified information or specified complexity was not widely shared. Part of the reason was that this conservation result required a fair amount of bookkeeping—keeping track of input and output probabilities, output specification, and then any pattern complexity being added by the function mapping inputs to outputs. But the bigger reason was that Darwinists regarded natural selection as a probability amplifier, rendering seemingly improbable events as highly probable. As statistician and neo-Darwinist R.A. Fisher put it back in the 1950s, writing about natural selection:
[I]t was Darwin’s chief contribution, not only to Biology but to the whole of natural science, to have brought to light a process by which contingencies a priori improbable, are given, in the process of time, an increasing probability, until it is their non-occurrence rather than their occurrence which becomes highly improbable. [For this quote and the full context and discussion surrounding it, see my Substack piece titled “The Intelligence of Natural Selection.”]
Consequently, no output of Darwinian evolution was ever highly improbable, and so it never produced real specified complexity, complexity here being understood as improbability (the greater the complexity, the smaller the corresponding probability). Supposedly, the improbability was apparent only until we understood the heavy probabilistic lifting that natural selection was doing. Consequently, “conservation of information” in my own writings and that of my colleagues who worked on it with me (Bob Marks, Winston Ewert, and George Montañez) came to mean something different by the term than what I had called conservation of information with respect to specified complexity, and which Wolfgang Smith called “Dembski’s Theorem.”
In effect, there ended up being two conservations of information, one for specified complexity and one for information generally. The one for specified complexity is nowadays largely forgotten because the one for information generally plugs the hole in the first, namely, by making sense of what happens informationally when processes raise probabilities and thereby eliminate what previously would have been specified complexity.
Displacement
To understand what conservation of information means today, I need to turn to an example from The Blind Watchmaker. It was this example that set me on a path to understanding conservation of information in its general form (not just with respect to specified complexity). In The Blind Watchmaker, Richard Dawkins describes a computer program that generates the phrase METHINKS IT IS LIKE A WEASEL from a random sequence of capital letters and spaces. Starting from a random sequence and then by making random changes, as intermediate sequences are rewarded for being, letter by letter, closer to the target sequence METHINKS IT IS LIKE A WEASEL, Dawkins shows how these letter sequences can converge (“evolve”) to the target in short order (roughly 40 steps) whereas it would take something like 10^40 steps on average to get this target sequence by pure chance. The much faster convergence to the target in the first scenario, Dawkins argues, demonstrates the power of natural selection.
Yet even the smallest reflection reveals that the target sequence was built into the algorithm that generates it, for how else could the algorithm determine proximity to the target sequence or reward increasing proximity? It became clear that Dawkins was engaged in a subterfuge, and that he had in fact smuggled in the information that he claimed to be getting out for free. The outputted sequence was made to seem like it had been produced for free. But in fact, the information outputted merely disguised the information inputted, building it into the algorithm. This, it seemed to me, was a perfectly general phenomenon with evolutionary computing, and so there would be no way for such algorithms to create novel information in the way Dawkins was claiming.
At the time I came to this conclusion in 1997, I was sharing office space with Paul Nelson in Evanston, Illinois. Paul is incredibly well read and keeps up with a wide range of literature relevant to intelligent design. As I was explaining to him the fallacy in Dawkins’ example, he recalled seeing a then recent paper by William Macready and David Wolpert on No Free Lunch theorems. These were information-theoretic results for searches in evolutionary computing (thus covering Dawkins’ example) showing that no search, on average across fitness landscapes, outperforms blind search. It was not a result that many wanted to here at the time, especially among those who thought evolutionary computing provided confirmation for biological evolution. But the math was solid, and there seemed no way around these results.
The No Free Lunch theorems, however, didn’t go far enough for me. It’s one thing to say that any search on average does not outperform blind search. The fact is, however, that for specific problems, some searches do outperform blind search. Dawkins’ algorithm generated METHINKS IT IS LIKE A WEASEL quickly with high probability because it was in effect a hill-climbing algorithm where the hill ascended with smooth gradients and the top of the hill corresponded to this sequence. And yet, any other sequence could likewise have been designated as the top of some hill. So the question, left unanswered by No Free Lunch, is what makes a search better than another, especially better than a baseline search that is purely random.
In Dawkins’ example, the difference maker that improved the search for METHINKS IT IS LIKE A WEASEL over blind search is the information inputted into the algorithm that directed it to reward sequences that, letter by letter, were closer to the target sequence. But was this smuggling in of information specific to Dawkins’ example and thus something that could be avoided in search more generally, or was it a property of any search that, if it improved on the probability of success in finding a target, it did so by incorporating novel information over and above what was available in a baseline search that had low probability of success. I suspected this was a general phenomenon, and in practice I could find no counterexample to it. I called that phenomenon “displacement”—the success of the search was displaced or offloaded to some item of information that was itself unaccounted for.
To illustrate this point, imagine trying to explain how Alice came into possession of $1,000,000. It would be one thing to say of Alice that, based on her own skills and effort, she earned $1,000,000. This would be like a baseline search, based on its inherent resources, achieving a target with high probability. But what if instead Alice’s $1,000,000 resulted from Bob giving her that amount? In that case, the natural question becomes where did Bob get the $1,000,000. The explanation of the $1,000,000 in Alice’s pocket is thus displaced and left unresolved. Likewise, if a baseline search gets supplanted by one that has much higher probability of successfully locating a target, where did it get the information to make itself so much more powerful a search?
Auditing Smuggled Information
The challenge for me, then, was to examine computational systems that seemed to be generating information in the form of successful search for free and show where the information had in fact been smuggled in. With Dawkins’ METHINKS IT IS LIKE A WEASEL, seeing where the information was smuggled in was straightforward. But with other examples, such as Thomas Schneider’s ev program, Adami et al.’s Avida program, and David Thomas’s Steiner Tree program, finding where the smuggling (displacement) occurred took more effort. It also helped to have the expertise of computer scientists to deconstruct these programs and show where exactly the information supposedly gotten for free had in fact been illicitly inputted.
One person early on whom I was able to enlist to deconstruct Schneider’s ev program was a UK programmer named Iain Strachan. He offered his deconstruction in an article that remains of historical interest from a now defunct journal. It is available through the Web Archive: “An Evaluation of ‘Ev’.” But his was a one-shot effort, and so I was looking to recruit others. Enter Robert J. Marks II, a distinguished professor of engineering at Baylor University, and students of his that he set to work on this project of deconstruction through his Evolutionary Informatics Lab (evoinfo.org).
And so, in the mid to late 2000s, it became a cottage industry for my colleagues and me at the Evolutionary Informatics Lab to take examples from the evolutionary computing literature that claimed it produced information for free (in analogy with Darwinian evolution) and show how, with a closer audit, the information had not been properly paid for. George Montañez and Winston Ewert did most of the heavy lifting in writing the programs and simulations to expose the illicit insertions of information in evolutionary computing programs that were supposed to generate them for free or from scratch, but in fact did nothing of the sort. All of their work in this area remains available at EvoInfo.org, and specifically on the Research Tools page.
Showing where the illicit information was inserted into such evolutionary computing programs played an important role in defending intelligent design and establishing its credibility in the 2000s. For instance, in 2003, Nature published “The Evolutionary Origins of Complex Features,” listing as authors Richard E. Lenski, Charles Ofria, Robert T. Pennock, and Christoph Adami. Pennock at the time was one of the most vocal critics of intelligent design. Even though that Nature paper studiously avoided citing Michael Behe, in his expert witness testimony in the Kitzmiller v. Dover case, Pennock made a point of saying that the Avida program, which was the subject of that Nature paper, refuted Behe’s claims about complex systems arising by Darwinian evolution.
In fact, the Nature paper did nothing of the sort, and it was cooked from the start to reward complexity for complexity’s own sake (moreover, the complexity was generic and not specifically irreducible complexity). The Avida program was thoroughly deconstructed at EvoInfo.org: see the article and simulation at https://www.evoinfo.org/minivida.
To get a flavor of how we deconstructed evolutionary algorithms that were used to argue for Darwinism, the case of physicist David Thomas is instructive. Thomas offered a Darwinian program for constructing Steiner trees (a type of graph that in some optimal way connects various vertices). He then posed the following challenge: “If you contend that this algorithm works only by sneaking in the answer (the Steiner shape) into the fitness test, please identify the precise code snippet where this frontloading is being performed.”
We found several such code snippets, but the most glaring is one that included the incriminating comment “over-ride!!!”:
x = (double)rand() / (double)RAND_MAX;
num = (int)((double)(m_varbnodes*x);
num = m_varbnodes; // over-ride!!!
As Winston Ewert, Bob Marks, and I pointed out on the EvoInfo.org website (https://evoinfo.org/papers/steiner.pdf):
The claim that no design was involved in the production of this algorithm is very hard to maintain given this section of code. The code picks a random count for the number of interchanges; however, immediately afterwards it throws away the randomly calculated value and replaces it with the maximum possible, in this case, 4. The code is marked with the comment “override!!!,” indicating that this was the intent of Thomas. It is the equivalent of saying “go east” and a moment later changing your mind and saying “go west.” The most likely occurrence is that Thomas was unhappy with the initial performance of his algorithm and thus had to tweak it.
The Evolutionary Informatics Lab thus became the go-to place to audit the books of the Darwinian evolutionary computing industry. In its wake, you nowadays find a lot less enthusiasm among Darwinists in touting the wonderworking evolutionary powers of algorithms in mimicking natural selection. This has all been to the good. But it also proved unsatisfying in the sense that our approach invited constantly new challenges from evolutionary algorithms that could become ever more clever and opaque at misrepresenting the information that was being outputted as though it had been created for free when in fact it had been surreptitiously inputted.
The situation was similar to people constructing ever more elaborate machines supposedly capable of perpetual motion and then the responsibility falling on patent examiners of having to show where they failed (such as by showing where they draw from solar energy to which they were not supposed to have access). The USPTO no longer accepts patents for perpetual motion machines because the second law says it can’t be done. We were looking for something like this for evolutionary search and indeed search in general.
My Revelation in Denmark
The challenge was therefore not merely to show, in case after case and without exception, that displacement was happening but that there were principled quantitative reasons for why information outputted could not exceed information inputted in the same way that the second law of thermodynamics provided principled quantitative reasons for precluding perpetual motion machines. The breakthrough for me came unexpectedly in 2004 in a whirlwind tour of academic institutions in Denmark.
Jakob Wolf, a professor of theology at the University of Copenhagen, had written a pro-ID book in Danish that was published in 2004: Rosens råb. Intelligent design i naturen. Opgør med darwinismen. (In English: The Rose’s Cry. Intelligent Design in Nature. A Clash with Darwinism.) Enthusiastic about advancing intelligent design in Denmark, he invited me to speak at a number of universities, meet with various faculty in more intimate settings, and show me some of the sites. I’m particularly grateful to him for showing me the Karen Blixen (Isak Dinesen) Museum.
The most significant stop for me in Denmark, however, was a lecture Jakob Wolf arranged for me to give at the Niels Bohr Institute. Interestingly, this was the very institute where my physics advisor Leo Kadanoff had done his first postdoc after getting his PhD in physics from Harvard in 1960—Bohr would have been alive at the time, not passing away until 1962.
In my lecture at the Niels Bohr Institute, I was going to do my usual presentation on displacement, arguing as a practical matter that highly improbable search output always required a corresponding input of information. But my argument in the past had been more case-by-case and qualitative. The night before my lecture it struck me that by representing search using probability measures, the relation between information out and information in could be made mathematically precise, with the amount of information out never exceeding the amount of information in.
All the pieces suddenly fell in place for me. In the actual lecture, I overpromised a bit, needing later to get out pencil and paper and calculate all the numbers precisely to confirm all that I was promising. But I knew I had made a major advance. And once I worked out the details, everything I said in the lecture proved true.
Searching Large Spaces
I wrote up the mathematical details of the insights I shared at the Niels Bohr Institute in a paper that I put up on my blog, designinference.com (I later moved that blog over to billdembski.com). The paper was titled “Searching Large Spaces: Displacement and the No Free Lunch Regress” (available here), which appeared in early 2005, and thus before the Evolutionary Informatics Lab got off the ground and before my collaborations with Bob Marks, Winston Ewert, and George Montañez.
So really, with this paper, the theoretical basis for why there would be no way around displacement in the evolutionary computing literature was made clear. Still, that paper needed to be situated within a proper information-theoretic framework. Bob Marks, with his extensive background in engineering and theoretical computer science, helped to provide just such a framework. At the same time, because of the mathematical demands the searching large spaces paper placed on readers (measure theory, functional analysis, and vector-valued integration), case-by-case deconstructions of inflated claims for the ability of natural selection to create information were, in addressing biologists and non-technical researchers, still needed.
The searching large spaces paper elicited from at least some readers a respect that I had not seen before. As happens in academic jockeying, it helps to be able to show off some expertise that others lack. This paper forced me to dust off some of the more advanced mathematics from my math grad student days going back about twenty years. I found writing up its results particularly satisfying. At least one technically proficient critic of intelligent design, Thomas English, who previously had used a pen name and who after this paper publicly identified himself, noted that this was the first paper that he thought showed intelligent design offered theoretical insights not easily dismissed.
Specified Complexity as an Actual Information Measure
Before describing the aftermath of this paper, I need to note another key development in my thinking about information that occurred in 2005. That year, my friend and colleague Jay Richards was guest editing a special double issue of Philosophia Christi, the journal of the Evangelical Philosophical Society. This issue focused on intelligent design. Jay wanted me to address how exactly specified complexity constituted a type of information. Till now, specified complexity was less a form of information and more the basis for a type of statistical inference. The challenge then was to say exactly how specified complexity or complex specified information was information in a conventional information-theoretic sense.
The paper I submitted to this special double issue was titled “Specification: The Pattern that Signifies Intelligence” (available here). Looking back at this paper, I don’t think it’s all that great. It rehashes a lot of what I had already written in my 2002 book No Free Lunch. What it calls specified complexity incorporates not only what has become today specified complexity in its full information-theoretic form but also incorporates what I defined as probabilistic resources, both replicational and specificational. If you will, I overcomplicated specified complexity in this paper.
Probabilistic resources denote the number of opportunities to bring about an otherwise chance event. The more such opportunities, the more likely the event is to happen regardless of the event’s inherent improbability. Probabilistic resources can be distinguished as replicational and specificational. Think of replicational resources as arrows in your quiver and specificational resources as targets so that hitting any of them counts as a success. Imagine, for instance, shooting an arrow into a forest and hitting someone. If the explanation for hitting someone is chance, the actual chance of hitting someone will depend on how many arrows are in your quiver (replicational resources — the number of shots you can get off) and also how many people are in the forest (specificational resources — the number of targets that can count as success).
As it is, I factored into my definition of specified complexity in the Philosophia Christi paper these probabilistic resources. This was ill-advised because it made specified complexity depend on the relevant probabilistic resources in a given situation, and these could vary. In the lead up to my definition of specified complexity in that paper, however, I defined what I called specificity, denoted by a Greek lower-case sigma:
The first term under the logarithm, the lower-case phi term, turned out to be an exponential of Kolmogorov information. The second term under the logarithm clearly denoted a probability. In fact, this definition proved to be equivalent to the definition of specified complexity that Winston Ewert articulated for a 2012 conference paper:
When Bob Marks shared with me Winston’s definition of specified complexity, I pointed out that my 2005 formulation was equivalent (the phi in my equation was the same as the K in his, albeit raised the power 2). Winston and Bob, who had written up the latter formulation of specified complexity, graciously added me as a co-author. I say graciously because even though what I had come up with was equivalent, it was not nearly as clean and clear as what they produced.
When Winston and I collaborated on the second edition of The Design Inference in 2023, we provided the latter definition of specified complexity, which is to say a difference between Shannon and Kolmogorov information (subject also to the condition that the underlying language was binary, prefix-free, and Turing complete). This definition gives specified complexity a fully mainstream information-theoretic formulation.
Breaking Into the Peer-Reviewed Engineering Literature
To return to my 2005 unpublished paper searching large spaces, it was after I posted it on my blog that Bob Marks and I started collaborating and subsequently that Winston Ewert and George Montañez came on board as graduate students of Bob’s. The idea in the searching large spaces paper was to substitute for searches probability measures that calculated the probability of successfully finding targets. The idea of displacement then took the form of searching for a search. The original search was thus displaced to searching for a search that with higher probability would conduct the original search. Thus, in place of finding a treasure on an island, a displaced search would be finding a map that helps you find the treasure. The point of conservation of information was then to show that the search for the search did not improve the probability of finding the original target.
Bob and I ended up reframing this search for the search idea and calculating information costs associated with it in terms of an information-theoretic apparatus that compared endogenous information, exogenous information, and active information. The two key papers we wrote in this regard were as follows (available here):
William A. Dembski and Robert J. Marks II “Conservation of Information in Search: Measuring the Cost of Success,” IEEE Transactions on Systems, Man and Cybernetics A, Systems & Humans, Vol.5, No.5, September 2009, pp.1051–1061
William A. Dembski and Robert J. Marks II, “The Search for a Search: Measuring the Information Cost of Higher Level Search,” Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.14, No.5, 2010, pp. 475–486.
The latter paper, in an appendix, provided the key proof from my 2005 searching large spaces paper, showing that the search for a search only exacerbates the information problem of lower-level search. The result here was formulated against a uniform probability baseline. In the paper immediately below, the result was generalized for arbitrary probabilistic baselines that do at least as well as uniform random search (it always being possible to do at least as well as uniform random search):
William A. Dembski, Winston Ewert, Robert J. Marks II, “A General Theory of Information Cost Incurred by Successful Search,” in R.J. Marks II, M.J. Behe, W.A. Dembski, B.L. Gordon, J.C. Sanford, eds, Biological Information: New Perspectives (Singapore: World Scientific, 2013).
About-Face from Springer, Acceptance by World Scientific
The latter paper was ready in 2011 and should have been published in 2012, except that Springer, which had agreed to publish Biological Information: New Perspectives, withdrew the book from publication a few weeks before it was scheduled to be published and made available for sale on Amazon. I mention this not because it is directly pertinent to the unfolding of my understanding of information, but to indicate the sort of opposition that intelligent design, even when focused on scientific issues such as information, engendered.
As it is, I had negotiated with a Heidelberg editor of Springer to have the proceedings of a 2011 Cornell University conference on information (organized by Cornell professor John Sanford) published through Springer. My Dad had done his PhD in biology at the University of Erlangen (a German university) and I had an uncle who had been on the faculty of the Technical University of Berlin. I also had lived in Germany for several years and was fluent in the language. So this Springer editor and I, because of my German background, hit it off and we agreed to have coffee or lunch if I should ever be his way.
Everything was set to go. The book had been typeset. Amazon announced it for preorder. And then, with less than six weeks to publication, anti-ID forces sprang into action and got Springer to pull the book. I’ve kept the high-res book cover as well as the Amazon screenshot of the book as it was set to be published by Springer (volume 38 in their series Intelligent Systems Reference Library). To this day, my colleagues and I in the ID community, when we are poised to get some major work published, whether with a secular book publisher or a peer-reviewed journal, are careful not to publicize the work until it is already in print (and sometimes not even after that if we are looking to publish a sequel). Here’s the Springer cover:
And here’s the Amazon screenshot (because my screenshot was an old html, I used AI to trim it and recover the book cover):
As a result of Springer pulling these proceedings, it took an extra fourteen months to get the book published. Fortunately, Bob Marks was able to persuade World Scientific in Singapore to publish the proceedings. They did a good job with the printing and, through some sort of subvention, the book was made available to download for free, at least for a time, which may have gotten the book into more people’s hands than with Springer. So it all worked out in the end. Here’s the cover of the book as it finally did appear in print:
By the early 2010s, much of the fundamental technical work for conservation of information was in place. Consequently, Bob Marks, Winston Ewert, and I published in 2017 the following book, which covered concrete examples of how evolutionary computing failed to create information from scratch and also presented the underlying theory for why this invariably happened: Robert J. Marks II, William A. Dembski, and Winston Ewert, Introduction to Evolutionary Informatics (Singapore: World Scientific, 2017). Note again World Scientific coming through for us.
My 2014 University of Chicago Talk
What I’ve recounted so far brings my history with information to about 2013. In 2014, my former physics supervisor Leo Kadanoff invited me to lecture on conservation of information at the University of Chicago’s Computations in Science Seminars. You can see the schedule for the seminars given in 2014 by clicking here—scroll down to August 13, 2014 to see my seminar, titled “Conservation of Information in Evolutionary Search.”
Although my talk that August day didn’t break any new conceptual ground, instead recapping previous work by colleagues and me, it did provide a suitable capstone for that work up to that date. Leo’s people arranged everything, including travel and lodging. At the talk itself, I was happy to see Paul Nelson in the audience, who used his Mac laptop to record the talk—which is available on YouTube and given at the end of this section. At the talk also was James Shapiro, a key figure in the Third Way of Evolution. He and I visited in his office after the talk. We had some history before this meeting, going back to 1997 when he and Mike Behe were speaking at Wheaton College together. In any case, it was refreshing to hear in his office his skepticism of Darwinism.
Leo himself was never an ID guy, but he was sympathetic, if not for the actual ideas of ID, then at least for the right of academics to engage the topic and what he also took to be the benefit to the academy of doing so. In the lead-up to my talk, after it had been announced on campus, Jerry Coyne, a prominent Darwinian biologist at the University of Chicago, tried to talk Leo into disinviting me. Leo shared the entire correspondence with me. The thing that Leo stressed to Jerry is that he thought members of the seminar would profit from hearing me. That for him settled the matter, and all Jerry’s efforts to disinvite me thus failed.
As I write this, I recall an exchange with Leo as I was graduating from the University of Chicago with my math PhD to take an NSF postdoc at MIT. He remarked, and it seemed almost cryptic at the time, that I needed to be prepared that the recognition of my work might fail to keep pace with the quality of my work. He said it differently, but that was the gist. I wondered at the time (1988) if his words might be prophetic.
Regarding my work on information, Leo never formally endorsed it. But in an email to me before the talk, he wrote this:
As you well know, information is the primary topic in 21st. Century science. Even my own work on phase transitions is often phrased as how one part of a material communicates with another. Like matter, information flow can in some situations be guided by intelligence and in others can flow without the aid of a guiding thought.
Amen to that.
At my talk, Leo introduced me and also wrapped things up. A question after my talk challenged me by claiming that evolution was not a search. I had argued that it was and that its targets could be identified independently of intelligence and thus didn’t stack the deck for or against intelligent design. I then went on to point out that if evolution was not a search, then evolutionists had no rigorous mathematical model for evolution, in which case they were not doing science but at best hoping to do science (I’m paraphrasing a bit here, but the gist in my answer was to put the onus on the evolutionists to explain exactly how what they are doing can rightly be regarded as scientific). Leo ended the seminar by saying that the burden was on my critics to respond. It was satisfying to hear this from a man for whom I had such enormous respect.
After the talk, Leo, a postdoc (I think it was a postdoc rather than faculty member or grad student), and I went out for a meal at a local Vietnamese restaurant. This was the last time Leo and I saw each other—he passed away the following year. Here is a picture of our time at the restaurant, taken by the postdoc:
As I mentioned, Paul Nelson took a video of my talk with his Mac laptop. He made it available to me and Bob Marks. Bob kindly inserted into Paul’s video my PowerPoint presentation so that my slides could be followed more easily by viewers. The talk, thus enhanced, was then uploaded onto YouTube. You can see it here:
The Metaphysics of Information
Besides my scientific work on information, in 2013 I began to write a book on the metaphysics of information. The background for this book goes back to 1998–99, when I applied for and received one of six Templeton Foundation book awards in science and religion, each worth $100,000. In applying for the award, I had to submit a detailed book proposal. Specifically, I proposed to write a book on the science and metaphysics of information. I decided to turn this into two books, one on the science of information, which appeared in 2002, titled No Free Lunch, and the other on the metaphysics of information, titled Being as Communion.
I kept delaying the latter book because it was meant to bridge information theory with evidence of purpose in nature, and that connection required intense preliminary research by me on the scientific underpinnings of information, research that came to fruition with colleagues at the Evolutionary Informatics Lab (as just described). Only with the science firmly in place did I feel justified pursuing its metaphysical implications. Thus, I finally felt in a position to write Being as Communion in 2013, wrote it in about twelve months, and then published it with Ashgate’s science and religion series in 2014. Being as Communion, many years overdue, finally made good on my Templeton award.
In this book, I present an informational realism in which the fundamental stuff of the world is information. Specifically, what demonstrates something’s reality is its ability to exchange information with other things. The ontology here is relational, but not purely relational in the sense of making relations more real than the things related. The things exchanging information are fundamental, but they make their reality felt through information.
My approach was neither Aristotelian nor Platonic. Contra Aristotle, I do not treat matter as an independent co-principle in which form inheres. Contra Plato, I do not posit a separate realm of static forms of which particulars are shadows. I take as my point of departure what Claude Shannon, in his classic 1949 book The Mathematical Theory of Communication called the “schematic diagram of a general communication system”:
This diagram is usually confined to humans sending signals across electronic communication channels. Nonetheless, it suggests a very general approach to information creation. According to Shannon’s diagram, information creation is essentially a speech act in which an intelligence, to advance a purpose, articulates an item of information and then broadcasts it. We obviously see this mode of creation in human speech acts. Nonetheless, speech acts, broadly construed, are able to characterize how intelligences create information quite generally. As I remark in chapter 20 of Being as Communion:
In our experience, information is created in any number of ways. Painters create information by applying dabs of paint to canvas, sculptors by chiseling away at stone, musicians by writing notes on lined sheets of paper, engineers by drawing up blueprints, etc. Yet, in general, all such acts of information creation can be represented in language. For instance, a sufficiently elaborate set of instructions from a natural language can, with arbitrary precision, tell the painter where to apply the colors, the sculptor how to chisel the stone, the musician where to place the notes, the engineer how to draw up the blueprints, etc. Language, conceived as a speech act, in this way becomes the universal medium for creating information and thereby bringing purposes to realization. This is a linguistic picture of teleology. In this picture, purposes remain inert, ineffective, in limbo until appropriate items of information that advance the purpose are articulated and broadcast. Only in this way do purposes move from potentiality to actuality.
I’m giving a flavor of the ontology I develop in Being as Communion. Ultimately I relate this ontology to the Trinitarian God, who becomes the source of all reality by forming intentions (as Father), concretizes these intentions in words (as the Son or Logos), and then empowers them through breath (as the Holy Spirit). In 2022, I applied this ontology to the mind-body problem in a chapter titled “How Informational Realism Dissolves the Mind-Body Problem,” which I contributed to the collection Minding the Brain: Models of the Mind, Information, and Empirical Science, edited by Angus Menuge, Brian Krouse, and Bob Marks.
The Design Inference’s Second Edition
After Introduction to Evolutionary Informatics and Being as Communion, I took a few years off from intelligent design to pursue business and other writing ventures. I felt I had become stale in my work on intelligent design and needed a break. Around 2020, I felt my enthusiasm for intelligent design return and was ready to get back in the saddle. One thing that spurred me on was the need finally to do a second edition of The Design Inference, published originally in 1998.
The big obstacle for me to do a second edition came from feeling at odds with my publisher, Cambridge University Press. My editor at Cambridge was Terence Moore. He was the business editor in New York, not the academic editor, Brian Skyrms, mentioned earlier, who had overseen the book’s peer review. Moore had been quite happy with the first edition of The Design Inference, which performed remarkably well. It had been the bestselling Cambridge philosophical monograph over the previous five years. I therefore approached him about whether Cambridge would be open to publishing a sequel that made explicit the connection between design inferences and biological evolution.
By that point, my own views in favor of intelligent design and critical of Darwinism were public knowledge. Pulling back the curtain on the inner workings of the press, Moore told me that members of the Cambridge Syndicate (especially one or two biologists) opposed my work on intelligent design, and that even if such a sequel were approved on this side of the Atlantic, the Syndicate would likely kill it once it went for approval on the other side of the Atlantic. Normally, the Syndicate rubberstamps books recommended on this side of the Atlantic. In my case, the Syndicate was going to make an exception.
Terence Moore was a stand-up guy, and I’m grateful for his plain speaking. Sadly, he died of cancer shortly after this exchange. As it is, I ended up publishing The Design Inference’s sequel, No Free Lunch, in 2002 with another press, Rowman and Littlefield. For many years, I didn’t look back. Instead, I simply kept extending the ideas and technical machinery inspired by the first edition of The Design Inference, without any thought of revising and upgrading it in a second edition.
But the thought that my publisher, Cambridge University Press, was not in my corner kept nagging me. Finally, in 2020, on impulse, I contacted Cambridge University Press and asked for the rights to the book back. Within a week, I had the rights back and now could do with the book whatever I wished. The sense of inertia I had felt about doing a second edition now vanished. To pursue the work in earnest, I therefore solicited a grant from Seattle’s Discovery Institute. They had supported the work on the first edition, and now generously provided a grant for the second. And so, the path was cleared for this second edition of The Design Inference.
The last four paragraphs above are virtually verbatim from the introduction to the second edition of The Design Inference, which was published in 2023, with Winston Ewert as co-author. In this second edition, all of the technical machinery from the first edition was cleaned up. Besides the design-inferential method described in detail in this new edition, we had hoped also to present our work on conservation of information. But the book had doubled in size from the first edition, and it seemed that a second book on conservation of information would better serve readers.
The Law of Conservation of Information
In 2023, when the second edition of The Design Inference was published, I would have said that all the important results about conservation of information were in place and that I would have been in a position to write a book on the topic simply on the basis of existing work that my colleagues and I had done. And I would have been wrong.
Conservation of information as I had developed it was a mathematical result that applied to search but where the actual mathematics had to be adapted to how the search itself was represented mathematically. There were thus different conservation of information theorems. There was a measure-theoretic version, as in my searching large spaces paper from 2005. There were also fitness- and function-theoretic versions, as stated and proved in a paper I did with Bob Marks titled “Life’s Conservation Law: Why Darwinian Evolution Cannot Create Biological Information,” available as a pdf here and published as a chapter in Bruce L. Gordon and William A. Dembski, eds., The Nature of Nature: Examining the Role of Naturalism in Science (Wilmington, DE: ISI Books, 2011). It may be my favorite information-theoretic paper from that period and, in my view, it is still worth reading.
Because there were so many conservation of information theorems, what I called the Law of Conservation of Information ended up not being a strictly mathematical result but more a meta-mathematical result, in the spirit of the Church-Turing thesis. As explained in “Life’s Conservation Law”:
The Law of Conservation of Information [LCI], in both scope and function, parallels the Church-Turing Thesis of theoretical computer science. The Church-Turing Thesis is a deep claim in mathematical logic about the nature of computation. LCI functions within the theory of intelligent design as the Church-Turing Thesis functions within the theory of computation. According to this thesis, given a procedure that is informally computable (i.e., that is finitely specifiable and rule-based), it can be coded as an algorithm running on a Turing machine. According to LCI, given an alternative search that does better than blind search, the alternative search was purchased at an information cost no less than I+. The task of the design theorist in that case is to “follow the information trail” and show where the information that this search outputs in locating a target was first inputted (much as the task of the computer scientist is to show how some procedure that is informally computable can be explicitly formulated as an algorithm capable of being run on a Turing machine).
It follows that there is no—and indeed can be no—strict mathematical proof of either the Church-Turing Thesis or LCI. Nonetheless, the two are subject to independent verification. With the Church-Turing Thesis, the challenge is to show that any operation that is informally computable is also, upon a close mathematical analysis, formally computable (to date, this has always been confirmed). Likewise with LCI, the challenge is to find where the information that improves an alternative search over a blind search was inputted and then to calculate the amount of this information, showing that it does not fall below I+ = log(q/p) (to date, this has also always been confirmed).
Until November of 2024, that was my position. Come up with any reasonable mathematical representation of a search, and a (indefinite article) conservation of information theorem can be proven for it and shown to apply to it. But then in November of 2024, I was in San Diego at an Evangelical Philosophical Society symposium on intelligent design moderated by John Bloom of Biola University. There both Doug Axe and I gave presentations.
At a moment in his talk that proved critical for me, Doug gave an example in which an event E denotes an arrow shot by an archer hitting a target and an event F denotes a wind blowing the arrow to the target. The events E and F don’t belong to the same probability spaces, E to a space characterizing an archer’s shooting, F to a space characterizing wind blowing. And yet, by thinking of F as a search for a search, I suddenly realized that the Law of Conservation of Information was not meta-mathematics in the sense of Church-Turing, but a direct consequence of a very simple probabilistic relationship, one I’ve dubbed the conservation of information inequality:
This inequality is a very elementary probabilistic result. And yet, because E and F belong to different probability spaces, this inequality needs to be interpreted in a way that allows probabilities to be applied to E and F in concert. As it is, Bayesian probabilities do just this in case of E varying over events and F varying over hypotheses. But if we think of events as hitting targets and hypotheses as characterizing search for a search, then this inequality captures the fundamental claim of conservation of information.
Accordingly, if we think of E as hitting a given target and having small probability, and if we think of F as a search for a search that makes it highly probable that the target will be found, then F must itself be highly improbable. All of this is captured in this inequality, which then ensures that conservation of information is a result of pure mathematics and not merely a type of mathematical result that has been widely verified in specific cases and has to date escaped all counterexamples. It is thus a law in the strongest possible sense.
There are a lot of details here, and the result is nontrivial even though the conservation of information inequality looks utterly elementary and straightforward. I wrote all of this up in a monograph published in BIO-Complexity that appeared in August of 2025. Here is the abstract for that monograph. I include this abstract here because it recaps much of our discussion about conservation of information in this history.
Conservation of information sparked scientific interest once a recurring pattern was noticed in the evolutionary computing literature. In grappling with the creation of information through evolutionary algorithms, this literature consistently revealed that the information outputted by such algorithms always needed first to be programmed into them. Thus, the primary goal of this literature—to uncover how information could be created from scratch or de novo—was shown to be misconceived: the information was not created but instead shuffled around or smuggled in, implying that it already existed in some form or other. Information output in these situations therefore always presupposed a counterbalancing input of prior information.
Once this pattern was seen, the next logical step was to quantify the amount of information inputted and outputted, demonstrating a consistent mathematical relation between the two. This led to the proof of a number of theorems about search. In these theorems, a baseline search with probability p of success gave way to an improved search with probability q of success. Typically p would be very small and close to zero, implying a practically impossible search (like searching for a needle in a haystack). By contrast, q would be much larger and close to one, implying an eminently doable search. The punchline of these theorems was that, as the improved search became itself the subject of a search (a search for a search, or S4S), the probability of finding it could not exceed p/q, rendering success of the improved search no more probable than success of the original baseline search, in effect filling one hole by digging another.
Such conservation-of-information theorems, as they came to be called, were search-space specific, adapted to different kinds of search across a range of search spaces. There was a measure-theoretic theorem in which probability measures guided search. There were also function-theoretic and fitness-theoretic theorems where mappings into the search space as well as fitness functions on the search space respectively guided search. The key insight of this paper is that all these conservation-of-information theorems are special cases of a simple probabilistic relation based on elementary probability theory. This paper identifies the underlying rationale that makes all the previous conservation-of-information theorems work. In so doing, it provides a straightforward proof and general formulation of what may rightly be called the Law of Conservation of Information.
A Theory of Intelligence Metrics
To round out my personal history with information, I need to mention my recent work on intelligence metrics. In 2023, I was invited to speak at the CELS (Conference on Engineering in Living Systems, 2023) outside Dallas, Texas. There I presented a PowerPoint presentation on intelligence metrics. These intelligence metrics assign numerical degrees of intelligence via probability and information measures. This September I published a long paper (36,000 words) arising from my presentation at that conference. It is titled “A Theory of Intelligence Metrics: Measuring the Degree of Intelligence in Search.” It is currently undergoing peer review. Here is the abstract, which for readers of this history, will by now touch many familiar themes:
The study of intelligence raises two key questions: (1) How can intelligence be detected or inferred from its effects, and (2) how can the degree of intelligence in effects be measured? To date, the first of these questions has received the bulk of scholarly attention. Thus, a standardized inferential method has emerged for SETI, archaeology, forensic science, and the historical sciences generally in which some trace, signal, or pattern is observed and the question then becomes whether it is best explained as the result of intelligence. This paper focuses on the second of these questions. It develops a general theory of intelligence metrics based on information theory. The proposed metrics assign numbers to the intelligence exhibited in successful search. This paper therefore treats search as the universal paradigm for intelligence. By framing the measurement of intelligence in terms of search, this paper avoids presupposing any particular view of intelligent causation. Instead, the theory presented here applies across diverse causal settings, including human problem solving, large language models, and evolutionary processes. It thus provides an overarching informational framework for comparing forms of intelligence, for assessing the efficacy with which intelligences use informational resources, and specifically for determining how much intelligence is manifested in successful search.
And that brings my personal history with information up to date.
Thank you for this excellent summary of your personal history with information. Reading this essay was a delight, as it put into perspective your life's work on information and how it evolved in your mind, which is outstanding.
Thanks for this excellent article! It’s fascinating to read of your journey with this important and groundbreaking work!