On the origins of creativity

I’m not a very creative guy. I had an idea back in the 1970s, but I managed not to do anything about it in time for someone else to do something with an almost identical idea. I think I dodged a bullet: once you come up with one great idea, they’re going to want you do come up with another, and so I would have ended up even more of a disappointment to my mother. It’s best not to raise expectations.

But John Cleese is somewhat creative, and so this video of a talk he gave in Germany recently is interesting. Cleese asserts that he gets his ideas on a postcard each Monday from some guy in Kent; in short that he doesn’t know the origins of ideas. Below the fold, I’m going to suggest one. It’s not original…

A similar problem arises in the context of evolutionary biology: what is the origin of novelty? It is a general problem of all historical processes. History is not just the recurrence of what has happened in the past, although if it doesn’t repeat, as Twain is reputed to have said, it rhymes. History is the rising of new things, and the novel arrangement of old things. At some point in the history of life, the first DNA molecule acting as a hereditable object arose. It was, of course, just what biochemistry, and ultimately physics, permitted (or it wouldn’t have been able to occur), but once it had, the elements of DNA were almost endlessly recombinable and rearrangeable, forming new and modified traits.

The “mother of AI” (a faintly sexist label, but commonly used) is Margaret Boden. Back in the 1990s she published an undeservedly unremarked book The Creative Mind: Myths and Mechanisms, and more recently Dimensions of Creativity. Boden’s analysis is that novelty comes in two “flavors” (my word): deep and combinatorial. Combinatorial novelty is the taking of elements of existing properties or ideas and combining them in novel ways. Einstein’s taking of Mach’s, Riemann’s, Lorenz’ and Duhem’s and others’ prior physical and mathematical speculations to generate special and later general relativity is a case in point. As significant as it was, his work would not have even been possible without conceptions going all the way back to Galileo.

Deep novelty, on the other hand, is where a truly new possibility arises. This is that “origins of ideas” that Cleese can’t account for. The “connectivity” account of historian Stephen Johnson which Maria Popova links to in the Cleese article is not deep novelty. It is combinatorial. Hunches are not explanations of deep novelty. They are just another way of saying we have them. How can we get a truly novel set of possibilities to explaore? Where do they “come” from?

I scare quoted “come” because that suggests they exist somewhere before we access them, in some Platonic realm. Platonism is an ever-present temptation in these matters, one that systematically misleads us. Novelty is not something that ideas have as a physical or metaphysical property as they insert themselves into our mundane world of problem solving, nor is it something objective in the biological world. The entire literature on “major transitions of evolution”, which John Maynard Smith and colleagues contributed to strikes me as fundamentally Platonic in that way. Stuff changes. It being a deep novelty is a fact not about the world, for anything that does happen necessarily could, but a fact about us (yet again: readers may have noted that many of the things we discuss on this blog turn out to be facts about us).

Deep novelty is this: we have a frame of prior contrasts in which we typically (and traditionally, since these are inherited from teachers and other cultural influences) set up our problems and thoughts. We may think of this as a Hilbert manifold; a space of many variables, possibly tens of thousands. It is a semantic space, set up by the contrasts we have previously employed to come to our beliefs. If you think that God may or may not exist, for example, then believing God does exist is to assert a coordinate in a binary space. If you think God’s existence is a matter of confidence or likelihood, then you settle on (if you do) a coordinate on a continuum.

Our semantic world is the sum of all the contrastive axes of that space. Our beliefs at any time are the coordinates we assert. Our total belief set is the set of all coordinates in that space. For simplicity, suppose there are only three such contrasts, and hence that the semantic space is a simple 3D space. If you believe that x = 3, y = 5, and z = 2, then your entire belief set is a triple: <3, 5, 2>.

Okay, having set up the technical machinery needed, what is a deep novelty? It is this: something is deeply novel if an entirely new contrast is added to the space. In our toy case, if the dimensionality moves from 3 to 4. Now you have all the possible coordinates of a quadruple: <x, y, z, a>. To be precise, though, deep novelty also arises when you reduce the semantic space. That there are fewer coordinates available means that the space behaves differently.

Similarly in biological evolution, novel states are simply physically attainable alternatives, made possible by there now being ploymers of nucleotides that pair and can be read and expressed in various ways. Is this a new form of “inheritance” or “information” as Jablonka and Lamb argued?Well, yes, because that is how we represent it in our semantic model of things.

One of John Cleese’s most loved and original sketches is the famous parrot sketch. In it, Cleese recites a number of synonyms for being dead:

‘E’s not pinin’! ‘E’s passed on! This parrot is no more! He has ceased to be! ‘E’s expired and gone to meet ‘is maker! ‘E’s a stiff! Bereft of life, ‘e rests in peace! If you hadn’t nailed ‘im to the perch ‘e’d be pushing up the daisies! ‘Is metabolic processes are now ‘istory! ‘E’s off the twig! ‘E’s kicked the bucket, ‘e’s shuffled off ‘is mortal coil, run down the curtain and joined the bleedin’ choir invisibile!! THIS IS AN EX-PARROT!!

The originality here has to have come from Cleese idly leafing through a thesaurus one day. The deep novelty is using a thesaurus to write a comedy sketch. If I did it, it would be merely combinatorial. And really really bad: substandard, poor, inferior, second-rate, second-class, unsatisfactory, inadequate, unacceptable, not up to scratch, not up to par, deficient, imperfect, defective, faulty, shoddy, amateurish, careless, negligent, miserable, sorry; incompetent, inept, inexpert, ineffectual; awful, atrocious, appalling, execrable, deplorable, terrible, abysmal, godawful; informal crummy, rotten, pathetic, useless, woeful, bum, lousy, and not up to snuff. This is an ex-post.

18 thoughts on “On the origins of creativity

  1. Um, I tend to define creativity a little more broadly. For example, a friend of mine from reasonably humble beginnings (and who could never satisfy her mother’s expectations) went on to raise her children to be healthy, happy, creative and confident. This is despite having to do the bulk of domestic chores inside and out, keep her whole family looking clean and well groomed, earn at least half of the family income, take regular courses to keep her qualifications current, manage some fairly serious personal health issues and put on a cheerful face to the world so nobody could tell she was often just holding things together. My friend is no rocket surgeon but, she used every ounce of her average intellect to manage this amazing feat by having the most incredible sense of focus to the task at hand. She has a number of creative talents of which I am only just managing to convince her and with a bit of luck she’ll be able to explore some of these now she has the time on her hands.

    I guess my point is, presiding over a vehicle like Evolving Thoughts is no small thing and very definitely creative. For the most part I despair at many of the things that are wrong with the world that could be better handled if only for the application of a bit more intellect and reason. While not everything in this journal flips my pancake it’s also true you write the occasional gem that provides me with some sense that the world isn’t inhabited entirely by bogans.

    While it’s great if we can at some point get our parents’ approval, the sad fact is some parents are never satisfied unless they can bask in the glow of a child who has somehow manage to achieve godlike proportions. While I was fortunate to have achieved a couple of things that thrilled my parents, I also engage in the occasional imaginary discussion with my late father while I’m doing something I think is creative and worthwhile. To date he has always approved and the exercise – although essentially a mental trick – goes someway to providing whatever it is that parental approval provides.

    Apologies for waxing sentimental. Your piece caught me during one of those moments.


  2. “Deep novelty, on the other hand, is where a truly new possibility arises.”
    OK. I’m stumped by the notion of a “new possibility.” I can easily grasp how something that was a “mere” possibility can become actual. If something is possible (or impossible), well, isn’t that the final word on the matter?


    1. If something is possible (or impossible), well, isn’t that the final word on the matter?

      It is, unless you were wrong.

      Bayesians tell each other to use Cromwell’s rule so as to believe everything possible that does not entail a logical contradiction. This is easy advice to follow, if you can actually conceive in advance of every possibility. John, it seems, is interested in the possibilities of which no one succeeded in conceiving in advance.

      Think of a category called “unimagined” which is shoehorned into a space of measure zero between “possible” and “impossible”. Sometimes things which were previously unimagined become imagined, and hence possible (or maybe impossible). This occurs as a result of an act of creativity.


      1. Yes. Bob, note that I rate originality and novelty as semantic properties – likewise I also think that possibility in the sense required here is also semantic. So a novel possibility is something that is a change in semantic space, that’s all.


  3. PiP – You’re remarks address only epistemic possibilities. What is possible to believe/conceive/imagine at T1 is (presumably) not the same as what is possible simpliciter. Consider John’s example of a transition from a 3-dimensional to a 4-dimensional system of representation. If such a transition is possible, then the 4-dimensional system and all its representational possibilities were already possibles prior to the transition. I’m not a modal logician, but I don’t think that possible possibilities are anything other than possibilities.


    1. PiP – You’re remarks address only epistemic possibilities.

      I know of no other kind. 🙂

      My point is that I should have allowed for the possibility of that 4th dimension in advance. But I didn’t, I thought it was impossible, but now realize my mistake. So, now what do I do?


      1. There have to be physical possibilities, in the sense of realisable physical states that are not realised. Of course, anything we can say about them are epistemic and semantic possibilities.


  4. Whether possible possibilities are just plain possibilities depends on the modal system you are using (at least as strong as S4, if I remember correctly); although determining what modal system is the right one to use in any given case is a tangled tangle of tangles in its own right.


  5. I want to highlight one aside in your post: “deep novelty also arises when you reduce the semantic space.” It seems to me that reduction is the usual mode of creativity. As Goethe wrote someplace, “Limitation makes the master.” The parrot bit illustrates the point because what is so very clever about Cleese’s language is not that it is exhaustive but that it creates the illusion of being exhausting without being very long at all. I think it was Umberto Eco who made a similar point by referring to a short story by Borges about Aleph, a place that contains all places. What was impressive to Eco was how Borges managed to evoke the notion of total plenitude with three or four examples. Less really is more, at least until we actually become a lot more intelligent than pigeons.

    I’m also impressed with the math of the thing. Assuming there is a finite set of objects in the universe (n), there is only one such set; but the power set, the number of subsets, is vastly bigger, specifically 2 to the nth. The ignorant may be benighted, but they have plenty of elbow room!

    Whether its music, with its systems of possible keys and modes, or poetry with its specialized vocabulary, rhyme schemes, or meters, or games with their arbitrary selection of rules, meaningful activities can often be understood as a combinatoric plus some rule of exclusion. Since there are immensely more possibilities than actualities, the problem is to explain how repetition is possible or even how we can find anybody else in this enormous desert. Foucault made this observation in his Archaeology of Knowledge, but nobody seems to have noticed.


    1. @Jim Harrison: I too would enjoy a guest post from you. until then:

      … or even how we can find anybody else in this enormous desert.

      well, the fruit doesn’t fall far from the tree, as it were: in addition to “a combinatoric plus some rule of exclusion” there is also a huge interlocking set of “initial conditions,” which include (a) the vast number of shared semantics that are the all-pervading field we make our distinctions out of (“shoulders of giants” comes to mind here), and (b) our actual “location” in the enormous range of possible possibles. getting to somewhere involves starting from somewhere, and the “distance” anyone can move is constrained.

      so we’re able to find each other because we just don’t get that far away.

      is there maybe some speed limit for us “moving” through the possibles? it wouldn’t surprise me to learn that, with our presumably finite brains, we are limited as to how much semantic change we can make / keep track of / agree on / utilize in a reasonable time frame. if so, I’d also expect a kind of observable-universe-style horizon within which we’re stuck: a “light-cone” of possible human understanding.

      I don’t think we come close to exploring that light-cone, by the way: as ever, creativity-absorbing stupidity surrounds us, and the cone itself is large. so we’ve got plenty of growth opportunities. but still, we seem, by virtue of our starting point and physical incarnation, to be limited to a much smaller region than any power set of possibilities.

      or, um, did I miss your point?


  6. Very interesting questions raised here. But I want to make two suggestions.

    First, I doubt there is the Boden distinction between deep and combinatorial novelty.

    Einstein’s and Newton’s novelties were taken as paradigms by Kuhn of new paradigms, by Feyerabend of systems logically incommensurable with what went before. In Newton, there were new notions e.g. of gravity (=weight; a body’s weight had been for over a millennium the paradigm of an intrinsic quality, with Newton it became a vector sum of weights-towards every other mass in the universe) and of mass etc., in Einstein new notions of e.g. time (relative simultaneity, anyone?). So, Feyerabend argued, the theories were not logically (deductively or inductively or even by logical clash) related to their predecessors.

    Newton, Darwin’s theory of evolution by natural selection, and quantum mechanics could be also taken as paradigms of new styles of explanation.

    But I think all these can be plausibly be presented as combinatorial novelties (I’m not on top of quantum mechanics to be sure enough of this).

    All that leads me to suspect that what is called deep novelty is just novelty whose genesis we haven’t yet understood enough.

    There are utterly intuitive arguments that there is deep novelty. How could one get entirely out of pre-linguistic elements? Or logical notions? Surely (pace Fodor) language, and logical notions, did originate. But how could these arise entirely, “combinatorially”, out of pre-linguistic, the pre-logical?

    Or consciousness, life, etc.? Which reminds us, surely, that evolutionists must be committed to there being answers to such questions.

    The second suggestion is somewhat Kantian; it’s that we have enormous difficulty understanding novelty, and that this is because of the way we understand: we understand by assimilating the new to the old, the unfamiliar to the familiar. Every judgment involves categorization, every categorization of something observed or heard of involves assimilating it to something previous. Hence e.g. the appearance of “universals”.

    Many have enormous difficulty in accepting that we ever decide or discover, because it seems unintelligible that we could; because only what is determined by what has gone before, or sheerly random, or some combination of the two, is considered intelligible. So it’s thought that real decision or real discovery must be illusory, hallucinatory. This is sometimes presented as mere opposition to one or other weird metaphysic of freewill; but it seems to me that it arises from assumptions that imply that the novel is unintelligible.

    I suspect that if there’s a deep problem here it’s that of providing a more adequate model, and even of developing more adequate ways, of understanding. But it can be done; it’s been done before, e.g. by Newton and Darwin.


    1. The Boden distinction is not, in my view, absolute. “Deep” novelty is, I believe a matter of representations only. But representations can evolve too. I don’t think we need Kantian a prioria (or, if we do, they are Lorenz’s evolutionary a posterioria). Something is a deep novelty when we haven’t previously conceived of it. I don’t think that is all that mysterious. Likewise, biological novelties are things that physics can do and happens to have for the first time on one planet. I must be very dense, because I don’t see the problem with consciousness and the like.


  7. FWIW, I think that possibilities were maximal at the moment “just prior to” the big bang, and everything that has happened since represents a reduction in possibilities. If that’s right, then whatever creativity might be, it isn’t an expansion of the range of possibilities — quite the opposite. Of course, I’m talking about de re possibility…


  8. These are just a few thoughts

    I think we need to be careful when referring to an idea as original or novel. Are we saying merely that is something we have never met before or is it something that has never occurred to any intelligent being anywhere in the Universe – something which we have no way of knowing at present?

    It seems to me also that it links to the origins of life the Universe and everything, what the IDeologues try to narrow down to the origins of information.

    As I understand it, contemporary physics is unable to tell us anything about the nature of the singularity that is hypothesized to have existed before the Big Bang. We don’t know why or how it went ‘bang!’ when it did. So we have no way of knowing whether the seeds of everything we see around us could be found in there.

    I am probably putting this badly but, as I understand it, after the period of rapid inflation the Universe comprised a staggeringly hot “quark soup”. If we had been able to observe it (without being instantly vaporized) could we have predicted the later states of the Universe, such as the emergence of hydrogen and the fundamental forces from that study, indeed, could they have been predicted at all?

    If we come further forward in time to when hydrogen and the four fundamental forces had ‘condensed’ out, could the present nature of the Universe, including us, have been predicted from those simple beginnings.

    In other words, is what is called combinatorial novelty a sufficient explanation? If it is not, then doesn’t that imply that something is being added from elsewhere and, if that is the case, doesn’t it raise the thorny questions of what and from where?


  9. I started thinking about this John, and . . . .

    You’re talking about semantic space as though it were a Platonic ideal somewhere out there in the ether. But real semantic space has to be physically embodied. And that’s tricky because there’s going to be a difference between the semantic space of a culture and the semantic space of an individual, but just how we talk about that is not clear to me. The only physically embodied semantic spaces are those in the brains of individuals. But they’re not going to be identical among individuals. No one individual will embody all the dimensions. The semantic space of the culture consists of all the dimensions existing in the semantic spaces of all the individuals in a culture.

    Given some one individual, that person can enlarge his semantic space either by acquiring a dimension from another individual (or individuals) or by introducing one that’s new to the culture. Presumably the latter is what you have in mind when you’re talking about deep creativity. [And forgetting about the case where a dimension is dropped.]

    Over time, as new dimensions are discovered and added to the collective space, the dimensionality of that space will increase. Sounds like ‘progressive’ evolution without a telos. Which is fine by me.

    As for the business of dropping dimensions. I note that we’d have to be talking about dropping some dimension from the space of every individual that embodies it. That’s the only way a dimension could be dropped from the culture. Beyond that . . . Well, I find it hard to use this particular metaphor.


    1. It might seem as if these spaces are platonic, but nothing is further from my mind. They are constructs (hence, “semantic”) of social contexts and conversations. In biology, they are merely physical states that were already potential, but only in the sense that any physically possible state is possible.

      So stick with semantic states. These do not exist in (or just in) the heads of agents in a culture. They exist primarily in the relations of meaningful behaviours, not as repositories in the minds of individuals. The coordinates as such are in the heads, or implicit therein, but not the spaces. So I think your objection fails. This was a view I argued for over 12 years ago.

      Of course, we can represent those semantic spaces in heads, and of course we do, but the representation and the spaces are not the same thing.

      As to whether an entire culture can drop a dimension here, consider the shift from pre-Milesian to Milesian philosophy. Previously, gods acted on a whim and the natures of things were arbitrary. Subsequently, gods were constrained by natures like everything else. Sure, it took a while for that view to spread (and arguably we haven’t eliminated the older view even now) but subcultures at the least (such as scientific cultures) have dropped these dimensions.

      I think we overestimate the accretion of dimensionality in culture. We are constrained by several factors: the time it takes to acquire a cultural tradition, the cost and rate of communicating it, and the number of heads which can be turned to that task. Overall, I think that culture has increased in diversity just to the extent that the population size has risen and our abilities to record and disseminate information have improved. All the rest is still making do with the usual human suite of capacities, and they haven’t changed significantly in the last 20,000 years.


      1. So stick with semantic states. These do not exist in (or just in) the heads of agents in a culture. They exist primarily in the relations of meaningful behaviours, not as repositories in the minds of individuals. The coordinates as such are in the heads, or implicit therein, but not the spaces.

        I’m afraid it sounds like either a Platonic ideal or an analytic fiction, which, of course, are different things.

        If you’re talking about the latter, then I want to know what it is that the deeply creative thinker does that requires the addition of a dimension to the analytic fiction. He does something to his own mind, surely. And if he doesn’t succeed in inducing others to do a similar something to their minds, well then, he can’t communicate his discovery. As far as the population is concerned, he’s talking gibberish.

        I think we overestimate the accretion of dimensionality in culture.

        As far as I can tell, we have no estimates whatsoever.

        There’s an empirical literature on cultural complexity that’s mostly about preliterate cultures. Various thinkers created various scales and then looked up 10, 50, 100, 300 cultures in the HRAF files and rated them on these scales. But, as far as I know, the scales are all ordinal scales. You can say that culture G is more complex than culture E, that culture E is more complex than culture B and that, therefore, G must be more complex than B. But nowhere can you say how much more complex one culture is than another. All you can do is say that you ran, say, 87 cultures on your scale, and this one can out at the low end and these three came out togother at the top end. The one at the bottom end might be a hunter-gatherer culture living in groups of 30 or 40 and with a total population of 1100. At the top you might have the Aztecs at the height of their development, with whatever their total population was and with some rather large cities. Further, you might have, say (I’m making this number up), 75 craft specialties at the top and none at the bottom (meaning that any adult is capable of making or doing whatever is necessary to live). So there’s a big difference between top and bottom. But we have no way to put dimensions onto these cultures. It may just be possible that the dimensionality is the same all the way through from our hunter-gatherers to our Aztecs.

        As for you example, I don’t know what Milesian philosophy is. Given your remarks, as far as i can tell it makes just as much sense to say that, with the addition of a dimension, it became possible systematically to constrain the behavior of the Gods. Without some fairly specific conventions about how objects and behaviors are to be represented in some formal system, I don’t see how to argue it one way or the other. So, you wave your hands in one way and say we’re dropping a dimension and I’ll wave my hands in another and say we’re adding one.


Leave a Reply