Pattern recognition: neither deduction nor induction

NewImage

It occurs to me as I read Rosenberg’s Philosophy of Science (2005), that we tend in that field to classify epistemic activities into two kinds: induction (about which we have many arguments as to its warrantability) and deduction (with many arguments about its applicability). But I believe there is something else that we do to learn about what exists in the world. In my forthcoming book, The Nature of Classification, coauthored with Malte Ebach, I argue that this is classification, but typically classification is seen as either of the other two kinds of inference. I think it is a third kind.

So what happens when we classify in the absence of theory? We aren’t yet inductively constructing theory, and we aren’t able to deduce from theory (since there isn’t any yet) the classes of objects in the domain we are investigating. We argue that what is happening here is pattern recognition (Bishop 1995). We are classifier systems. It is one of the distinguishing features of neural network (NN) systems such as those between our ears that they will classify patterns. They do so in an interesting fashion. Rather than being cued by theory or explanatory goals, NNs are cued by stereotypical “training sets”. In effect, in order to see patterns, you need to have prior patterns to train your NN.

Where do these come from? I think that there are several sources. One is evolution: we are observer/classifier systems of a certain kind. This gives us a host of cue types to which we respond by training our stereotype classifier system. For example, we respond to movement of large objects, to differences in colour and shade, and so on, in our optical system. Quine (1953) referred to this as our “quality spaces” – these are fields of discriminata, to which we (in Quine’s view, behaviouralistically) react. They are adaptations to the exigencies of survival for organisms of the kind that we are. The problem is that so long as our survival and reproductive success is ensured, evolution cannot guarantee us access to the way things “really” are. At best it gives us a good balance between false positives and false negatives. It is good enough, as it were, for government work (Godfrey-Smith 1991). But is it good enough for science and metaphysics?

One of the standard accounts of the success of science is that it increasingly approaches the truth. This is called the Ultimate Argument for Scientific Realism by van Fraassen (1980) and the Miracle Argument by Putnam (1975, that unless science does converge on reality, science would be a miracle). It is quite clear that the received dispositions evolution has bequeathed our cognitive capacities is not enough. While one might reject the Plantingan argument against all naturalism based on this insufficiency of our evolved cognitive powers (Plantinga 2002), there is a problem. How do we come to identify aspects of the world reliably and properly?

Science proceeds by refining its categories of what exists in the world based on two main sources. These are evidence, and explanatory force. In the case of a domain of investigation for which there is as yet no explanation, all we have is evidence, but apart from our evolved dispositions to respond to certain stimuli, how do we identify the salient aspects of evidence? There is an almost infinite amount of possible information we might make use of, and so we must glean the right sources of information. One source is economic necessity. Over time, farmers and hunters will tend to respond to the features of the things they are engaged in acquiring and using that are more or less important for success, because those features which are not salient will impose a cost of time and effort that tends to reduce success. This is a process very like natural selection, and has been the basis for what came to be known as evolutionary epistemology, in which a parallel process to biological evolution occurs in the domain of knowledge. Cognitive traditions become better at acquiring reliable knowledge because ideas and approaches that do not aid this goal are costly and are abandoned.

However, we have a superfluity of cognitive and conceptual resources. We can retain ideas and practices that are not really relevant for social reasons, such as rituals and “explanations” that have no counterpart with the reality being dealt with. So the fact that a particular culture is successful at farming by relying upon a ritual calendar (as in pre-Indonesian Bali) doesn’t warrant belief in Hindu gods. The functional aspects of the rituals acts to transmit the information even if nobody in the culture (or in Western agribusiness) fully understands why those rituals make farming successful (Lansing 2007).

So when a classifier recognises patterns in economic circumstances, what counts is not the conceptual superstructure, the theories and ideologies, but the categories of what matters – in this case of water, soil, and landscapes. How might this explain the success of science?

Taxonomists are classifiers in a particular economic situation: professional science. When a taxonomist encounters organisms in the wild, they are in the same situation as when a hunter hunts in that ecology. To succeed at taxonomy, as to succeed at hunting, the agent must know the right things about the target objects. A hunter that doesn’t know what different species of bird look like and how they behave and where they live is in exactly the same economic conditions as a taxonomist who also lacks knowledge. Neither will end up with dinner on the plate (qua hunter or taxonomist). In the case of the taxonomist, the gap between failure and hunger is somewhat more distal than for the hunter (but hunters typically get most of their food from foraging rather than hunting anyway, courtesy of the non hunters, mostly women, in their village), but ultimately economic success depends directly upon correct pattern recognition.

Ernst Mayr was fond of telling the story of how when he visited Papua in the 1930s, he and the local hunters identified the same species of bird, with an exception where western ornithologists also disagreed, and he used this as justification for the reality of those (and all) species. He made the inference that science was able to discover kinds of things that were real in the world, and he may have been right (many biologists and philosophers believe species are not real), but it was not, I think, because of the pattern recognition abilities of humans per se to see species. When Ed Wilson tried the same experiment about ants, a subject he knows intimately, instead of the locals counting the same species he did (several dozens) he got something like “the black ones, the bitey ones and the red ones”. Why did Mayr’s informants know their birds while WIlson’s did not know their ants? The answer is that birds, but not ants, were of economic importance to the locals, while ants were of economic importance to Wilson and entomologists only.

By “economic” I do not mean fiscal, but the acquisition of resources, success at which gives the person investigating a living. What distinguishes scientific success is a unique socioeconomic system of professionalism, credit in society, and access to funds and resources like labs, students, and equipment. The motivations of the individuals concerned are several, often (but not always) based on personal curiosity, but curiosity is not enough if you don’t get the resources to do the work.

So we are very good at turning our perceptual pattern recognition systems to scientific work. What evolution provides, science refines. It happens that pattern recognition and the subsequent classificatory activities can deliver reliable knowledge of the world when it matters. But being as it is parasitic upon those evolved capacities, and being as scientists are social organisms, this is not without its failures. Social influences, particularly the inherited traditions of ritual and conception that history bequeaths, can skew and bias our categories about the world. This is where theory and experiment come in.

Science, by way of its historical accidents, also seeks to explain things in ways that can be tested. Here the ordinary philosophical issues come into play – we inductively generalise based on the patterns we have recognised, and form hypotheses, and from those hypotheses we derive deductive consequences, which we can test in ways that are not circular, which do not rely upon our original observations. As T. H. Huxley once said, nature whispers yes, but shouts NO! And so we can eliminate hypotheses that are not fit to the facts, more or less. This is what the evolutionary epistemologists, and philosophers like Popper, built their views upon. What evolutionary epistemology never explained, nor Popper, was how we came up with those hypotheses in the first place. Pattern recognition does.

For a half century or more we have had the view that observation is theory laden. As I have argued before (and which is part of our forthcoming book), observation need not be laden with theory of the domain under investigation. And what evolution has bequeathed need not be in the slightest theoretical, nor even reliable (as the massive literature on illusions shows us). We can naively observe things that we know little about, but we never start knowing, or at least being disposed to know, nothing.

So can we say that science is adequate to tell us the true nature of the world? Putnam’s miracle argument indicates a reason for thinking the world is knowable. If we could not know the world, there would be no reason to think that success indicated anything. And while success is not a guarantee of truth, it is as good as fallible knowers can ever achieve. In the end, I think that truth is, as the pragmatists said, what works. More than that is restricted to gods, demons and mathematicians.

References

Bishop, Christopher M. 1995. Neural networks for pattern recognition. Oxford, New York: Clarendon Press; Oxford University Press. [Sorry, I forgot to put this in]

Van Fraassen, Bas C. 1980. The scientific image. Oxford: Clarendon Press.

Godfrey-Smith, Peter. 1991. Signal, decision, action. Journal of Philosophy 88:709-722.

Lansing, J. Stephen. 2007. Priests and programmers: technologies of power in the engineered landscape of Bali. Princeton NJ: Princeton University Press.

Plantinga, Alvin. 2002. The Evolutionary Argument against Naturalism. In Naturalism Defeated? Essays on Plantinga’s Evolutionary Argument against Naturalism, edited by J. K. Beilby. Ithaca, NY: Cornell University Press:1-13.

Putnam, Hilary. 1975. Mind, language, and reality, His Philosophical papers v. 2. Cambridge Eng. ; New York: Cambridge University Press.

Quine, Willard Van Orman. 1953. From a logical point of view: 9 logico-philosophical essays. Cambridge MA: Harvard University Press.

Rosenberg, Alexander. 2005. Philosophy of science: a contemporary introduction. 2nd ed, Routledge contemporary introductions to philosophy. New York; London: Routledge.

24 thoughts on “Pattern recognition: neither deduction nor induction

  1. I’m rather fond of the idea of abductive reasoning. Broadly speaking I can see the possibility that evolutionary processes have primed us to abduce cause from effect – abduce the most likely cause, often without rational thought. Perhaps it is pattern matching from a different viewpoint.

    So… walking along a forest path we notice (perhaps unconsciously) a strange movement of grass or branches. We abduce that the cause of the strange movement may be a dangerous animal. Abducing potentially dangerous causes carries an evolutionary premium. We stop, alerted. In the absence of a mass of ravening teeth and claws and the later presence of a smiling face we have time to consciously deduce that our joker friend was playing a trick. We then infer that by pretending we were not really afraid he will not feel rewarded for playing the trick.

    A somewhat artificial example, but abducing (a) cause from effect might explain the false positives generated by our pre-conscious brain processes – which our rational brain processes then have to present to our social peers. Our own self also tries to build the explanations into some sort of coherent narrative…

    We abduce agency and purpose as the (evolutionarily justified) most likely cause for significant events we don’t understand, and thus magic and the supernatural is born.

       0 likes

    1. In my view, abduction is larger in scope than pattern recognition (PR). PR provides the foundation, but abductive reasoning leaps (usually on the basis of very few observations) to a causal argument or inference, while I am merely talking about the PR itself. PR presents the explicandum for inductive, abductive and deductive explanation.

      So I have a much smaller target here. However, I should have thought (and written) about abductive reasoning more. Let me now. Abduction is sometimes called “inference to the best explanation”. Recognition of species, for example, is not, I think, explanatory, but it sets up a problem for the pattern recogniser: why is that pattern there? The usual answer (leapt to immediately on the basis of prior knowledge) is that there is some reproductive power that makes progeny resemble parents. This is the abduction, not the recognition of a pattern. It is the “best explanation” based on a host of prior assumptions and knowledge claims. If we had never seen a living thing (if we were a Matrix style computer), we might not leap to the explanation, but I think we would still recognise the pattern.

      Of course the economic argument would not apply, and would rely upon other criteria of salience (maybe the Matrix needs to categorise objects that have functional roles in the simulation).

         0 likes

  2. To the limit of my understanding and knowledge: the Standard model of particle physics consists of nothing but a classification system that allows extremely good predictions without a trace of an underlying theory, although folks hope that one of the string theories or its relatives may eventually “explain” the observations.

       0 likes

  3. You know, John, the sorts of things you’re arguing under the rubrics of classification and observation, I’ve been arguing in terms of description. So, one can describe objects in some domain without having a (full-blown) theory of them. And that has CERTAINLY been the case with biology. Descriptions, as you say of observation, aren’t built of nothing. But they aren’t necessarily built of theory either.

       0 likes

    1. The reason why I don’t discuss description is twofold: one is that the philosophical tradition since the linguistic turn seems to treat everything as being about words. Frame it in terms of descriptors and we get into the O-language/T-language dichotomy I’m trying to avoid. The other is that then we are inclined to think the descriptive vocabulary is prior to observation, and that leads inexorably to theory-ladenness of observation, another dichotomy I’m trying to avoid…

      That said, we can return to description as the recording of observations, so it is correct, I think, to see it in this way.

         0 likes

      1. Note that, as far as I’m concerned, a drawing is a description, as is a measurement. And the digital humanists who use the statistical techniques of corpus linguistics are involved in a very sophisticated descriptive activity.

           0 likes

  4. Nice read. I enjoyed the economic argument.

    “We can retain ideas and practices that are not really relevant for social reasons, such as rituals and “explanations”

    I would want to reverse that in the case of one Medieval example and look to socioeconomic system of professionalism in his day and audience expectation of what is relevant to explain differences that you flagged in the last post.

    But I may be growing a pair of asses ears on that one. Not being culturally disposed to gelotophobia, I find checking the mirror for said appendages every once in a while amusing and helpful.

       0 likes

      1. Was rather Helpful.

        Be interesting to see if I can use local U.K. ethnography past and present rather than having to rely exclusively on more far flung examples when dealing with Gerald. Folklore of birds and four footed beasts will be easy, fish and insects bit more difficult.

        I may even be forced to do a bit of work on the ground here on modern local variation in insect names. Most of the common ones we come into contact with have pretty universal names throughout the U.K; a few do not. I have never had to drop the very localized name for a woodlouse I use even though I have not lived in that environment in about 37 years. An indication of how often it comes up in conversation.

        Main idea I take away from this

           0 likes

  5. I spent my career as a taxonomist identifying fish species. I’ve read a lot of taxonomic literature and know many taxonomists who study various groups. I’ve heard of the idea of nonreality of species, but know little about it. I think most species are real, although there are some involved in speciation, or something, but that doesn’t seem to be real common.

    I think doing taxonomy is a basic characteristic of living things. The simplest of living things, on encountering another living thing, has do make an identification which informs it as to: do I try to eat it, do I make love to it, do I run away as fast as I can, or do I ignore it?

       0 likes

  6. Pattern recognition: if you don’t recognize the predator/food source, you die.
    Induction/deduction/abduction: if you can’t predict the whereabouts of the predator/food source more efficiently than your competitor, you die.
    Academia: if you can’t publish your descriptions/predictions, you perish.
    Politics: if you can’t blame the presence/absence of the predator/food source on your political opponent, you perish.

       0 likes

  7. We know more and more about the mathematics (ie computation) of classifying images from the real world. Recent neural networks eg the Google image classifier learn as they go without a presorted or prelabelled
    training set:

    “it is at least in principle possible that a baby learns to group faces into one class because it has seen many of them and not because it is guided by supervision or rewards.”

    http://arxiv.org/abs/1112.6209

    And instead of “economics”, “salience” is very similar concept, I think.

       0 likes

    1. The issue is why is something salient to a classifier system. Just because it is unsupervised doesn’t mean that it is without a training set. If, in the case of the paper you cite, you present only a couple of faces in amongst pictures of jam jars, the system would, I think, classify jam jars, so the implicit supervision lies in the presentation of the objects that become salient.

      Economics is, in my view, one way things become salient as a matter of fact in systematics. There will be others (see above on the four Fs)…

         0 likes

      1. “the system would, I think, classify jam jars”

        Indeed, as it does. It is unsupervised – the faces are the target of the experimenters’ analysis of the system’s performance, not the target of the system, which “merely” abstracts and classifies the “invariant” higher order features in whatever is presented to it, in the same way that factor analysis summarize multivariate data. The outputs in this example are an automatically produced grandmother neuron, a cat neuron etc. The point of the Google paper is a demonstration that relatively straightforward approaches perform surprisingly well when scaled up to include a billion parameters, one millionth the size of the human visual cortex.

        Anyway, my point was that some of the structure of one’s classifications can come in from the world itself; they are mathematical models implemented as neural networks, and the relative importance of the various features (as a summary of what has been presented) falls out naturally from the algorithm.

           0 likes

        1. “Anyway, my point was that some of the structure of one’s classifications can come in from the world itself…”

          YES. I note that one of the standard techniques taught in Composition 101 is comparison and contrast. It seems to be a good way of discovering the salient features of a collection of objects in terms suggested by the objects themselves.

             0 likes

    2. I think Google conclusively proves that the human brain is not using naive algorithms. For example, Google Translate has a huge corpus of bilingual texts in many languages—much more than any one person could expect to read, speak, or hear in a lifetime—and yet its translations are not very good compared to someone who has lived in the two societies involved for, say, a decade or two each. Therefore, the human brain must be using shortcuts that Google’s simple translation algorithm isn’t.

         0 likes

  8. “Epistemic activities”…? Does experimentation come into this anywhere? Surely that is something we do to learn about what exists in the world too.

       0 likes

    1. Experimentation is one way to learn about the world. Another is to observe passively, or any mix in between. I am rejecting the idea that only experiment is the source of empirical knowledge.

         0 likes

  9. “observe passively”

    I think that is what my university put at the top of it’s teacher training manual ‘the ideal student will observe passively’

    I must confess to being a massive cultural disappointment, I think it is all about joining in.

    It is teaching at its most lazy and lowest level.

       0 likes

  10. If pattern-finding in the data is neither induction nor deduction, then most molecular biologists of my acquaintance are *not* using these core logical methods. It seems that pattern finding is often the *only* thing many of these biologists do, except for a tiny soupçon of ad hoc, abductive Inference to best explanation. The genomics and systems biology large data set approach makes this problem worse.

    You may be surprised by how “robust theory-free” (but hidden, shaky paradigm-full) the practice of much of biomedical and molecular biology research is.

    As an example, most of these seen unacquainted with models from the Modern Synthesis, let alone more recent molecular evolutionary theories. We have, as a specific example, people finding “patterns” from Genome-wide Association Studies who are unacquainted with the underlying models from these theories.that underlie their computational pattern finding tools. .

    So…either their research is not really a robust science or there might be a problem in a philosophy of science that does not have a name for what a very large cohort of the NIH-funded, MRC-funded, HHMI-funded biologists actually do..

    I fear the problem may lie more in the former (the practice of some biologists), then in the latter (the philosophy)!..

       1 likes

    1. I worked as head of communications at a medical research institute for ten years. I have some idea of these problems, yes. You might say they inform some of my asseverations…

         0 likes

Leave a Reply