Eight to Late

Sensemaking and Analytics for Organizations

Peirce, Holmes and a gold chain – an essay on abductive reasoning

with 8 comments

It has long been an axiom of mine that the little things are infinitely the most important.” – Sir Arthur Conan Doyle (A Case of Identity)

The scientific method is a systematic approach to acquiring and establishing knowledge about how the world works. A scientific investigation typically starts with the formulation of a hypothesis – an educated, evidence-based guess about the mechanism behind the phenomenon being studied – and proceeds by testing how well the hypothesis holds up against experiments designed to disconfirm it.

Although many philosophers have waxed eloquent about the scientific method, very few of them have talked about the process of hypothesis generation. Indeed, most scientists will recognise a good hypothesis when they stumble upon one, but they usually will not be able to say how they came upon it. Hypothesis generation is essentially a creative act that requires a deep familiarity with the phenomenon in question and a spark of intuition. The latter is absolutely essential, a point captured eloquently in the following lines attributed to Einstein:

“…[Man] makes this cosmos and its construction the pivot of his emotional life in order to find in this way the peace and serenity which he cannot find in the narrow whirlpool of personal experience.  The supreme task is to arrive at those universal elementary laws from which the cosmos can be built up by pure deduction. There is no logical path to these laws; only intuition, resting on sympathetic understanding of experience can reach them…”  – quoted from Zen and The Art of Motorcycle Maintenance by Robert Pirsig.

The American philosopher, Charles Peirce, recognised that hypothesis generation involves a special kind of reasoning, one that enables the investigator to zero in on a small set of relevant facts out of an infinity of possibilities.

Charles Sanders Peirce

As Pierce wrote in one of his papers:

“A given object presents an extraordinary combination of characters of which we should like to have an explanation. That there is any explanation of them is a pure assumption; and if there be, it is [a single] fact which explains them; while there are, perhaps, a million other possible ways of explaining them, if they were not all, unfortunately, false.

A man is found in the streets of New York stabbed in the back. The chief of police might open a directory and put his finger on any name and guess that that is the name of the murderer. How much would such a guess be worth? But the number of names in the directory does not approach the multitude of possible laws of attraction which could have accounted for Kepler’s law of planetary motion and, in advance of verification by predications of perturbations etc., would have accounted for them to perfection. Newton, you will say, assumed that the law would be a simple one. But what was that but piling guess on guess? Surely vastly more phenomena in nature are complex than simple…” – quoted from this paper by Thomas Sebeok.

Peirce coined the term abduction (as opposed to induction or deduction) to refer to the creative act of hypothesis generation.  In the present day, the term is used to the process of justifying hypotheses rather than generating them (see this article for more on the distinction). In the remainder of this piece I will use the term in its Peircean sense.

–x–

Contrary to what is commonly stated, Arthur Conan Doyle’s fictional detective employed abductive rather than deductive methods in his cases.  Consider the following lines, taken from an early paragraph of Sherlock Holmes’ most celebrated exploit, The Adventure of the Speckled Band. We join Holmes in conversation with a lady who has come to him for assistance:

…You have come in by train this morning, I see.”

“You know me, then?”

“No, but I observe the second half of a return ticket in the palm of your left glove. You must have started early, and yet you had a good drive in a dog-cart, along heavy roads, before you reached the station.”

The lady gave a violent start and stared in bewilderment at my companion.

“There is no mystery, my dear madam,” said he, smiling. “The left arm of your jacket is spattered with mud in no less than seven places. The marks are perfectly fresh. There is no vehicle save a dog-cart which throws up mud in that way, and then only when you sit on the left-hand side of the driver.”

“Whatever your reasons may be, you are perfectly correct,” said she. “I started from home before six, reached Leatherhead at twenty past, and came in by the first train to Waterloo…

Notice what Holmes does: he forms hypotheses about what the lady did, based on a selective observation of facts. Nothing is said about why he picks those particular facts – the ticket stub and the freshness / location of mud spatters on the lady’s jacket.  Indeed, as Holmes says in another story, “You know my method. It is founded upon the observation of trifles.”

Abductive reasoning is essentially about recognising which trifles are important.

–x–

I have a gold chain that my mum gave me many years ago.  I’ve worn it for so long that now I barely notice it. The only time I’m consciously aware that I’m wearing the chain is when I finger it around my neck, a (nervous?) habit I have developed over time.

As you might imagine, I’m quite attached to my gold chain. So, when I discovered it was missing a few days ago, my first reaction was near panic,  I felt like a part of me had gone missing.

Since I hardly ever take the chain off, I could not think of any plausible explanation for how I might have lost it. Indeed, the only time I have had to consistently take the chain off is when going in for an X-ray or, on occasion, through airport security.

Where could it have gone?

After mulling over it for a while, the only plausible explanation I could think of is that I had taken it off at airport security when returning from an overseas trip a week earlier, and had somehow forgotten to collect it on the other side.  Realising that it would be near impossible to recover it, I told myself to get used to the idea that it was probably gone for good.

That Sunday, I went for a swim. After doing my laps, I went to the side of the pool for my customary shower. Now, anyone who has taken off a rash vest after a swim will know that it can be a struggle. I suspect this is because water trapped between skin and fabric forms a thin adhesive layer (a manifestation of surface tension perhaps?).  Anyway, I wrestled the garment over my head and it eventually came free with a snap, generating a spray of droplets that gleamed in reflected light.

Later in the day, I was at the movies. For some reason, when coming out of the cinema, I remembered the rash vest and the flash of droplets.  Hmm, I thought, “a gleam of gold….”

…A near forgotten memory: I vaguely recalled a flash of gold while taking of my rash vest in the pool shower some days ago. Was it after my previous swim or the week earlier, I couldn’t be sure. But I distinctly remembered it had bothered me enough to check the floor of the cubicle cursorily. Finding nothing, I had completely forgotten about it and moved on.

Could  it have come off there?

As I thought about it some more, possibility turned to plausibility: I was convinced it was what had happened. Although unlikely I would find it there now,  it was worth a try on the hope that someone had found the chain and turned it in as lost property.

I stopped over at the pool on my way back from the movies and asked at reception.

“A gold chain? Hmm, I think you may be in luck,” he said, “I was doing an inventory of lost property last week and came across a chain. I was wondering why no one had come in to claim something so valuable.”

“You’re kidding,” I said, incredulous. “You mean you have a gold chain?”

“Yeah, and I’m pretty sure it will still be there unless someone else has claimed it,” he replied. “I’ll have a look in the safe. Can you describe it for me?”

I described it down to the brief inscription on the clasp.

“Wait here,” he said, “I’ll be a sec”

It took longer than that but he soon emerged, chain in hand.

I could not believe my eyes; I had given up on ever recovering it. “Thanks, so much” I said fervently, “you won’t believe how much it means to me to have found this.”

“No worries mate,” he said, smiling broadly. “Happy to have helped.”

–x–

Endnote: in case you haven’t read it yet, I recommend you take ten minutes to read Sherlock Holmes’ finest adventure and abductive masterpiece, The Adventure of the Speckled Band.

Written by K

December 4, 2018 at 6:05 am

8 Responses

Subscribe to comments with RSS.

  1. Love your work, you make me think.

    Like

    Glyn Davies

    December 4, 2018 at 6:23 am

    • Thanks so much, you’ve made my day!

      Like

      K

      December 4, 2018 at 6:24 am

  2. Dear Kailash,

    This is a very interesting essay, full of relevant insights and great links, thanks for writing it. In the spirit of sharing and commenting upon Peirce’s work, after reading your post I’m reminded of his article on “How to Make Our Ideas Clear” (1878) (available at http://www.peirce.org/writings/p119.html), where he raises this question:

    “And what, then, is belief? It is the demi-cadence which closes a musical phrase in the symphony of our intellectual life.”

    From the perspective of social scientists, it seems to me this is a critical question which is rarely entertained nowadays, but may be worth to keep in mind and continue learning from the work of people like Peirce, Sir Arthur Conan Doyle, and your work.

    Always looking forward to your next post!

    Best,
    Francisco

    Liked by 1 person

    Francisco Aldape

    December 4, 2018 at 2:45 pm

    • Hi Francisco,

      First up, thanks so much for reading and for your very kind remarks!

      Thanks too for the link to Peirce’s article. What a brilliant way to frame the notion of belief – as resolution of doubt. It makes it crystal clear as to why faith-based frameworks – such as religion – are so compelling: they enable the believer to resolve all those discomforting doubts in a flash. I’ve only skimmed the article and must go back and read it more carefully.

      Thanks again!

      Regards,

      Kailash.

      Like

      K

      December 4, 2018 at 5:29 pm

  3. Kailash, thanks for this little meditation on Peirce, abduction, etc. What Francisco noted in his comment above about one of Peirce’s articles is a theme that runs through many of Peirce’s writings and was continued by Peirce’s student John Dewey (1859–1952) and by others. (I mentioned Peirce and Dewey in a previous comment.) What you said in your reply to Franciso about frameworks that “enable the believer to resolve all those discomforting doubts” was well described by John Dewey in his book The Quest for Certainty (1929):

    “Tendency to premature judgment, jumping at conclusions, excessive love of simplicity, making over of evidence to suit desire, taking the familiar for the clear, etc., all spring from confusing the feeling of certitude with a certified situation. Thought hastens toward the settled and is only too likely to force the pace. The natural man dislikes the dis-ease which accompanies the doubtful and is ready to take almost any means to end it. Uncertainty is got rid of by fair means or foul…. The scientific attitude may almost be defined as that which is capable of enjoying the doubtful; scientific method is, in one aspect, a technique for making a productive use of doubt by converting it into operations of definite inquiry. No one gets far intellectually who does not ‘love to think’, and no one loves to think who does not have an interest in problems as such. Being on the alert for problems signifies that mere organic curiosity, the restless disposition to meddle and reach out, has become a truly intellectual curiosity, one that protects a person from hurrying to a conclusion and that induces him to undertake active search for new facts and ideas. Skepticism that is not such a search is as much a personal emotional indulgence as is dogmatism. Attainment of the relatively secure and settled takes place, however, only with respect to specified problematic situations; quest for certainty that is universal, applying to everything, is a compensatory perversion.”

    Peirce’s “first rule of reason” was: “Do not block the way of inquiry”! (Susan Haack borrowed this phrase as the title of a 2014 article on Peirce which explained why “Peirce’s warning against blocking the way of inquiry is no less important, given the condition of philosophy today, than it was when he offered it more than a century ago—perhaps even more so.”)

    At the start of your post you said that “a scientific investigation typically starts with the formulation of a hypothesis”; it’s important to note that it could also start from evidence (e.g., your “spray of droplets that gleamed in reflected light”). Donald T. Campbell (1916–1996) once wrote:

    “More and more I have come to the conclusion that the core of the scientific method is not experimentation per se but rather the strategy connoted by the phrase ‘plausible rival hypotheses’. This strategy may start its puzzle solving with evidence, or it may start with hypothesis. Rather than presenting this hypothesis or evidence in the context-independent manner of positivistic confirmation (or even of postpositivistic corroboration), it is presented instead in extended networks of implications that (although never complete) are nonetheless crucial to its scientific evaluation. This strategy includes making explicit other implications of the hypothesis for other available data and reporting how these fit. It also includes seeking out rival explanations of the focal evidence and examining their plausibility. The plausibility of these rivals is usually reduced by ramification extinction, that is, by looking at their other implications on other data sets and seeing how well these fit. How far these two potentially endless tasks are carried depends on the scientific community of the time and what implications and plausible rival hypotheses have been made explicit.”

    Notice Campbell’s phrase “extended networks of implications”, which sounds a lot like Werner Kunz & Horst Rittel’s IBIS/issue mapping, Jeff Conklin’s dialogue mapping, Tim van Gelder’s hypothesis mapping, etc. As Jeff Conklin, and later you and Paul Culmsee, pointed out, one of the virtues of IBIS or dialogue mapping is that you can start your inquiry at any point: as Campbell said, you can start from hypotheses, or you can start from evidence, and proceed to construct networks of implications, adding hypotheses or arguments or evidence (etc.) in arbitrary sequence.

    There is much more that I could say about all this, but I’ll stop myself from carrying on!

    Liked by 1 person

    Nathan

    December 5, 2018 at 6:09 am

    • Hi Nathan,

      Thanks so much for another characteristically insightful comment that offers much to think about.

      The quote from Dewey’s book is brilliant; the following lines resonate, especially in today’s world – “Attainment of the relatively secure and settled takes place, however, only with respect to specified problematic situations; quest for certainty that is universal, applying to everything, is a compensatory perversion.” The problem is that people not only do not listen, they actively avoid encounters with contrary opinions.

      Many thanks for pointing out the importance of evidence in hypothesis formulation – indeed it is educated guesswork, backed by evidence (I have amended that line in my post). I should add though, that the evidence or facts are there for all to see. The creative aspect of hypothesis formulation lies in how that evidence is interpreted.

      The dialectical aspect of refining ideas collaboratively through argumentation cannot be overemphasised. I don’t know if you’re aware of Tim van Gelder’s recent work on this. If you’re not, it’s worth a look: https://www.swarmproject.info/

      Thanks again for taking the time to read and write in – I learn a lot from your comments.

      Regards,

      Kailash.

      Like

      K

      December 6, 2018 at 7:34 pm

  4. So are you saying it’s mind / logic ( whatever
    New name…) plus intuition that led you to find that gold chain ?
    Why must you find or give a name of a complicated method / process to a simple event ? When we desire something badly/ truely- we usually get it.this is my experience
    Logic is a part of mind.Usually when we transcend mind- creativity happens.e.g.you analyse some problem intellectually without success.Tired you sleep
    On getting up often solution appears- that’s when mind was not working.Advait Vedanta says- everything is One not two…..on this hypothesis (!) We are bound to discover all aspects of the One….if we truely look for it Lives of saints the world over are an example.In crude scientific terms- Its energy transference based on sincere seeking.Look & you shall find! All answers are available to all But each one’s understanding ( and the yetms we use) is as per his peculiar “prakruti” ( satvagun, rajogun, tamogun)

    Like

    Kranthi Shastro

    December 6, 2018 at 7:12 pm

    • Hi Kranthi,

      Thanks for your comment. There are two recognised modes of reasoning: deductive and inductive. The former is the process of arriving at specific conclusions from general premises (Example: All humans are mortal. Socrates is human. Hence Socrates is mortal) whereas the latter is a process of generalisation from a number of instances (Example: All swans we have seen are white. Therefore, we expect that all swans are white). The latter leads to provisional conclusions unlike the former, which leads to watertight conclusions as long as the premise(s) are true.

      Coming to the article, abductive reasoning is qualitatively different from the above because it requires a creative leap. There was no logical reason for me to make the connection between the memory of the flash of droplets and my gold chain. In retrospect one might attribute the memory to desire or anything else, but the fact remains that it is essentially outside the realm of pure logic. The best word for it, IMO, is intuition – a creative leap – much like Kekule’s dream – which is the subject of this essay by Cormac McCarthy. I’ll leave you to read the full essay for yourself, but I think it is apt to end my comment with the following excerpt from it:

      “Problems in general are often well posed in terms of language and language remains a handy tool for explaining them. But the actual process of thinking—in any discipline—is largely an unconscious affair. Language can be used to sum up some point at which one has arrived—a sort of milepost—so as to gain a fresh starting point. But if you believe that you actually use language in the solving of problems I wish that you would write to me and tell me how you go about it.

      I’ve pointed out to some of my mathematical friends that the unconscious appears to be better at math than they are. My friend George Zweig calls this the Night Shift. Bear in mind that the unconscious has no pencil or notepad and certainly no eraser. That it does solve problems in mathematics is indisputable. How does it go about it? When I’ve suggested to my friends that it may well do it without using numbers, most of them thought—after a while—that this was a possibility. How, we dont know….”

      …and that, perhaps, is the most eloquent description of what I have, rather lamely, called intuition.

      Regards,

      Kailash.

      Like

      K

      December 6, 2018 at 8:35 pm


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: