Karl Popper and the Beginning of Infinity

In conversation with David Deutsch

 

When one of your firm scholarly influences is Jacob Bronowski, it is hard to avoid seeing the world in large, encompassing strokes.

So David Deutsch starts early… with Us! All the way back to the beginning of our species in the Great Rift Valley. It is a history of pain, suffering and death – sheer horror in every waking and sleeping moment. These people were family. They birthed us through generations. We remain genetically alike in every way, and yet completely different in the only way that actually matters. That we escaped their horror, that we survived and can now look back upon them in disbelief from our high-water mark of progress and comfort, is something only explainable by reference to Karl Popper.

The problem was always knowledge. Through all that terror and pain, those ancestors of ours all wanted the same thing: to know! They wanted to know how to avoid large predators, how to avoid illness, how to hunt more efficiently, how to build better shelters and stay warm and dry. In every aspect they wanted to improve their lives – they were desperate to know! And yet despite this life-or-death motivation, they failed – almost universally – to do so.

From the anthropological record we can now see that change happened so rarely, that the vast majority of people died in societies that were technologically the same as those that they were born into. The world never changed. They lived lives of complete stagnation and stasis – nothing got any better, in any way: “For thousands of years at a time, the rhythm, the content of human life was unchanging”.

But it didn’t have to be this way. How hospitable any given environment is, is simply a matter of knowledge. Our ancestors had the same ability to make progress as we do today, they just didn’t know how to. So there is a mystery of sorts here that needs answering: why did this capacity to create new knowledge sit largely unused for so long, at such horrific cost? Why are we the only survivors and why did all of our cousin species die out?

That answer was cultural, it was carved into their traditions. What they inherited and lived within involved a very common idea about knowledge: that it comes from authorities such as tribal leaders, community elders, from religion or superstition. These were societies finely tuned to avoid change, and therefore also finely tuned to embrace error.

To the rescue came empiricism – the idea that knowledge instead comes from the senses and from observation. It wasn’t true, but it was incredibly helpful. By thinking about knowledge in this way it became possible for people to reject those traditional authorities that had held back progress for so long.

The next problem was predictable. The rejection of authority was a hugely important development, but it also wasn’t an altogether new thing. It was actually quite common for authorities to be challenged and removed, the trouble is they were then replaced by new, different authorities, continuing the mistakes of the previous ones. Deutsch quotes from a poem by William Butler Yeats:

A beggar upon horseback lashes a beggar on foot.
Hurrah for revolution and cannon come again!
The beggars have changed places, but the lash goes on.

Something else was needed, a revolution that would put knowledge at the centre of things and not people. Empiricism wasn’t going to do it, but it did have instinct on its side. It seemed to make sense. Somehow, in our bones, it feels like knowledge comes to us directly from the world we see, touch, feel and experience. To learn about something, you only need to look at it long enough, and through that observation comes knowledge. But this needs an explanation – something that describes how truth radiates outwards from objects and then into people.

And in searching for this, it doesn’t help that, as Deutsch tells us, “the vast majority of things that you see from when you open your eyes in the morning, to when you fall asleep at night, you have never seen before”. Sure, there is often familiarities about them, but that’s all there is. You may see the same sun rise each day, but it is never exactly the same.

To the salvation of empiricism came induction – the theory that the unseen resembles the seen, and that the future resembles the past. But David Hume was quick to shoot this down. Just like with empiricism, our instincts fill-out the details. It feels correct. Hume’s objection was simple: why should a sequence of propositions inform us in anyway about the next proposition? Why should the next premise be deduced from the former?

David Deutsch has a problem with the software in his phone. His voice-command program – SIRI – comes to life when you say its name (as it is intended to). Appropriate to the nature of his work, Deutsch says the word theory a lot. And perhaps due to his accent, his intonation, or errors in the design, SIRI is constantly mistaking theory for its name; then springing into action and asking David what he would like it to do. The more that SIRI makes this mistake, the more SIRI thinks that it isn’t – that when David says theory he is really saying SIRI. The program keeps confirming its error.

This is why induction doesn’t work. Just like SIRI, the only reason that we think that we are seeing or hearing the same things is because we already have a theory within our minds about what those things look like and sound like. A filter of importance that focuses on regularities to the detriment of the vast majority of other possibilities. Induction is self-confirming, and therefore also wrong.

What Karl Popper showed, was that in all cases what actually comes first is the theory, not the observation. Still people had – and often still hold – a strange attachment to empiricism and induction. Both were wrong and invalid, but scientists and philosophers continued to entertain the idea that perhaps we do them anyway, and perhaps they work regardless. So Popper would have to stretch-out toward a fairly unreasonable standard of proof. He would have to show that not only do such theories not work, but that we are not actually doing them at all; we only think that we are.

So how do we get started? How do we begin finding truth and knowledge? With two theories that conflict with each other, or with something that we want to improve (a factual theory conflicting with a moral theory). In short, we start with a problem – unjustified knowledge and a problem. What we do next is a little uncomfortable to say out loud, because it feels deeply unscientific: we guess! We hope for a solution, or in Popper’s words, a conjecture. It is only then that observation has its place, through testing our conjecture against reality.

Why does it have to be this way? “Well”, Deutsch calmly explains, “there is nothing else.”

These conjectures need explanations to go with them. It is the quality of the explanation that makes a conjecture more or less likely to be true, or even worth testing. Then occasionally, we hit epistemological gold. We produce certain conjectures that have generality – that reach into other aspects of knowledge. And it is from this that we leap forward and make rapid progress.

So much of philosophy and reason has involved the search for epistemological bedrock. That one solid foundation from which all other knowledge can be built. Again our instincts are the problem here. It feels like knowledge builds upward upon itself, and so must also start from certain key first principles – a place that constitutes the groundwork, something unchanging and true. This is a continuing mistake. It depends of course on that foundation being true (and beyond criticism), but also on it containing all other possible knowledge. It is a retrograde mindset that seeks to drag us back to religion, to tradition and to holy books. And as Deutsch explains, every existing claim to a foundation has so far proven to be false, all the way down to mathematics.

It is at this point in the process that postmodernism leaps onto the stage and declares that objective truth therefore cannot exist, that there is no way of judging one’s theory to be any more correct than any other. And they do have a very slight point. What the postmodernists call truth – “justified true belief” – really doesn’t exist. It is language again causing us problems. Knowledge is never justified by anything. We are always likely to be wrong, and “there is no limit to how misled we can be”.

But to do away with absolute truth, isn’t to do away with truth itself. Deutsch uses the example of wanting to visit a shop but being told that it is unfortunately closed. This is a claim that is capable of being true or false, the shop might be closed as you are told, or it might be open. Perhaps you then go and see for yourself and it does in fact look open. Then you get to the door and it is locked. Perhaps it is actually closed? But wait, you can see people inside. Are they customers, or perhaps just employees doing maintenance? It is easy to be misled, and you may be confused, but that doesn’t mean that there is no truth to the matter. The shop really is either open or closed.

This is where Popper made all the difference. Seeking truth boils down to correcting errors and solving problems – and there will always be new errors and new problems.

And this is one of those generality-rich conjectures, a conjecture that has reach. If someone is claiming to have authority about knowledge, then they are inevitably also claiming authority about politics. It might sound like an overstatement at first, but once it sinks-in it becomes unavoidable that “the doctrine that the truth is manifest, is the source of all tyranny”. All assertions of infallibilism lead to violence because “if you are obviously right, then someone who contradicts you is obviously wrong”. This false surety licenses the use of force – why bother to argue, debate and convince someone when you already know beyond any doubt that they are not correct?

As he did for epistemology, Karl Popper shifted everything that people thought they knew about politics and democracy. Just as we cannot predict the future growth of knowledge, we also cannot know who makes a good leader, or what makes a good policy, in advance of it happening.

In science, Popper changed the question that everyone was asking from how do we get to theories from observations? To instead how can we improve on the theories we have? In politics he changed it from who should rule? To how do we remove bad leaders and bad policies without violence? And so he starts again in the only place that we can: with a problem. And democracy – through consent of the voting public – is the best way to solve this problem.

Just as with epistemology, the way that our democratic systems work just seems to pass most people by, even those of us lucky enough to live and vote within them. Our language gets in the way again. To counter this, Deutsch thinks it would be helpful with science and philosophy to replace our common use of the word theory with misconception: the misconception of evolution, the big bang misconception… Perhaps if we did this for the official titles of our politicians we wouldn’t slip into error as much as we do: Presidential Mistake Joe Biden, the Misconceived Senatorial Candidate for New Hampshire, Congressional Blunder Alexandria Ocasio-Cortez, Error-prone Supreme Court Justice John Roberts. And newly minted policies would read as constant reminders of their fallibility: The Faulty Stimulus Package, The Miscalculated Climate Policy, The Flawed Defence Bill, or The Delusional Health Care Plan.

If it is all about error correction, then the best systems of government are those that error-correct the fastest and most efficiently. But with this comes an odd truism: systems that are good at correcting mistakes also make more mistakes. More small mistakes that is! They methodically seek out missteps, they loudly challenge and criticise everything, and so they also might seem a little messy, complicated and unpleasant. This is what democracy is, our best attempt – so far – at political error-correction.

Perhaps it is because most people still don’t understand this, or perhaps because they simply lack the stomach for it, that a strange adoration for authoritarianism exists within democratic populations today. It is a fad of our age and a luxury of progress. By suppressing criticism and resisting change, oppressive regimes can often appear a lot cleaner and more harmonious, with everyone pulling in the same direction. But they also entrench error. And so sooner or later a problem will arise that requires criticism to find and creativity to solve, a problem whereby everyone turns out to be pulling in the wrong direction, and then – with criticism and creativity suppressed – the whole thing falls over. As you would expect, most authoritarian regimes are incredibly short lived.

For all of this to work, however, we need something more from democracy than just its name. We need a system that makes a sharp movement from elections to governance, with the only meaningful restraint being the next election. Popular ideas like compromise and consensus belong in the dust bin. A compromise is by definition something that nobody really wants, and a consensus means that no one is ever proven wrong. Once elected our leaders need to be able to actually govern and implement the policies that they campaigned on. It is only then that we can judge their success and choose to vote them out at the next election if we deem them to have failed.

Of course on the other side of this, politicians tend to love compromise, consensus and proportional representation. It means that they can avoid being held responsible for their mistakes and evade accountability by saying, in all honesty, that “not only does the buck not stop here, but it doesn’t stop anywhere, because there was nobody who advocated that in the first place.

It is worth taking a deep breath at this point, and avoiding the popular worry that modern democracy is – somehow – at risk. That under a wash of fake news, disinformation and polarisation, our systems of government are witnessing a unique type of stress, or otherwise completely failing. For Deutsch, this is not a crisis of democracy, but a crisis of hyperbole where fairly trivial concerns are being elevated as threats to civilization:

People seem to like the idea that they are living in a time of momentous challenge, where the stakes are exactly like, or analogous to, what the stakes were in the Second World War, where it was good against evil, where if evil wins it is the end of civilization. And therefore, fighting against that is glorious and it is worthwhile, it gives meaning to life. And so the more you can talk in terms of these hyperboles, the more life seems worthwhile. It’s as if making rapid, quiet, peaceful progress, which is what’s actually going on all the time right now, is not exciting enough for people when they are in political mode.

All of our failure, including this one, is due to lack of knowledge. And so it is only here that our species is confronted by existential crises. There is no limit (other than the laws of physics) to what we can solve, shape and create – in fact our very survival relies upon this. Enemies of civilisation really do exist, and they are out there hoping to bring everything we have to a sudden end; but they also have one important thing in common: they are wrong! They seek to supress knowledge, not to cultivate it – they try to avoid problems, rather than to solve them.

There is nothing inevitable about our continued existence. If we want to avoid the fate of our cousin species we will have to work at it, and we will have to move fast. This means making our mistakes as quickly as possible, because it is the only way to stay ahead of those bad actors, as well as new – unforeseen – problems. Rapid open-ended progress is not only possible, but necessary.

What we need is a novel type of cultural heritage, one that values only the creation of new knowledge: what Karl Popper called a tradition of criticism.

 

*** The Popperian Podcast #1 – David Deutsch – ‘Karl Popper and the Beginning of Infinity’ (The Popperian Podcast: The Popperian Podcast #1 – David Deutsch – ‘Karl Popper and the Beginning of Infinity’ (libsyn.com)).

My Misconceptions (Introduction)

Karl Popper makes me feel like a lapsed alcoholic – someone clinging, slipping, painfully from sobriety. Always uncomfortable and always angry at myself. Critical rationalism – or Popper’s method of conjecture and refutation – bends back and circles inward, it loudly demands on being attacked. The tougher and more rigorous the beating, the more bloodied and broken, the better! Behind the theory is a smiling, smug kind of masochism. It doesn’t just concede that it might be wrong, but insists upon it! And welcomes the fight.

As far as philosophy goes, as far as philosophy can be popular, Popper isn’t (perhaps only Friedrich Nietzsche can still claim such a thing)! But a large number of people do pay a misconnected lip service to his work; as if they know they should utter his name in certain moments, and nod along knowingly when they hear it back at them. Most people with even the slightest finger on the pulse of philosophy and science fall into this category: they understand that Popper has done something important, but are often at a complete loss to explain what that is!

Everyone else (a much smaller group) seems to have wonderful stories of personal enlightenment – stories of how they discovered Popper’s philosophy in a haze of opening light; something profound, life changing and positive. It was not this way for me. I found him frozen in time, still deep in the trenches – a veteran, scarred and embittered and fighting a war that by all accounts should have been over long ago:

I have solved a major philosophical problem: the problem of induction. (I must have reached the solution in 1927 or thereabouts.) This solution has been extremely fruitful, and it has enabled me to solve a good number of other philosophical problems. However, few philosophers would support the thesis that I have solved the problem of induction. Few philosophers have taken the trouble to study-or even to criticize-my views on this problem, or have taken notice of the fact that I have done some work on it. Many books have been published quite recently on the subject which do not refer to any of my work, although most of them show signs of having been influenced by some very indirect echoes of my ideas; and those works which take notice of my ideas usually ascribe views to me which I have never held, or criticize me on the basis of straightforward misunderstandings or misreadings, or with invalid arguments.

This unconcealed grievance is not the tone that you would ordinarily expect from a philosophical book – in fact it is the type of tone that causes professors around the world to routinely reject undergraduate essays at a glance, or hush-away the arguments of students around tutorial tables. It reveals a painful amount of personal investment and emotion, and seems to say a lot more about the person than the problem.

Objective Knowledge was first published in English in 1972, and it was certainly not Popper’s best book. But many years later these were the first words of his that I would read, and so I saw the man and his frustration before I saw, or understood, his work. But I also knew of the “major philosophical problem” that he claimed to have solved in such simple, matter of fact, language. Deep in the pages of Jean-Paul Sartre at the time, the consolation that I found from this clear, unphilosophical tone, cannot be understated. Moving a little higher in my chair, I read on…

The problem of induction (or David Hume’s problem) sat at the pointy end of the most important – and most troublesome – question in the history of philosophy and reason: how is it that we can know anything to be true? The implications of such an answer were – and still are – obvious. The sheer size of intellectual energy that was thrown at finding a solution is hard to comprehend – almost as much as the combined impact, and despair, caused by the constant failure to do so.

So here was Popper, living and breathing within the academic circles that prized this epistemological answer so highly, and yet 45 years after delivering it he was again putting pen to paper in the desperate hope that someone, anyone, would listen. He continued: “This chapter is an attempt to explain my views afresh, and in a way which contains a full answer to my critics.” And so that anger and impatience begins to make a little more sense.  

The unusual fizz about Karl Popper sits somewhere between what he did and how he was treated. Popper clearly expected more of a celebration, or at the bare minimum a type of noisy recognition – after all the most important question in all of philosophy now had an answer. That he couldn’t see how badly his solution would be received is a little hard to understand. It was as if Popper were a doctor treating a young and excited woman. She walks into his clinic, affectionately rubbing her swollen belly, and tells him that she is five months pregnant and would like to know the sex of her unborn baby. Popper runs a few tests and tells her that she is not pregnant at all, just fat! She storms out, and Popper is left bemused, thinking that she would have been happy to learn the truth.

Before Karl Popper came along the world of philosophy had a monstrous hole in it, after Popper the world of philosophy was left wishing it still had the hole. People weren’t just expecting an answer, they were expecting a certain kind of answer. Something a little more beautiful.

And it is here where things become uncomfortable for me. Unlike most Popperians I can see and sympathise with this – I was also very slow to recognise my mistake and to grasp the richer beauty within Popper’s theory. It is a mistake that I continue to make… and one that I don’t always catch. I hinge on other people and their good sense to kick me back into shape. As much as this troubles me, I feel that I am in good company: Karl Popper himself was, it seems, a bad Popperian.

From those lucky enough to have met Popper, there are no shortage of stories that start something like this: I confronted Popper about…, I challenged Popper on…, I criticised Popper for… And invariably such stories end with: not only did he ignore my ideas, but he did so in the petulant tone of a parent annoyed with their child for speaking out of turn.

Central to critical rationalism is a happy embrace of error correction – a desire to be proven wrong and to then change one’s mind. This, and only this, is how progress happens and how we can edge closer to truth. Popper of course was riddled with the failures that his theory lacked – he was human, and he didn’t seem to like criticism very much. Strangely perhaps, this makes me happy – I prefer him flawed and fallible, down in the mud with the rest of us, and with knowledge itself. It also depersonalises the man from the theory, which can’t be a bad thing.

There are a few ways for a philosopher to become immortal, to live on after their death. They could have historical importance like Plato or Aristotle, they could shake imaginations through bombast and excitement like Friedrich Nietzsche or Karl Marx, or they could be right! And what they said could be true!

The mathematician and astronomer, Sir Hermann Bondi, once proudly looked upon the place that all his effort, toil, creativity, success and reputation had come from and described it like this: “There is no more to science than its method, and there is no more to its method than Popper has said”. Here all that discomfort rushes back at me again. The philosophy of Karl Popper has reach… real-world reach. From critical rationalism and the answer to how knowledge is created, we naturally shift into a theory of science; then it is on to art, to morality, to economics, to society, to policy and politics (bad science and bad systems of government have exactly the same thing in common).

Popper will survive because what he said works, and it works in every realm of life where truth matters. And yet anything this big, with such a nerve-rackingly large scope onto the world, and with such success, also, quite unavoidably, has the ring of a fraud to it. Trained through experience the mind unpleasantly jumps to the back-woods footprints and late night crop circles of conspiracy theory.

The line that matters here is between science and pseudo-science; with Popper’s anger now steeled onto Karl Marx, Georg Wilhelm Friedrich Hegel and Sigmund Freud. The demarcation that Popper cut between these two types of knowledge-seeking is most easily understood in terms of behaviour: how do theories deal with the prospect of being proven wrong. Everything that has ever come from Popper not only opens itself to criticism and dissolution, but also in new, hearty, and enterprising ways that were made possible only by Popper himself. Popper may not have been very good at dealing with criticism, but his theory is!

To learn that you are wrong should be a very liberating experience, because to learn that you are wrong is to also learn how and where you can improve. It stops you from continuing in error. It also means that objective truth – whether or not we can ever get to it – must exist, because if it is possible to be wrong then it is also possible to be right. And the bigger our mistakes, the bigger the possible leap forward.

By turning around the science of Francis Bacon in this way, and forcing people to abandon their long-trained instincts (criticism is often a deeply unpleasant and resented experience for most people, regardless of how mild it is) Popper was never likely to win many quick friends. He would have to wait, much too long perhaps, for people to slowly recognise the wealth of his philosophy within their individual lives.

Most troubling for many though, is likely the problem of what happens next. No matter how much effort and criticism and destruction of false theories we move through, there can never be a point of revelation and light where finally the words: this I know is certainly true, can ever be said. Every truth about the world that we hold dear, and seems unshakably solid, is always just delicately balanced on the precipice of refutation and collapse. It means that no matter where we step-to after discovering and correcting a mistake, that new place, that new strong-hold, not only might be a blunder of some kind as well, but almost definitely is. The only hope we have, is that this new mistake turns out to be also a slightly better mistake than the last one; and that we discover this new error as quickly as possible, so that we can move on to the next one after that.

Popper demands something painful from us – he demands that we become careful and well-suffering adults. Here is where things become most uncomfortable for me. I don’t match-up well to this standard. I am persuaded, the ideas sink-in and capture me, but each day and with most of my language, I fail! The automatic vocabulary of our societies and our everyday instincts fall well short of Popper, and it drags me back-under and into error. Words like certainty, authority, validity and surety have no place when it comes to objective truth, and yet I use them all the time!

All this despite the fact that Popper writes so clearly and with such straightforward prose.

It is hard to think of truth as something out there to be found, something that we can get closer to, and yet also something that we can never actually grasp (even if it were discovered, we would have no way of recognising it as such). Especially while also accepting that the quest for truth – through Popperian conjecture and refutation – isn’t wasted, in fact it is the complete opposite. This endless struggle is how we improve things, how we make progress, how we survive and make life increasingly rich.

This is a lived-in philosophy, a philosophy of big questions and real world problems; it is a rejection of Ludwig Wittgenstein and a return to what matters. But philosophy, science, reason, truth, are only possible as a social activity – it cannot be done well by oneself. You need to be imposed upon by new directions and perspectives, to be poked at and punched with all the things that you hadn’t thought of by yourself. It all needs to be adversarial.

You need the voices of other people.

Here they are…

Karl Popper’s Epistemological Swamp (Prologue)

Groggy and bruised, you wake up somewhere in the depths of a dark swamp. You have no memory, no sense of who you are or how you got there, and no indication or clues for how to get out. To make matters worse, as you are pulling yourself together you discover that you are also, now, blind.

Through the darkness and the fear, you wait, scared to move and hoping for some elegant form of rescue. And as you wait, things begin to get worse. The horrible smell catches your nose, mud slides further and further up your shins, leeches start to burrow into your skin, the waist deep water feels sharp and cold, and you begin to sense the close movement of large predators.

Everything that you touch is painful and life threatening. Standing up as tall as you can, you are sure of only one thing: you want to improve your situation, you want to get out. And so you must do something, no matter what it is. To stay where you are is to suffer and soon die.

Not knowing where to go is one thing, but you also don’t know where you are, where it is that you are starting from. There are no landmarks, no map, no compass bearing, no possible foundation that could work as a guide or a starting point. You could be anywhere in this swamp – close to an edge, in the absolute middle, or somewhere in-between.

Scared of what you don’t know, you take a tentative step forward, and stop. You try to feel around for changes in the consistency of the mud, the objects under foot, the temperature of the water, the amount of unsettling animal activity – anything by which you might judge whether that step was an improvement or a mistake; anything to judge whether you are going in the right direction.

You take another step. And another.

You never know whether you are taking the correct path to freedom, but you always know when you aren’t (you tread on something sharp, the mud feels thicker, something blocks your way, an animal bites you…).

In such moments your first instinct is to turn around and retrace your steps. But soon you learn to apply a little more nuance. Where you started from was also uncomfortable, also unsafe, so going back doesn’t get you any closer to escaping from, nor surviving, this wetland. Increasingly you respond to obstacles and dangers by changing direction altogether and plotting new courses.

It is painstaking, frustrating and slow. But with each new mistake and misstep you learn a little more about your environment. A map of where you have been begins to build in your mind, but more importantly you begin to develop a feel for your new home, new ways to judge the things you encounter, new ways to understand the swamp and its challenges.

Every time you solve a problem you seem to discover a new one. But with this bank of knowledge behind you, each new problem also feels a little better than the last: how you treated your last illness helps you treat the next one, insights from difficult landscapes helps you to traverse future terrain, and encounters with old predators helps you to understand and avoid new ones.

Soon you are thriving, crisscrossing your way through the swamp with evermore speed and evermore comfort. With so many mistakes behind you and so many problems solved, everything you are now doing feels like progress, like you are finally going in the right direction.

The ground under your feet is more solid, the mud less and less thick, the mosquitoes less aggressive, and large animals see you more and more as a threat and not as a food source. You are sure of it, you are winning. You are walking your way up an incline and out of the marsh once and for all, saying to yourself in dumb amazement, “of course this is the way, it is obvious. How could I have ever thought otherwise?”

Then with your next step the ground disappears under your feet, and you plunge below the water. That steady incline that you thought was the way out, was in fact just a sandbar in an otherwise deeper and more inhospitable part of the swamp. And you have just gone over its edge.

Up to your neck, cold, struggling to breathe, and swimming just to stay above the sludge, you desperately start feeling around in the darkness for something, anything that might be an improvement. Again, as before, and as always, to stay where you are is to suffer and soon die…

But you are lucky! You have made another mistake! And so despite how bad things might feel right now, you are better off than you were before you made it. Because you now also know more than you did before.