The End of Knowledge Management

[I]nformation that cannot be reliably retrieved is not really being stored. - David Deutsch, The Beginning of Infinity

I've been assigned knowledge management projects four different times in my career. I'm 0 for 4 on cracking the code. The smidgen of success I did have was with a start-up in the late 90's. The company was sold to GE before the project had a chance to mature, but it looked promising. In hindsight, the fact this was a smaller team is probably what made the prospect for success more promising.

But perhaps it's much more than that. Are the problems of knowledge management best thought of as a reflection of our limited skill for handling the quantity and nature of information available to us in the 21st century? For millennia our cognitive capabilities have been up to the task of wading around in the ankle deep information born from extremely limited technology. We off-loaded much of the explanations for strange phenomenon to superstition and religion. Today, we're bobbing and drifting around in a turbulent ocean of information, unable to see beyond the next wave and lacking the tools and skills to navigate anywhere meaningful. We're needing skills and cognitive capabilities from an evolutionary process that cannot deliver them fast enough.

In the past knowledge accumulated and evolved at a pace we could more readily assimilate. We could harness and control the pool of relatively limited information we had. The very phrase itself, "knowledge management," reflects an outdated relationship to information and knowledge. Knowledge is something we expect to manage, to control. But knowledge today accumulates and evolves at an accelerated pace. And it's increasingly more abstract. It's a world of complex supply chains, unprecedented waste, and a near complete disregard for first order effects. A world of "fake, but accurate," "fungible facts," "lived experiences," and 24/7 infotainment feeds. The idea of filtering, sorting, and managing any of this into some sort of stable or usable repository seems a fool's errand.

My sense is there is something missing in the modern evolutionary trail from data to information to knowledge to wisdom. We're good at the data - generating it, storing it, and shuffling it around. Maybe AI will help us get to better information, but what that looks like is uncertain. The missing "something" is between the transition from data to information and from information to knowledge. An educated speculation on what's missing:

1. Our error correction skills are so overwhelmed they are effectively shut down.

2. Same for our skills for filtering signal from noise. We no longer respond. We simply react.

3. There is no data or knowledge "apoptosis" - all we seem to do is collect data, never throw it away.

Whatever this something is, it's influencing the transition in a way that fogs our understanding of what the data actually mean and consequently decreases the quality of the decisions we're making. We, as a population, as a species, are in dire need of upping our game with system thinking.

I cast around to a few Masters of Data I know to hear their thinking on various dimensions of knowledge management in the 21st century. One common thread seems to be that the business side of the relationship is stuck on collecting data. They want bigger and bigger mountains of data from which they can mine deeper and deeper holes to nowhere. What they pull out is increasingly vast amounts of low grade ore with trace amounts of value and contaminated with confirmation and availability bias. For businesses and individuals who recognize the limits and keep it simple, good things can result. For others, data informs quality decisions less and fuels motivated reasoning more.

Another common thread was concern over the impact all this has on our humanity. Before the WWW, we not only relied on the expertise of travel agents, bank tellers, tax advisors and numerous other service professions, we benefited from the social interactions and connections when interacting with the people who provided these services. We're long past any single person's ability to understanding the complex systems that have replaced these and many other professions. Consequently, locked within these systems is the wisdom needed to create them in the first place. How, then, are we to adapt them to respond to new challenges and serve emergent needs without great expense?

Where I've landed with all this is the conclusion none of this is solvable in a way that adds value and meaning in my life. The early days of the World Wide Web briefly held a promise for building a usable knowledgebase. But somewhere around 2005 this promise began to unravel along an acceleration curve. A progression as sad as it is unsurprising. That Twitter would happen could have been predicted by anyone familiar with the history of the telegraph in the 19th century. Flame wars are flame wars regardless the medium - Morse code for the telegraph, command line text for Usenet, or smartphone apps for Twitter. The only thing that's changed over the years are the barriers for entry and the extent to which the noise is amplified.

My "knowledge management" efforts focus on developing a personal system that is less about management and more about connecting information. Intentionally built into this system are ways to increase the adjacent possible and include information sources and perspectives outside my bubble. The membrane is permeable, flexible, responsive, and adaptable in ways no longer possible with traditional knowledge systems. Developing my skill with modern day tools has helped with the data and information apoptosis - it's pretty clear what does and doesn't matter. This also helps with filtering. Given the sorry state of data quality in the world today, however, error correction is still in tatters. And this is a major concern.

---

Photo by Max Langelott on Unsplash