The importance of seeing what isn’t there

by Eric Drexler on 2010/01/17

The Edge Annual Question — 2010 asks “How is the Internet changing the way you think?”, with answers by (to borrow from the Edge description) “an array of world-class scientists, artists, and creative thinkers” that includes technology analyst Nicholas Carr, social software guru Clay Shirky, science historian George Dyson, and Web 2.0 pioneer Tim O’Reilly, among many others (Richard Dawkins, Nicholas Taleb, Marin Rees, Sean Carroll…). The landscape of social cognition is changing, and the authors offer many views and maps.

In my answer I discuss how the Internet boosts the growth of human knowledge in a way that is powerful and yet — by nature — almost invisible: It helps us see what’s missing:


THE WEB HELPS US SEE WHAT ISN’T THERE

As the Web becomes more comprehensive and searchable, it helps us see what’s missing in the world. The emergence of more effective ways to detect the absence of a piece of knowledge is a subtle and slowly emerging contribution of the Web, yet important to the growth of human knowledge. I think we all use absence-detection when we try to squeeze information out of the Web. I think it’s worth considering both how it works and how it could be be made more reliable and user-friendly.

The contributions of absence-detection to the growth of shared knowledge are relatively subtle. Absences themselves are invisible, and when they are recognized (often tentatively), they usually operate indirectly, by influencing the thinking of people who create and evaluate knowledge. Nonetheless, the potential benefits of better absence-detection can be measured on the same scale as the most important questions of our time, because improved absence-detection could help societies blunder toward somewhat better decisions about those questions.

Absence-detection boosts the growth of shared human knowledge in at least three ways:

Development of knowledge: Generally, for shared knowledge to grow, someone must invest effort to develop a novel idea into something more substantial (resulting in a blog post, a doctoral dissertation, or whatever). A potential knowledge-creator may need some degree of confidence that the expected result doesn’t already exist. Better absence-detection can help build that confidence — or drop it to zero and abort a costly duplication.

Validation of knowledge: For shared knowledge to grow, something that looks like knowledge must gain enough credibility to be treated as knowledge. Some knowledge is born with credibility, inherited from a credible source, yet new knowledge, supported by evidence, can be discredited by arguments backed by nothing but noise. A crucial form of evidence for a proposition is sometimes the absence of credible evidence against it.

Destruction of anti-knowledge: Shared knowledge can also grow through removal of of anti-knowledge, for example, by discrediting false ideas that had displaced or discredited true ones. Mirroring validation, a crucial form of evidence against the credibility of a proposition is sometimes the absence of credible evidence for it.

Identifying what is absent by observation is inherently more difficult than identifying what is present, and conclusions about absences are usually substantially less certain. The very idea runs counter to the adage, being based on the principle that absence of evidence sometimes is evidence of absence. This can be obvious: What makes you think there’s no elephant in your room? Of course, good intellectual housekeeping demands that reasoning of this sort be used with care. Perceptible evidence must be comprehensive enough that a particular absence, in a particular place, is significant: I’m not at all sure that there’s no gnat in my room, and can’t be entirely sure that there’s no elephant in my neighbor’s yard.

Reasonably reliable absence-detection through the Web requires both good search and dense information, and this is one reason why the Web becomes effective for the task only slowly, unevenly, and almost imperceptibly. Early on, an absence in the Web shows a gap in the Web; only later does an absence begin to suggest a gap in the world itself.

I think there’s a better way to detect absences, one that bypasses ad hocsearch by creating a public place where knowledge comes into focus:

We could benefit immensely from a medium that is as good at representing factual controversies as Wikipedia is at representing factual consensus.

What I mean by this is a social software system and community much like Wikipedia — perhaps an organic offshoot — that would operate to draw forth and present what is, roughly speaking, the best evidence on each side of a factual controversy. To function well would require a core community that shares many of the Wikipedia norms, but would invite advocates to present a far-from-neutral point of view. In an effective system of this sort, competitive pressures would drive competent advocates to participate, and incentives and constraints inherent in the dynamics and structure of the medium would drive advocates to pit their best arguments head-to-head and point-by-point against the other side’s best arguments. Ignoring or caricaturing opposing arguments simply wouldn’t work, and unsupported arguments would become more recognizable.

Success in such an innovation would provide a single place to look for the best arguments that support a point in a debate, and with these, the best counter-arguments — a single place where the absence of a good argument would be good reason to think that none exists.

The most important debates could be expected to gain traction early. The science of climate change comes to mind, but there are many others. The benefits of more effective absence-detection could be immense and concrete.


See also:


{ 5 comments… read them below or add one }

Chris January 17, 2010 at 9:38 pm UTC

http://canonizer.com/ seems a similar idea.

Eric Drexler January 17, 2010 at 10:07 pm UTC

@ Chris — This is deeply different from the Wikipedia-like social software that I envision. Rather than being structured to bring forth the best evidence for and against controversial factual propositions, it is instead about people and their shared beliefs and values. From the description:

Personal Canonized Values
To find out what someone’s beliefs and values are, click on their ID to get to their personal canonized values page. These personal pages have a list of all supported camps….

The system is intended to be competitive with petitions, rather than being on a par with, for example, the content of scientific review articles.

I am speaking of a medium focused on marshaling factual evidence on factual questions. This is radically different from a focus on values, opinions, and people.

Valkyrie Ice January 20, 2010 at 5:27 am UTC

The problem I see with the idea is how do you control a majority faction from preventing any dissenting views? Wikipedia is rife with topics of scientifically controversial theories which are dismissed often times with little more than the opening sentence “this discredited theory” which then proceeds to give only the majority opinion, without examining any of the supporting evidence for the alternate view. Any site dedicated to examining the evidence for both sides of a case would have to be able to control the majority consensus to even allow a dissenting view. Considering the recent “Climategate” issue, it seems such a site is desperately needed in our highly politicized peer review system, but keeping it truly open to all sides would be problematic.

Take for example Astrophysics, which has recently been running into more and more problems with observations not matching mathematical predictions, creating new theories like dark flow and dark energy. There are dissenting views, such as those of Tom Van Flandern or Halton Arp, who point out that in a very real sense, astrophysics has ceased to allow examination of any evidence which could refute the Big Bang Theory, such as the existence of high redshift objects IN FRONT of lower redshift objects, or criticisms of the mathematical existence of singularities in violation of the laws of thermodynamics. While I would not presume to claim I possess the knowledge to debate such topics myself, as a casual investigator, it does seem as if extremely logical, strongly evidenced cases have been made only to be dismissed by “thats not a consensus view so it’s not worth investigating”

Considering a recent report of the enormous number of anomalous experiments that go unrecorded and unreported, it seems such dissenting viewpoints are in dire need of investigation, but will not be because of a mainstream view which controls funding and access to research equipment.

SudarshanP January 25, 2010 at 7:45 am UTC

http://ideagra.ph/ is something different. But still might evolve into such a project :)

Woozle March 28, 2010 at 1:54 am UTC

Well, what you describe sounds a lot like one of the major functions of issuepedia — which I realize is quite flawed in its current implementation, but perhaps with some constructive criticism and additional contributors (I’m basically running and writing it alone, while dealing with various unrelated crises) it could become better-defined and more of a useful resource.

I’ve posted more of an introduction to it over at Less Wrong.

Leave a Comment

Previous post:

Next post: