Decentralization to Centralize Human Thought & Universalize Access to Knowledge

5:30p 2015-11-30 Mon

Read on Facebook

A little-spoken about problem of life is, the world is already quite a
decentralized system. Surprisingly, we haven't figured out how to
centralize it. I don't mean decentralization as to cite the big "E"
Economy, or politics/voting, or propagation of news (though these are
all symptomatic examples / offshoots of the underlying problem). I
mean fundamentally that...

We have 3B+ people[1] with internet access, each with their own brain,
each thinking of ideas separately, taking notes separately, sometimes
even across several different programs (google docs, evernote,
etherpad, github, facebook, email). You likely have thought before,
"Shoot, I've stumbled across some excellent resources on the web which
are buried" or "I can't believe this idea was discovered 20 years
ago". The fact is, the web (and pretty much no system) is not set up
to be centralized (for instance, there's no single source of trust for
authentication/identity, no single source of all academic papers in
the world, and no entity resolution or algorithm to perfectly maintain
a record/index of all the services even which try to address these
centralization efforts!)

Don't get me wrong, decentralization has many favorable qualities. It
can offer a strategy for a degree fault tolerance and permanence,
discourage monopolization, and can provide efficient dispatch,
computation, and retrieval of (especially too large for an individual
commodity machine) tasks/data which are well suited for parallel
computation or which can benefit from spatial and or temporal
locality.

But the most important element of decentralization is that it
decentralizes something meant to be centralized (by definition). And
this requires, by necessity of definition, that we indeed have a
solution for how centralization is achieved. And if you ask me, we're
not especially good at this. Again, the Economy, politics/voting,
authentication as examples.

Why is it imperative to address this today? Because, we're creating
more and more data, but our mechanisms for improving the web (as a
protocol) are not keeping up. Right now, Google (and several other
"indexing search engines") are retroactively turning a distributed
system into a centralized one by running imperfect algorithms over all
documents in the web. This is partly an issue of trust -- Google et
al. do this because people are afraid that there should be any single
institution physically capable of monopolizing an index of the web (I
say physically as opposed to practically as Google eta al. do
practically control this -- just not to the mutual exclusion of any
other party).

Thus, today there is no perfect ledger of all existing documents
(other than these artificially produced, imperfect ones). The obvious
solution (which is not so obvious to implement and get adopted) is
building "decentralized" centralization into the protocol-level
itself. That is, for decentralized agents (necessitated by the
logistic enormity of the task) to maintain a centralized ledger, at a
protocol level. IPFS (https://ipfs.io) is one interesting attempt to
achieve this. Bitcoin and Ethereum are interesting approaches as well,
which not only replace a middleware, but also provide a more
comprehensive real-time solution.

The question remains, even with a framework of decentralization to
power a the centralized solution we need, how do we incentive
innovation of this framework (e.g. Filecoin; http://filecoin.io/) and
what might a decentralized-centralized web look like or enable us to
accomplish?  Imagine if there was a single note taking database for
the whole world. And as you type a note, the service compares and
entity resolves (matches) your note against every other note in this
universal database. You see an auto-complete dropdown box, just like
when you're tagging someone in facebook, and it allows you to link
your idea or add your ideas to an existing record. Instead of Google
trying to reverse engineer search engine result pages, we'd have a
real web of knowledge. All original ideas and notes would exist only
once, people would spend less time seeing what's been done, and
spending more time adding value to the universal graph.

Of course a lot has to happen (and other infrastructure is required)
before this vision becomes a reality, but it's achievable in our life
time. A real universalized, decentralized-centralized framework /
index, not owned by any one person; a single stream of consciousness
of all worldly knowledge, accessible to all.

P.S. I am attempting to build something like this (but only for my
blog) to allow me to register ideas, notes, todo items in a universal
database. I will be the first to gladly admit Mark Carranza has had
this built[2] and has been using it for himself for *years*, and only
today did I really understand its full potential (though I know there
are things he wishes it had, like better semantic tagging, todo
functionality, and an API into other service -- some of which he may
have added since we last spoke) which will be fun to work on.

cc: Mark P Xu Neyer, Bram Cohen, Bob Ippolito, Brent Goldman, Juan
Batiz-Benet, Stephen Balaban, Drew Winget, Gregory Price, Bernat
Fortet Unanue, Akhil Aryan, Jan Paul Posma, Michael Noveck

EDIT (adding citations): 
[1] http://www.statista.com/…/number-of-internet-users-worldwi…/ 
[2] http://quantifiedself.com/2009/09/the-social-memex/