The emperor's new database

Technology is breathing new life into an old Chinese project to capture the best of all human knowledge, says Peter Cochrane

Prof. Peter Cochrane

DURING his long life, Emperor Qian Long never lost a conflict and was the victor of 10 major battles. But in his early sixties, he decided he did not want to be remembered for his military prowess alone and embarked on a project to record all human knowledge worthy of preservation. By his decree, all the books in China were collected under pain of death. He recruited a 300-strong editorial board comprising his most eminent scholars, supported by countless scribes and clerks. They catalogued and sorted more than 100,000 manuscripts into the four divisions of Chinese literature: classics, philosophy, history and letters. Between 1773 and 1782, they selected the best versions and accounts from all the books, and compiled the Sikuquanshu, or Complete Library in Four Branches. By edict, any writings detrimental to Qian Long's reputation were destroyed . . . well, no one is perfect.

The final version comprised some 3,460 works on 36,000 scrolls, having roughly 800,000,000 characters - the single biggest work ever. Even today, such a task looks formidable, but most impressive is the precision and style. Every page is laid out on a regular grid with standardised characters. Each work has a descriptive note, brief biography of the author, summary of contents, examples of strong and weak points, and a critical evaluation. If people did this today, we would save a great deal of time in the reading.

The Sikuquanshu charts 5,000 years of Chinese civilisation, and only seven copies were produced; today just three survive, locked away in dark vaults. Good-quality paper has a half-life of about 500 years and is biodegradable, so this work will be irrecoverable in about 300 years. In anticipation of some accident, photocopies have been produced and are available to scholars.

Of course, no one has read all of the Sikuquanshu because it is effectively an infinite work, and is generally considered far too complex for non-academic interests. Obviously, it lacks a search engine, and it is impossible to navigate, to know where everything is and how things are related. But now, a new project has started: the Sikuquanshu is being digitised. Because of the standardised format, Optical Character Recognition can be used to capture every page. In its final form, the 4,600,000 pages will be scanned on to 175 fully indexed and searchable CD-Roms.

So, an electronic Sikuquanshu will soon be available, and it will be possible to search and cross-reference everything with a minimum of fuss. Who knows what secrets it will reveal? One thing is certain: this version will be closer to the original intention of making all known knowledge available. I suspect Qian Long would be impressed.

Since 1782, we have generated far more fundamental scientific knowledge, philosophical works, history and letters than contained in the Sikuquanshu. Many millions of original volumes now fill our library shelves. Should we not be digitising all this for posterity and reference, too? I think so; we would undoubtedly benefit from cross-referencing and searching all the works of our species. But we would now need more talent to read, filter and select the best versions (especially on the Net) than there are humans on the planet. It looks like we will have to wait for a digital Qian Long to evolve and clean up our 21st-century Sikuquanshu.

Peter Cochrane holds the Collier Chair for the Public Understanding of Science & Technology at the University of Bristol.

The above article was published in Electronic Telegraph on Thursday 27 May 1999 Issue 1462.
We would like to express our gratitude to Prof. Peter Cochrane for his permission to publish this article here.