Digital data for the ages

Digital durability is the ability of digital data and media to maintain their symbolic meaning over a sustained period of time. Contrary to popular belief, this is no trivial matter. There are several properties of digitally stored information, inherent to their very nature, that have a severely limiting effect on their lifespan.

They are:

Physical decay of the medium

This happens when the medium on which the data are stored expire due to external influences. Common examples are oxydation of magnetic tape or disks, exposure of magnetic media to radiation or magnetic fields, wear and tear, etc.

Obsolescence of the storage technology

This happens when certain methods of storing digital information that are common practice today, get outdated. This always happens to technology unfortunately. A good example are floppy disks: even if you have one in your home right now, chances are you don't have acces to an old disk drive to read them. If you have, there might be a problem installing one in a modern computer, getting them to run under a new operating system , etc.

Loss of symbolic meaning of the data

Even if we will be able to read a Zip disk in 2098, chances are dim that anyone will remember what NTFS or an mp3 file was, let alone how to decode one. If you think that plain text is forever and ASCII will never die, guess again.

An interesting example is the census conducted by the United States Census in the 1970's. The resulting data was stored on magnetic tape and they are essentially lost. The tapes still exist but the data they contain cannot be recovered. The census data of 1790 however, which were written on paper, are still available today!

Legal issues

Last but not least, there are legal issues that fuel the fire. A document that was made in a digital manner is only a valid legal document if it is stored in the original file format, or if it is validated in another way. That rules out periodic conversion of files. Once your convert your ASCII docs to XML you will have to validate them to keep them legal. Keeping documents legal can be very important to a company. Especially companies in the pharmaceutical business facing an audit by the FDA. How one validates a converted document is a question that cannot be answered at this time.

A note on standards: they just tend to complicate the matter. The problem here is that standards only seem to last until something better comes along. In that case, Backward compatibility is something one can only hope for. As was the case with SGML, XML and HTML there tend to be just enough differences for it not to work properly under all circumstances.


Update: The Brazillian and German governments have independently announced their support of new electronic methods for document authorization. Both methods consist of a personal private and public key system that allows either an individual or a government official to certify a document. Authorization is achieved by attaching a digital autograph to a file. The certified document then has the same legal status as a signed paper document.