January 12, 2008 at 2:33 pm
Comments posted to this topic are about the item Digital Storage
January 14, 2008 at 12:32 am
There is an ISO standard for the long term archival of documents. It's called PDF/A (I assume the 'A' stands for archival, but I'm not sure.
See http://en.wikipedia.org/wiki/PDF/A for more info
January 14, 2008 at 3:29 am
Several years ago (10?), when I was on a local township board, I attended a session put on by the State of Michigan Archives discussing long-term archival of documents.
At the time, digital storage required CDs of a certain quality, with all data to be re-read and re-stored on new CDs at no more than 5-year intervals.
But (and this is what I found most interesting) all documents ALSO required stored in some HUMAN-READABLE format. The archivists understood that the ability to decipher the digital format may become lost over time. Once lost, the digital format is no good. But the human-readable format would still be legible and useful.
January 14, 2008 at 5:33 am
Steve,
How ironic that your editorial is on the same day as the
Worth a Second Look - Running a Query Using a Text File for Input is in the featured articles. I've worked for a large regional bank and as a consultant for a major Aluminum manufacturer. Both are required to keep financial, tax and production information for an exceptional number of years to guard against litigation. (Digital storage manufacturers benefitted from the tobacco industries problems too.) We utilized the pdf/a for reporting purposes but trusted them only a little. For data that you really truly need to recover, no matter what digital technology is in play export to a delimited file. Provided the world doesn't completely lose all power sources and reenter the dark ages we should as data professionals always be able to import the data again. (As an alternative energy geek myself I don't see that happening.)
So that's how we got the raw data out of the database.
We then stored this exported data on CD and then DVD media because of it's stable long term capabilities. AND the data was rolled to archive tape. But the archive tape was required to be restored and the consistency checked every 5 years on data with a high risk footprint. The restored export files were then archived again in the newer archival solution. This way we were covered for degradation in media as well as expiring technology. (The business process owners were responsible for assessing the risk footprint.)
January 14, 2008 at 7:37 am
Bob, that's an interesting solution, but it has the problem of you constantly having to read and store back more information on a regular basis. That could end up being a full-time job over time. Maybe even a job for a few people.
January 14, 2008 at 10:18 am
I've thought about this over the years for any number of reasons. The "best" idea I came up with was to use "printed digital information". Some years back I read a (very small) article in Scientific American where Xerox had come up with this idea to implant the digital information of a graph in the graph itself, based on a printed pattern. Essentially something like / = 0 and \ = 1.
If you take that to a DPI level high enough you start to look at holding very large amounts of data; no where near digital storage (I think), but still respectable. If you keep the resolution "low enough" so that a low tech approach could be used to read it (magnifying glass or rough microscope) you add "backwards compatibility" if needed. Going in the direction of "higher tech", a scanner / ETL program could interpret the "data sheet" and produce the output automatically.
With the quality of printers / ink increasing as they have (I have an Epson Stylus 4500 with "archival" ink rated for 90+ years) you could in theory print the data archive on (photo?) paper, then store the paper. The volume of raw data would increase compared to written text. The potential for extracting the data again if desired is present (scanner, etc), and it has the possibility to be "backwards compatible" to lower tech retrieval (if needed).
I kept wanting to add to the density of storage by a couple of ways. The idea of a "denser" arrangement of pixels was one (the / and \ in a 2 by 2 pixel array uses half the available pixels). I kept thinking that using all 4 would be better (a nibble for all us old timers out there). If you add to that the idea of using color in the encoding, you should be able to go with some multiple of 4 bits quite nicely.
There are other aspects to this idea as well; adding instruction in the beginning to help decode (if needed), noting the standards used to encode data; define MP3, mpeg4, etc.
Beer's Law: Absolutum obsoletum
"if it works it's out-of-date"
January 14, 2008 at 10:33 am
The problem with archiving data is not just that you archive the application that consumes it but that you have the required hardware that it runs on. I hate to think of how many digital tapes there are stored about the world where there is no longer the appropriate hardware to read them.
Paul Richmond
IS Project Leader
L.D. McFarland Co
January 14, 2008 at 3:56 pm
After reading many of Steve Jones' editorials I can see that Steve himself is suffering from the most basic of digital malaise viz relying on his spell checker. In todays editorial Steve has spelled struggling and later incorrectly. No big deal really but this is a document that is being distributed to 600,000 plus people around the world.
My old teacher once told me - "accuracy in spelling reflects accuracy in thought" - and the main issue with complaining about spell checkers is that the complainer himself has to be without sin so that the first stone can be cast.
If there are any spilling mistaeks in this post please let me kno.
Regards
Tim Brimelow
PS the spell checker missed the incorrect spelling of spelling above (I typed it as spilling which is a legitimate word)
January 14, 2008 at 4:09 pm
Any one have a way to read the following?
1. 800 BPI 9 track reel-to-reel tapes formatted to hold punch card images.
2. 8 inch floppy disks formatted for a Honeywell DPS-6 mini-computer
3. Cassette tapes formated for a VIC-20
4. DLT-1 backup tapes for a VAX 3100
5. Paper tapes for an Altair
Viewing 9 posts - 1 through 8 (of 8 total)
You must be logged in to reply to this topic. Login to reply