As I make my way through Daniel Brin's The Transparent Society, this article from CIO.com on near term availability petabyte levels of storage takes on a new meaning. Among some of the scenarios that they paint, consider the following:
Suppose you came to work one day, took off your jacket, loosened your tie, sat down and found a letter on your desk from corporate counsel advising you that a court has just ruled that corporations are now responsible for retaining all business-related phone conversations for one calendar year.
Under this letter is a memo from the PR department (labeled URGENT) advising you that marketing's plan to use face recognition technology to identify prime customers entering the store may be getting some negative reviews in the press, with phrases like invasion of privacy and Big Brother being tossed around.
And under that is a directive from management asking for a technical analysis of a market simulator that may be able to predict to 5 percent accuracy the response to a new product of single parents between the ages of 29 and 33 living in Illinois suburbs with incomes in the third decile.
When I was building eCommerce systems, one of our big problems was storing and analyzing the mountains of data that we could collect with even a moderately busy eCommerce site. I've no doubt that this problem will shrink as we get petabyte stores. Think of the privacy concerns that petabyte stores introduce and, if Brin is right, the sheer inevitability of having more and more of our lives recorded.
The article describes a petabyte this way:
A terabyte is 12 zeros. You need that to express the number of square inches (4,872,996,000,000) in Rhode Island. A petabyte is 15 zeros. You need that to do the same for Mexico (3,057,065,000,000,000).
I remember reading a book from ACM a few years ago that discussed petabyte stores and some of the research issues surrounding them. I can't find it on my shelf tonight. We will soon be in reach of economically being able to store a video record of each persons entire life. How's that for scary, not to mention boring?
