Just know when that is prudent and legal.
The subject of this month’s education campaign is Data Management, a subject very close to me as my partner of many years is a Records Manager for a large pharmaceutical manufacturer. She has a Master’s Degree in Library Sciences and daily I hear of her challenges sorting, collecting, storing, recovering, and destroying data in a Fortune 100 company. It is not an easy job.
While the challenges she faces are typical for multinational, Fortune 100 enterprises, the issues will be familiar to IT professionals in small and mid-sized companies. Everyone, regardless of size is dealing with rapidly growing volumes of data, the challenge of where to store files, how to ensure they remain accessible, and finally how to efficiently destroy them at the end of their legal or useful life. Large enterprises have the luxury of hiring professionally trained records managers, but for SMBs it is IT that has to deal with the challenge.
One of the biggest challenges faced by all organizations is that everyone thinks they are an expert in data management. There is a natural, human inclination to save all your data in the belief that it will needed or become reusable at some point in the future (or just to cover you’re a**). The fact that it rarely happens doesn’t deter anyone from becoming a data hoarder.
People all over the organization become their own Records Managers by creating repositories of all data that crosses their paths. Some people go so far as to save literally everything they ever create, including all emails (some including auto replies), attachments, docs, PPTs and spreadsheets, regardless of any direction they receive from corporate. IT can tell you that entire storage arrays are crammed with old files users will go to great lengths to protect. Access to a cloud has only made this worse.
Professional records managers will tell you that only 2-3% of corporate information truly needs to be preserved, mostly information on patents, intellectual property and legal actions. The rest is called “transitory data”, defined as data that is only important around the time of its creation. The vast, vast amount of information created (saved by data hoarders) is really transitory data.
Old data can only hurt you
The flaw in this thinking is that early on lawyers and records managers learned that “Old data can only hurt you.” Old data has a habit of surfacing at just the wrong time.
When a company gets sued, one of the first actions of the court is to order a search all corporate documents for any relevant information. This is a legal action during the discovery phase of any proceeding. Any documents or file found that are remotely applicable are required to be placed “on-hold” and not destroyed for any reason. Lawyers for both sides then review the information to assess its value, and irrelevant data is then taken off hold and allowed to resume its normal life cycle. This is very expensive in both time and resources.
No one assumes that enterprises save all data forever. Courts understand that corporations have data retention policies and those policies include when data should be destroyed. If the contested event occurred years ago, past the formal, corporate-mandated destruction date for corporate records there is no penalty for not having files for the lawyers to review. The court proceeds with the available data.
But if files are later found, or leaked (as happens more frequently) from a private stash then the company is potentially in violation of a court order. Even if the law suit is unsuccessful the corporation still has to deal with the court violation. This is called “spoliation” – the intentional, reckless, or negligent withholding, hiding, altering, fabricating, or destroying evidence relevant to a legal proceeding, and courts take this very seriously.
Also now there are privacy laws that make data deletion of personal or identifiable data a legal requirement.
Backup appliances have data deletion options for a reason
Enterprises of all sizes need to have some form of data management. This includes protecting mandated data for only the correct period of time. Companies need to archive tax information, financial reporting data, health records, credit transactions, etc. for the legally mandated time. But just as important, to delete those files when the mandates expire.
Leading edge backup appliances have the ability to automatically manage backups through their life cycle. A comprehensive Archiving User Interface (UI) can automate a GFS (Grandfather, Father, Son) archiving strategy. As data ages it can be set to automatically move to remote devices, locations and/or the cloud. Old data can be consolidated into monthly or yearly compilations with the oldest files automatically deleted as the data passes retention mandates.
The delete function is a beautiful thing. It legally and prudently destroys data files that, at best are just taking up space, but potentially can cause harm to the enterprise.
Gone are the days of managing backups just for recovery.