Earlier I wrote a post entitled Backup, Deduplication, and Affordable Data Protection. This post is a follow-up to that one. In my previous post I pointed out that data reduction was a more complex subject – particularly when taking into account cost – than some vendors led their buyers to believe.
I got a question over e-mail concerning this – basically asking me what my position was concerning deduplication in general both now and in the future.
Here’s my position. I think that deduplication has to get smarter. Right now, deduplication works great if you buy a Data Domain NAS or VTL appliance. There are a few other deduplication NAS appliance vendors who are getting it right. And EMC Avamar is a decent backup product with some really interesting underlying technology that works well in very modern environments but has some problems with restore speed, scalability, and client-load.
But there are an awfully lot of backup software vendors – with more getting added every day – that have jumped on the deduplication bandwagon. And their claims tend to defy physics. When you look closer, what they’re really trying to do is to be able to allow buyers to “check-off” the feature without really explaining the trade-offs at either the target or the source level in terms of price and performance.
Deduplication and the various deduplication implementation schemes are evolving. And as in any evolutionary process, there are some dead-ends. Deduplication has to – and will – do a better job in the future of adapting. The right deduplication backup solutions have to adapt to the hardware on which they are operating and on the user’s expectations with respect to backup performance versus network utilization, with respect to client load versus network utilization, with respect to ingest rate and target load, and with respect to varying forms of backup (masters, incrementals, differentials, etc.)
