With strong demand for products featuring data de-duplication systems, the move represents an effort by EMC to make the most of the technology it acquired when it bought Avamar Technologies Inc for $165m cash last year.
W. Curtis Preston, vice president at storage consultancy Glasshouse said: I’ve never seen a technology that only emerged two years ago that is being deployed by so many major companies. Corporations that are usually risk averse and would wait for 10,000 other companies to test the water are running de-dupe betas.
EMC’s integration does not turn NetBackup into a de-duplicating backup system, and it does not alter the applications for which Avamar’ de-duplication is best suited.
But it does allow Avamar backup operations to be managed from NetWorker. What were two separate Avamar and NetWorker clients have been merged into one client, and the NetWorker management console has been extended so that it can administer Avamar de-duped backups.
Avamar de-duplicates data at the backup source as well as at the target, because it was designed principally to allow remote back-up of branch offices. By de-duping data at the source, backup volumes are reduced, allowing them to be sent back to headquarters data centers over affordably thin network connections.
Today’s news doesn’t change the use case for Avamar, which is still for backup of remote offices, or LAN-constrained servers in the data center, said Rob Emsley, senior marketing director at EMC.
Even though the Avamar and NetWorker clients have been integrated, de-duplicated backup data is still directed at a different Avamar target than non-de-duplicated backups, which continue to be sent to a NetWorker target.
When will EMC integrate Avamar de-duplication directly into the NetBackup? It takes time to achieve that level of integration, said Yueh.
EMC’s integration invites a comparison with the efforts of its rival Symantec, whose PureDisk de-duplicating system was also designed for remote office backup, with de-duplication at source and target.
Symantec has worked from the other direction, and has integrated its PureDisk de-dupe target side system into its mainstream NetBackup backup system, but says that it is not likely to integrate its PureDisk and NetBackup clients for around a year to eighteen months. So although Symantec says that its customers will not be able to manage PureDisk remote backups from NetBackup consoles until then, its customers will very soon be able to perform dedupe backups without changing any of their policies, using software that will ship this year or early next year.
The process of de-duplication involves the processor-intensive process breaking data into chunks that are then finger-printed in ordered to spot the duplicate chunks that can be discarded.
According to Emsley, source de-duplication is not suitable for large volumes of fast changing data – such as transactional databases — which are better handled by systems that only de-duplicate data at the target, and do so after data has been ingested, out-of-band.
But according to Jed Yueh, EMC vice president of product management, it is not cycle consumption that makes source de-duplication unsuitable for large transactional databases, but the fact that there is not that much duplicate data in such data sets.
Because the rate of data change is so high, there aren’t that many dollars to be saved doing it, Yueh said, adding that some customers are however using Avamar to backup high-end data center databases.
But EMC’s next move will be to be to deliver some form of target-only de-duplication, as part of its FalconStor-powered virtual tape library. EMC is eschewing the de-duplicating engine that FalconStor has written as an extension to its VTL software. And it is re-writing its own Avamar code.
That will be target-based de-duplication. We’ll certainly leverage the Avamar intellectual property, but it’ll be a specifically target-based system, Emsley said. According to Yueh, the company is only re-writing the software in order to adapt it to the VTL.
