ZFS has received built-in deduplication [http://blogs.sun.com/bonwick/en_US/entry/zfs_dedup]. "Deduplication is the process of eliminating duplicate copies of data. Dedup is generally either file-level, block-level, or byte-level. Chunks of data - files, blocks, or byte ranges - are checksummed using some hash function that uniquely identifies data with very high probability. Chunks of data are remembered in a table of some sort that maps the data's checksum to its storage location and reference count. When you store another copy of existing data, instead of allocating new space on disk, the dedup code just increments the reference count on the existing data. When data is highly replicated, which is typical of backup servers, virtual machine images, and source code repositories, deduplication can reduce space consumption not just by percentages, but by multiples."

Original story page here.

Copyright OSNews.com 1997-2006. All Rights Reserved. OSNews and the OSNews logo are trademarks of OSNews.
All trademarks, icons, and logos, shown or mentioned in this web site, are the property of their respective owners.
Reproduction of OSNews stories is granted only by explicitly receiving authorization from OSNews and if credit is given to OSNews.
Privacy statement   -   Notice to Bulk Emailers