BitSpread

Zenith's BitSpread technology ensures data is preserved the moment it is written. As virtual machines commit data to the Solid State Disk, the Solid State Disk immediately writes the data to another Solid State Disk in a different node. The Solid State Disk then disperses the data to random hard drives throughout all attached nodes.


For example, if there are four hard drives per node and three nodes are deployed, then a Solid State Disk has twelve SATA II drives to write to randomly. The Solid State Disk may select one of the four located in the same node, but also has the random ability to write to any of the other eight drives located on other attached nodes.


By writing data using BitSpread, the probability of loosing data is severely lowered. Data is written using Forward Error Correction (FEC) where each data write includes redundant data, known as Error Correction Code (ECC). The ECC can be used to reconstitute lost data. This helps prevent unnecessarily resending the same data should a transmission fail.


Common BitSpread configurations allow for an (n/2n) or 50% disk failure rate. Most common RAID setups allow for a single disk failure in the array with a limited option allowing for 2 or more.


Back to our example, with BitSpread in a three node environment with four disks per node, a total of six disks can be lost out of the twelve before data loss occurs.


BitSpread allows for hot adding storage capacity with no recreation required. Rebuilds do not have a performance impact on the virtual machines running. As more nodes are added to Zenith Private Cloud, the capacity and the reliability both increase.


BitStream is compatible with Zenith's iSCSI storage offering.


Think of BitSpread as a cross-word puzzle or, more appropriately, a Suduko grid. With limited information available the BitSpread policy can use the remaining data to reconstruct the original data.


For each node stack, a disk policy has been defined:


Two Nodes


6/10 Disk Failure Policy
BitSpread writes a Virtual Machine's data over 10 hard drives. Out of those 10 hard drives, 6 can fail before data is lost.
Three or more Nodes


7/15 Disk Failure Policy
BitSpread writes a Virtual Machine's data over 15 hard drives. Out of those 15 hard drives, 7 can fail before data is lost.
The algorithms behind BitSpread are based on coding theory allowing for error detection and correction. As hard drives in the spread fail the algorithms continue to recreate the lost data. As hard drives are added back into the equation, they are once again incorporated into the BitSpread. Data is regenerated over time.


Benefits


Low Disk Overhead
Build-in High Availability

Ability for 6 (two node) or 7 (three node+) hard drive failures

====================================================

BitSpread: Enabling Unbreakable Storage for high-density disk drives.

Amplidata has created distributed storage based on the BitSpread next-generation erasure-coding technology, that can enable the highest levels of data durability: ten 9’s, fifteen 9’s or beyond – as specified by a user policy. BitSpread is a smart software codec that replaces the use of RAID controllers or software to provide much higher data availability and durability than achievable with RAID and is optimized specifically for big data stored on high-density (multi-terabyte) disk drives, exactly where RAID encounters limitations.

BitSpread splits and encodes data objects into thousands of check blocks that directly encode redundancy into a single instance of the data, with dramatically reduced overhead as compared to mirroring schemes. These check-blocks are spread widely over the entire storage pool. The codec requires only a subset of the check blocks to retrieve the original data object, as determined by the user-specified policy. For each data set, the administrator can specify a durability policy with control over several parameters:

• The number of disks that are to be selected in the data spread (typically 16-20 drives)
• The number of simultaneous failures in this spread that the system needs to be able to survive (typically 4, 6 or 8 drives or modules)
• Whether to enforce spreads across disks, storage modules or racks
• Geographic spreading “GeoSpread” rules, for spreading a single instance across multiple Sites for site level protection

This first of all enables data protection way beyond RAID6’s two-disk failure protection, to 4, 6, 8 or more failures. In this way, multiple devices can fail simultaneously without data loss or affecting data– enabling any statistical level of data durability. In large scaled out systems, storage modules, full racks or even data centers can fail and data can be accessed and reconstructed from the remaining blocks.

To protect data against bit errors (BER), BitSpread uses a fine-grained checksum protection scheme to protect each check block on disk. This fine-grained approach provides superior protection against sector level bit errors, as contrasted with a single top-level stronger checksum such as an MD5. Where other erasure coding technologies only protect up to only a few small number of simultaneous bit errors, BitSpread protects against more than 1000 simultaneous bit errors per single object, making it hundreds of times more reliable than other algorithms.

BitSpread has also been tuned for very high-throughput, making it possible to use in performance critical Big File applications in Media and for Online applications. Aggregate throughput levels that drive many Gigabytes per second can be achieved, and can be scaled up as more BitSpread codecs are deployed across AmpliStor controllers.

BitSpread works in collaboration with the other key Amplidata software component, BitDynamics, which is responsible for continual data integrity verification and assurance by sweeping the storage pool for checksum mismatches. BitDynamics is also responsible for automatic healing of objects that are either affected by bit errors or loss due to component failures, and doing so in parallel across the entire storage pool.