IntegriFS

IntegriFS is a bit different to most RAID setups. Normally, you’d decide:

• how much disk space you need
• how much redundancy you need
• how many drives you can afford to buy
• what sort of RAID to use on them (RAID 1, RAID 5, RAID 6)
• how to allocate drives to RAID volumes
• what filesystem to layer on top of it
• whether you intend to expand the array in the future
• how you’re going to handle expansion and merge the new drives with the old drives

IntegriFS streamlines all of this. Instead, you specify:

• which drives to use
• how many drive failures you want to tolerate

and IntegriFS designs and sets up the array for you. You can add and remove drives at any time and IntegriFS will resize for you. If you decide that you need to be able to tolerate two hard drive failures, it can redesign and rebuild the array for you with no downtime. If there’s a power failure at any time - even during array rebuilding - it will handle this with no data loss. Even if your hard drives ‘rot’ - bad sectors, cosmic rays flipping bits - IntegriFS will detect this with its end-to-end checksumming and correct the errors.

A really crazy, insane example

Say you buy a 400G drive and put IntegriFS on it. Obviously, at this point, a drive failure will destroy the entire array; there’s no getting around this. IntegriFS will still do what it can to protect your data, though. It will:

• checksum everything so that errors can be detected
• keep redundant copies of everything within the drive so a small error on the disk won’t damage your data

As you fill up the disk, IntegriFS will start removing redundant copies of your old data. It’ll give you as much space as it can while maximising redundancy.

Let’s add another disk to the array. This will let us tolerate one disk failure - either disk can fail and the other will take over.

As soon as you add the new disk, IntegriFS will start replicating your data to the new drive. I guess this is sort of like RAID 1 - each disk contains an identical copy of the data. Unlike RAID 1, however, IntegriFS can do this while the array is live. There’s no downtime. It just keeps going.

We have a safety factor of 1 here. If we lose any single drive, the array will keep working. Any more than that and we might start losing data.

Let’s get crazy. Let’s add another drive.

Now you have the option: more safety (we could increase the safety factor to 2) or more space. Keeping the safety factor at 1, we could do a RAID 5 scheme where two drives contain the data and one drive contains an XOR stripe.

With a traditional RAID scheme, you’re stuck at this point. You can’t change from RAID 1 to RAID 5 on the fly. IntegriFS can. In fact, it can have multiple RAID levels on the same disk. When you add the third disk, it won’t start using RAID 5-style storage straight away. It’ll just replicate your data onto the third disk. Safety first. When you have more data that can fit on a RAID 1-style array, it’ll automatically start rewriting portions using a RAID 5 scheme.

You never have to worry about any of this - it all happens ‘under the hood’. This adds another important level of safety - it removes human error from the equation. You can’t accidentally add a device twice to the same array or mix up the order, trashing the array in the process. IntegriFS takes care of all of that for you.