Thursday, 31 March 2011

Automatic recovery of RAID 5 with delayed parity

Recently we set to develop the algorithm to automatically recover RAID 5 array with delayed parity, as implemented by HP SmartArrays. Today we have mostly finalized the work. From this day on (build 306 onwards) ReclaiMe Free RAID Recovery is capable of recovering such arrays.

Monday, 28 March 2011

RAID 5 of two disks

The common wisdom says that RAID 5 requires a minimum of three disks.
If one thinks of it, it is not true.
You can create a RAID 5 of two disks. With even parity, the array becomes a mirror (RAID 1), and with odd parity (although nobody ever uses that), becomes a RAID 4 of two disks.

And yes, the same applies to a RAID 4 of two disks.

Saturday, 26 March 2011

More modeling issues

I recall we discussed that earlier, that our ReclaiMe Free RAID recovery software (and pretty much any generic RAID recovery software, like Runtime's RAID Reconstructor) does not actually work with RAIDs. It works with models of RAIDs instead. Data recovery service view on RAID5 more like "disks (by specific vendor), controller (also by specific vendor), and cables". The software sees it like

1 2 P
3 P 4
P 5 6

and pretty much nothing else.

The problem arises when the actual data does not match the model used in software. The synchronous array (below) cannot be described in terms of the asynchronous model (above).

1 2 P
4 P 3
P 5 6

So the software has to acount for all possible models of the RAID 5, of which there are quite a number.

We're currently working on automatic analysis of the so-called delayed parity arrays, used in HP SmartArray controllers. This subtype of RAID5 does effectively have two distinct stripe sizes, one for data and the other (larger) one for parity, with the array looking like

1 2 P
3 4 P
5 P 6
7 P 8

Would be nice to account for that, because I'm not aware of anything capable of recovering this type of array automatically.

Thursday, 24 March 2011

Even in 2011..

... people still believe you can get an electron microscope and recover overwritten data.
  1. It is not an electron microscope, it should be an MFM, Magnetic Force Microscope. Electron beams are no good against the hard drive platter. Probably no harm either, just useless.
  2. Even with MFM, no recovery of overwritten data on a modern hard drive is possible, because of various aspects of applied physics.

Sunday, 20 March 2011

"Write hole" in filesystems

Unsurprisingly, filesystems are also suspectible to damage when the power failure occurs during write. The most simple example is the file being deleted. If the clusters are deallocated (marked free) first, and then a power failure occurs before the file record is removed, then we got a file having its data stored in free clusters on the disk. If a new file is subsequently created and uses the same cluster, the cross-link siutation occurs, potentialy leading to data loss.

There are several ways around this problem.

Careful write ordering. The sequence of operations can be ordered in such a way that the damage due to the incomplete write is predictable, easy to repair, and confined to a single file. This is the cheapest option. It does not require any change to the on-disk structures if you want to implement it on the existing filesystem.

Multisector transfer protections (used e.g. in NTFS). If several sectors are to be written out as a group, each sector in a group stores a specific signature. When the group is later read, the driver verifies signatures in all sectors of the group. Should the signatures not match, the data is rejected as corrupt. This only allows for error detection, but not correction.

Intention logging is the most complex option, similar to a database transaction logging. The filesystem driver ensures so called atomicity of certain operations, meaning that the operation either completes entirely, or no change occurs at all. This option is implemented in most modern high-capacity filesystems, most widespread being NTFS and ext3/ext4.

Wednesday, 16 March 2011

Write hole in RAID 1

Actually, RAID 1 has the same write hole problem as RAID 5 does. Should the power fail after one disk is updated, but the other is not yet updated, and then the first disk fails, there will be data corruption.

As usual, scheduled synchronizations of the array reduce probability of this effect causing any practical trouble.

Sunday, 6 March 2011


Improved memory usage in ReclaiMe Free RAID Recovery (, by the factor of ten. The update is not yet live, but will be out shortly, probably no later than tomorrow. Surprisingly, there was not at all that much loss of speed.

Saturday, 5 March 2011

Exotic RAID types

With a RAID recovery, certain exotic RAID types can be recovered using the same basic algorithms, because these exotic RAID types can be reduced to one of the three basic types (RAID0, RAID1, and RAID5).
The list goes like this:
  • RAID 1+0 or 0+1 can either be reduced to RAID 0 by removing mirrors (near layout), or can be recovered as a RAID 0 straight away (far layout).
  • RAID 5E or RAID 6E can be recovered as a RAID 5 or RAID 6 respectively, because all the extra data is at the end of the array.
  • RAID 5EE can be recovered as RAID 6 with one of the sets of parity corrupt.
  • RAID 4 is actually a variation of RAID 5 where parity does not change position across rows.
The array types requiring special processing are RAID 1+0 using offset layout and RAID 1E.

Wednesday, 2 March 2011

Data recovery - probably cheaper than backup, ...

... but less than 100% reliable, though.

See this -
Given that you can literally burn the laptop in fire, and still recover all the significant data, why would anyone bother with backups at all?

Disclaimer: just kidding