Data Recovery Weekly

If you are doing a RAID Recovery and the software has the capability to limit the allowed block sizes for search (which is quite often actually, ReclaiMe Pro has it, Runtime has it, ZAR has it, and perhaps R-Studio has too), and if you happen to know the block size exactly, do not set the limiter to exact block size. If you know the block size is 128 whatever units, set limits to 64 low and 256 high ( of the same units , repeat, the same units ). Otherwise, if the automatic detection gets you the value at one of the edges of the range, you do not know if it is because the value is correct, or because it hit the limit and was not able to further change the block size. The final block size must be inside the allowed range, not on the edge.

Theoretically, data recovery tools are read-only that means that their usage cannot cause any damage. However, in practice, when recovering data you may observe the effect as if the hard disks are being destroyed mechanically. For example, we took four disks from the NAS, connected them to a PC and launched the RAID recovery tool. Immediately S.M.A.R.T. monitoring software (Cropel) raised alarm because too many seek errors arose. Indeed, these seek errors were caused by the disk vibration that was provoked by the RAID recovery command to move disk heads on all the disks simultaneously. To tell the truth, a NAS device does the same when reading data from an array. However, a regular NAS device is equipped with more vibration-resistant drive mounts and fastenings. So, when transferring disks from the device managing RAID array to a regular PC, you may get alerts from S.M.A.R.T. monitoring software telling that the values of the Seek Error Rate attribute have changed significant...

Based on this MS blog post , once you lost a storage space configuration, it is just lost. At the moment, there is a significant difficulty recovering JBODs automatically. What the Storage Spaces subsystem does is, in effect, creation of JBODs of 256MB blocks. So in the end you can have just a plain JBOD, RAID 1 over JBODs, or RAID 5 over JBOD configuration. The capability to put 256MB blocks back into the pool and then use them again as needed, probably for another volume, leads to fragmentation on the pool level. That is, the volume itself gets fragmented, and obvously alignment gets out of window. This means no RAID recovery on Storage Spaces unless some significant breakthroughs are made. Even simple volumes will not be recoverable if they got fragmented.

If in RAID recovery the software says the block size is X, while customer says it is Y, X and Y being diffferent, the software is correct at least nine times out of ten. Sometimes, that annoys the customer, but there is little we can do about it.

Human vs. computer battle. As far as I am aware there is no way to rebuild HFS+ RAID from file-system analysis. NTFS is simple because has good system counters to use, same as EXT. But HFS+ requires good knowledge of RAID distributions. As you see, people rely on some property of the filesystem being recovered to produce a strong signal that we can use to determine the correct RAID configuration. NTFS provides plenty of those. FAT provides even more. With EXT, hddguy probably knows more than I do, because I'm not aware of any strong singal in EXT. By and large, humans perfer to find a small bit of data with high signal-to-noise ratio, and use it. Understandable because it is limits amount of effort involved. The RAID recovery software , on the other hand, mostly works with weaker signals. Weaker signals have far worse SNR, but they are in plenty. For example, you can calculate entropy values for any data. Obvious computer strength is to quickly process large arrays of data, which ...

Some do actually like non-standard hardware and software setups. If we build a 16 TB RAID 5 (9x 2TB), can we then install Windows on it? Probably yes, with some U/EFI trickery, but then troubleshooting this contraption if hardware ever dies would be a nightmare with 9 drives. Now another try We have a leftover of drives, like all sorts of 160GB to 2TB Parallel ATA, all sorts of Serial ATA, five RAID/HBA controllers, and a motherboard. We thought of putting it all together and deploying ZFS over it. Do you think it is a good idea? Actually, no. The complexity of the failure modes for the proposed design is just mind-boggling. First of all, when ZFS crashes, there is no reliable data recovery for it. Then, multiple HBA/RAID cards from different vendors in the same system are not going to work stable. More then, with a different size drives, no common RAID scheme can be applied. Should the RAID fail, the system is not recoverable. OK you can go with ZFS hybrid filesystem-RAID capability, ...

Every once in a while we do a remote recovery session via TeamViewer. The most annoying thing in remote recoveries actually is not knowing who is in control . TeamViewer does not provide any feedback when the other party is going to take over by pulling a mouse cursor away. The fact that someone is standing on the remote side watching what you doing, and you cannot even tell if they are there or not, is not very comforting but acceptable. With remote recoveries, it is a part of a job, actually. Someone may choose to ride the shotgun with you. The real problem starts when they interfere with what you are doing and there is no way to stop it. This is not really because people on the remote side are specifically evil, just because there is no convenient way to establish who is in control, and how to request or how to relinquish it. Damn annoying still.

If there is a set of disks, but the RAID type is not known, how do we determine what type of RAID is that? Most of the RAID recovery programs, including ours at www.FreeRaidRecovery.com , require the RAID type to be provided by the operator. In a most simple case, where all disks are available, one can get the idea of the RAID type by just plugging all the disks and looking at the Disk Management data. The following cases are most typical, 1. One or multiple partitions on exactly one of the disks. This is a RAID 0 or a RAID 5, more likely RAID 0. 2. One or multiple partitions, with two identical sets of partitions on two disks. With three disks, this is a RAID 5. With four or more disks, this is either a RAID 5 or a RAID 10. The above does not account for RAID 6 or exotics like RAID 3, and assumes MBR-style partitioning on the array, but nevertheless makes for a good start when working with an array of unknown type.

A full, start-to-end RAID recovery is generally a three part process. Determine status of the member disks and make clones when required. Detect RAID parameters and perform destriping If the destriped volume is not readily mountable, perform filesystem recovery on it to pump out the data Now, if the above three steps fail to produce correct data, the question is how do we tell if it is RAID recovery part, or filesystem analysis part that failed ? We tell if the RAID recovery is OK by looking at the sizes of the recovered files. If there are multiple good files recovered which are larger than twice the full row size (i.e. larger than 2 * block_size * num_disks), then the RAID recovery is almost certainly OK. However, if all good files are of the small size, the RAID parameters should be investigated. This also applies to the files found by raw scan; however, keep in mind that file sizes produced by raw scan are not reliable.

The customer walks in and says something along the lines of you should have more input options for your RAID Recovery app . Unfortunately, it just does not work that way. As you add more options and combinations thereof, people starting to get lost among these fast. Interestingly, I recall once considering an automatic software to detect all RAIDs attached to the system with just one click of button. Something more along the lines of All your RAID are belong to us , which would eliminate the requirements both to specify array type and to select a disk set. Just do all the probing and produce all possible RAIDs. Unfortunately, this did not work out for technical reasons. Actually, even providing a correct RAID type may prove difficult if the array was created five years ago, the person setting up the system retired four years ago, and noone even noticed the RAID until it failed. So now you have four disks, some of which may or may not work; so, RAID 0, RAID 10, RAID 5, or RAID 6? ...

A customer walks in and says: We need to recover a FATX volume, can you do that? - Sorry, no. Various exotic filesystems, btrfs, logfs, and even ReiserFs have always been considered a job for a data recovery serivce, not an automated software. Software is cheap, but only resolves common cases by applying typical solutions. Data recovery service is expensive, and applies its high fees toward the difficult cases, e.g. writing custom software to deal with just one specific case. Lately, there is an influx of requests for something nonstandard. The latest hit was - We have ReiserFs on the RAID5. - Okay, no problem. Turns out there was a problem. The RAID5 was using 512 bytes per block. JMB 393 controller. Oops. So far we have delayed parity (from HP SmartArray), Promise RAID 6 with its non-standard Reed-Solomon, Promise 1E interleaved layout, exFAT filesystem recovery . The capability to recover RAID with a block size of 512 bytes is in the pipeline, currently undergoing testi...

Every once in a while, we get a feature request for our RAID recovery software ( http://www.freeraidrecovery.com/ ) to implement the ability to interrupt the analysis midway and get a list of possible solutions, sorted by confidence level . There is some strong reservation against this would-be feature. Although it looks like a good idea, a very nice thing to have, it has some undesired consequences we cannot allow. The confidence thresholds are there for reason, and we put an effort to ensure they are balanced between faster analysis (lower thresholds) and reliability (higher thresholds). Once we make incomplete solutions accessible, people will start using these solutions on real data. Sooner than later, someone is going to destroy their RAID 5 by reassembling it on the controller using wrong parameters set. In RAID0, this would be no harm (just re-assemble again in correct order), but with RAID 5, incorrect assembly (automatically followed by a rebuild) destroys the array beyo...

Did a RAID 6 test on an Intel SRCSASBB8I controller (LSI 1078 chipset, also used in Dell PERC 6/i), and it was successful save for a couple minor issues. First we got the R-Studio RAID matrix rendered wrong (data blocks are off by one), and also under the hood it looks like the analysis can use some more noise reduction. Still, the output image is OK. This applies to ReclaiMe Free RAID Recovery build 397 at www.freeraidrecovery.com .

If you want some unusual RAID layouts, call Promise. So far, Promise controller has the most bizzare RAID 6 variation we've seen, and also gave us a nasty surprise with RAID 1E. Typically, RAID 1E would be laid out as follows 1 1 2 2 3 3 4 4 5 5 6 6 (the above is so-called NEAR layout) or 1 2 3 4 5 6 7 8 9 3 1 2 6 4 5 9 7 8 (the above is FAR layout). Now Promise combines the best of both worlds to produce a third variation (Promise 1E layout) 1 2 3 3 1 2 4 5 6 6 4 5 Looks promising , doesn't it? So, the implementation of RAID 1E recovery capability in ReclaiMe Free RAID Recovery ( www.FreeRaidRecovery.com ) is delayed until we figure out how to handle this one.

Recently, a question arose about our RAID recovery software ( www.freeraidrecovery.com ), along the lines of " There is a RAID5 array, but I get some results if I try to recover RAID0 as well, what's wrong? " Actually, there is nothing wrong. RAID Recovery requires the array type as outside input, along with the list of member disks. It has no way of determining if the provided array type is correct or not. Therefore, it tries to produce the closest possible layout for a given array type. There are certain cases (mostly involving RAID6) were nothing meaningful can be constructed, and it gives up with the appropriate error message. However, in most cases some layout will be produced. If the original array type is not readily known and it is not possible to infer it from the number of disks and available capacity, the only solution is trial-and-error testing of all possible layouts (of which there are about four fundamentally different, RAID5/5E being practically equal).

We have a 5TB raid 5 ... I think the MBR got messed up and windows server 2003 no longer is able to view the contents of the virtual disk... ... recovery software ... will fail due to the large amount of files we have, about 64 million files. Some raid software will save the recovered files to another disk.. however 5TB is a large amount of data to save and we do not have any disks or other virtual disks that large. (from ServerFault ) I suppose we can do that 64 million files, should be interesting to test. As far as pricing goes, 5 TB swap space is just 3x 2TB WD Caviar Green in RAID 0. At a cost of 4x Caviar Greens you get the same swap space protected by RAID5. At about $60 a pop, the entire lot would cost about $250. The initial cost of purchase is then further offset by putting the temporary drives to use as a backup media.

The most significant problem we facing now is the lack of feedback. As it was earlier discussed, the RAID recovery software works with models, not actual data. Now the difficulty is to determine how good these models are in real-world application. Relying on people feedback does not look very promising. We'll consider incorporating certain automatic dial-home system sending statistical info we can use to get a clearer view of things. The obvious alternative of buying a sample of all possible RAID controllers and NAS units for tests looks prohibitively expensive, both in terms of money and effort.

Recently we set to develop the algorithm to automatically recover RAID 5 array with delayed parity , as implemented by HP SmartArrays. Today we have mostly finalized the work. From this day on (build 306 onwards) ReclaiMe Free RAID Recovery is capable of recovering such arrays.

Improved memory usage in ReclaiMe Free RAID Recovery ( www.FreeRaidRecovery.com ), by the factor of ten. The update is not yet live, but will be out shortly, probably no later than tomorrow. Surprisingly, there was not at all that much loss of speed.

With a RAID recovery , certain exotic RAID types can be recovered using the same basic algorithms, because these exotic RAID types can be reduced to one of the three basic types (RAID0, RAID1, and RAID5). The list goes like this: RAID 1+0 or 0+1 can either be reduced to RAID 0 by removing mirrors ( near layout), or can be recovered as a RAID 0 straight away ( far layout). RAID 5E or RAID 6E can be recovered as a RAID 5 or RAID 6 respectively, because all the extra data is at the end of the array. RAID 5EE can be recovered as RAID 6 with one of the sets of parity corrupt. RAID 4 is actually a variation of RAID 5 where parity does not change position across rows. The array types requiring special processing are RAID 1+0 using offset layout and RAID 1E.