3Ware 9650SE-16ML Default Config

[Jared]

I'm currently working on a RAID 50 that used this controller which failed. I'm ordering one just so I can play with it's settings to see, but until it arrives I'm wondering if anyone can help point me in the right direction about its default stripe size, parity delay (if any), etc. It's a big array so any possibilities I can eliminate will be a big help.

Thanks!

 

[Jared]

Update: OK, so I found a good long manual and looks like default stripe size is 64kb, and looks like right synchronous rotation judging by the graphics in the manual. Now I just have to figure out how it splits ups the drives into the two RAID 5 groups.... fun, fun, fun.

 

[Jared]

Update 2: Confirmed with their tech support that it is right synchronous parity rotation, 64Kb stripe across the RAID 5's and 128 across the RAID 0, and that it doesn't employ any parity delay. I think I'm about halfway through figuring out the drive order now. Fun, fun, fun.

 

[hddguy]

Let me know if you need any help with it, I can look at it remotely if you cant figure it out. You need to identify where the parity blocks are at each drive to work out the direction and length of it, also you can identify 2 disks with XOR at the same position then these are from opposing RAID sets. Sometimes you can use differentiate which set by looking at the pattern of data but additionally there may be some metadata at the ends of the disks that can differentiate the two RAID5 sets.

 

[Jared]

Let me know if you need any help with it, I can look at it remotely if you cant figure it out. You need to identify where the parity blocks are at each drive to work out the direction and length of it, also you can identify 2 disks with XOR at the same position then these are from opposing RAID sets. Sometimes you can use differentiate which set by looking at the pattern of data but additionally there may be some metadata at the ends of the disks that can differentiate the two RAID5 sets.

I've already identified which 8 drives go with which half of the array, by XOR'ing all the values across them. I had a pretty good idea by looking at the controller cabling, but there were two that were switched from their physical locations it looks like. In any event I now have two sets of 8 that XOR to each other confirming that they are RAID 5's.

However I don't think your idea about them being in the same position on opposing RAID sets will work. Keep in mind that it's RAID 0 over the RAID 5, so none of the data is duplicated or parity'ed to opposing sets. That's a trick you can use on RAID 10, but not 50, I wish you could, I'm pretty sure I've figured out the order of one set, would be nice if the other matched up too.

 

[hddguy]

What I mean is that XOR will be physically present at 2 disks at any time, but not within the same RAID5 set, so identifying the pairs of disks with XOR patterns at the same offsets will confirm they are from opposing RAID5 sets within the top level RAID0.

 

[Jared]

What I mean is that XOR will be physically present at 2 disks at any time, but not within the same RAID5 set, so identifying the pairs of disks with XOR patterns at the same offsets will confirm they are from opposing RAID5 sets within the top level RAID0.

Now I understand what you mean. I'm pretty sure I've identified the two sets, and even the drive order of one set. I can even browse the filesystem now. But, I'm having a strange issue which must have something to do with stripe size or the RAID 0 or something strange with the parity rotation. I'm analyzing jpeg files, finding where they start and where they end (using every possible drive to follow in the rotation until I find it) but it seems like there are times where it should be in the same set but instead is in the other or vice versa.

 

[hddguy]

What file system is it? File systems like EXT and NTFS have filesystem 'counters' you can use that may help. Also, in some nested arrays the block size of the top level stripe is the total number of sectors in a single stripe of data the RAID5 (excluding the parity block of XOR). At 64KB RAID5 stripe with 4 disks at each RAID5 set, try block-size of 64KB*3 192KB).

Also, depending on how you determined which disks belong to which RAID5 set, you could have one or more disks in the wrong set. Just to be sure run Raid Reconstructor and select 4 disks in one RAID5 set and run XOR test. If it passes with all green then its right, if it does not then adjust the members in this specific set until it does show green (will only work on each RAID5 set, not on the entire array with all disks).

 

[acelab]

Hello Jared,

Maybe you have configuration like this:

It can be called “RAID 0 with block size 128KB, one is made on two RAID 5 RS with block size 64KB”. Pay attention to the data blocks 4, 5, 6, 7. They have a nonstandard position. We have faced RAID like this in practical work.

By the way... If you find out exactly what kind of blocks are XOR, then you can easily define using Data Extractor RAID Edition what part of RAID50 is this drive. Just try to autodetect indexes for ReedSolomon (yes, we know, that there’s no RS blocks, but XOR is particular case of RS).

This is complete algorithm:
1. Substitute of RS blocks for XOR;
2. Add all data blocks to the RS sequence;
3. Try to find the area where all members contain non-zero data (just try different RAID LBA);
4. Run autodetection of RS indexes;
5. Data Extractor RAID Edition should recognize the type of XOR-block and which data-blocks are used to calculate this XOR.

Before:

After:

 

[Jared]

Thanks, you're making me wish I had DE RAID edition. The diagram you show makes sense to what I'm finding, so I think that may be it (just bigger due to being 16 total drives). I managed to pull off the recovery using a less standard method, but was effective nonetheless.