Mark's blog

Entries from September 2009

Quicksearch

Syndicate This Blog

Dell MD3000i

I've just got a new array to play with at work for a small Xen virtualisation setup. It's the Dell MD3000i, which I've seen a few posts about before but though I'd chime in with my experiences. It is a budget array, but I have to say for the price it's not a bad bit of kit.

We've got it configured with dual controllers, 8x300Gb and 7x146GB 15k SAS drives. Throughput is around GigE wire speed - 110MB/s for both reads and writes. I'm also seeing a respectable IOPS figure depending on workloads: During an iozone run, I could see it sustaining around 1.5k IOPS.

True, the management features fall a little short when compared to the usual Sun and HP storage kit I'm used to, but it does the job. My main gripes are :

No built in graphing (seriously, Dell - WTF?), but you can do it from the CLI - see here.
Can't resize or change the I/O profile of a virtual disk once it's setup. This is a real pain, so make sure you set things up correctly the first time! You can however change the RAID level of a disk group once it's been created.
You need a Windows or RHEL box to run the administration GUI on - I'm sure you can probably hack a way to get the CLI running under Debian, but I haven't tried. You're probably straight out of luck if you want to run it on anything else like Solaris.
Can't mix SAS and SATA in the same enclosure. The controllers do support SATA as well as SAS, although SATA drives don't show up as options in the Dell pricing configuration thingy. Our account manager advised us that although technically you can mix SAS and SATA in the same enclosure, they'd experienced a higher than average number of disk failures in that configuration, due to the vibration patterns created by disks spinning at different rates (15K SAS and 7.2K SATA). If you need to mix the two types, your only real option is to attach a MD1000 array to the back (you can add up to two of these) and have each chassis filled with just one type of drive.

The hardware failover works nicely - the array is active/passive for each virtual disk, as both controllers are typically active, each handling separate virtual disks for load-balancing purposes. When a controller fails, the remaining "good" controller takes over the virtual disks or disk groups from the failed controller. Failback is pretty transparent - the GUI guides you through the steps, but I found that simply inserting a replacement HD/Controller/etc. just did the job automagically.

Multipath support under RHEL/CentOS with multipath-tools (dm-multipath) works fine with some tweaking - it uses the RDAC modules which lead to some oddness on CentOS 5.3. What tends to happen is that the first time device mapper picks up the paths, RDAC doesn't get a chance to initialise things properly (scsi_dh_rdac module isn't loaded) so you end up with all sorts of SCSI errors showing up in your logs. After flushing your paths (multipath -F) and restarting multipathd, things are OK. This is apparently fixed in RHEL 5.4, so should make it's way out to CentOS from there. I'm unsure what the status is on other distros, though.

My multipath.conf contains the following :

devices {
        device {
                vendor "DELL"
                product "MD3000i"
                product_blacklist "Universal Xport"
                path_grouping_policy group_by_prio
                getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
                path_checker rdac
                prio_callout "/sbin/mpath_prio_rdac /dev/%n"
                hardware_handler "1 rdac"
                failback immediate
        }
}

And with everything working, multipath -ll shows :

360026b90002ab6f40000056a4aa9e87b dm-12 DELL,MD3000i
[size=409G][features=0][hwhandler=1 rdac][rw]
_ round-robin 0 [prio=200][active]
 _ 21:0:0:1  sdi 8:128 [active][ready]
 _ 22:0:0:1  sdj 8:144 [active][ready]
_ round-robin 0 [prio=0][enabled]
 _ 20:0:0:1  sdg 8:96  [active][ghost]
 _ 23:0:0:1  sdh 8:112 [active][ghost]

Update: It looks like the admin tool and SMcli are just shell script wrappers that run Java apps. I tried a quick'n'dirty hack of installing everything under RHEL, tarring up /opt/dell and /var/opt/SM and then transferring them over to a Debian Lenny host. All I had to change was the #!/bin/sh to #!/bin/bash at the top of the SMcli and SMclient wrappers, and they seem to work. I haven't put them through any serious testing though...

Posted by Mark Round on Monday, September 14. 2009 at 12:08 in Sysadmin
no comments yet, be the first! Trackbacks (0)