Adventures in LSI-land

I bought an IBM M1015 raid card. Which is actually an LSI 9240, containing a SAS2008 chip. It is a basic card, with no battery backup. It has two mini-SAS ports, that can be split to 8 SAS/SATA devices (four per port), and then configured to either RAID0 or RAID1. With a feature-key (some little IBM fob that is slapped onto the card), you can also get RAID5, and other modes. RAID1, however, is what I wanted.

So turns out the IBM M1015 does not get detected by three of the four motherboards I tested it on. It worked on an Intel reference board (?) in some old workstation, but not on:

• MSI MS-7497
• MSI M2N-E
• Asus P8Z68-V GEN3

Out of those, the 7497 and the Asus did not even boot, just a blank screen if the card was in (any PCI-E slot). I also couldn’t find any BIOS settings that would make it boot. Nothing would ever come on screen. The M2N booted just fine, but no card was detected.

So, instead of giving up, I decided to try and flash the card with a different firmware. In a process known as crossflashing, you essentially clean up the firmware on the card, and flash something that wasn’t originally intended to go on the card.

For this card, there are three alternatives (at least):

• Original IBM M1015 (LSI 9240) firmware, as provided by IBM
• LSI 9211-8i IT-firmware (tried this one too, machine booted fine and detected the drives behind the card)
• LSI 9211-8i IR-firmware (i picked this one)

The latter two being provided by the actual manufacturer of the card, LSI. The differences (as listed by this site) are as follows:

IBM M1015 firmware

• Can do RAID0 and RAID1. Contains Web-Bios for controlling settings, perhaps other IBM branding

LSI IT-firmware

• Can do straight passthrough, without RAID. Apparently ideal for ZFS for instance.

LSI IR-firmware

• Can do RAID0 and RAID1, 1E and 10, as well as passthrough

The flashage!

So, to flash the card, you need a machine that (obviously) can recognize the card. I had two of them, the Intel reference motherboard-box, and an IBM x3690 X5 server (UEFI, more on this later). On top of this, you need a bootable USB stick. I used a Kingston U3 USB stick, which is recognized by most machines, and works great. On that stick, I have FreeDOS, the LSI Megacli/megarec tools, as well as the required firmware- and bios-images. I can make a package of the files that I have, so you can slap em on a card. To get FreeDOS on a stick, check here and then here, for instance. I also needed the UEFI Shell file, again, more on that later. But you might as well put that file on your stick too.

To start off, you need to clean out whatever is on the 16MB flash chip on the card. Boot up to freedos, and move to the directory where you have megacli and the firmware files. First, get the ‘SAS Address’ of the card, either by looking at the card physically, or by running:

megarec -writesbr 0 sbrempty.bin
megarec -cleanflash 0

After this, you have a card with pretty much nothing on it. If you do not flash a firmware on the chip, you have what is effectively a dead card. Reboot the computer. You should not see the machine detect the card, as it will not load the BIOS of the card. Now, there are two ways to move forward. Either boot back to the FreeDOS environment, and flash the correct firmware to the card, or load up an UEFI Shell (depending on your hardware) and do the flashing from there. You should start with the FreeDOS-way:

sas2flsh -o -f 2118ir.bin -b mptsas2.rom
sas2flsh -o -sasadd put_your_SAS_Address_here

A note about the first command: Chose which firmware you want to flash, either it or ir. Note, that you can flash between any of the firmwares after the fact, just do megarec -cleanflash0, and then proceed to the second step with the new firmware that you chose. You can leave out the -b mptsas2.rom command. This is the BIOS of the card, which you do not need, if you do not intend to boot off a RAID-array which is behind the card. Boot times will be faster if the BIOS isn’t loaded. I put it in just for good measure (and yes, the boot slow-down is noticeable).

The UEFI caveat

If when running the first command you get: “ERROR: Failed to initialize PAL. Exiting program.”, there is a problem with your motherboard’s BIOS and/or you have UEFI instead of BIOS. I can confirm that this happened on a regular old workstation (3 years old maybe?) which does not have UEFI (or then I’m blind and dumb), so I’m not exactly sure as to what is causing the error in this case. In either case, I had to move the card to a server that actually has UEFI, in my case the IBM x3690 X5 server. This server, however, does not have UEFI shell, for some inconceivable reason. But, I was able to boot to the UEFI Shell .efi file that I downloaded previously, by going into UEFI, going to Boot Manager, and selecting Boot From File. Then I navigated to the USB stick where the .efi file was, hit enter, and soon I was in the UEFI shell.

Some notes about the shell. It’s Unix-like, but has some select commands that you need here. Firstly, to navigate to different disks that it detected, use fs0:, fs1: etc. In my case, the USB stick was fs0. After that, you can use either standard DOS or Unix commands to list files; so either ls or dir. Navigate to the directory where you have your megacli and whatnot using cd, as usual. There, you can use the following commands to flash the card (and BIOS):

sas2flash.efi -o -f 2118ir.bin -b mptsas2.rom
sas2flash.efi -o -sasadd put_your_SAS_Address_here

Again, you can leave out the -b mptsas2.rom if you don’t need the BIOS. This time, I had success in flashing the card.

After the commands are done, reboot the machine. You should now see it loading LSI 2008 whatever, instead of IBM M1015. You can use Ctrl-I to enter the configuration management, where you can set card options and create RAID arrays.

Performance and management

A note about performance: When I created the RAID 1 array (consisting of two WD 1TB Red drives), the background initialization which started (There was apparently also a fast initialization option) had significant performance impact. Running Crystal Disk Mark x64, I got around 85MB/s sequential reads. When the init was done, these were the figures:

Noteworthy is the write-performance. After the Init was done, I got a log entry stating the Write Cache is disabled. Since this card has no battery backup (being an entry level card), Write Cache probably should be disabled. If it were enabled (I might try this later) from the card options, write perofmance will be significantly better. But, since this is mostly for storage (more reads than writes by far), this is not of concern to me. Data integrity is more important.

In linux, you can use the same MegaCLI from LSI to manage and view the card status. In Windows, you can use a similar graphical program called MegaRAID Storage Manager, from LSI (on the search page, pick Host Bus Adapters -> LSI SAS 9211-8i -> Management Software and Tools), which supports most versions of Windows desktop and server. To download either of these, visit here and select Host Bus Adapters, then LSI SAS 9211-8i, and your relevant download category. Also get the driver for your operating system from the same site, even though Windows 7 and Linux both supported this out of the box.

Oh. also in case you were wondering, the cable I got to hook up the SATA-drives to the card was this one, the DeLOCK Mini SAS 36 pin (SFF-8087) -> 4 SATA cable. The price at the time of this article was 17,90€.

Sources:

I would like to thank the following pages. Without them, this would not have progressed.

http://blog.monsted.dk/?q=node/5
http://www.servethehome.com/ibm-serveraid-m1015-part-4/