Einarc: Reference manual


Table of Contents

1. Concept
2. Architecture
3. Supported devices
4. Installation
4.1. Manual installation
4.2. Package installation
5. Command-line interface (CLI)
6. Application programming interface (API)
6.1. Add libraries
6.2. Listing available adapters
6.3. Create adapter objects
6.4. Use adapter object methods
6.5. Error handling
7. Object interface
7.1. query (special)
7.2. adapter
7.3. log
7.4. physical
7.5. logical
7.6. task
7.7. firmware
8. Planned features
8.1. More devices
8.2. General features

1. Concept

Hardware RAID controllers are complex devices: essentially, it’s a computer inside a computer. It has it’s own processor (or may be processors), it usually has it’s own RAM, and, most important, it runs its own software, which is usually a fairly complex operating system that manages physical discs, hides all the RAID magic behind the curtains and allows the host operating system to see only few logical discs.

Unfortunately, most of the RAID controllers are proprietary. It’s a well-known fact that physical discs used on controller A are incompatible with some other controller B (even worse, they can be incompatible between various revisions of firmware of controller A). RAID’s firmware hides all the intricate management of discs from the system administrator. Administrator can control RAID using either:

  • BIOS OPROM, which is available by pressing some magic key when the computer’s booting.

    • Pros: it’s basic and solid method, it works, it doesn’t require any OS installed (can be done before OS installation), it works even if OS is broken, etc.
    • Cons: it requires rebooting of computer when trying to edit RAID or even to check/monitor its state (which is hard to do remotely without KVM-over-IP or similar solution), it requires manual intervention, it’s not scriptable (= it’s hard to create the same RAID configuration on 1000 machines), and, most of all, every RAID controller model or vendor has his own version of such OPROM utility, incompatible with each other even in terms they use.
  • Some management utilities, running in OS. Such utilities are mostly proprietary, provided by vendors, available in binary-only form.

    • Pros: they usually include command-line interface (they’re scriptable), they’re suitable to call regularly and automatically to monitor status of discs, they don’t require rebooting, etc.
    • Cons: of course, they require booted OS to run, and, worst of all, they’re also very incompatible with each other.

Einarc project tries to solve such problem: provide a unified, solid interface to manage all possible storage devices.

2. Architecture

Einarc unifies various RAIDs and storage devices, providing a simple object interface for end-user. Inside, it works as a proxy, translating commands from that object interface to RAID-specific commands for it’s proprietary CLI, and back, it translates response from proprietary CLI into unified answer.

Figure 1. Einarc architecture diagram

Einarc architecture diagram

Support for various RAIDs is implemented as classes. Each class has to implement a couple of methods that would translate requests.

  • Common unit of size used is Megabyte = 1024 * 1024 bytes. Various adapters provide various interfaces and size units, so X megabytes on one adapter may differ from X megabytes on some other adapter.

3. Supported devices

Currently, there are following modules available:

  • adaptec_aaccli — supports older Adaptec SCSI RAID controllers, such as Adaptec SCSI RAID 2230SLP, Adaptec SCSI RAID 2130SLP, Adaptec SCSI RAID 2120S, Adaptec SCSI RAID 2020ZCR, Adaptec SCSI RAID 2025ZCR.
  • adaptec_arcconf — supports newer Adaptec SAS/SATA RAID controllers, such as Adaptec RAID 5405, Adaptec RAID 5445, Adaptec RAID 5805, Adaptec RAID 5085, Adaptec RAID 51245, Adaptec RAID 51645, Adaptec RAID 52445, Adaptec RAID 3405, Adaptec RAID 3085, Adaptec RAID 3805, Adaptec RAID 31205, Adaptec RAID 31605, Adaptec RAID 2405, Adaptec RAID 2045, Adaptec RAID 4000, Adaptec RAID 4800SAS, Adaptec RAID 4805SAS.
  • amcc — supports all AMCC/3ware controllers, such as 3ware 7000 series, 3ware 8000 series, 3ware 9690SA, 3ware 9650SE, 3ware 9550SX, 3ware 9500S, 3ware 8506.
  • areca — supports all Areca SATA RAID controllers, such as ARC-1110, ARC-1120, ARC-1130, ARC-1160, ARC-1130ML, ARC-1160ML, ARC-1170, ARC-1200, ARC-1210, ARC-1220, ARC-1230, ARC-1260, ARC-1231ML, ARC-1261ML, ARC-1280ML, ARC-1280.
  • lsi_megacli — supports newer LSI MegaRAID SAS/SATA RAID controllers, such as LSI MegaRAID SAS 8208ELP, LSI MegaRAID SAS 8208XLP, LSI MegaRAID SAS 8204ELP, LSI MegaRAID SAS 8204XLP, LSI MegaRAID SAS 84016E, LSI MegaRAID SAS 8704ELP, LSI MegaRAID SAS 8708ELP, LSI MegaRAID SAS 8300XLP, LSI MegaRAID SAS 8708EM2, LSI MegaRAID SAS 8888ELP, LSI MegaRAID SAS 8880EM2, as well as rebranded copies of these controllers from Dell, HP and Intel, as well as all on-board or daughterboard products based on LSI SAS 1078 chipset.
  • lsi_megarc — support older LSI MegaRAID SCSI/SATA RAID controllers, such as LSI MegaRAID SATA 150-4 (523), LSI MegaRAID SATA 150-6 (523), LSI MegaRAID SATA 300-8X, LSI MegaRAID SATA 300-8XLP, LSI MegaRAID SATA 300-4XLP, LSI MegaRAID SCSI 320-0 (520-0 CH), LSI MegaRAID SCSI 320-0X (530), LSI MegaRAID SCSI 320-1 (520-1CH), LSI MegaRAID SCSI 320-1LP (520-1 CH LP), LSI MegaRAID SCSI 320-2 (518), LSI MegaRAID SCSI 320-2E, LSI MegaRAID SCSI 320-2X (532), as well as rebranded copies of these controllers from Dell, HP and Intel, as well as all on-board or daughterboard products based on LSI MegaRAID 160/320 or 150/300 chipset.
  • lsi_mpt — Fusion/MPT, now branded as LSI MPT design of HBAs, available in such adapters as LSI20160, LSIU320, LSI20320-R, LSI20320IE, LSI21320-R, LSI22320-R, LSI22320SE.
  • software — Linux software RAID device (md), for a sake of completeness.

This is not a complete list. To check if your adapter is supported, just run ./configure or einarc-install without options to select all available modules. There’s a good chance that your adapter would be supported too.

4. Installation

Currently, Einarc is a sub-project of Inquisitor hardware testing framework and can be downloaded either bundled with Inquisitor itself or from its own mini-homepage. Alternatively, your favourite distributed may provide you with pre-built package.

4.1. Manual installation

After downloading, extract the contents of tarball (this would yield a directory with sources), then run ./configure. You can use the following options:

\--target=TARGET
target to build for; default is selected by looking up architecture of current host
\--bindir=DIR
where to place user executables
\--rubysharedir=DIR
ruby share directory
\--rubylibdir=DIR
ruby binary extensions directory
\--einarclibdir=DIR
proprietary extensions directory
\--modules=LIST

a comma-separated list of storage support modules to build:

  • all — SPECIAL: build all available modules, default
  • auto — SPECIAL: autodetect which modules are required — experimental feature, added since v1.4, tries to make an automatic guess by analysing PCI IDs of machine where you run ./configure; requires lspci to run. If in doubt, just use "all" — it will download everything, all CLIs not usable on current machine just won’t be used.
  • lsi_megarc — older LSI MegaRAID SCSI/SATA adapters
  • adaptec_arcconf — newer Adaptec adapters that use arcconf
  • amcc — 3Ware/AMCC RAID 7/8/9xxx/95xxx series controllers that use tw_cli
  • lsi_megacli — newest LSI MegaRAID SAS adapters
  • adaptec_aaccli — older Adaptec SCSI adapters that use aaccli
  • areca — Areca adapters
-h, \--help
show help

After running ./configure with options, file Makefile.config would be created. You can check it out and edit by hand if necessary. Then run make to start a build process. In fact, Einarc is written in a script language and does not need build, but we use the build stage to download all the proprietary CLIs that would be required to support requested controllers.

After downloading, unpacking and preparing all required tools, you can run make install (usually requires root privileges) to install Einarc in your system.

4.2. Package installation

Since version 1.4, it’s possible to package and install Einarc as pre-built binary package (rpm, deb) from major distribution repositories. Recommended name for a package is einarc.

After installation of package, all Einarc modules that don’t require proprietary CLIs are available (so far it’s only software module to control Linux software RAIDs), so in most case you’ll still have to download necessary proprietary CLIs to make Einarc useful. It can be done automatically on target machine by running einarc-install. It can be called with the same --modules option as in ./configure:

\--modules=LIST

a comma-separated list of storage support modules to build:

  • all — SPECIAL: download and build all available modules, default
  • auto — SPECIAL: autodetect which modules are required — experimental feature, added since v1.4, tries to make an automatic guess by analysing PCI IDs of machine where you run ./configure; requires lspci to run. If in doubt, just use "all" — it will download everything, all CLIs not usable on current machine just won’t be used.
  • lsi_megarc — older LSI MegaRAID SCSI/SATA adapters
  • adaptec_arcconf — newer Adaptec adapters that use arcconf
  • amcc — 3Ware/AMCC RAID 7/8/9xxx/95xxx series controllers that use tw_cli
  • lsi_megacli — newest LSI MegaRAID SAS adapters
  • adaptec_aaccli — older Adaptec SCSI adapters that use aaccli
  • areca — Areca adapters

5. Command-line interface (CLI)

Simplest way to use Einarc is to use command-line interface. Einarc can be invoked using:

einarc [options] <object> <method> <parameters>

Options:

-l, \--list
list all adapters found in system and exit
-V, \--version
print program version and exit
\--help
show short help text and exit
-h, \--human
output human-readable information
-t, \--type=TYPE
type (manufacturer) of RAID adapter
-a, \--adapter=NUM
select the adapter if multiple adapters are present

Refer to object interface description for particular object.

Note that command-line interface is always ready to provide a hint. For example, one can forget almost everything and start with just calling "einarc". Consider this session:

$ einarc
Adapter type not specified: use --list to list all adapters and choose one
$ einarc --list
areca   1       ARC-1160        PCI
$ einarc -h -t areca
Object not specified; available objects: logical, log, adapter, task, physical

Let’s start with showing all logical drives. We don’t know what method we should choose, but Einarc will give a hint:

$ einarc -h -t areca logical
Command not specified; available commands: list, add, delete, clear, get, set
$ einarc -h -t areca logical list
#  RAID level   Physical drives                 Capacity     Device  State
1  5            0:1,0:2,0:4,0:5,0:8            762939.45 MB          normal
2  1+0          0:6,0:7                        190734.86 MB          normal
3  5            0:9,0:10,0:11,0:12,0:13       1907348.63 MB          normal
4  1+0          0:15,0:16                      476837.16 MB          normal

More complex example: let’s check if a physical drive is hotspare drive or not. Such simple boolean flag is available as a property of physical object, and Einarc will give a hint on how to get it and when to add a reference to particular physical disc ID:

$ einarc -h -t areca physical
Command not specified; available commands: list, get, set
$ einarc -h -t areca physical get
Object identifier not specified
$ einarc -h -t areca physical get 0:1
Property not specified; available properties: hotspare
$ einarc -h -t areca physical get 0:1 hotspare
0

Getting a property of adapter is in fact even simpler: there shouldn’t be any IDs. Here we’ll get a list of all RAID levels a particular adapter supports:

$ einarc -h -t areca adapter
Command not specified; available commands: info, restart, get, set
$ einarc -h -t areca adapter get
Property not specified; available properties: raidlevels
$ einarc -h -t areca adapter get raidlevels
0
1
1E
10
3
5
6

6. Application programming interface (API)

Einarc’s libraries can be called directly with almost the same object terms as from the command-line interface.

Einarc is written in Ruby and thus can be used easily from this script language. Using Einarc API in your program takes a few simple steps:

6.1. Add libraries

Einarc modules should be loaded before use. Adding the following require statement would automatically load all available RAID support modules:

require 'raid/baseraid'

6.2. Listing available adapters

To find out which adapters are available, use:

RAID::BaseRaid.query_adapters

This would return you an array of hashes, one element of array for each adapter found, for example:

[{:driver=>"areca", :version=>"PCI", :num=>1, :model=>"ARC-1160"}]

6.3. Create adapter objects

Instantiate a particular adapter object, for example:

a = RAID::Areca.new

6.4. Use adapter object methods

After adapter object a is created, one can use all the methods described in object interface using a simple convention.

  • If method does not return anything, it’s just

    a.object_method(all, other, arguments, ...)

    for example:

    a.logical_clear
  • If method should return its data as Ruby objects (a hash, an array, an array of hashes, etc), then it should be called as

    a._object_method(all, other, arguments, ...)

    Note the beginning underscore. An example:

    a._adapter_info
    => {"PCI product ID"=>"1160", "System Memory"=>"256MB/333MHz",
        "Current IP Address"=>"192.168.0.1", "Controller Name"=>"ARC-1160",
        "BOOT ROM Version"=>"V1.39 2006-1-4  ", "CPU DCache Size"=>"32KB",
        "Main Processor"=>"500MHz", "Firmware version"=>"V1.39 2006-2-9  ",
        "PCI vendor ID"=>"17d3", "CPU ICache Size"=>"32KB",
        "Serial number"=>"XXXXXXXXXXXXXXXX"}
  • If method should print whatever strings Einarc CLI displays, it should be called as:

    a.object_method(all, other, arguments, ...)

    For example:

    a.adapter_info
    PCI product ID          1160
    System Memory           56MB/333MHz
    Current IP Address      192.168.10.111
    ...

6.5. Error handling

All RAID-related activities raise an exception of class RAID::Error on error.

7. Object interface

Einarc exposes a relatively simple and straight-forward object model for an end-user.

Figure 2. Einarc objects

Einarc objects

Every computer (host) may have one or more adapters (object adapter). Adapters provide RAID services as logical discs (object logical) that are usually seen in OS as individual devnodes. Logical discs may include one or more physical discs (object physical). Physical discs are usually hidden from the OS and are accessible only through proprietary CLI. Adapters also may keep some history of what events happened in the past and what objects were involved (object log). Finally, adapters being little computers with their own operating system and multi-tasking may perform separate background tasks, such as building/rebuilding or checking of the array (object task).

Each of these objects have a couple of methods described below. Some of these objects have named properties that can be read using get and written using set. Some of the properties are read-only (only get would work), some are write-only (only set would work).

7.1. query (special)

A special function that lists available adapters of current model. This function is not called directly through object model, but called indirectly, iterating over all support drivers when performing an -l (--list) command line command to list all available and supported adapters.

Output is tab-separated strings in format:

Driver Adapter ID Adapter model Adapter serial number / version
lsi_megarc 00 MegaRAID SCSI 320-1 40LD/8SPAN
areca 01 ARC-1120 PCI

7.2. adapter

"adapter" is an object related to the controller card. It stores various settings with properties that can be read and wriiten with get and set and can be examined for general info with info command.

Properties:

alarm
(mute, disable, enable) — mode of adapter’s beeper alarm
coercion
(0, 1)
rebuildrate
(0-3) — a priority of rebuilding and other background tasks, 0 being the lowest (least percent of time slices goes to that tasks), 3 being the highest.

7.2.1. info

Gets general information about adapter. Information is returned as key-value pairs. Some keys are fixed and are consistent across all adapters:

  • Serial number
  • Firmware version
  • PCI vendor ID
  • PCI product ID
  • PCI subvendor ID
  • PCI subproduct ID

Other keys are adapter- and implementation-specific and thus vary.

Input
none
Output

several tab-separated lines in format

Name Value
Firmware Version 1L37
BIOS Version G119
DRAM 64MB

7.2.2. get

Input
name of property
Output
value of property

7.2.3. set

Input
  • name of property
  • value of property
Output
none normally, error message if error happened

7.3. log

"log" is an object to access adapter’s event log in NVRAM.

7.3.1. clear

Clears event log in NVRAM.

Input
none
Output
none normally, error message if error happened

7.3.2. list

Display full event log in NVRAM.

Input
none
Output

log entries, one per line, in following tab-separated format:

Entry ID YYYY-MM-DD HH:MI:SE Where it happened What happened
0 2006-12-04 14:13:24 ARC-1120-VOL#00 Create Volume
1 2006-12-04 14:13:24 Raid Set # 00 Create RaidSet

7.3.3. test

Posts a test message in log. Can be later read back to see that logging works.

Input
none
Output
none normally, error message if error happened; log list would now return at least this test message.

7.4. physical

"physical" is an object to access physical disc drives attached to adapter. It’s possible to get list with list and read and write it’s properties and settings with get and set.

Physical disc address is a:b, where a is channel number and b is a device number on that channel. Usually, it’s best to assume that it’s a string to pass through.

Properties:

  • hotspare (0, 1) — if true, a physical drive is considered a global hotspare.
  • state (free, hotspare, logical drive ID) — shows that drive is either free, reserved for hotspare, or belongs to the one of the logical drives (numeric).

7.4.1. list

Shows a list of all physical disc drives attached to adapter.

Input
none
Output

info on one disc drive per line, tab-separated:

Physical disc address Drive vendor and model Drive version, revision, firmware, misc info Drive serial Drive size Drive state
0:1 WDC WD800JD-00J 05.01C05 WD-WMAM97310565 80000 hotspare
0:2 WDC WD1600JB-00EVA0 15.05R15 WD-WMAEK1016963 160000 0
1:7 WDC WD3200JD-00KLB0 08.05J08 WD-WMAMR1259708 320000 free

7.4.2. get

Input
  • physical disc address
  • name of property
Output
value of property

7.4.3. set

Input
  • physical disc address
  • name of property
  • value of property
Output
none normally, error message if error happened

7.5. logical

Logical drive is a virtual drive, set up on regions of one or several physical drives. It is referenced with a logical drive ID, a plain integer number.

Properties:

  • raidlevel (passthrough, 0, 1, 5) — a RAID level of array or passthrough — a special logical disc without any RAIDs that will directly access single physical disc.
  • stripe — stripe size of logical drive in kilobytes; possible values vary with adapter, 64 is typical default.
  • cache (writeback, writethrough) — a write cache mode.
  • state (normal, degraded, initializing, migrating, rebuilding, checking) — state of array.

7.5.1. list

Displays a list of logical drives.

Input
none
Output

info on one logical drive per line, tab-separated:

Logical drive ID RAID level Physical discs included, comma-separated Capacity (MB) State
1 0 0:1 80000 normal
2 5 0:2,0:3,0:4 10000 degraded
3 passthrough 0:7 73400 normal

7.5.2. add

Creates one or more new logical drives.

Input
  • RAID level (as in raidlevel property), for example 5
  • (optional) Physical discs to include, comma-separated, for example 0:2,0:3,0:4. By default, use all available physical discs. Various RAID levels place limits on what arrays can be created with a given set of discs. Obviously, passthrough can be created only on one physical disc.
  • (optional) Size of logical discs to create, comma-separated, for example 200,3500. By default, create one logical disc with maximal size. Not applicable while creating passthrough disc.
  • (optional) Properties, comma-separated pairs of name=value, for example stripe=64,cache=readahead.
Output
none normally, error message if error happened.

7.5.3. delete

Deletes a logical drive.

Input
logical drive ID.
Output
none normally, error message if error happened.

7.5.4. clear

Deletes all logical drives. Leaves array of free physical discs, ready for creation of new array.

Input
none.
Output
none normally, error message if error happened.

7.5.5. get

Input
  • logical drive ID
  • name of property
Output
value of property

7.5.6. set

Input
  • logical drive ID
  • name of property
  • value of property
Output
none normally, error message if error happened.

7.6. task

Tasks are processes that are executed in background on an adapter, such as building or checking the logical disc.

7.6.1. list

Displays a list of tasks.

Input
none.
Output

info on one task per line, tab-separated:

Task ID Where What Progress
112 0 Bld/Vfy 0.0%
113 1:1 Checking 24.0%

7.6.2. wait

Pauses further execution until all tasks are finished (it does so by waiting while regularly polling the task list for changes).

Input
none
Output
none

7.7. firmware

A special object that deals with adapter-related firmware reading/writing operations.

7.7.1. read

Extracts a firmware from the adapter into a file.

Input
filename (would be overwritten)
Output
none normally, error message if error happened.

7.7.2. write

Flashes a firmware from a file into the adapter.

Input
filename
Output
none normally, error message if error happened.

8. Planned features

8.1. More devices

It’s possible to implement support for the following storage devices in Einarc:

  • FreeBSD’s GEOM software RAID
  • Adaptec raidutils
  • Chelsio TOE
  • Emulex dfc / lputil
  • ICP Vortex RAID Storage Console — icpcon
  • HighPoint RocketRAID hptraidconf
  • Some HP rebranded controllers using HP Array Configuration Utility CLI
  • LSI (former IBM, former Mylex) DAC960/DAC1100
  • Promise CLI
  • QLogic SANsurferCLI / qlremote

8.2. General features

Moreover, Einarc would be nice with:

  • Graphical UI
  • Web-based remote-accessible UI
  • Monitoring subsystem or plugin for Nagios