<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi,<br>
<br>
As part of my recent unintentional multi-week experiment getting
some BlueOnyx servers set up, I ran into an esoteric bug or two.<br>
<br>
I've been imaging servers by using RAID 1 to duplicate hard drives
across machines, starting from one, and then creating another and
another as necessary. I have four Sun Fire X2200 servers. They all
have four ethernet ports in the back: two are controlled by an
nVidia chipset, and two by a Broadcom chipset. They are labeled on
the physical case as eth0, eth1, eth2 and eth3.<br>
<br>
The way the ports get labeled in the operating system, using the
same terms (eth0 through eth3) is pretty haphazard, meaning that the
OS labels definitely do not correspond to the physical port labels.
Suddenly eth3 means eth0. It's very confusing. I doubt there's much
that can be done about this unless the hardware itself provides
metadata about which port is which, and I don't know if it does.<br>
<br>
At first I thought that this was just a minor annoyance, but it
turns out that it matters when combined with the RAID 1 imaging
method I'd been using, because when you take an imaged drive from
another CentOS server and plug it into a new chassis (with new
ethernet adapters that each has a new MAC address), what happens is
not quite what you'd expect: rather than replace the four old MAC
addresses with four new MAC addresses, udev just concatenates them
in /etc/udev/rules.d/70-persistent-net.rules. So you end up with
eth0 with eth7, none of which correspond to anything in reality.
This makes it really hard to get an internet connection up and
running, by the way.<br>
<br>
It gets better. When you go to change your IP address in the
BlueOnyx control panel, you are presented with four sets of fields
corresponding to your four ethernet adapters. But you can't save.
Why can't you save? Because there's a JavaScript error. Why is there
a JavaScript error? Because on line 101 of
/usr/sausalito/ui/web/base/network/ethernet.php, the PHP variable
$interfaces is getting its data from $cceClient->findx('Network',
array('real' => 1), array(), 'ascii', 'device');, and that's
reporting back that there are in fact eight ethernet adapters. There
aren't, but as soon as JavaScript hits $interfaces[4], it can't find
the HTML inputs because they don't exist. (This makes it REALLY hard
to get an internet connection up and running, by the way.)<br>
<br>
At the root level the problem is with udev being dumb, but having
two different ways of determining the number of ethernet adapters
(there are only four input blocks, which is correct, versus the
eight JavaScript array elements) does not help.<br>
<br>
Another fun fact about using RAID 1 to clone a clean BlueOnyx
install image: it kills your swap partition. If you ever want that
back instead of a lame md127 array that comes out of nowhere because
the mdadm-style (versus the other style, who thought of this
insanity and why are there two styles?) UUID of the array doesn't
match the contents of /etc/mdadm.conf, you'll have to do the
following:<br>
<br>
mdadm --manage --stop /dev/md127<br>
mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sda3
/dev/sdb3<br>
mdadm --misc --detail /dev/md2 (tells you the UUID)<br>
vim /etc/mdadm.conf (paste the UUID in the proper place)<br>
mkswap /dev/md2<br>
swapon /dev/md2<br>
<br>
I'm using /dev/md2 here for the array and /dev/sda and /dev/sdb for
my drives, but fill in your own device handlers as necessary.<br>
<br>
Aaron<br>
<br>
<div class="moz-signature">
<table border="0" cellpadding="0" cellspacing="0" width="100%">
<tbody>
<tr>
<td colspan="4">
<hr style="color: #404040; background-color: #404040;
height: 1px; border: #404040 solid 0px;"></td>
</tr>
<tr>
<td style="padding-left: 8px; padding-top: 8px;"
valign="top"><a href="http://www.thinkcomputer.com"><img
src="cid:part1.07090507.00070702@thinkcomputer.com"
alt="Think Computer Corporation" align="bottom"
border="0" height="96" width="66"></a></td>
<td style="padding-right: 8px;" align="right" valign="top"
width="380">
<font color="#606060" face="Helvetica,Arial" size="-1">
<p>
<b>Aaron Greenspan</b><br>
President & CEO<br>
Think Computer Corporation</p>
<font style="text-decoration: none;" color="#808080"
face="Tahoma,Helvetica,Arial" size="-2">
<p>
<i>telephone</i> +1 415 670 9350<br>
<i>toll free</i> +1 888 815 8599<br>
<i>fax</i> +1 415 373 3959<br>
<i>e-mail</i> <a class="moz-txt-link-abbreviated" href="mailto:aarong@thinkcomputer.com">aarong@thinkcomputer.com</a><br>
<i>web</i> <a class="moz-txt-link-freetext" href="http://www.thinkcomputer.com">http://www.thinkcomputer.com</a></p>
</font>
</font>
</td>
</tr>
</tbody>
</table>
</div>
</body>
</html>