XEN on 32 Bit with 8 GB Ram Odyssey

Last week I've ordered a new shiny root server at hetzner. A nice 8gb Server begging me to install XEN on its virgin 750GB Raid HD. Hell yeah, but wait, I need to run 32 Bit XEN-Guests on that one. After a quick glance at the current Debian XEN packages I decided not to install the 64 bit variant of Debian as the Domain-0. Unfortunately Debian is still stuck at 3.0.x which doesn't allow 32-on-64 setups (which I have found no documentation for anyway).

Ok, 32 bit it is. To my very disappointment I had to discover the lack of PAE extensions in the XEN-Kernel. But why-o-why is the hypervisor package flagged with PAE extensions but not the kernel? So, what now? One option has been to go with a XEN Setup from the sources. For obvious reasons I'd rather like to have a distribution run solely on official stable packages.

I’ve always wanted to take a look at Debian's younger brother - Ubuntu. Thank god Hetzner did offer easy and convenient image-base setup for various Linux distributions, including Ubuntu 7.10. Setup and first XEN steps went like a breeze. Soon I have found myself running XEN on 32 Bit with full PAE support being able to address all of my tiny little ram bits. YAY!

Time to migrate some XEN-U guests to the new machine. Fortunately the old server was also located at the same collocation center, so the copy process did not take too long. The fun did not last any longer though. The very first boot of the guest OS resulted in a frozen console. No login. WTF!? After fiddling with various Kernel options and googling myself into madness I have found some very interesting facts.

To get a login console using the XEN console you need to add extra='XENcons=tty1'to your XEN guest-config. Getting ssh working was a whole different issue. Trying to ssh into the guest OS you will get stuck with an stdin: is not a tty error message on the screen. Frozen. Again. One proposed solution to this issue was to use the udev filesystem on the guest. The others did not work for me at all (using xvc0 instead of tty1, removing event.d/tty*).

So, after installing udev I had to notice that my network device is gone. Great! Theoretically I *could* ssh into the box now. It seems udev re-recognizes the NIC device each time the machine gets a new random NIC assigned. XEN will do such things to virtual network devices. This will result in plenty dead NIC devices on the guest, with the very last (for instance eth9) active. The solution to this issue was rather simple; remove /etc/udev/rules.d/z45_persistent-net-generator.rules and you are ready to go.

Finally!

Sources: