Getting to grips with Xen paravirtualization

Updated: October 3, 2011

Let me warn you before we start; if you try using Xen without hardware virtualization, your journey will be rift with problems. This is a not so subtle hint that you should not try Xen if you don't have a modern CPU with virtualization extensions. A far cry from home solutions like VirtualBox or perhaps VMware Player, but then, Xen aims at a completely different market segment.

But say you're stubborn like me and want to conquer the world with tutorials. Then you will try Xen paravirtualization and fail. The biggest obstacle is getting your Xen guests to boot from CDs and CD images, so you can start the installation. And paravirtualized networking is no joy either. Well, today, I'll try to make things simpler. The emphasis is on try.

We will learn what problems we face, what the symptoms are, how to get around them, if at all, how to work with paravirtualized kernel and initrd files, and how to configure bridged network adapters for our guests. All of these and more, wrapped in uncertainty. Follow me.


Why paravirtualization can be a pain

Here's the official explanation:

Paravirtualization is an efficient and lightweight virtualization technique introduced by Xen, later adopted also by other virtualization solutions. Paravirtualization doesn't require virtualization extensions from the host CPU. However paravirtualized guests require special kernel that is ported to run natively on Xen, so the guests are aware of the hypervisor and can run efficiently without emulation or virtual emulated hardware.

The emphasis is on words guests require special kernel. If you don't know what this is or how to get one, you will need to work hard to get things going. Most notably, you'll see problems when trying to boot from live CD image, which ought to be the first step in a system installation.

Boot from CD or ISO image

The problem will begin when you try to setup your fresh new virtual machine. Like we did a million times before, we want to have our guest boot from CD first.

Create CD


But then you get this kind of ugly error, even if you try using an ISO:

New configuration


If you hit the Intertubes, you will find seven million recipes telling you how this can or can't be done. But it's never a one-liner, never plug and play. Most common solutions vary between extracting the image and uploading it to an HTTP server and mounting the image as a loopback device.

P.S. Please take a look at the More reading section, too. The reference material explains why Ubuntu, which we have chosen for the testing above, won't really boot as a paravirtualized guest without extra hard work and modifications.

So what should you do?

First of all, nothing. Few of the suggested methods are easily implemented. And they don't quite work really. What you need to do is examine the error message. Click on Details and hope for the best. The description might be useful.

Without elaborating on all possible cases, the main problems with paravirtualization narrow down to two core issues: 1) you need the right kernel and initrd to boot the guest system 2) paravirtualized network interfaces are bad. And the combination of these two will prevent you from booting your virtual machines from CD/DVD.

Kernel & initrd image

With full virtualization, it's quite simple. In the virtual machine configuration file, you need to specify the builder, kernel and boot directives properly. The hvmloader kernel is always available, so you're all good there.


There's no such kernel or initrd for paravirtualized guests. So you will need to improvise a little. When you try launching your virtual machine after creating it with the GUI wizard, you'll get the CD-ROM error. But the configuration file will be written to the disk.

Go to /etc/xen/vm directory and open the file in a text editor. Notice the kernel and initrd lines, pointing to temporary kernel and initrd images under /tmp. You can change these two entries to point to permanent Xen kernel and initrd files, but this can be cumbersome, as you may need to download these separately or build them on your own. You may also want to check the contents of the CD image and look for xen-related content, which you can then copy and use to boot the guest. Here's a sample:


You can also use the bootloader directive with the proper arguments, but again, this is an exhausting process, as every virtual machine will require a different subset. However, for most Linux guests, the following ought to work, with a big, implied maybe:

bootloader = '/usr/lib/xen/boot/'
bootentry = '--entry-=xvda1:/boot/vmlinuz-xen,/boot/initrd-xen'

You will of course need to adjust the virtual device from xvda1 to whatever you're using in your setup. This is an ugly hack and will not necessarily work.

Paravirtualized network interface

Another obstacle is the vif directive. Xen paravirtualized guests may not like your setup. The reason is, you cannot directly use your network adapters without bridging them. You won't be able to use devices like eth0 or wlan0, you will have to use xenbr0 or br0 and similar. If you try to start a virtual machine from the command line, the error message will be loud and clear:

Network error

As a temporary workaround, delete or comment the directive and your virtual machine should boot. Again, unlike most of my tutorials, which use a positive, definite yes, here we are dealing with frustrating maybes. Replace generic vif with any kind of directive you may have in your configuration file.


And then, the magic console:

Creating virtual machine

Booting Scientific

Scientific running

And you'll hit the network snag sooner or later:

Configure network

In general, you may get away with no network, But this is hardly useful, as we want to have a connection for our guests. Therefore, you will need to bridge your adapters. Some of your network interfaces might not support bridging, so take that into account.

Bridge, new

Bridge details

Again, this may or may not help. There's virtually no guarantee. For example, you may get to be unlucky and see this kind of message telling you: you should quit now. Or you can try manually creating a bridge using the brctl utility.

No bridge

What do we have here?

This guide is a little confusing, so let's see what we have so far. You have two core issues: one, the bootloader slash kernel for your guest; two, the network interface. Running in paravirtualized mode, Xen requires the right kernel and initrd files for your guests to be located somewhere on the disk. It also requires bridged network adapters.

You can solve the first by letting the virtual machine wizard create temporary entries for you on the hard disk, but this works only for some types of guest operating systems. For instance, Scientific Linux 6, which is based on RedHat 6, explicitly supports this kind of thing, so you're lucky. See below for more details. Alternatively, you can manually place the proper Xen kernel and initrd somewhere on your hard disk. As a last option, you can also use the bootloader directly, again with the right files.

The second issue takes an extra step; you must configure a network bridge. You can bridge both wired and Wireless adapters, however some hardware may not support this.

Even so, things may not work. And they probably won't. Bottom line, Xen paravirtualization is a pain; best avoided. If you compare the difficulty in getting your virtual machines running this way when pitted against pretty much any alternative, the answer is quite obvious.

More reading

If you're still brave after this guide, then you may want to consider reading a few more articles presenting possible solutions to booting your guests from CD-ROM. Or rather, showing you why it doesn't work and why you shouldn't bother.

Xen domU support - Popular choices like Ubuntu and Fedora are a no-go, yet

How to boot SLES para-virtual domU as a rescue system


Working with Xen in paravirtualized mode is not a pleasant task. Depending on how advanced your guest systems are in supporting para-domU, you may get lucky and boot from a CD or an ISO image. But you will also require bridged networking, which may not always work, especially with Wireless adapters.

Moreover, you may need to hunt for Xen kernel and initrd files for your guest system, including even compiling your own custom version. This kind of work is too much for most people, both in terms of skill, patience and administrative overhead, especially since other virtualization solutions offer as much without breaking into sweat.

Hopefully, this guide clears the fog a little. Web server, loopback device, physical drive, it makes no difference. You need special kernel and initrd images and bridged networking. You may be lucky and have guests that support domU paravirtualization. If not, you're in for a lot of trouble. My warmest advice is, avoid if you can, go for alternative solutions. However, in its full hardware virtualization mode, Xen is rather cool.


You may also like: