<!> Note: This is a development release, only meant for testing purposes. We do not recommend anyone deploy production systems using the content mentioned here.

Overview

Standard CentOS-6 does not have the capabilities to run Xen as a virtualisation host due to upstream vendor not shipping Xen in EL6. We are undertaking an effort to try and fix that, with the aim of providing people using Xen on CentOS-5 an upgrade path and also to provide people with an option should they not find KVM to be a suiteable Virtualisation technology in their workloads.

At this time, our efforts are to try and use as much of the content already in the distro, but also to bring in upstream traction from the xen.org project as well as use the new linux kernel content in the Linux-3.4 tree, which is the kernel.org longterm supported stable kernel. Hence the code in this repo is based on :

Its important that you take on board the fact that when you are done with this setup, you will no longer be running the standard CentOS linux kernel. The kernel we provide has not been processed by our upstream vendor and has not been through their QA process, however we are still confident that it should generally work for most people. Note that even when the 3.4.x kernel seems more recent than the 2.6.32 kernel in EL6, it might still have older versions of some drivers, resulting in different behaviour between the stock el6 kernel and the 3.4 kernel used for dom0, so you might need to update certain device drivers manually.

Note that this is only targetting CentOS-6/x86_64. We are not attemping a Xen-4 build for CentOS-6/i386 or for CentOS-5 at this point.

Known limitations :

Known problems :

Install Process

A repository with all the required code is available at : http://dev.centos.org/centos/6/xen-c6/ . To set this up on your machine :

cd /etc/yum.repos.d/
wget http://dev.centos.org/centos/6/xen-c6/xen-c6.repo
yum repolist

Firstly, we need to install the newer kernel. To achieve that :

yum install kernel kernel-firmware

Ensure that the 3.4.x kernel and the corresponding kernel-firmware rpms are installed. And check that /etc/grub.conf is setup to boot this new kernel as default.

Second step is to install Xen :

yum install xen

Once that completes, check /boot/ for the presence of xen-4.2.1.gz and edit your /etc/grub.conf to add a stanza like :

title xen
        root (hd0,0)
        kernel /xen.gz dom0_mem=256M,max:256M loglvl=all guest_loglvl=all
        module /vmlinuz-3.4.24-4.el6.centos.alt.x86_64 ro root=/dev/mapper/vg_cs-lv_root rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=uk LANG=en_US.UTF-8 rd_LVM_LV=vg_cs/lv_root rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto rd_NO_DM rd_LVM_LV=vg_cs/lv_swap
        module /initramfs-3.4.24-4.el6.centos.alt.x86_64.img

Ensure that the path to /xen, /vmlinux and /initramfs are correct ( based on the other entries already in /etc/grub.conf ). Also consider tweaking the 256M dom0_mem parameter to match your site requirements. In most cases, a minimal install CentOS-6, with Xen-4 will be fine with 256 Mbytes of RAM for dom0 (see Bug 6893, so dom0_mem=256M,max:512M would get rid of the error message in that bug).

You should now be able to reboot the machine. When the machine comes back, check that the new kernel is running and xen is active by executing :

uname -a ; xl info 

Setting up serial console

title xen with serial console
        root (hd0,0)
        kernel /xen.gz dom0_mem=1024M,max:1024M loglvl=all guest_loglvl=all com1=115200,8n1 console=com1
        module /vmlinuz-3.4.24-4.el6.centos.alt.x86_64 earlyprintk=xen console=hvc0 ro root=/dev/mapper/vg_cs-lv_root rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=uk LANG=en_US.UTF-8 rd_LVM_LV=vg_cs/lv_root rd_NO_MD SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto rd_NO_DM rd_LVM_LV=vg_cs/lv_swap
        module /initramfs-3.4.24-4.el6.centos.alt.x86_64.img

We're setting up Xen hypervisor (xen.gz) to use and log to the serial port, and dom0 Linux kernel (vmlinuz) is configured to log to the serial console through Xen hypervisor (console=hvc0), so both the Xen hypervisor and dom0 Linux kernel console messages appear on the serial console. For more information about setting up Xen serial console with non-standard serial ports etc: http://wiki.xen.org/wiki/Xen_Serial_Console .

If you're experiencing trouble booting up the system please set up the serial console and capture the full boot logs for analysis. Most servers offer serial port in their management processor (iLO/iDRAC/IPMI/BMC), which you can actually use through the network (SOL = Serial Over LAN).

Note: see Bug 6893, so dom0_mem=1024M,max:1536M would get rid of the error message in the bug.

Installing a VM

At this point the libvirt tools do not work with the Xen-4.2 stack, so the distro virt tools might not work. The best way to install a new VM is to bootstrap using the pxe images. This process will work for both paravirt and full / hypervirt environments.

For CentOS-6 these images are located at http://mirror.centos.org/centos/6/os/x86_64/images/pxeboot/ and http://mirror.centos.org/centos/6/os/i686/images/pxeboot/ for 64bit and 32bit respectively. For CentOS-5 these images are located at http://mirror.centos.org/centos/5/os/x86_64/images/xen/ and http://mirror.centos.org/centos/5/os/i386/images/xen/ for 64bit and 32 bit respectively.

  1. Download the pxe images for the distro and arch you require
  2. Do any network setup needed ( you will need a bridge setup, refer to the example files in /etc/xen/ for more info )
  3. Setup storage you are going to use for the VM. eg, if using lvm and the VM is going to have a lv backing store : lvcreate -L 20G -n lv_dom1 /dev/vg_storage
  4. Create a basic config file that looks something like this
    name = "domain-1"
    memory = "768"
    kernel = "/tmp/vmlinuz"
    ramdisk = "/tmp/initrd.img"
    disk = [ 'tap2:aio:/var/lib/xen/images/disk.0,xvda,w', ]
    vif = [ 'bridge=virbr0', ]
    vcpus=1
    on_reboot = 'destroy'
    on_crash = 'destroy'
    

Note that the kernel and ramdisk lines point to the pxe boot images downloaded in step 2; if you have a kickstart file or want to send some specific parameters to the installer, you can use the 'extra' keyword in there as well.

You should now be able to 'xl create -c <config file>' and start the install process.

Once the install is done, you need to change the kernel line, remove the ramdisk link and add another extra line in the config file to fall back to the pv-grub process. These lines would now look something like this :

kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0,0)/grub/menu.lst"

At this point you should be able to start the VM / DomU with 'xl create <configfile>' and check its running.

QaWiki/Xen4 (last edited 2014-01-10 13:09:56 by JohnnyHughes)