Soon after I started to work on my bachelors thesis, I had to figure out a practical and portable way to run a Xen hypervisor on which I could test my development builds. Initially I just created a simple Ubuntu virtual machine on my Desktop PC and installed the latest available version of Xen by hand. Unfortunately at the time Ubuntu only shipped Xen until version, 4.9 but I needed access to features that were only available in versions 4.10 and later.

So I switched to Alpine Linux which has a very nice version with the latest Xen already installed and set up to work as dom0. This served me well for some time but things quickly became messy once I started to also develop on my Notebook. Keeping manually set up VMs in sync across multiple machines turned out to be very difficult and annoying. I had some prior experience with Vagrant, that’s why I decided to create a Vagrant Box which includes an already configured installation of Xen.

Vagrant is an open source tool for building and managing virtual machine environments based on VirtualBox, VMWare, KVM and many more. It is developed by HashiCorp and was first released in 2010.

With the help of Vagrant, it is possible to create the same exact virtual machine containing the Xen hypervisor on different computers. Every branch of my HermitCore fork contains a vagrant directory, in which the configuration for the virtual machine is stored. Using the Vagrantfile inside this directory, it is possible to create a virtual machine based on the latest Ubuntu version 18.04. Xen and all it’s dependencies get automatically installed on the creation of the machine.

The Vagrantfile for my setup looks like this:

Vagrant.configure("2") do |config|
  config.vm.box = "generic/ubuntu1804"
  config.vm.hostname = "xen"
  config.vm.define "xen"
  config.vm.network :private_network, :ip => "192.168.50.11"
  config.vm.synced_folder '../', '/HermitCore', type: "rsync", rsync__exclude: ".git/"
  config.vm.provider :libvirt do |libvirt|
    libvirt.cpus = 2
    libvirt.cpu_mode = "host-model"
    libvirt.nested = true
    libvirt.memory = 2048
    libvirt.keymap = "de"
  end
  config.vm.provider "virtualbox" do |vb|
    vb.memory = "2048"
    vb.cpus = "2"
    vb.customize ["modifyvm", :id, "--paravirtprovider", "kvm"]
    vb.customize ["modifyvm", :id, "--hwvirtex", "on"]
    vb.customize ["modifyvm", :id, "--nestedpaging", "on"]
    vb.customize ["modifyvm", :id, "--cpu-profile", "host"]
    vb.customize ["modifyvm", :id, "--uart1", "0x3F8", "4"]
    vb.customize ["modifyvm", :id, "--uartmode1", "file",  "./xen.log"]
  end
  config.vm.provision "shell", inline: <<-SHELL
     export DEBIAN_FRONTEND=noninteractive
     apt-get update
     apt-get upgrade -qy
     apt-get install libyajl2 qemu-system-x86 xorriso bridge-utils -qy
     apt-get install /HermitCore/vagrant/files/xen-upstream-4.11.0.deb -qy
     ldconfig
     mv /etc/grub.d/10_linux /etc/grub.d/50_linux
     mv /HermitCore/vagrant/files/grub /etc/default/grub
     mv /HermitCore/vagrant/files/50-vagrant.yaml /etc/netplan/50-vagrant.yaml
     update-grub2
     systemctl enable xen-qemu-dom0-disk-backend.service
     systemctl enable xen-init-dom0.service
     systemctl enable xenconsoled.service
     systemctl enable xendomains.service
     systemctl enable xen-watchdog.service
     echo "Reboot VM!"
     reboot
  SHELL
end

You can choose to either start a VM via VirtualBox or libvirt. Sadly VirtualBox has no support for nested virtualization, that’s why I’d recommend to use libvirt as the default virtualization provider. To use Vagrant with libvirt, you first have to install the necessary plugin:

vagrant plugin install vagrant-libvirt

Afterwards you can start the VM with:

vagrant up --provider=libvirt

This will spin up a Vagrant Box with 2 GiB of memory and two virtual CPUs. After Vagrant created the VM, the whole parent folder containing the Git repository including the built files of HermitCore will be copied into the virtual machine and are available inside the /HermitCore directory. It will also run the inline shell script that is included in the Vagrantfile. It installs a custom built version of Xen with debug output enabled and reboots the VM. Afterwards everything is set up and ready to go.

One very nice thing Vagrant brings with it, is to automatically rsync folders into the running Vagrant Box. So when I made some changes to HermitCore and built them, I only had to run

vagrant rsync

from within the vagrant folder and I could start the latest build inside the VM without much effort.

This setup proved to be very convenient to use, even across multiple systems. I just had to clone the repo, start the Vagrant Box and could dive right into developing. I hope you found this post interesting and will also consider to use Vagrant to provide you with a portable development environment for your projects

So long, Jan