Getting Start with tlc-kvm
Version 0.1, 5/25/13
By Kasidit Chanchio
Copyright 2013 Vasabilab. Kasidit Chanchio
Vasabilab, Department of Computer Science, Faculty of Science & Technology, Thammasat University.
Preface: This Getting Start document intends to demonstrate the easiest method to migrate a VM using the Thread-Based Live Migration (TLM) mechanism. In tlc-kvm software, users have to define a set of TLM migration parameters prior to migration. In this document, however, we want to make tlc-kvm easy to use for users who are already familiar with qemu live migration. In doing so, we have made tlc-kvm accept the traditional qemu live migration command and perform TLM live migration without having to define any TLM parameter. This document describes how to install tlc-kvm on your Ubuntu 12.04 systems and use the traditional qemu monitor commands to perform TLM migration.
Table of Contents
2. Run tlc-kvm
3. Migrate a VM
To install tlc-kvm, you have to download the tlc-kvm debian package from our website (http://vasabilab.cs.tu.ac.th/projects/TLM.html). Be sure to use a package version that match the flavor of your ubuntu OS. At this time, we have tested our packages on ubuntu 12.04 desktop and ubuntu 12.04 server. After download, you have to do the following to install the software. Note that it is preferable that your ubuntu OS does not have qemu or kvm package installed previously. In case you have installed qemu or kvm earlier, you need to consult the trouble shooting section below to either install tlc-kvm with the “--force-all” option or remove those packages before installing tlc-kvm.
On ubuntu 12.04 desktop:
$ sudo apt-get install libsdl1.2debian
$ sudo dpkg –i tlc-kvm_0.1.0-0ubuntu1_amd64.deb
On ubuntu 12.04 server:
$ sudo dpkg –i tlc-kvm-server_0.1.1-0ubuntu1_amd64.deb
On ubuntu 13.04, we found some issues with the kvm kernel module and are working on a new tlc-kvm package that is compatible with Ubuntu 13.04 kernel.
To verify the installation, you can use dpkg-query utility below.
$ dpkg-query –L tlc-kvm | more
$ ls –l /usr/bin/qemu-*
You should also see qemu-ga, qemu-io, qemu-img, qemu-nbd, qemu-system-x86_64 executable files installed in the /usr/bin directory. Now you are ready to run tlc-kvm. The tlc-kvm package must be installed on the source and destination computers of the migration. Before running tlc-kvm, issue the following commands to load kvm driver to the kernel on an intel or amd machine, respectively.
$ sudo modprobe kvm-intel
$ sudo modprobe kvm-amd
To remove tlc-kvm, you may use the following command.
$ sudo dpkg –r tlc-kvm
1. Before installation, you may want to check if some packages have already installed the above executable files in the /usr/bin directory. If so, the dpkg command may not install tlc-kvm and report errors like “error overwriting /usr/bin/qemu-*”. To fix the problems, you may use one of the following solutions.
· You can use dpkg with “--force-all” option to force dpkg installation to replace the existing files and status information. For instance,
$ sudo dpkg --force-all –i tlc-kvm-server_0.1.1-0ubuntu1_amd64.deb
In case you want to install the package on a different directory tree, you can add “--instdir=dir” or “--root=dir” options to the above command. For example, the following command would install tlc-kvm files in the /usr/local/software directory
$ sudo dpkg --force-all --instdir=/usr/local/software –i tlc-kvm-server_0.1.1-0ubuntu1_amd64.deb
· Alternatively, you may remove previously install qemu and kvm packages using
$ sudo apt-get remove qemu*; sudo apt-get remove kvm*
before installing a tlc-kvm debian package.
2.1 Run a VM on the source computer
To run tlc-kvm, you can open a terminal and issue the traditional qemu-kvm command. Since tlc-kvm is a modified version of qemu-kvm-1.0.0 [1, 2], all qemu monitor commands are supported (with some changes to those related to VM migration). In order to use tlc-kvm, you have to make sure that port 12300 on the source and destination computers is not used by any other program.
First, let’s run a VM on a source computer. In this example, we assume that an image file Ubuntu1004.img has already been created in a shared file system such as NFS that is accessible from the source and destination computers. We suppose the image Ubuntu1004.img already has Ubuntu 10.04 OS and application programs installed, and the IP addresses of the source and destination computers are 192.168.1.12 and 192.168.1.13, respectively. It is worth noting that users should choose appropriate network bandwidths for the VM workload they want to migrate. From our experiences, the migration of a VM with large benchmark programs such as NPB Kernel MG Class D, IS Class D, and SP Class D that has working set sizes of 27.3 GB, 34.1 GB, and 12.1 GB, respectively, would perform best with a 10 Gbps network or higher. Migrating benchmark programs with lower memory usage requirements would perform just fine with a 1 Gigabit network.
$ /bin/qemu-system-x86_64 –smp 2 –m 2048 –hda Ubuntu1004.img –net nic –net user
Since we run the above command on an ubuntu desktop OS, you should see a qemu GUI console on the screen as shown in Figure 1.
Figure 1. Launched an Ubuntu VM
Supposed that the NAS Parallel Benchmark programs have been installed as illustrated in Figure 2. In the Figure, we open the system monitor utility and a new command-line terminal to view a list of benchmark programs.
Figure 2. The VM runs the system monitor and lists NPB benchmark programs installed on the VM.
2.2 Prepare a VM on the destination computer
Before performing VM migration, you have to prepare the followings. First, you have to install tlc-kvm on a destination computer as explained in Section 1. Next, you have to run the following command.
$ /bin/qemu-system-x86_64 –smp 2 –m 2048 –hda Ubuntu1004.img –net nic –net user –incoming tcp:192.168.1.13:9007
Note that 192.168.1.13 is the IP address of the destination computer. This command will pop up a window with dark background as shown in Figure 3. The window is in fact the console of the qemu-system-x86_64 hypervisor that is waiting for state transfer from a migration source.
Figure 3. A black window waiting for migration information. (The window on the right shows the command that launched the black window).
This section show the easiest way for users to perform TLM live migration using the same command as in the traditional kvm software using “tcp” protocol. In the display terminal you have already opened inside the VM on the source computer in Figure 2, issue a command to invoke the ft.C.x (OpenMP) NPB benchmark (see Figure 4 below).
Figure 4. Running a benchmark program.
To migrate the VM, you have to press Ctrl-Alt-2 to switch qemu-kvm console to qemu-monitor console and enter the following command. Note that tlc-kvm does not support the “migrate –d” option at the moment.
(qemu) migrate tcp:192.168.1.13:9007
The VM will perform a migration and report debugging information on the terminal on the source computer that you invoke the qemu-system-x86-64 command. I will explain their meanings next time.
After finished, you should see the black blank window in Figure 3 turns to a window full of stuffs in Figure 5.
Figure 5. Destination VM activates and continues computation.
1. TLM Part 1: tlc-kvm Installation (http://www.youtube.com/watch?v=MnJi0k_uFLk)
2. TLM Part 2: VM Migration (http://www.youtube.com/watch?v=AjOX5Yy_3vA)
3. TLM migration of NAS Parallel Benchmark kernel MG Class D, IS Class D, SP Class D, and BT Class D