Debugging Qemu

Sometimes you must be able to analyse Qemu more in-depth than the log files allow to.

Using GDB

For this, one can attach a GDB instance to a running Qemu process as long as it is started with the -s option.

To pass the required option from libvirt down to Qemu, one can use an additional namespace to pass along options to Qemu as described in the libvirt wiki.

Simply replace at the beginning of the XML (for example when editing using virsh edit):

<domain type='kvm'>

by

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <qemu:commandline><qemu:arg value='-s'/></qemu:commandline>

and then start the VM.

The GDB session to attach to the process and continue to run it until you hit a bug:

~ # gdb
GNU gdb (Gentoo 7.6.2 p1) 7.6.2
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>.
(gdb) file /usr/bin/qemu-system-
/usr/bin/qemu-system-: No such file or directory.
(gdb) file /usr/bin/qemu-system-x86_64 
Reading symbols from /usr/bin/qemu-system-x86_64...Reading symbols from /usr/lib64/debug/usr/bin/qemu-system-x86_64.debug...done.
done.
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib64/debug/lib64/ld-2.17.so.debug...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
0xfffff8003d7ab25f in _start () from /lib64/ld-linux-x86-64.so.2
(gdb) c
Continuing.

Generate core dumps

mkdir -p /var/virtualization/core-dumps
chmod 1777 /var/virtualization/core-dumps
 
echo "/var/virtualization/core-dumps/core.%e.%p.%h.%t" > /proc/sys/kernel/core_pattern
ulimit -c unlimited
 
/etc/init.d/libvirtd reload

... after that, all newly started Qemu processes will generate a (unique) core dump file in /var/virtualization/core-dumps in case they crash