This is partially a followup to Creating disk images for gem5 and partially how to setup x86 full system for gem5. In this post, I'll discuss how to create a disk image from scratch and start using it with gem5.
It is important for computer architecture research to use the most up-to-date software on the systems we are simulating. Too much computer architecture research reports results using kernels from 5+ years ago or ancient system software Hopefully, this post will help others be able to keep up with the ever-changing system software. This way, researchers can use up-to-date versions of Linux and easily update their kernels.
This post takes a different approach than Creating disk images for gem5. Instead of using the gem5 tools, this post uses qemu to create, edit, and set up the disk for gem5 usage.
This post assumes that you have installed qemu on your system. In Ubuntu, this can be done with
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils
I also assume you have downloaded and built gem5. All of the full system examples use the simple full system scripts that are covered in Learning gem5.
Step 1: Create an empty disk
Using the qemu disk tools, create a blank raw disk image. In this case, I chose to create a disk named "ubuntu-test.img" that is 8GB.
qemu-img create ubuntu-test.img 8G
Step 2: Install ubuntu with qemu
Now that we have a blank disk, we are going to use qemu to install Ubuntu on the disk. I would encourage you to use the server version of Ubuntu since gem5 does not have great support for displays. Thus, the desktop environment isn't very useful.
First, you need to download the installation CD image from the Ubuntu website.
Next, use qemu to boot off of the CD image, and set the disk in the system to be the blank disk you created above. Ubuntu needs at least 1GB of memory to install correctly, so be sure to configure qemu to use at least 1GB memory.
qemu-system-x86_64 -hda ../gem5-fs-testing/ubuntu-test.img -cdrom ubuntu-16.04.1-server-amd64.iso -m 1024 -enable-kvm -boot d
With this, you can simply follow the on-screen directions to install Ubuntu to the disk image. The only gotcha in the installation is that gem5's IDE drivers don't seem to play nicely with logical paritions. Thus, during the Ubuntu install, be sure to manually partition the disk and remove any logical partitions. You don't need any swap space on the disk anyway, unless you're doing something specifically with swap space.
Step 3: Boot up and install needed software
Once you have installed Ubuntu on the disk, quit qemu and remove the -boot d option so that you are not booting off of the CD anymore. Now, you can again boot off of the main disk image you have installed Ubuntu on.
Since we're using qemu, you should have a network connection (although ping won't work). When booting in qemu, you can just use sudo apt-get install and install any software you need on your disk.
qemu-system-x86_64 -hda ../gem5-fs-testing/ubuntu-test.img -cdrom ubuntu-16.04.1-server-amd64.iso -m 1024 -enable-kvm
Step 4: Build a kernel
Next, you need to build a Linux kernel. Unfortunately, the out-of-the-box Ubuntu kernel doesn't play well with gem5. See the error below.
First, you need to download latest kernel from kernel.org. Then, to build the kernel, you are going to want to start with a known-good config file.The config file that I'm used for kernel version 4.8.13 can be downloaded here. Then, you need to move the good config to .config and the run make oldconfig which starts the kernel configuration process with an existing config file.
At this point you can select any extra drivers you want to build into the kernel. Note: You cannot use any kernel modules unless you are planning on copying the modules onto the guest disk at the correct location. All drivers must be built into the kernel binary.
It may be possible to use modules by compiling the binary on the guest disk via qemu, but I have not tested this.
Finally, you need to build the kernel.
Step 5: Update init script
By default, gem5 expects a modified init script which loads a script off of the host to execute in the guest. To use this feature, you need to follow the steps below.
Alternatively, you can install the precompiled binaries for x86 found on my website: From qemu, you can run the following, which completes the above steps for you.
wget http://cs.wisc.edu/~powerjg/files/gem5-guest-tools-x86.tgz
tar xzvf gem5-guest-tools-x86.tgz
cd gem5-guest-tools/
sudo ./install
Now, you can use the system.readfile parameter in your Python config scripts. This file will automatically be loaded (by the gem5init script) and executed.
Manually installing the gem5 init script
First, build the m5 binary on the host.
cd util/m5
make -f Makefile.x86
Then, copy this binary to the guest and put it in /sbin. Also, create a link from /sbin/gem5.
Then, to get the init script to execute when gem5 boots, create file /lib/systemd/system/gem5.service with the following:
[Unit]
Description=gem5 init script
Documentation=http://gem5.org
After=getty.target
[Service]
Type=idle
ExecStart=/sbin/gem5init
StandardOutput=tty
StandardInput=tty-force
StandardError=tty
[Install]
WantedBy=default.target
Enable the gem5 service and disable the ttyS0 service.
systemctl enable gem5.service
Finally, create the init script that is executed by the service. In /sbin/gem5init:
#!/bin/bash -
CPU=`cat /proc/cpuinfo | grep vendor_id | head -n 1 | cut -d ' ' -f2-`
echo "Got CPU type: $CPU"
if [ "$CPU" != "M5 Simulator" ];
then
echo "Not in gem5. Not loading script"
exit 0
fi
# Try to read in the script from the host system
/sbin/m5 readfile > /tmp/script
chmod 755 /tmp/script
if [ -s /tmp/script ]
then
# If there is a script, execute the script and then exit the simulation
su root -c '/tmp/script' # gives script full privileges as root user in multi-user mode
sync
sleep 10
/sbin/m5 exit
fi
echo "No script found"
Problems and (some) solutions
Failed to early mount API filesystems
Write protecting the kernel read-only data: 8192k Freeing unused kernel memory: 1956K (ffff880001417000 - ffff880001600000) Freeing unused kernel memory: 456K (ffff88000178e000 - ffff880001800000) [!!!!!!] Failed to early mount API filesystems, freezing.
Solutions tried: Enable cgroups in the kernel. I think. Nope! I think this is the same as the problem below mount-problem.
Can't mount /dev
Failed to mount devtmpfs at /dev: No such device Freezing execution.
Something like the above (this was taken from arch linux boot). The problem is that that the right devfs is not compiled into the kernel. You need to make sure that devtmpfs is enabled.
panic: KVM: Unexpected exit (exit_reason: 8)
Exit reason 8 is "shutdown". See http://lxr.free-electrons.com/source/include/uapi/linux/kvm.h#L188. This seems to happen when there is a triple fault: http://lxr.free-electrons.com/source/arch/x86/kvm/x86.c#L6498
I get this error every time I try to boot the unmodified Ubuntu kernel. I don't know how to solve this problem. Instead of trying to solve the problem, I used a different config file for "oldconfig" when I compiled the kernel from scratch.
Slow boot
[ TIME ] Timed out waiting for device dev-di...\x2da115\x2de3f263d7b53a.device. [DEPEND] Dependency failed for /dev/disk/by-...382-f41d-4c99-a115-e3f263d7b53a. [DEPEND] Dependency failed for Swap.
This may happen if you have changed the disk without updating the fstab on the disk. To fix it, you can boot the disk in qemu and update fstab with the correct UUID.
I ran into this when I was resizing the disk.
Disk is too small for what you want to do
Resizing an iso is pretty easy. You can use the same method you would if you wanted to resize a partition on a regular hard drive.
First, you need to resize the iso with qemu-image:
qemu-img resize ubuntu-test.img +8G
Now, you have a disk that has 8 GB of free space at the end of the disk. You need to resize the partitions to use this free space. To do this, I suggest using gparted just like you would for a real hard drive.
You can download a gparted ISO from http://gparted.org/livecd.php. Once you download the ISO, you can boot it with qemu the same way as we booted the installation CD. Then, once its booted you can select the disk you want to modify and follow the howto (http://gparted.org/display-doc.php%3Fname%3Dhelp-manual).
Comments !