Custom Compiled Kernel with PV-GRUB on Debian & Ubuntu

Note

This guide mentions Xen architecture. However, all Linode instances now use KVM architecture. This guide may be updated or replaced in the future; or it may be retained for historical purposes.

Running a custom-compiled Linux kernel is useful if you need to enable or disable certain kernel features that are unavailable in Linode-supplied or distribution-supplied kernels. For example, some users desire SELinux support, which is not enabled in stock Linode kernels, and may not be enabled in some distribution-supplied kernels.

If you’d rather run a distribution-supplied kernel instead, please follow our guide for Running a Distribution-Supplied Kernel.

Prior to these instructions, follow the steps outlined in our Setting Up and Securing a Compute Instance guide. Then, make sure you are logged into your Linode as the root user.

Prepare the System

Update your package repositories and installed packages, install the development tools required for compiling a kernel, and install the ncurses library.

apt-get update
apt-get upgrade
apt-get install -y build-essential libncurses5-dev gcc

If this is the first time compiling a kernel on the Linode, issue the following command to remove any existing files in the /boot directory. This helps avoid confusion later, as certain distributions install a pre-compiled kernel package along with their development packages.

rm -rf /boot/*

Compile and Install the Kernel

Download Kernel Sources

1. Download the latest 3.x kernel sources from kernel.org. A conventional location to download to is /usr/src/.

   wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.19.3.tar.xz
   

2. Expand the archived file and change directories:

   tar -xvf linux-3.19.3.tar.xz
   cd linux-3.19.3
   

Default Kernel Configuration

The kernel must be properly configured to run under the Linode environment. Some required configuration options may include:

• CONFIG_PARAVIRT=y
• CONFIG_PARAVIRT_GUEST=y
• CONFIG_PARAVIRT_CLOCK=y
• CONFIG_XEN=y
• CONFIG_XEN_BLKDEV_FRONTEND=y
• CONFIG_XEN_NETDEV_FRONTEND=y
• CONFIG_XEN_SCRUB_PAGES=y
• CONFIG_HVC_XEN=y

It’s recommended to start with a kernel config from a running Linode kernel. All Linode kernels expose their configuration via /proc/config.gz. For example:

zcat /proc/config.gz > .config
make oldconfig

make oldconfig prompts the user to answer any new configuration options not present in the old configuration file.

Changes to the kernel’s configuration can be made with the menuconfig command. Enable any additional options, making sure to leave filesystem support (likely ext3 or ext4) compiled into the kernel (not configured as a module). For example, to enable SELinux support, check the option “Security options –> NSA SELinux Support” in the configuration interface:

make menuconfig

Once your configuration options are set, exit the configuration interface and answer “y” when asked whether you would like to save the new kernel configuration.

Build the Kernel

1. Compile and install the kernel and modules:

   make -j3 bzImage
   make -j3 modules
   make
   make install
   make modules_install
   

2. PV-GRUB looks for menu.lst in the directory /boot/grub. Create this directory with the following command:

   mkdir /boot/grub
   

3. Create a menu.lst file with the following contents. Adjust the “title” and “kernel” lines to reflect the actual filenames found in the /boot directory.

   File: /boot/grub/menu.lst

   1 2 3 4 5 6

   timeout 5 title Custom Compiled, kernel 3.19.3-custom root (hd0) kernel /boot/vmlinuz-3.19.3 root=/dev/xvda ro quiet initrd /boot/initrd.img-3.19.3

Configure for PV-GRUB

In the Linode Manager, edit your Linode’s configuration profile to use pv-grub-x86_64 as the “Kernel”. Make sure the root device is specified as xvda. Save the changes by clicking Save Profile at the bottom of the page, and reboot your Linode from the “Dashboard” tab.

Once the Linode has rebooted, log back into it and issue the command uname -a. You should see output similar to the following, indicating you’re running the custom kernel:

Linux li175-165 3.19.3 #1 SMP Fri Apr 3 11:50:04 EDT 2015 x86_64 GNU/Linux

Note that if you install an updated kernel, you need to add an entry for it to your menu.lst file. By default, the first kernel in the list is booted. If you have multiple kernels installed, you can choose which kernel your Linode uses to boot by watching for the kernel list in the Lish console (see the “Console” tab in the Linode Manager). Congratulations, you’ve booted your Linode using a custom-compiled kernel!