Updated: April 11, 2011
Linux kernel compilation is a process where you compile a kernel from sources. Now, this sounds posh and geeky. Indeed, as a home user, there are very few reasons why you would ever want to consider doing this. But let's put philosophy aside for later.
Today, I would like to show you the basics of Linux kernel compilation. What it is, how it works, what the pros and cons are, how you can manage the process in a safe and sane manner. This is by no means an easy job. In fact, kernel compilation is a serious and complex task, which can easily turn into chaos and despair. Furthermore, to get most from a kernel compilation, you will need intimate knowledge with how things works under the hood. Anyhow, I promised you a tutorial in my Linux Kernel Crash Book, so here it is.
The reasons to compile kernels are far and few in between. You may want to consider doing this if you have exotic architecture or hardware that the mainline kernel does not support, which is a rarity today. Another reason may be performance benefits. If you have a very old machine, creating a custom kernel with a minimal disk and memory footprint could be your thing. Similarly, if you need a kernel for an odd target device that has constrained resources, like a router or such, the only way to get past the limitations would be by compiling your own kernel.
Other reasons include study and exploration, glory, boasting, ego, showing everyone how your superior kernel outmatches the vanilla, and so forth.
A real example where we needed kernel compilation was when we studied Kdump. The kernel crash mechanism requires that the kernel be compiled with very specific options. If you wanted to use Kdump and your kernel did not have these, you were kind of stuck without a compilation.
But this is something home users will rarely do. This is more typical for businesses and companies. Then again, paying customers will usually have enterprise or corporate support from distribution vendors, who will take care of kernel compilation and any other problems.
As a home user, you will possibly gain some CPU speed and memory by using a custom kernel, but in most cases, the benefits will be unnoticeable. On the other hand, every single tweak introduced in the kernel configuration could lead to problems down the road. It's very similar to Windows tweaks; overdo them at your own peril.
Most users will never need more than the very flexible and generous defaults. Moreover, they will have no idea how to make changes that could make their system work faster or smarter. Kernel optimization is a delicate art best done by experts, and even then, it often includes lots of magic and experiments and somewhat empirical luck.
For instance, you may want to make your kernel smaller by removing Bluetooth support. OK, so you disable the Bluetooth module. No worries. But what do you do if one day you need Bluetooth? Are you going to compile from scratch?
Things only get worse when you start playing with dozens of options, some related to hardware, other related to filesystem and network behavior, etc. It becomes a mess, with unpredictable results and dire consequences. Worse yet, the complex interaction between individual parameters on top of user chaos and lack of knowledge can render the delightful exercise of kernel compilation into a nightmare.
You're not Linus Torvalds, give it a rest.
If you're really keen on minimalism, you can use GRUB to pass kernel commandline parameters, which enable or disable certain parameters, so that if you need them one day, you have them. All right, we know what we're up against. Let's go through the technical details.
To get kernel compiled, you will need the kernel source, the build tools, the kernel configuration file, and root or sudo privileges for the final stages of the process.
You can find all of the kernels you need on kernel.org. Whether you want the latest release candidates or very old kernels, all of the sources are there, just waiting for you. Grab the one you require and let it download to your hard disk. If you want to use a kernel that matches your own running version, you can download the sources using the package manager. For instance, using RPM-based yum on Fedora:
yum install kernel-source-`uname -r`
We did something similar in the older Cambridge version.
Now, on Ubuntu, things are very similar. You can use the package manager to get only the sources. The apt-get command below will download the source to your current directory, without installing the sources under /usr/src. You will need to have the Sources repository enabled, though. Furthermore, if you need a kernel that is newer than the currently supported version, this method won't really work. You will have to go with the traditional manual download.
apt-get source linux-image-$(uname -r)
Sources, as shown in the Software Center:
For more details about the apt-get functionality, please check this tutorial.
To compile, you will need a handful of tools, namely make and gcc. Depending on your Linux distributions, these may be available to you by default. If not, you will need to install them. I have shown you this procedure many times, including when we had to compile VMware Tools on Linux Mint. Anyhow, on any Debian-based distribution, you can install the build packages by running apt-get install build-essential.
apt-get install build-essential
You may also need other packages, so if something does not quite work, you should try installing the following; some of these may be installed and/or unnecessary.
apt-get install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge git-core libncurses5 libncurses5-dev libelf-dev asciidoc binutils-dev
Replace apt-get install with equivalent command for yum or zypper or any other package manager you may be using. Please note that crash and kexec-tools were used for crash dumping and analysis, as I've shown you in the book. Things kind of seem to connect in one big friendly circle of love and understanding, don't they.
After you download and extract the kernel archive, cd into it. Inside, there's a handful of directories and files, nothing remarkable. This is your entire kernel source. It needs to be built against a configuration file that dictates what, who and when will be included in the final kernel image.
The configuration file is a hidden file called .config and it's a line-delimited list of entries with kernel parameters and values. Again, using my crash analysis series as a reference, we saw this already. We tackled the kernel module compilation and handled the issues of missing configuration and Makefile.
By default, you will not have a configuration if you've just download a kernel. You can use your own, which ships with your distribution. On Ubuntu and RedHat, you will find the configuration under /boot. on SUSE, it's located under /proc/config.gz.
To get the configuration, just copy the config file from under /boot or redirect the output of config.gz to a file, as I've already shown you in my crash tutorial.
Now, you need to toggle what settings you want to use. You can manually go through the configuration file and mark different entries, but this is tedious and not really useful if you're not a super-expert. The friendlier alternative is to launch a configuration menu for kernel compilation, which is invoking in different ways, depending on what interface you want to use.
make menuconfig is the simple option, with ncurses text interface inside terminal. make gconfig and make xconfig are graphical interfaces with mouse support. You can use those, too, if you must, but I don't see why.
make menuconfig is the most sensible option; you will need the ncurses packages installed, which we did earlier. Now, let's launch the configuration wizard and start toggling options on and off.
This is where fun - and chaos - begin. Do not just randomly select options. Work slowly. Explore every option in detail, and when in doubt, just leave it alone. Like I mentioned earlier, there's no easy or fast way about kernel compilation. If you're really serious about it, the configuration process can take hours, maybe even days.
The next step is to compile everything. This is done by running make all command and then waiting several minutes to hours for the procedure to complete, hopefully with success.
If your configuration does not contain answers for all of the options, especially if they are new and not currently included in your running kernel, you will need to answer the prompts for these options. There's no easy way about it.
Actually, there is an easier way, but it's still fairly geeky. You can run make oldconfig, a command that will prompt you only for new changes that are not marked in your existing configuration file. If you don't do that, you will have to answer prompts in real time.
The compilation will then start and churn your CPU for a while:
Once this is done, you will need to either manually copy the new kernel to /boot, create the initrd file and change the GRUB menu or use the make install option to get things done automatically.
So the manual way to get this done is by first copying the kernel image:
cp <new kernel image> /boot/
Next, create initrd file; more about initrd in yonder article:
mkinitrd -o initrd.img-<kernel version>
make modules_install install
This last step needs root or sudo privileges.
The really last step is to reboot and hope for the best. If you didn't go wild with hardware support and drivers, you will probably see your machine reach the desktop without any issues. Whether you will see any benefits in the short and long run depends entirely on your configuration. And we're done.
A handful of worthy articles that discuss Linux compilation:
An older piece on 2.4 kernels:
I must stress my dire objection to kernel compilation at home. It's a futile exercise in code, with virtually no benefits, save for a few corner cases. For the absolute majority of users, there is no added value to using a custom kernel. Any problem that may arise on your system will forever bear the mark of doubt, as you won't really know if this is something broken by design or something you broke.
Anyhow, this tutorial completes two objectives: one, it teaches you the basics of Linux compilation; two, it settles an old score, my promise on kernel compilation. There you go, it took a while, but I did it.
To wrap it up, let me go quickly through what we did today: get the sources and the build tools, prepare the correct configuration, compile everything, make the system use the new kernel image. In a nutshell, it sounds easy. But with thousands of options available, you are up against a mountain of vast and complex knowledge. But then, if you're not afraid of learning new things, this could be a fabulous journey.