Technology may have changed in the meantime.
This is a FreeBSD tutorial; it’s useless for Linux.
The FreeBSD handbook describes how to build a custom kernel. However, this documentation assumes that you have access to the boot menu in case the new kernel won’t boot. And on a remote server this is not always an option.
So I will describe the process for building a custom kernel on a remote server here. For clarity and sysadmin friendliness I will start at the beginning, instead of only explaining how to boot the old, working kernel if something is wrong with your new kernel.
This procedure will take some time, so you may wish to use a tool like sysutils/screen or sysutils/tmux to avoid having to start all over in case you lose the connection to your server.
Before doing anything else, make a copy of the kernel that’s currently in use. This way we can always go back to now.
# cp -rp /boot/kernel /boot/kernel.works
And make sure all installed software is up to date. It’s always better to work on a clean system.
# freebsd-update fetch # freebsd-update install # pkg autoremove # portsnap fetch # portsnap update # less /usr/ports/UPDATING # portupgrade -aR # pkg autoremove # portsclean -CDDLP
I ran pkg autoremove twice to avoid updating software I don’t need; I’m hoping to save some time this way. Then again, it is very well possible that software that was removed with the first run, is re-installed by portupgrade, and then removed again with the second run (build dependencies). If you want, you can skip the first run.
If you’re used to other tools to update your ports collection and upgrade your ports, then you should obviously feel free to use those.
If you prefer to upgrade from packages instead of ports, go right ahead.
The preferred way for retrieving the FreeBSD kernel sources is Git, so if this isn’t installed yet, this is the time to do so.
# make -C /usr/ports/devel/git install clean
or
# pkg install git
Before you checkout the source tree, make sure that freebsd-update cannot mess with the source or the kernel; open the file /etc/freebsd-update.conf in your favorite text editor and make sure src and kernel are not listed in the Components option.
Then, determine the git branch you need, based on your FreeBSD version and this table in the FreeBSD handbook, move the old source tree (if any) out of the way, and checkout the source.
DO NOT BLINDLY COPY THE LINES BELOW, BUT MAKE SURE TO CHECK THE TABLE THAT I LINKED TO IN THE PREVIOUS SENTENCE.
# vim /etc/freebsd-update.conf # uname -r 13.0-RELEASE-p4 # rm -rf /usr/src # git clone --branch releng/13.0 https://git.freebsd.org/src.git /usr/src
Alternatively, if /usr/src is populated already, and the current content was retrieved using Git, you can save some time by updating it instead.
# git -C /usr/src pull
Then, change to the kernel configuration directory for your machine’s architecture.
# uname -m amd64 # cd /usr/src/sys/amd64/conf
There, copy the file named GENERIC to a directory where you will be able to find it again, and link that copy back to this directory. This file will contain the configuration for the new kernel. You can use whatever names you like for both the copied file and the link (but don’t use the name GENERIC, unless you’re looking to confuse yourself). The file is a plaintext file, so you can store it in a version control system like RCS, Subversion or Git if you want.
# cp GENERIC /root/KERNEL-CONFIG # ln -s /root/KERNEL-CONFIG ./NEWKERNEL
And then it’s time to open the file in your favorite text editor, and configure your own custom kernel.
Personally, I prefer to edit the configuration file in 2 runs. In the first run I remove all support I don’t need, and in the second run I add the support I do need. (Do with that info what you want. 🙂 )
Remove all devices and options you do not need. This blog post is about a remote server, so you probably won’t use the floppy drive and the USB and parallel ports; remove the support from the kernel, and nobody will be able to load malicious software onto your server using those devices. The GENERIC kernel supports many network cards; if you keep only the drivers for the network cards that are actually in the server, your kernel will be smaller and use less memory. And so on…
However, if you’re in doubt about an option or device, do not remove them; you don’t want to risk removing too much. (If you do accidentally remove an option or device that you need, you will be able to repair it later, but that would mean recompiling the kernel, which takes time.)
Look at the files ./NOTES and ../../conf/NOTES for information about available options and devices. Information about devices present in your system can be found in the file /var/run/dmesg.boot and the output of pciconf -lv.
Also look at the output of kldstat to see which modules are currently loaded; if they are loaded now, your system possibly needs them (but not necessarily; ultimately, you decide).
# kldstat Id Refs Address Size Name 1 32 0xffffffff80200000 1f11ef8 kernel 2 1 0xffffffff82320000 3250 ichsmb.ko 3 1 0xffffffff82324000 2180 smbus.ko 4 1 0xffffffff82327000 2240 pflog.ko 5 1 0xffffffff8232a000 36f70 pf.ko 6 1 0xffffffff82361000 2340 uhid.ko 7 1 0xffffffff82364000 4350 ums.ko 8 1 0xffffffff82369000 3380 usbhid.ko 9 1 0xffffffff8236d000 31f8 hidbus.ko 10 1 0xffffffff82371000 3530 fdescfs.ko 11 1 0xffffffff82375000 2a08 mac_ntpd.ko
Use the module names without the .ko suffix.
Most kernel modules come with a man page (in section 4), so to get more information about a module you could do something like
# man 4 smbus # man 4 uhid
Make sure to add everything you need to the kernel configuration, so that it is all compiled into the kernel. At the end of this article we will configure the system to forbid the loading of kernel modules, so all needed support should be in the kernel.
When the configuration is complete, it is time to compile the new kernel.
# cd /usr/src # make buildkernel KERNCONF=NEWKERNEL
Replace NEWKERNEL with the name of the symlink you created in the conf directory.
Compiling a kernel will take quite some time.
Once it’s done, the kernel can be installed.
# make installkernel KERNCONF=NEWKERNEL
This will rename the current kernel from /boot/kernel to /boot/kernel.old, and install the new kernel in /boot/kernel.
The FreeBSD handbook suggests a reboot now, to find out whether the new kernel works.
However, if the new kernel won’t boot, or we’ve included support for the wrong network card for example, we have a problem: this is a remote server, so we can’t access it if it doesn’t boot correctly.
Luckily, FreeBSD has a command that allows us to boot an alternative kernel just once. If there is any problem, it suffices to power cycle the machine (which you can probably do through the data center’s web interface), and it will boot up with the original kernel.
For this to work, the new kernel must be renamed to something arbitrary, and the old kernel must be renamed to what it was before.
# mv /boot/kernel /boot/kernel.new # mv /boot/kernel.old /boot/kernel
The nextboot command can now be used to configure the system to boot the new kernel on the next boot. After that, the system will revert to the kernel in /boot/kernel (which is the old, working kernel).
# nextboot -k kernel.new # reboot
If the new kernel boots correctly, it is time to test if everything works. I can’t really tell you what to do here, because you know what functionalities should be in your kernel, and I don’t. Verify that all desired filesystems work, see that the firewall works, test virtualization if you configured that, etc…
Make sure to also verify that the kernel does not load any kernel modules (kldstat); if it does, you may need to add some more support to the kernel configuration (and then recompile and reinstall the new kernel).
If everything works as expected, the new kernel can be made the default kernel.
# mv /boot/kernel /boot/kernel.old # mv /boot/kernel.new /boot/kernel # reboot
However, if things do not work as expected, you simply go back to the old kernel.
# rm -rf /boot/kernel.new # reboot
And then you can re-edit the kernel configuration, and rebuild and reinstall the new kernel.
# cd /usr/src # make clean # cd sys/<arch>/conf # vim ./NEWKERNEL # # ...etc...
And obviously, if you ever need to go back to the old kernel for some reason, you can do
# nextboot -k kernel.old # reboot
With all the necessary modules compiled into the kernel, we can now instruct the system to forbid the loading of kernel modules, as an additional security measure.
# sysctl kern.securelevel=1
To have this set automatically at boot time, add these 2 lines to /etc/rc.conf:
kern_securelevel_enable="YES" kern_securelevel="1"
Now, for kernel modules to be loaded, kern_securelevel_enable must be set to NO, and the server must be rebooted; this should prevent an intruder from loading undesirable kernel modules. Obviously, you are expected to have monitoring in place that alerts you if and when the server reboots.
See security(7) for the other consequences of setting the securelevel to 1, and for the other securelevels that exist.
And this concludes the installation of a custom kernel on a remote server.
If somewhere in this process you really screw up and/or you lose track of which are the old, and which the new kernels, remember that you copied the working kernel at the beginning of this article. In the worst case scenario you can always restart from scratch.
# rm -rf /boot/kernel # cp -rp /boot/kernel.works /boot/kernel # reboot # rm -rf /usr/src # git clone --branch <branch> https://git.freebsd.org/src.git /usr/src # # ...etc...
And once you’re confident enough, you may want to replace that working kernel copy with a copy of the new kernel.
If at some point you’d want to build a new kernel again (new functionality, newer versions, etc.), you’d do something like this:
# cd /usr/src # make clean # git pull # cd sys/<arch>/conf # cp /root/KERNEL-CONFIG /root/KERNEL-CONFIG2 # ln -s /root/KERNEL-CONFIG2 ./NEWKERNEL2 # vim ./NEWKERNEL2 # # ...etc...
Or, if the kernel configuration is under version control, you can just continue with the same file. In that case, make sure to tag the versions that were used to build a kernel.