3. Building and installing 64-bit Fedora 24 kernel on Raspberry Pi-3

First version 2016.09.13

Rev 2016.09.18

Background

I started this page as a guide for others who wanted to build the 64-bit kernel for Fedora 24.  However, the page had a short lifetime of usefulness -- some problems that prompted me to recompile were corrected by Kraxel in bringing the update to kernel 4.7.3-2-main of 64-bit Fedora 24.   But other problems continue even with the corrected updates and in recompiled kernels.   So I'll leave the page around (and try to keep it updated) for those who might need to build the kernel on the RPi-3 in the future.

I installed 64-bit Fedora 24 on a Raspberry Pi-3B using Will Foster's guide for the 32-bit version as a procedural outline [ https://hobo.house/2016/03/13/installing-fedora-linux-on-the-raspberry-pi-3/ ].  The FC24 distribution repository version at kraxel.org, dated 20160623, installs 4.6.2 of the kernel.  After installing FC24 on a Raspberry Pi-3B, with a resulting system that was reasonably functional, I did a "dnf update" that brought the system to kernel 4.7.1 (initially -- now 4.7.4) and resulted in at least one key component (polkit) not functioning.  SirSpudd identified the problem as a change in a kernel configuration parameter (VA_BITS set to 48 rather than 39) and reported that by recompiling the kernel with the original  value (39), the polkit (and other) errors were resolved.  He also noted that it might be some time before that correction migrated into the mainstream distribution.

And he provided instructions for compiling (cross-compiling in his case) and installing the corrected kernel.  It looked fairly straightforward, so I thought I'd try it to see if the corrected kernel would eliminate some of my problems.

It has been a while since I'd compiled a Fedora kernel, and compiling for the Raspberry Pi booting environment (in which the graphical coprocessors actually load the ARM OS code) makes this installation a bit more complicated.

The result was a painful (re-)learning experience, but I did eventually compile and install a kernel that would actually boot and run Fedora 24.  And it eliminated the polkit memory addressing crashes that had made some of the Fedora services inaccessible.  It did not eliminate all my problems, but they may be caused by other issues: the obvious problems were resolved.

So, in case others get to the point where they need to compile and install the 64-bit Fedora kernel, here is an expanded version of SirSpudd's instructions, modified slightly since I compiled on the RPi3 itself rather than cross-compiled.

My Pi-3 Configuration

In case my documentation below is confusing, it might help to know the environment in which my Pi-3B operates:

• RPi-3B running 64-bit Fedora 24 (see part 2 of this blog to set that up)
• Boot to a Samsung 32GB µSD with FC24 and three partitions: /boot, /swap, and /
• Logitech K400r wireless single-unit keyboard+trackpad -- uses a USB port of the Pi-3 (the only device on the USB system)
• Samsung HDMI TV as display -- also supplies the power to the Pi-3 through its 1A USB port
• Wired Ethernet connection to Cisco Powerline adapter (when needed)
• Mac Mini as remote computer when needed (used to access RPi-3 via ssh/sftp) 
• WiFi connection to Cisco Powerline adapter (once it is configured)

Context

I installed 64-bit Fedora 24 on the Pi-3 (part 2 of this blog).  I worked in multi-user mode (terminal) rather than graphical mode.

I followed SirSpudd's guide (reference hobo.house link above) except that I did a native compile on the RPi-3 rather than a cross-compile.  The RPi-3 is fast enough that with a quad-core system the compilation of the kernel requires about 50 minutes (wall clock).  I decided it would be more efficient (in my time) to build it on the RPi-3 than to install, configure, and then learn about cross-compiling on my Mac.

The process is mostly automated and quite straightforward, except at the final installation, where the make file misses some steps that are likely unique to the Raspberry Pi booting environment.  I haven't tried to amend the make file -- I just handle the steps manually (since I don't expect to do this often!).

Preparation

To start this process, you'll download a .rpm file with the source code for the kernel.  You'll need to then unpack and apply any patches before configuring and compiling the kernel.  To unpack, you can use "bsdtar" (which is not the same as "tar"; "tar" doesn't know how to read .rpm files).  And you'll need "patch".  You'll also need to have installed compilers and development headers, if you haven't done so.  So here are the steps you'll need to do in preparation for the real work:

1. Connect your Pi-3B to the Internet (Ethernet or WiFi).
2. Log into your own account.
3. cd (to connect to your home directory)
4. sudo dnf install bsdtar patch ncurses-devel
5. sudo dnf groupinstall "system tools" "C development tools and libraries"

These are quick and straightforward.

Procedure

You'll need root access for a few of the steps, but most of this process is done in non-privileged mode from your own account.  Don't put your system at risk by doing this as root.

1. Log into your own account.
   Don't su.  You may want to do this from a remote computer via ssh into your RPi-3.  The kernel source download is about 140MB and may take a while to download.  The kernel compile itself takes about 50 minutes.  So you may want to be doing something else while waiting for those tasks to complete.
2. Make a directory for the build and connect into it.
   cd; mkdir FC24; cd FC24
3. Get a copy of the source repository.
   dnf download --source kernel-main
   After this step, you'll have a .rpm file in your FC24 directory with a name looking something like kernel-main-4.7.4-2.src.rpm.  Next you'll unpack that .rpm distribution file, then unzip and unpack the .tar file that contains the source code.
4. Unpack the .rpm file and extract its contents:
   bsdtar -xvf kernel-main-x.x.x-x.src.rpm
gunzip linux-x.x.x.tar.gz
tar -xf linux-x.x.x.tar
5. Next apply the patches to that source code (in the linux-x.x.x directory):
   cd linux-x.x.x
cat ../*.patch | patch -p 1
6. Make sure to start with a cleaned-up source tree:
   make mrproper
7. Get a  copy of the .config file that generated your running OS:
   zcat /proc/config.gz > .config
8. Make/remake the .config file
   I believe that make wants to validate the configuration and write its own copy before you can actually start building the kernel.   If you just do a make, it'll run the configuration program and then start the build.  But you might want to double-check the .config file before starting the hour-long build, so I've divided this into steps.   The simplest approach is to simply validate the .config file, verify it visually, and then do the make.  If you want to use make menuconfig here instead so you get an item-by-item check-box-style menu for selecting configuration parameters, you'll need ncurses-devel to have been installed, and we already did that under Preparation.  So this step is simply:
   make oldconfig
   unless you want to do the complete configuration setup yourself, in which case you'd use make menuconfig.  At the end of this step, the configuration program will tell you that it has written the configuration to .config.
9. Label your kernel and verify the .config file.
   You'll want to give your kernel a local version number that identifies it as yours rather than from the repositories, and you may want to verify some of the parameter settings. I went through this exercise because VA_BITS was set to 48 rather than 39, so I check the .config file to verify that the parameter is set correctly before proceeding with the build.  At the very least, edit the file, locate the string CONFIG_LOCALVERSION=, and change the distribution value (something like "-2-main") to an identifying string in place of "main" that will be unique to you (something like "-2-yih3" -- using Your Initials Here and a sequence number).  While you're in the file, you might check around to confirm that other parameters are set as you intended, but don't arbitrarily change parameters:  Go through make menuconfig if you want to make other changes as there may be inter-dependencies of which you're unaware.
10. Build the kernel
    I like to track the time it takes to build the kernel, so I time the build.  Omit it if you don't care. Since the RPi-3 has 4 cores and I don't do any other work during the build, I tell make to keep 4 streams running (the -j 4 part).
    time make -j 4
    It'll take about 50 minutes to complete.
11. Make the modules
    make modules
12. Install the kernel and modules (most of it)
    This part has to be done as root, hence the sudo's.
    sudo make modules_install
sudo make install
    These steps install most of what's needed in /boot and in /lib/modules/.   If you ls /boot, you'll see some files there identified with your unique identifier (e.g., "-2-yih3"). Ditto if you check /lib/modules.
    
    But unfortunately the makefile misses a couple of steps for the RPi, and we need to finish up manually. The key parts are:
    • Copy the device tree directory from the build tree to /boot
    • Copy the bcm2837 device tree file to a place bootcode will find it
    • Edit /boot/extlinux/extlinux.conf to have bootcode let you choose your new kernel when doing the boot
    
    Kraxel pointed out that there are a couple of lines we can add to /boot/extlinux/extlinux.conf that will allow you to choose at boot time the kernel you want to run. We'll install two lines at the beginning of that file, just once, and then we can choose whether to boot our new kernel or revert back to the older, distribution version -- always a helpful feature in case your new version doesn't boot correctly.  With your LOCALVERSION identifier that you edited into the .config file, the build process created a boot image and related files with a corresponding unique image identifier, something like 4.7.3-2-yih3, that includes the kernel version from the distribution appended with your unique identifier.   Key files and directories will use that identifier in their names.  In the steps below, we'll use the string "<imageid>".  Use your unique image identifier, 4.7.3-2-yih3 or whatever, wherever you see "<imageid>" below.
13. Copy the device tree and bcm2837 .dtb file to expected locations
    cd /boot/dtbs
sudo mkdir <imageid>
cd <imageid>
sudo cp -rL <your home>/FC24/linux-x.x.x/arch/arm64/boot/dts/* .
sudo cp broadcom/bcm2837-rpi-3-b.dtb .
14. Tell bootcode about the new kernel
    sudo edit /boot/extlinux/extlinux.conf

    Just by looking at that file, you'll see what the general layout of a boot kernel description entry is:
    
    label kernel-4.7.3-2-main
  kernel /vmlinux-4.7.3-2-main
  initrd /initramfs-4.7.3-2-main.img
  fdtdir /dtbs/4.7.3-2-main/
  append dwc_otg.lpm_enable=0 console=ttyAMA0,115200 \ 
    console=tty1 elevator=deadline root=/dev/mmcblk0p3 \
    rootfstype=ext4 rootwait
    You're just going to replicate that at the bottom of the file with the <imageid> of your new kernel image as the kernel name.  So open the file with your favorite editor, and before the first line, insert the following just the first time you edit the file:
    
    menu title pick kernel 
timeout 100
    
    Now go to the bottom of the file and insert the description of your new kernel with your unique identifier (something like 4.7.3-2-yih3) in place of <imageid>:
    
    label kernel-<imageid>
  kernel /vmlinuz-<imageid>
  initrd /initramfs-<imageid>.img
  fdtdir /dtbs/<imageid>/
  append dwc_otg.lpm_enable=0 console=ttyAMA0,115200 \
    console=tty1 elevator=deadline root=/dev/mmcblk0p3 \
    rootfstype=ext4 rootwait
    
    NOTE that the last line ("append ...") should all be on one line of the file (no continuation "\").
    
    ALSO NOTE that the kernel file is named vmlinuz rather than vmlinux: the make install gets the compressed boot file and names it as a compressed image when it copies it to /boot.
15. Save the file and exit the editor.
16. Double check
    cat /boot/extlinux/extlinux.conf and make sure that each of the files or directories mentioned in the section for your kernel is actually in place in /boot, and make sure that the kernel module directory is in place as /lib/modules/<imageid>.
17. Reboot.  Issue a "sudo reboot" command to initiate the reboot.  You'll now get a menu allowing you to pick the kernel you want to use: type in the number of the kernel with your unique identifier.  After you've booted up and logged in, verify that your kernel is running by typing uname -a and make sure the name string has your <imageid> in it.  
• If it booted up but didn't boot into your kernel, repeat the double-check in 16: typographical errors are easy to make but fatal to the boot process.  
• If it appears that everything is in place but your kernel isn't coming up, watch the beginning of the boot process carefully to see if you can determine which step is failing, then correct it. 
• If the boot process is hanging completely when you specify your kernel, reboot by power cycling and select a distribution kernel (rather than the one you just built).  Again, do the double-check in 16.  If there are no typos, review your configuration (make menuconfig) to make sure you have a valid configuration.  Compare your .config with /proc/config.gz to identify inadvertent changes.  And, finally, go back to "make mrproper" and start again.