June 12, 2017

Customizing Ubuntu/Debian kernels on i.MX6/7 boards

 This post is an update to our previous Ubuntunizing i.MX6 kernels in order to allow you to customize your image.

Also I try to give you a guide how to modify and customize the system image on the host PC before installing it on SD card. You can automatize the process if you want and you’ll get the a customized and ready to work system image as result.

1. Preparing and building kernel

As usual, the first step is cloning and building the kernel as it was described earlier in our Cross Compiling Kernels post:

~/$ git clone https://github.com/boundarydevices/linux-imx6.git && cd linux-imx6
~/linux-imx6$ export KERNEL_SRC=$PWD
~/linux-imx6$ export INSTALL_MOD_PATH=$KERNEL_SRC/ubuntunize/linux-staging
~/linux-imx6$ export ARCH=arm
~/linux-imx6$ export CROSS_COMPILE=arm-linux-gnueabihf-
~/linux-imx6$ git checkout boundary-imx_4.1.15_2.0.0_ga
~/linux-imx6$ make -C ubuntunize prerequisites 
~/linux-imx6$ make boundary_defconfig
... make your customization, code or configuration changes here.
~/linux-imx6$ make zImage modules dtbs -j8
~/linux-imx6$ make -C ubuntunize tarball
~/linux-imx6$ cd ..

Let’s see line by line:

  1.  Clone Boundary’s linux-imx6 repository.
  2.  Set KERNEL_SRC variable, it’s needed for building modules.
  3.  Set INSTALL_MOD_PATH variable, it’s needed for installing modules here and later too.
  4.  Set the target architecture.
  5.  Set the CROSS_COMPILE prefix.
    • We recommend using the official cross-toolchain that comes with Ubuntu/Debian.
  6.  Checkout the kernel branch you need. At the time of this writing, you can choose between the following versions (later this list will be extended): 
    • 3.14.28_1.0.0_ga
    • 3.14.52_1.1.0_ga
    • 4.1.15_2.0.0_ga
  7.  Install the armhf cross-toolchain.
  8.  Create the config file. If you have one of these boards, you need to use boundary_defconfig. Custom designs have their own specific defconfig.
  9.  Now you can customize the kernel configuration by running make menuconfig enable/disable options/modules as you wish. 
  10.  Build the kernel image (zImage), the modules (.ko) and the device tree blob (.dtb) files using 8 CPU cores.
  11.  This command copies the new files in a directory named linux-staging, and then creates a tarball of it.
    • Although the folder name is “ubuntunize”, this process also works with Debian images.
    • The kernel’s tarball is placed into the ubuntunize/ directory, and is named with the kernel version. If you modified the kernel source, then a -dirty flag will be appended to the name.
~/linux-imx6$ ls -l ubuntunize
total 39852
-rwxr-xr-x 1 tele tele      242 Jun 2 08:49 apt-upgrade.sh
-rw-rw-r-- 1 tele tele     2194 Jun 2 17:05 common.mk
-rwxr-xr-x 1 tele tele      118 Jun 2 05:02 create-initrd.sh.in
-rw-rw-r-- 1 tele tele 40778707 Jun 2 08:43 linux-4.1.15-gff461ec-dirty.tar.gz
drwxrwxr-x 4 tele tele     4096 Jun 2 08:43 linux-staging
-rw-r--r-- 1 tele tele     3296 Jun 2 06:14 Makefile
-rw-rw-r-- 1 tele tele     2767 Jun 2 18:00 Merge.mk
-rwxr-xr-x 1 tele tele      204 Jun 2 05:37 uninstall-kernel.sh

2. Building galcore and other modules

The galcore is a kernel module which is necessary to run Vivante GPU acceleration packages. Since the galcore module version must match exactly the version of the libraries in the rootfs, it is important to be able to build it independently from the kernel. You have to build this external module only if the kernel version is 4.1.15 or higher. Earlier system images with earlier kernels contain built-in galcore module.

Here is the procedure to generate the module using the version of your choosing.

~/$ git clone https://github.com/Freescale/kernel-module-imx-gpu-viv.git && cd kernel-module-imx-gpu-viv
~/kernel-module-imx-gpu-viv$ git checkout upstream/5.0.11.p7.4
~/kernel-module-imx-gpu-viv$ sed 's,-Werror,-Werror -Wno-error=misleading-indentation,g' -i ./kernel-module-imx-gpu-viv-src/Kbuild
~/kernel-module-imx-gpu-viv$ make -j8
~/kernel-module-imx-gpu-viv$ make modules_install
~/kernel-module-imx-gpu-viv$ cd ..

Before you start building this module you have to select the galcore version, it could be 5.0.11.p7.4 or 5.0.11.p8.6 or above version in the future.

It depends on rootfs you are going to use, so you have to check that first. Please boot the original Ubuntu/Debian system (what you are going to modify) on your board. Open a console window and type:

~/$ dpkg -l | grep imx-gpu-viv

In the 2nd column you’ll find the version. As of this writing, systems with X11 desktop usually use the 5.0.11.p7.4 driver and some console images use 5.0.11.p8.6.

Here is a line by line details of the procedure:

  1.  Clone the galcore repository from Freescale’s Github.
  2.  Set the matching branch as described above.
  3.  This sed command is only necessary if you use gcc 6 or higher, which will be used in Debian Stretch for instance.
    • This is a new kind of warning, but it would be considered as error, because of strict module building rules, so it must be disabled.
  4.  In this line we build the module. You don’t need to worry about KERNEL_SRC variable, because it was set earlier. We use 8 CPU cores.
  5.  Install the galcore module in place. You don’t need to worry about INSTALL_MOD_PATH variable, it was set earlier.
    • The module will be copied to a directory named extra.

Similarly to galcore, you can build any other kernel module. For example if you have a Silex WiFi module, you’ll have to build it’s driver as follows:

~/$ git clone https://github.com/boundarydevices/qcacld-2.0.git && cd qcacld-2.0
~/qcacld-2.0$ export CONFIG_CLD_HL_SDIO_CORE=y
~/qcacld-2.0$ git checkout boundary-LNX.LEH.4.2.2.2
~/qcacld-2.0$ make -j8
~/$qcacld-2.0 make modules_install
~/qcacld-2.0$ cd ..

You don’t have to build the above Silex  module on Ubuntu Trusty, but only on Ubuntu Xenial or Debian Jessie or Debian Stretch.

Based on the above examples you can build your own kernel modules as well. Fetch your sources to a working directory, set your environment variables and build flags as necessary, then simply run make and make modules_install. The environment variables KERNEL_SRC and  INSTALL_MOD_PATH has already been set earlier, you don’t need to worry about them.

After you have built all modules you wanted you need to refresh the tarball:

~/$ make -C $KERNEL_SRC/ubuntunize targz

3. Merging rootfs with kernel and modules

In this step we will merge the new kernel and modules and the original rootfs to a complete image tarball. I assume that you have already downloaded a Ubuntu or Debian system image from this page:

You have two options to merge the new kernel parts into the image; either on the target or on the host PC.

3.1 Merging on the target

Please create the SD card as usual, as described in the paragraph “Programming the image” in the system image’s blog post.

Then copy the tarball file what you have just created (linux-<version>.tar.gz) to the SD card’s /home directory. Don’t forget to type sync to finish copy. Now insert the card in the target board’s socket, and boot the new system up. After booting please do this:

~/$ sudo apt-get purge linux-boundary-* linux-header-* linux-image-* qcacld-module
~/$ sudo tar --numeric-owner -xf /home/linux-4.1.15-gff461ec-dirty.tar.gz -C /
~/$ sudo update-initramfs -c -k4.1.15-gff461ec-dirty
~/$ sudo apt-get update && sudo apt-get dist-upgrade
~/$ sync && sudo reboot
  1. Uninstall the old kernel first
  2. Extract the new kernel
  3. Create an initrd.img ramdisk
  4. Refresh the system, this is optional, it might take about 15-20 minutes
  5. Reboot

3.2 Merging on the host PC

After extracting, we will mount the tarball on a loop device and we modify it with pbuilder. Let’s see it first, then I explain it in details line by line.

~/$ export ROOTFS_IMG=~/Downloads/20160913-nitrogen-4.1.15_1.2.0_ga-xenial-en_US-mate_armhf.img
~/$ export IMG_MOUNT=/tmp/image-loop && mkdir -p $IMG_MOUNT
~/$ export LOOP_DEV=`sudo losetup -f`
~/$ export UBUNTUNIZE=$KERNEL_SRC/ubuntunize
~/$ export PB_EXEC=sudo pbuilder --execute --configfile $UBUNTUNIZE/pbuilderrc --no-targz --buildplace $IMG_MOUNT --bindmounts "/tmp $UBUNTUNIZE"
~/$ export KERNEL_REL=`cat $KERNEL_SRC/include/config/kernel.release`
~/$ make -C $UBUNTUNIZE -f Merge.mk prerequisites
~/$ gunzip $ROOTFS_IMG.gz
~/$ sudo losetup -o 1048576 $LOOP_DEV $ROOTFS_IMG
~/$ sudo mount -t ext4 $LOOP_DEV $IMG_MOUNT
~/$ $PB_EXEC -- $UBUNTUNIZE/uninstall-kernel.sh
~/$ sudo cp -a $UBUNTUNIZE/linux-staging/* $IMG_MOUNT
~/$ cp $UBUNTUNIZE/create-initrd.sh.in $UBUNTUNIZE/create-initrd.sh && sed "s,@KERNEL_REL@,${KERNEL_REL},g" -i $UBUNTUNIZE/create-initrd.sh
~/$ $PB_EXEC -- $UBUNTUNIZE/create-initrd.sh
~/$ $PB_EXEC -- $UBUNTUNIZE/apt-upgrade.sh
~/$ sudo umount $IMG_MOUNT
~/$ sudo losetup -d $LOOP_DEV
~/$ pigz $ROOTFS_IMG

Line by line explanation:

  1. Define the downloaded system image. We want to change it’s kernel and add modules and optionally we upgrade the whole system with apt-upgrade. You have to define the filename without the ending .gz or .xz
  2. Define the directory where the system image will be mounted up and we create it in the same step.
  3. Look for the first available loop device.
  4. Define the pbuilder mounting point. When pbuilder invokes chroot, this directory will be mounted up, so the system is able to see and execute the scripts.
    • KERNEL_SRC was defined earlier, in the kernel building section.
  5. Define the repeating part of the pbuilder command, we don’t want to type it again and again.
  6. Define the current kernel release, its needed to create the proper initrd.img in the create-initrd.sh script.
  7. Install prerequisites for the following tasks.
    • Including pbuilder, developer scripts and qemu for armhf emulation.
  8. Extract the gzipped tarball. If the tarball compression was not .gz but .xz then you have to type unxz $ROOTFS_IMG.xz instead.
  9. Set the loop device up at the offset 1048576 of the image file.
  10. Mount the loop device to the image mounting directory. At this moment we can see the content of the partition 1 (at offset 2048 sector : 2048 x 512 = 1048576 ) in that directory. And we can modify it if we want.
  11. Run pbuilder to execute a script inside the chroot. The script will run with qemu arm emulation.
    • This script uninstalls the original kernel packages and qcacld module if they exist, because we are going to replace them.
  12. Copy the new kernel and modules in place.
  13. Create the initrd creator script and we e set the current kernel.release in it.
  14. Run pbuilder to execute a script inside the chroot. This script will create an initrd.img ramdisk.
  15. Run pbuilder to execute a script inside the chroot. This script will run an apt update/upgrade on the system to make it up to date.
    • This is optional, you can skip this step if you want.
    • The qemu arm emulation is somewhat slower than native run on target, and running natively is better anyway, so that is the preferred way.
  16. Finish modifying image so unmount the loop device.
  17. Free the used loop device up. The image has been modified inside the extracted .img file.
  18. Compress the image back to .gz . Of course this is necessray only if you want to move/copy/transfer the image.
    • You can write the image directly to an SD card, in that case compressing is not necessary.
    • We use pigz here, its a gzip compatible compressor, but it makes use of multiple cores, so its much faster at big files than gzip.

I used the $HOME (~) directory as working directory, you can replace it with your preferred working directory. Although I used shell commands, you can easily put these lines in a shell script, or you can convert it to a makefile if you like that better. Check the .sh script files in the ubuntunize directory, those commands will be executed on the system image. Really simple scripts, nothing sophisticated. Feel free to change them for your needs.

If this merging process above looks too long or complicated, you can do this in a simpler way :

~/$ export ROOTFS_IMG=~/Downloads/20160913-nitrogen-4.1.15_1.2.0_ga-xenial-en_US-mate_armhf.img
~/$ make -C $KERNEL_SRC/ubuntunize -f Merge.mk

And that will do the same as above.

Final notes

I hope these tools make it easier for you to customize your environment to match a particular board or application use case, but they’re certainly not a panacea. I’ve added an example script file in the ubuntunize directory, named customize-kernel.sh , you can check it to see a typical using. Check it, modify it for your tasks.

You still need to be careful to match the needs of the image you’re tweaking, and need to align your kernel version and configuration with the O/S image used. This is especially true if you need graphics or video acceleration provided by our GPU and VPU packages.

As always, let us know how/if this works for you.