June 18, 2013

U-Boot on i.MX6

There are a lot of posts in this blog that describe the state of U-Boot for our i.MX6 boards, but most of them describe the history. They were designed to help the early adopters make transitions as we switched from the Freescale U-Boot 2009.08 to main-line and added display support.

Things have stabilized, and in this post, we’ll recap the current state of affairs to provide the new user a quick Getting started guide.

Here are some things you should know about the U-Boot that we ship with our i.MX6 boards:

  • Our boards boot to a serial EEPROM. The i.MX6 processor has a set of internal fuses that control the boot device, and we program these for SPI-NOR. There are a variety of ways to over-ride this, but we don’t recommend them except for very specific needs.
  • The primary U-Boot console is the serial port on a DB-9 labelled console. It’s configured for direct connection to common USB serial adapters (i.e. as DCE). The baud rate is 115200. 8 data bits, no parity, no flow control. If you hit the any key right after power-on, you’ll abort the automatic boot process and get a U-Boot prompt on the serial port.
  • U-Boot images have support for USB keyboard. See this post for details. If you boot without a working 6x_bootscript on SD card or SATA, you’ll get a prompt on the display and USB keyboard in addition to the serial port.
  • Our images will boot from either SD card slot or SATA. The default bootcmd string is set to iterate through all of the boot media on your board, looking for the file 6x_bootscript in partition 1. If found, the script will be run (and won’t return if successful). Note that some userspaces may require the use of a certain boot media. Consult the corresponding documentation for details.
  • Our images will boot from FAT, ext2, ext3, or ext4 filesystems. The default bootcmd variable will try FAT first, then ext2 (which supports ext3 and ext4).
  • U-Boot images include the USB Mass Storage Gadget. This allows the board to present a block device (such as an SD card) to the host as a mass storage device via the USB OTG port.
  • If you power-on a device with one of our displays connected, you should get an on-screen image. Display support has been in U-Boot for a while now, and you should see a display without any SD card present.
  • Our default bootcmd is set to run 6x_bootscript. To elaborate, we leave the precise boot instructions to the userspace image itself, since different distributions package things differently and require at least different kernel command-lines. We accomplish this by having a bootcmd which simply loads a distribution-specific startup script (6x_bootscript) from the first partition of either SD card or SATA drive.
  • Linux kernel 3.10.31 introduces incompatible naming of SD card devices. As detailed in this post, the root= clause in the kernel command line may need to be altered to accommodate these changes. The new sdphys variable in our boot scripts can be used to select the old or new numbering scheme.
  • We provide sample boot scripts for Android and typical non-Android use. You can find these in the directory corresponding to your board in files named 6x_bootscript*. These are just samples though. You will likely need to tailor them for your use in production, because
  • The default boot scripts try to detect your display. There are some notes in this post but from a high-level, suffice it to say that these boot scripts try to provide an easy out-of-the-box experience.
  • Boot scripts are compiled text files consisting of a series of U-Boot commands. The language supports the use of conditionals, so you can test for the presence of files, environment variables, and the like.
  • We have an online tool for compiling boot scripts here that takes care of the arcane mkimage command-line and removes carriage-returns, which the U-Boot hush parser doesn’t like.
  • Don’t assume that you need to re-compile and re-install U-Boot to match your userspace. We routinely boot Linaro, Buildroot, LTIB, Ubuntu, Debian, Timesys, Android, QNX, and Windows Embedded Compact 7 operating systems without re-programming U-Boot. You should only need to re-install U-boot to take advantage of a new feature or to fix a bug. The U in U-Boot stands for Universal and it comes pretty darn close. Many userspace builders will build a U-Boot image for you, but that’s primarily because other platforms are configured for boot to raw SD card and require it. Our boards don’t.
  • Use the 6x_upgrade script and the upgradeu environment variable if you do need an upgrade. You can copy 6x_upgrade and u-boot.imx to an SD card, and upgrade like this:
    U-Boot > run upgradeu
    ...
    check U-Boot
    320748 bytes read in 145 ms (2.1 MiB/s)
    read 0x4e4ec bytes from SD card
    SF: Detected SST25VF016B with page size 4 KiB, total 2 MiB
    probed SPI ROM
    byte at 0x1200000d (0xf0) != byte at 0x1240000d (0xfc)
    Total of 13 byte(s) were the same
    Need U-Boot upgrade
    Program in 5 seconds
    5
    4
    3
    2
    1
    erasing
    programming
    verifying
    Total of 320748 byte(s) were the same
    ---- U-Boot upgraded. reset
    
  • U-Boot packages – The example of upgradeu above works when you’re compiling U-Boot yourself, as it makes reference to u-boot.imx. If you’re using our production binary package, you’ll either need to re-name one of the files in the package to u-boot.imx or over-ride the boot file like so:
    U-Boot > bootfile=u-boot.nitrogen6q ; run upgradeu
    ...
    check U-Boot
    ...
    Total of 320748 byte(s) were the same
    ---- U-Boot upgraded. reset
    

    Refer to this post for information about the various files and the naming conventions.

  • If you’re booting to SD card or SATA, you shouldn’t need to set any environment variables before booting. Our sample boot scripts configure most things automatically, and you should generally do the same. If you need to customize the boot environment (usually the bootargs variable), you’ll generally want to hack the boot script instead. This is easier to copy to other machines, and fits nicely into a source repository.
  • If you’re booting over NFS during development, you should hack your environment variables. See the notes below for typical usage.
  • Restore your factory defaults with clearenv.
    U-Boot > run clearenv
    

    Or better yet, use env default -f -a. We only learned of this nice feature recently, and will likely discard clearenv in the future.

    U-Boot > env default -f -a
    
  • Our U-Boot source code is on Github. We aim to have all of the code for our standard products in main-line U-Boot, but generally have at least a few patches in flight.
  • Our production branch contains the version that we’re currently shipping for new orders and is typically the most stable and recommended branch.
  • Our staging branch contains the version that we’re prepping for the next release. It may have some bugs, but also may have some new features of interest.



Boot flow

The process of booting an i.MX6 involves multiple steps:

  1. At power-on or reset, the processor starts executing code from on-chip ROM. This code reads the Boot Mode switches, and if they’re in the normal position (Boot from fuses), goes on to read the fuses to find out which media to search for a bootable image.
    In our case, it will see that we have the fuses programmed for serial EEPROM, and should find a U-Boot image at offset 0x400 in that device.
  2. The code in on-chip ROM will read the U-Boot preamble, configuring registers for DDR, then copy U-Boot into DDR and begin U-Boot execution.
  3. U-Boot will wait for bootdelay seconds for a user to abort the boot with a keystroke on the console (the serial port).
  4. If no keystroke is present, U-Boot will execute a set of commands stored in the environment variable bootcmd.
  5. As mentioned above, our default bootcmd is configured to iterate through all bootable media (SATA and both SD cards), looking for 6x_bootscript. If found, the commands inside are executed.
    In the normal case, these commands will never return because an O/S will be launched.

Unbricking

If step 1 above sees the Boot mode pins in the Serial Boot position, or doesn’t find a valid image in the serial EEPROM, the code in the on-chip ROM will enter serial download mode.


This mode allows a user to download a valid boot image over the USB OTG port, providing a robust means of recovery.

If you connect the USB OTG port to a Linux-based machine (including i.MX6 devices), you can see this in the output of lsusb:

~/$ lsusb
...
Bus 001 Device 009: ID 15a2:0054 Freescale Semiconductor, Inc. i.MX6Q SystemOnChip in RecoveryMode

We wrote a Linux-based tool called imx_usb that supports this protocol, so you can supply a U-Boot image to the device from the command-line like so:

~/imx_usb_loader$ sudo ./imx_usb u-boot.imx

The sudo is needed to provide access to the raw USB device, and the command-line parameter can be any fully-formed i.MX6 image. In the example above, we’re supplying the U-Boot binary.

You can find more details in the our post on un-bricking an i.MX6.

The Freescale Manufacturing tool does something similar in the first stage, but requires USB OTG support in U-Boot itself, and this is not yet supported in the main-line code base.

The notes below provide some additional details for advanced users. Most users can ignore them.

How to build

Assuming that you have a cross-compiler for armv7 named arm-none-linux-gnueabi-gcc, you can get and compile U-Boot like this:

~$ git clone git://github.com/boundarydevices/u-boot-imx6.git
...
Resolving deltas: 100% (156593/156593), done.
~$ cd u-boot-imx6
~/u-boot-imx6$ git checkout origin/production -b production
~/u-boot-imx6$ export ARCH=arm
~/u-boot-imx6$ export CROSS_COMPILE=arm-none-linux-gnueabi-
~/u-boot-imx6$ make nitrogen6q_config
Configuring for nitrogen6q - Board: nitrogen6x, Options: IMX_CONFIG=board/boundary/nitrogen6x/nitrogen6q.cfg,MX6Q,DDR_MB=1024
~/u-boot-imx6$ make all
Generating include/autoconf.mk
...
~/u-boot-imx6$ ls -l u-boot.imx
-rw-rw-r-- 1 user group 312572 Nov 26 11:48 u-boot.imx

Key bits embedded in the snippet above include:

  • We selected nitrogen6q_config, which is the standard configuration for our most popular boards, the Nitrogen6X and SABRE Lite. If you’re compiling for another board or have a non-standard memory configuration, please refer to the table in this post.
  • We selected the production branch.
  • The output is in the file u-boot.imx

Refer to this post for more detail.

Details about branches on Github


In general, our U-Boot Github repository is used to contain our additions to the upstream U-Boot repository at Denx. Our aim is to stay as close to main-line as possible for our standard boards, and we submit changes whenever we can for those boards.

A number of custom boards aren’t suitable for up-streaming though, and we do generally have additions in our repository that either aren’t worth the noise of pushing up-stream (notably boot script updates) or have not yet made their way through the approval process into the main-line U-Boot code base.

We try to update our repository with the latest from up-stream as quickly as possible after each release cycle, so you’ll have access to all of the efforts by others in the community. This process largely entails moving or re-basing our branch onto the latest version from Denx, but we do usually take steps to reduce the patch set (and clean things up) during those efforts, so we lose a little of the history.

Because many of our customers (especially customers using Nitrogen6X-SOM) create their own branches based on our versions, we’ll keep each of of our branches indefinitely,
but they might be re-named. Please shoot us an e-mail if you do this.

Our production branch will move with each official release from us. If you’re trying to decide what branch to build, this is always the one you want. It should represent the most stable, most full-featured code base at any time. As a consequence, the branch name is also unstable. In other words, it will move over time as we jump from one version of U-Boot to the next.

To provide more stable branch names, we’ll also create branches named vYYYY.MM-yyyymmdd each time we complete a re-base. The YYYY.MM will represent the U-Boot version number and yyyymmdd will represent the date on which we re-based. If you need a stable branch name, as the Yocto Project does, you’ll want to use these instead of production. Additional patches may be added to a vYYYY.MM-yyyymmdd branch, but the history won’t be lost.

Basics of launching Linux

In order to boot Linux using U-Boot, you usually need only two things:

  • A working kernel, and
  • A filesystem containing init

But you’ll generally need a third:

  • A set of kernel parameters passed via the bootargs environment variable in U-Boot.

The kernel image is typically stored in a file named uImage and is most commonly stored in the /boot directory of a filesystem, but these are guidelines, not rules. The only important thing is that U-Boot is able to load the kernel into RAM.

The filesystem used at startup (generally referred to as the root filesystem) could be a typical ext2, ext3, ext4 filesystem on SD card or SATA, an accessible NFS share, or a RAM disk image. The only important piece is that you can tell the kernel how to find it at startup time.

Kernel startup is almost always invoked using the bootm command under U-Boot. The first parameter to bootm is the address of a kernel as shown in this example:

U-Boot > mmc dev 0
U-Boot > ext2load mmc 0 10800000 /boot/uImage
U-Boot > bootm 10800000

This simple example shows how to load /boot/uImage into memory address 0x10800000 and launch it.

But wait. We haven’t provided a filesystem reference in this example.

When you invoke bootm using a single parameter, you’ll need to specify the root filesystem indirectly through the bootargs environment variable as shown below.
This example tells the kernel to wait for the root filesystem to become available, and that the root filesystem is the partition /dev/mmcblk0p1 (the first partition of the first SD card enumerated on the system).

U-Boot > setenv bootargs $bootargs rootwait root=/dev/mmcblk0p1
U-Boot > mmc dev 0
U-Boot > ext2load mmc 0 10800000 /boot/uImage
U-Boot > bootm 10800000

This is precisely what’s done in the default boot script board/boundary/nitrogen6x/6x_bootscript.txt.

To boot a RAM disk for the root filesystem, you add a second parameter to the bootm command with the load address of the RAM disk and omit the root= clause from bootargs. The following example illustrates this:

U-Boot > mmc dev 0
U-Boot > ext2load mmc 0 10800000 /boot/uImage
U-Boot > ext2load mmc 0 12800000 /boot/uramdisk.img
U-Boot > bootm 10800000 12800000

RAM disks are used in the default Android boot script (board/boundary/nitrogen6x/6x_bootscript_android.txt), but are also useful for other distributions. We used one to put together this image for exposing storage across USB.

Note that there is a third parameter for bootm that’s required for use of the main-line Linux kernel. It supplies something called a device tree, which is used to make the kernel a bit more generic. If you’re working with main-line, it’s likely that you have access to this, so we won’t go into the details here.

How to boot over NFS

Booting over NFS is a straightforward extension of this that simply replaces the root= clause in bootargs with root=/dev/nfs. It does require a couple of additional parameters to the kernel, though:

parameter typical value
nfsroot hostip:/path/to/rootfs,options
ip dhcp

The first parameter, nfsroot= tells the kernel where to find a root filesystem and what options to pass to the NFS mount process. A typical value for the entire clause might be
nfsroot=192.168.0.42:/home/user/imx-android,v3,tcp.

The second parameter, ip= is needed because the kernel needs to have an IP address in order to access the network. An IP address is normally done as a part of the userspace startup, but in the case of an NFS root, we can’t wait for that because of the chicken-and-egg problem.

Putting it all together, we can boot over NFS like this:

U-Boot > setenv bootargs $bootargs rootwait root=/dev/nfs
U-Boot > setenv bootargs $bootargs nfsroot=192.168.0.42:/home/user/imx-android,v3,tcp
U-Boot > setenv bootargs $bootargs ip=dhcp
U-Boot > mmc dev 0
U-Boot > ext2load mmc 0 10800000 /boot/uImage
U-Boot > bootm 10800000

Note that this example doesn’t completely boot over the network, though. The kernel is still loaded from an ext2/3/4 filesystem on partition 1 of the SD card. This brings up the next question:

Loading a file over TFTP

U-Boot contains a number of networking commands, and support for a number of protocols.

The most common are the dhcp and tftp commands, and we generally use the dhcp command to acquire an IP address and transfer a file in a single command like so:

U-Boot > dhcp 10800000 192.168.0.62:uImage

This command will acquire an IP address using DHCP, then request a file named uImage and load it into memory address 0x10800000.

When used in conjunction with the NFS boot arguments, this provides a single, relatively command line to be used for booting:

U-Boot > dhcp 10800000 192.168.0.62:uImage && bootm 10800000

You may now be understanding why I mentioned the saving of environment variables when booting NFS. There are a number of parameters to provide, including

  • the IP address of the TFTP server, and
  • the IP address and path to the NFS filesystem

If you’re going to save all of these, you might as well just over-write the bootargs variable entirely, and the bootcmd variable while you’re at it.

You can always run clearenv when you’re done.

Important clauses for bootargs:

To recap and expand on the notes above, here are a set of known variables that you might want to set in bootargs under U-Boot:

name typical value Notes
console ttymxc1,115200 This tells the kernel to send kernel messages to /dev/ttymxc1, the serial console port. You almost always want this set during development, though you might consider turning it off in production, since it can slow the boot process.
enable_wait_mode false This variable controls the behavior of the idle loop in the Linux kernel and you may see system stalls without this value set.
root= /dev/mmcblk0p1

/dev/sda1
See notes above
video= mxcfb0:dev=hdmi,1280x720M@60,if=RGB24

mxcfb0:dev=ldb,LDB-XGA,if=RGB666

mxcfb0:dev=ldb,1024x600M@60,if=RGB666

mxcfb0:dev=lcd,CLAA-WVGA,if=RGB666
Defines the display(s). These should be numbered starting at mxcfb0 through mxcfb2 and will translate into a number of device nodes as described in this post.
consoleblank 0 This variable controls the amount of idle time before the console device goes to sleep. Use consoleblank=0 to disable blanking on idle.
vmalloc 400M This controls the amount of memory available to the kernel for dynamic allocation. You’ll normally want this to be 400M or so on a system running a graphical U/I with accelerated video and graphics.
fbmem 28M This controls the amount of memory allocated for each frame-buffer.
ip dhcp

192.168.0.44:::255.255.255.0
Refer to the documentation for details.
nfsroot 192.168.0.62:/path/to/rootfs Refer to the documentation for details.
androidboot.console ttymxc1 Tells Android about the console
androidboot.hardware freescale Android needs this. Details elsewhere

Booting Yocto

Thanks largely to the efforts of the team at O.S. Systems, the current builds of Yocto using the meta-fsl repositories have a custom boot script that specifies partition 2 for the root filesystem instead of partition 1.

Their efforts show the right thing to do: have the boot script built as a part of the userspace, but not U-Boot itself.

Booting Windows Embedded Compact 7

Coming soon

Booting QNX

Details coming soon, but the general process involves the use of the gocommand:

mmc dev ${disk} && ext2load mmc ${disk} 10800000 /path/to/ifs.ifs && go 10800000

Booting Debian

Details coming soon, but also available on eewiki.

.