October 4, 2013

i.MX6 Video acceleration on Ubuntu Raring (and Debian)

When we recently posted Linaro Raring images for i.MX6, we were surprised at the number responses regarding hardware acceleration, and especially video acceleration.

For a long time now, we’ve been saying that it could be done, but leaving it as an exercise for the reader. In other words, go read the manuals and figure it out. Based on the response, it’s probably time to show some details.

It’s not very pretty, but the following will show you how it can be done. I recommend a full cup of coffee and a SATA drive, because almost all of this will occur on the device, and native compilation is very slow on SD card.

Also note that these notes apply and have been tested on Debian Wheezy, except that the names of some of the prerequisites are different. Also note that I’m starting with raring-nano to make sure I capture all of them.

First, the pre-requisites

First, make sure your system is up-to-date:
root@linaro-nano:~# apt-get update
Then you’ll need a compiler:
root@linaro-nano:~# apt-get install build-essential
And the ability to compile gstreamer plugins:
root@linaro-nano:~# apt-get install libgstreamer-plugins-base0.10-dev
root@linaro-nano:~# apt-get build-dep gst-plugins-base0.10
During the steps above, you’re going to get messages like this:
/sbin/ldconfig.real: /usr/lib/libOpenVG.so is not a symbolic link
In English, this is a warning that the Vivante libraries we installed don’t follow proper Linux shared-library versioning standards (where .so files are supposed to be symlinks to .so.version).

You can safely ignore them.

Then get the packages

The packages required for building the gstreamer plugins are as follows:
  • A set of kernel headers
  • firmware-imx-3.5.7-1.0.0
  • imx-lib-3.5.7-1.0.0
  • fsl-alsa-plugins-3.5.7-1.0.0
  • libfslcodec-3.5.7-1.0.0
  • libfslparser-3.5.7-1.0.0
  • libfslvpuwrap-3.5.7-1.0.0
  • gst-fsl-plugins-3.5.7-1.0.0

Kernel headers

Because some of these packages use hardware-specific APIs, they expect some i.MX6-specific header files to be present in /usr/include on the build machine.

There are two simple ways to accomplish this:
  1. If you’ve already built the kernel, you can use the headers_install target in the kernel build tree to create a set of headers and simply copy them into /usr/include, or
  2. You can use the Debian approach of make-kpkg to build a linux-headers-versionnum.deb package and use dpkg -i to install it.
To make this easy, we’ve uploaded a package to our cloud storage site with a recent version of the tar-ball version that you can grab and extract as shown below. We had some difficulty with make-kpkg generating an armhf package with a cross-compiler.
root@linaro-nano:~# wget http://commondatastorage.googleapis.com/boundarydevices.com/linux-headers-3.0.35-02829-gac24896_4.1.0.tar.gz
root@linaro-nano:~# tar zxvf linux-headers-3.0.35-02829-gac24896_4.1.0.tar.gz -C /usr

Getting the other packages from Freescale – the LTIB way

There are two routes to the other packages, which are distributed by Freescale. The first is to download two packages from the Freescale 3.0.35_4.1.0 release page:

Inside the L3.0.35_4.1.0_ER_SOURCE_BSP package, you’ll find tar-balls for both the imx-lib-3.0.35-4.1.0 and firmware-imx-3.0.35-4.1.0 packages, and you’ll find tar-balls of the other packages inside L3.0.35_4.1.0_MM_CODECS package in the ltib codecs directory. Note that although there are .deb packages inside the codec package, we weren’t successful in compiling from them.

Getting the other packages from Freescale – the Yocto way

Another way to get the codec and BSP packages is to download them from the Freescale Yocto repository at http://www.freescale.com/lgfiles/NMG/MAD/YOCTO/. This location allows for automated download, but because some of the sources require acceptance of a EULA, they’re packaged as self-extracting shell scripts (.bin files).

Getting the packages is very easy using this route:
# first, the open-source codec package:
root@linaro-nano:~# wget http://www.freescale.com/lgfiles/NMG/MAD/YOCTO/gst-fsl-plugins-3.5.7-1.0.0.tar.gz
# then the restricted files
root@linaro-nano:~# for pkg in firmware-imx 
                             imx-lib 
                             fsl-alsa-plugins 
                             libfslcodec 
                             libfslparser 
                             libfslvpuwrap ; do 
                         wget http://www.freescale.com/lgfiles/NMG/MAD/YOCTO/${pkg}-3.5.7-1.0.0.bin ; 
                     done
But extracting is a bit more complicated, requiring you to accept the EULA for each:
root@linaro-nano:~# tar zxf gst-fsl-plugins-3.5.7-1.0.0.tar.gz
root@linaro-nano:~# for pkg in firmware-imx 
                             imx-lib 
                             fsl-alsa-plugins 
                             libfslcodec 
                             libfslparser 
                             libfslvpuwrap ; do 
                         sh ${pkg}-3.5.7-1.0.0.bin ; 
                     done

Building the packages

Building (or installing) the packages is relatively straightforward once you know the flags required for each.

The firmware-imx package simply requires copying files to the /lib/firmware directory:
root@linaro-nano:~# cp -ravf firmware-imx-3.5.7-1.0.0/firmware/* /lib/firmware/
The imx-lib package seems to have failures building the rng library, so we’ve skipped it. It also requires the PLATFORM=IMX6Q flag:
root@linaro-nano:~# cd imx-lib-3.5.7-1.0.0/
root@linaro-nano:~/imx-lib-3.5.7-1.0.0# rm -rf rng
root@linaro-nano:~/imx-lib-3.5.7-1.0.0# make PLATFORM=IMX6Q all && make PLATFORM=IMX6Q install
The fsl-alsa-plugins package requires some autotools-fu before compilation:
root@linaro-nano:~# cd fsl-alsa-plugins-3.5.7-1.0.0/
root@linaro-nano:~/fsl-alsa-plugins-3.5.7-1.0.0# aclocal && autoconf && automake --add-missing
root@linaro-nano:~/fsl-alsa-plugins-3.5.7-1.0.0# ./configure --prefix=/usr
root@linaro-nano:~/fsl-alsa-plugins-3.5.7-1.0.0# make all && make DESTDIR=/ install
The libfslparser, libfslvpuwrap, and gst-fsl-plugins packages each have an autogen.sh script to take care of the autotools stuff:
root@linaro-nano:~# cd libfslcodec-3.5.7-1.0.0
root@linaro-nano:~/libfslcodec-3.5.7-1.0.0# ./autogen.sh --prefix=/usr && make all && make DESTDIR=/ install
root@linaro-nano:~/libfslcodec-3.5.7-1.0.0# cd ../libfslparser-3.5.7-1.0.0
root@linaro-nano:~/libfslparser-3.5.7-1.0.0# ./autogen.sh --prefix=/usr && make all && make DESTDIR=/ install
root@linaro-nano:~/libfslparser-3.5.7-1.0.0# cd ../libfslvpuwrap-3.5.7-1.0.0
root@linaro-nano:~/libfslvpuwrap-3.5.7-1.0.0# ./autogen.sh --prefix=/usr && make all && make DESTDIR=/ install
root@linaro-nano:~/libfslvpuwrap-3.5.7-1.0.0# cd ../gst-fsl-plugins-3.5.7-1.0.0
root@linaro-nano:~/gst-fsl-plugins-3.5.7-1.0.0# ./autogen.sh PLATFORM=MX6 --prefix=/usr
root@linaro-nano:~/gst-fsl-plugins-3.5.7-1.0.0# make all && make DESTDIR=/ install

Testing things out

To test things out, we downloaded the Sintel 1080P video file from the Blender Foundation’s Sintel web-site and played the video using gplay. Note that before doing so, we also installed the base, good, bad, and ugly gstreamer plugins, and that we flushed the disk cache prior to starting playback:
root@linaro-nano:~# for t in base good bad ugly ; do 
         apt-get install gstreamer0.10-plugins-${t} ; 
done
root@linaro-nano:~# echo 3 > /proc/sys/vm/drop_caches
root@linaro-nano:~# gplay Sintel.2010.1080p.mkv
We also tested out camera capture for both our OV5642 5MP parallel camera and OV5640 5MP MIPI camera using the mfw_v4lsrc element:
root@linaro-nano:~# modprobe mxc_v4l2_capture
root@linaro-nano:~# gst-launch-0.10 mfw_v4lsrc capture-mode=4 ! mfw_v4lsink

Binary distribution

I’m sure a lot of you patient enough to get this far saying “I just want it to work!”.

I hear you, and I’m getting there.

If you look closely above, you’ll see that everything except firmware-imx uses make install to install things into the root filesystem, and that most of the packages specify the DESTDIR variable to say where the output should go. The one exception is imx-lib, which wants the PLATFORM=IMX6Q flag and a destination directory of DEST_DIR.

We can use this to create an overlay of the binary bits in a temporary directory:
root@linaro-nano:~# mkdir -p /root/overlay/lib/firmware/
root@linaro-nano:~# cp -ravf firmware-imx-3.5.7-1.0.0/firmware/* 
                        /root/overlay/lib/firmware/
root@linaro-nano:~# for pkg in fsl-alsa-plugins 
      gst-fsl-plugins 
      libfslcodec 
      libfslparser 
      libfslvpuwrap ; do 
      cd ${pkg}-3.5.7-1.0.0 ; 
      make DESTDIR=/root/overlay/ install ; 
      cd ../ ; 
done
root@linaro-nano:~# cd imx-lib-3.5.7-1.0.0
root@linaro-nano:~/imx-lib-3.5.7-1.0.0# make DEST_DIR=/root/overlay PLATFORM=IMX6Q install
And create a tar-ball of the result like so:
root@linaro-nano:~# cd overlay
root@linaro-nano:~/overlay# tar czvf ../imx6-gstreamer-plugins-raring-20131003.tar.gz *
We’ve uploaded the result of this, and you can grab it at: If you extract this on top of one of the previously uploaded images, you’ll need to install gstreamer and the gst-plugins- packages as described above.

Recap and commentary

To recap, this post described the steps needed to get, build, and install most of the gstreamer plugins for i.MX6.

Note that there is one component not covered: the OpenGL image sink. That will have to wait for another day.

We also haven’t yet tested all of the functionality to look for gaps between this and other platforms. It’s entirely possible that there’s some breakage because we haven’t included patches like this one to the core GStreamer build.