Playing multiple HD videos on i.MX6

This is just a quick post with some details of some gstreamer-fu to decode and play back multiple HD videos using the VPU hardware acceleration on the i.MX6. The command-lines below use the mfw_isink gstreamer element to display the output from the 1920×816 Kung Fu Panda trailer from pocketmovies.net into three windows on a display configured at 1080P.

The only real trick to this is setting the display-width, display-height, axis-top, and axis-left parameters. For what it’s worth, this was tested on our LTIB-No X userspace.

Window 1 at 0:0

root@boundary ~/$ gst-launch filesrc location=kungfu.flv \
        ! 'video/quicktime' \
        ! aiurdemux name=demux demux. \
        ! queue max-size-buffers=0 max-size-time=0 \
        ! vpudec \
        ! mfw_isink disp-width=960 disp-height=408 demux. \
        ! queue max-size-buffers=0 max-size-time=0 \
        ! mfw_aacdecoder ! audioconvert \
        ! 'audio/x-raw-int,channels=2' \
        ! alsasink

Window 2 at 960:480

root@boundary ~/$ gst-launch filesrc location=kungfu.flv \
        ! 'video/quicktime' \
        ! aiurdemux name=demux demux. \
        ! queue max-size-buffers=0 max-size-time=0 \
        ! vpudec \
        ! mfw_isink axis-top=408 axis-left=960 disp-width=960 disp-height=408 demux. \
        ! queue max-size-buffers=0 max-size-time=0 \
        ! mfw_aacdecoder ! audioconvert \
        ! 'audio/x-raw-int,channels=2' \
        ! alsasink

Window 3 at 0:672

root@boundary ~/$ gst-launch filesrc location=kungfu.flv \
        ! 'video/quicktime' \
        ! aiurdemux name=demux demux. \
        ! queue max-size-buffers=0 max-size-time=0 \
        ! vpudec \
        ! mfw_isink axis-top=672 axis-left=0 disp-width=960 disp-height=408 demux. \
        ! queue max-size-buffers=0 max-size-time=0 \
        ! mfw_aacdecoder ! audioconvert \
        ! 'audio/x-raw-int,channels=2' \
        ! alsasink
Note that audio blending from the three videos also just works.

I know this begs for a video, but it’s late in the day and we thought this might be useful.

Tags:

14 Responses to “Playing multiple HD videos on i.MX6”

  1. Derek Dai January 31, 2013 11:04 am #

    How about gstreamer with X? Will it impact performance?

    If I wanna draw something on video (or blend GUI layer with video layer. eg. DVD menu, subpicture, video control interface), what kind of sink should I use? Thanks!

    • ericn January 31, 2013 11:16 am #

      Hi Derek,

      The IPU and V4L sinks use the YUV layer, which supports alpha-blending and color keying.

      There’s also a new GPU sink in the 01.01 package, though we haven’t spent any time with it. Check out the Multimedia docs in the release package this release page on i.MX Community for details.

      • Derek Dai January 31, 2013 11:25 am #

        Thanks. I know that v4lsink on i.MX5 can supports alpha-blending, but only one global alpha can be specified. What I want is per pixel alpha. Can i.MX6 do that?

        • ericn January 31, 2013 11:35 am #

          The YUV overlay hardware and drivers support per-pixel alpha, but I’m not certain about how/if you can invoke it using the gstreamer plugins.

  2. mach February 23, 2013 5:55 am #

    Hello,

    I’m trying this test on LTIB without X (http://boundarydevices.com/ltib-on-i-mx6-without-x/)

    root@freescale ~$ gst-launch filesrc location=kungfu.flv \
    > ‘video/quicktime’ \
    > aiurdemux name=demux demux. \
    > queue max-size-buffers=0 max-size-time=0 \
    > vpudec \
    > mfw_isink disp-width=960 disp-height=408 demux. \
    > queue max-size-buffers=0 max-size-time=0 \
    > mfw_aacdecoder ! audioconvert \
    > ‘audio/x-raw-int,channels=2′ \
    > alsasink

    But I’ve faced this message.

    MFW_GST_ISINK_PLUGIN 3.0.2 build on Dec 19 2012 10:56:54.
    BLN_MAD-MMCODECS_AACD_ARM_03.07.00 build on Jul 17 2012 17:47:46.
    MFW_GST_AAC_PLUGIN 3.0.2 build on Dec 19 2012 10:56:37.
    WARNING: erroneous pipeline: no element “video”

    Can I get some hints..?

    Thanks.

    • ericn February 23, 2013 9:45 am #

      You need some exclamation points as separators here.

      I suggest reading the Linux_Multimedia_Framework_User_Guide.pdf in the Freescale MMDOCS package for background.

  3. rcagley May 1, 2013 2:35 pm #

    I’m trying to use gstreamer with buildroot for the nitrogen6x board. Basic commands like
    gst-launch videotestsrc ! mfw_v4lsink
    don’t output to the display. Does anything besides configuring video displays in u-boot need to be done?

    • ericn May 1, 2013 2:39 pm #

      How did you get mfw_v4lsink installed? I believe some bits are closed-source,
      so you’d have to copy the shared library files.

      • rcagley May 1, 2013 2:45 pm #

        gst-fsl plugins is a package in buildroot…I just added it. Note I do have gpu-viv-bin-mx6q-12.0.01 installed. I did have to manually install that.

        • rcagley May 1, 2013 2:54 pm #

          on a lark I did this
          gst-launch videotestsrc ! mfw_v4lsink device=/dev/video0
          weird that video0 isn’t default…but that did the trick.

          • ericn May 1, 2013 3:10 pm
            #

            video0?

            The playback devices should be showing up as /dev/video16…

            root@boundary ~$ gst-inspect mfw_v4lsink | grep -A 3 device
            ...
              device              : V4L device name
                                    flags: readable, writable
                                    String. Default: null Current: "/dev/video17"
            

            Are you using mdev?

        • ericn May 1, 2013 3:08 pm #

          Cool!

  4. Tarek May 13, 2013 7:51 am #

    Hi Eric,

    What about 1920 x 1080 videos? I tried the same instructions with 3 x 1080p streams but it couldn’t play the 3rd one. Even when I lowered the frame rate to 7 fps. Is that what we expect from the HW?

    Thanks

  5. Mark June 25, 2013 1:00 pm #

    Hey, trying something along these lines, only going out to multiple displays. Works well until trying to have a single source and using a tee. Eventually, the sink on the hdmi gets choppy. Usually starts happening after 2 or 3 test runs. Dropping back to a single source/sink test, and things are fine, but until rebooting the test with the tee will be choppy until rebooting.

Leave a Reply