HVX/P2/MXF Media in Linux

I’ve been doing some transcoding lately for a client, who shot some industrial footage on an HVX200 and wanted it transcoded to an alternate format for editing (I guess they didn’t have DVCPROHD codecs). Anyway, it took a bit of figuring out, but here is what I found out.

FFMPEG supports MXF, but apparently not the packaged MXF files that are stored on the P2 cards. There is a handy C++ library and accompanying tools from http://www.freemxf.org, that can do a handful of operations on MXF files.
It’s not terribly intuitive, but apparently, if you use the ‘mxfsplit’ command on the MXF files in the /CONTENTS/VIDEO folder, it will generate a subsequent MXF ‘stream’ that, counter-intuitively contains the video and all the audio streams in a single MXF file, which can then be processed by FFMPEG. Unfortunately, mxfsplit doesn’t let you specify an output filename; it generates one from the original MXF filename that is anything but, well, intuitive (I like that word today, apparently). So I wrote up a quick script to convert a named MXF file into a FFMPEG compatible MXF file (with the same name). You can then use the ‘find’ command to process all the files on your media. I recommend making a backup before you do any of this.

The first thing you’ll need to do is go download the MXFLIB package from FreeMXF (http://www.freemxf.org/).  Download, untar, and do the usual build stuff:

unzip mxflib-beta-1.0.1-gcc4.3-v2.tar.gz
tar -xvf mxflib-beta-1.0.1-gcc4.3-v2.tar
cd mxflib-beta-1.0.1-gcc4.3-v2/
./configure
make
sudo make install

Once we have that, we need the following script, which will call mxfsplit with the path to the video file to convert, and output a new MXF stream, which will contain all the audio as well.  The script will move/rename the stream file, to a more friendly name/location.

vi mxf-fixit.sh

Below is an updated script… an unknowning collaboration as a result of a number of comments, and my own attempt to re-use my original script…  The following should be a lot more robust.

Paste the following into vi:

#!/bin/bash
for i in $1/VIDEO/*.MXF
do
STREAM=`mxfsplit -m $i | awk '{ FS="( |=)"; if ( $0 ~ /File=/ ) { print $6 } }'`
OUTPUT="`basename $i .MXF`"
mv $STREAM $OUTPUT.MXF
done

Don’t forget to mark the script as executable:

chmod +x ./mxf-fixit.sh

Pass as the one and only argument to the script, the location of your CONTENTS directory of your MXF tree.  It will make a NEW COPY of your MXF files, located in your current working directory (and will NOT replace/modify your originals).

./mxf-fixit.sh /media/P2MEDIA/CONTENTS

Make sure you have enough free disk space for a copy of all your media.

Once that is finished, you should have a new batch of MXF files that are readable by FFMPEG.  From here you can do any transcoding necessary.  If you are particularly sharp, you might consider modifying the script above to do your ffmpeg transcoding on the fly (instead of calling “mv $i…”  you’d call “ffmpeg $i -target ntsc-dvd $OUTPUT.mp2”).

ffmpeg -i myp2media.mxf -target ntsc-dvd p2media.mp2

or play directly via ffplay:

ffplay myp2media.mxf
Advertisements

~ by kylemallory on May 28, 2010.

18 Responses to “HVX/P2/MXF Media in Linux”

  1. I’d love to be able to try the script. Any timeline for the code left at home? 🙂

    Looks promising.

    • Ha! Thanks for the reminder. Apparently, the script is on the drive with the footage, that’s been filed. In the meantime, here is what I was able to piece back together going through my shell history:

      #!/bin/bash
      STREAM=`mxfsplit -m $1 | grep “File=” | cut -c 31-52`
      mv $STREAM $1

      I seem to remember the script being slightly more complicated, but running that on some other P2 media I have here works for me.

      NOTE: THIS WILL OVERWRITE THE ORIGINAL MXF FILES, so make sure you have a backup.

      • I’ll look into that. Currently I’m creating read-only disk image from the P2 cards, so I need to either write out to a new image or make additional working copies.

        Short story? I want dailies from P2 cards. Shooting on a Panasonic that has proxies isn’t always a given and I haven’t found a straight-forward solution. Knowing the power of FFMpeg, I think it’s the best bet, but I’m just starting to discover all of the intricacies of MXF and the Panasonic implementation.

        Have you looked it opening up a stream for FFMpeg to access, but not writing it to disc?

        Thanks for the notes.

  2. Hey Wes, sorry. Somehow, I either don’t get notified of half of the comments posted, or the email notifications get filtered by my email spam filters or such.

    Dailies for MXF would be nice, but I don’t do enough with the P2 media to devote much time to perfecting a workflow. The limitation I see with the streaming, is that the mxfsplit doesn’t seem to offer any kind of output options (not even specifying the filename, let alone redirecting it to a pipe or something). Source code is provided with the MXF utilities, so if you were really motivated, I’m sure you could whip up a custom MXF tool that pushed the stream into FFMPEG (or ffplay).

  3. I get the error:

    mv: manca l’operando per il file di destinazione dopo “0001DY.MXF”
    (missing operator for dest file after $1)

  4. I got the same error so I changed the script to read:

    mv *.Stream “`basename $1 .MXF`converted.MXF”

    that makes new files with the word “converted” added just before the extension in the directory where the script is ran from. I ran it in my home directory and it worked. I will be happy to help anyone who needs more help but to avoid spam I can’t give my email out here. Please use the contact form on my multimedia company’s website at http://burningtokenmedia.com/contact. I see all messages sent there.

    I hope this comment helps someone. I’ve been searching on and off for over a year for this solution and today’s the day!

    Cheers,
    Loop Duplicate

  5. Oh Yeah,

    Thanks so much for this blog post. You are awesome!!!!!!!!!!!!

    Loop

  6. I tried using your technique, but I’m getting hung up on audio. My P2 data looks like this:

    `-- CONTENTS
        |-- AUDIO
        |   |-- 0004FW00.MXF
        |   |-- 0004FW01.MXF
        |   |-- 0004FW02.MXF
        |   |-- 0004FW03.MXF
        |   |-- 0004FW04.MXF
        |   |-- 0004FW05.MXF
        |   |-- 0004FW06.MXF
        |   `-- 0004FW07.MXF
        |-- CLIP
        |   `-- 0004FW.XML
        |-- ICON
        |   `-- 0004FW.BMP
        |-- PROXY
        |-- VIDEO
        |   `-- 0004FW.MXF
        `-- VOICE
    

    If I do a

    find ./CONTENTS -name "*.MXF" -exec /tmp/mxf-fixit.sh {} /tmp/fixed \;
    

    I get 9 new files, 8 audio and 1 video. But ffmpeg doesn’t see an audio track in the video file.

    What am I missing?

    • Jason, use: find ./CONTENTS/VIDEO -name “*.MXF”

      Notice the /VIDEO on the end. It seems that the mxf-tools will dereference the audio from the video, all by itself.

      –Kyle

  7. Thanks for the reply, Kyle. I just tried it again, and still nothing. I wonder if we’re dealing with a different version of mxflib? My mxfsplit doesn’t have a -m option, which your shell script relies on. I’ve got version 1.0.1 of mxflib…

  8. It does appear that the current mxfsplit does not have the -m option; instead it looks like it should be -w:1, as in

    mxfsplit -w:1 foo.MXF

    But even that doesn’t work for me — it doesn’t seem to have any awareness of the audio files.

  9. I do it in one script now. Without mxffixit.sh, I don’t have to have an EXT4 drive to convert videos.
    cd /media/temppp/scar_pink/mxffix/
    for i in /media/temppp/scar_pink/p2/CONTENTS/VIDEO/*.MXF
    do
    STREAM=`mxfsplit -m $i | grep “File=” | cut -c 31-52`
    mv *.Stream "`basename $i .MXF`converted.MXF"
    done

  10. Just want to say thanks – this helped a lot in trying to convert some MXF files from a HVX200 into something I could actually use.

  11. For anyone following the comments… I updated the mxf-fixit.sh script to use awk instead of grep/cut to extract the name of the output stream. It should be a bit more reliable now… hopefully.

  12. Thanks for this thing, it definitely saves me a lot of time in getting useful data from my P2HD camera.
    There is however still an error in your script:

    mv $i $OUTPUT.MXF
    should be
    mv $STREAM $OUTPUT.MXF

  13. Thanks for this information. But I have a small problems. The video playback from VLC or ffplay run too fast… Is there a way to deal with it when making the mxf file ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: