IMPORTANT: Blu-ray Authoring in Linux needs your help! It has come to my attention that UDF 2.5 support, which is required for authoring Blu-ray content is currently not supported at a writeable filesystem in Linux. The current UDF tools (discussed below) haven’t been maintained in years, and appear to be a dead project. In order to make Linux Blu-ray authoring a reality, we need to revive this project (udftools), and continue the development of UDF 2.5 and UDF 2.6 support. But!!! All is not lost. There are workable solutions, that range from commercial (yet still accessible) software, to hoping that your Blu-ray player doesn’t care about UDF 2.5 support any more than you do… like mine did.
In the wake of producing ‘Reco‘ this summer, I’ve taken on the responsibility of DVD and Blu-ray authoring for the film. DVD authoring on Linux has mostly evolved to the point of being within the reach of the average user, with a number of solutions for DVD authoring. Blu-ray on the other hand is a whole different story. Blu-ray authoring can be a very complicated process, especially when you get into menus and such. Lucky for me, we are only producing Blu-ray discs for exhibition as festivals, which means we don’t care about fancy menus. Just put the disc in, and go…
This post will chronicle my attempts at BD authoring towards that goal, keeping the entire workflow limited to OFS, and Linux as the authoring platform.
So far, I found these useful tools…
ffmpeg (to encode the streams, of course!)
tsMuxeR (to mux the streams and generate the BD folder structure)
mkudfiso (to generate an ISO of the BD folder structure)
udftools (to generate the UDF image, this is in the Ubuntu repos)
dvd+rw-tools (to actually burn the ISO to a BD-R disc).
First things first… export your movie at the highest quality possible. Spare no expense here, just keep it in a format that FFMPEG will like. In my case, I export from Piranha using the Quicktime plugin, which supports HUFFYUV (lossless), and 24bit, 48k audio.
Then using ffmpeg, we have to convert this to H.264. ffmpeg uses libx264, which has some nice presets to make things easy. For me, quality if everything, so time isn’t an issue. I’ll 2-pass, please. I made a little shell script to actually do the encoding for me.
ffmpeg -i $1 -f rawvideo -an -pass 1 -vcodec libx264 -vpre slowfirstpass -b $3 -bt $3 -threads 0 -passlogfile $2.log -y /dev/null
ffmpeg -i $1 -an -pass 2 -vcodec libx264 -vpre hq -b $3 -bt $3 -threads 0 -passlogfile $2.log $2.h264
ffmpeg -i $1 -acodec pcm_s24le $2.wav
I invoke it like this:
$ blu-ray_encode.sh bluray-movie.mov bluray-movie 36M
The first parameter is the input Quicktime that I created from Piranha. The second parameter is the output filename, minus extension, for the resulting H.264 stream and WAV file. The third parameter is the encoding bitrate, in this case 36Mb/s. Blu-ray supports allows upto 54Mb/s total, so this should give me a very high-quality encoding, but still afford me the bandwidth for the audio, and still not push everything to the max.
With this done, we should have two files, a raw H.264 stream, the corresponding 24bit WAV file.
Addendum, courtesy of Dkottmair (and edited courtesy of me):
In your x264-settings, the x264 guys recommend very specific settings for Bluray (or AVCHD) in order to be fully compatible on any player. For example there is one chipset used in players that cannot do weighted p-frames (–weightp option in x264), even though it should according to the standard (yes, I also have no idea how they got this chip approved then, but thanks to this shitty chip all Bluray- or AVCHD-H.264 now cannot use weighted pframes!). I’ve seen two players during my try-outs that seemed to have this chip, one was a Sony, can’t remember the other one. It causes a lot of block artefacts.
These are the settings they recommend for x264: –preset slow –tune film –weightp 0 –nal-hrd vbr –vbv-maxrate 15000 –vbv-bufsize 15000 –aud –keyint 24 –bframes 3 –slices 4 –level 4.1 –b-pyramid strict
You can add bitrate control via -b or -crf, and also add reference-frames: -r 6 for 720p and -r 4 for 1080p. Also, –vbv-maxrate should be 40000 and –vbv-bufsize 30000 for Bluray, 15000 is for AVCHD – but works on Bluray, too.
“This is all true to spec”, Dkottmair says, having done “a lot of research on the subject, even before x264 got BD-certified! ;-)” Also, it seems, a new site was put up specifically dealing with X264 and Bluray encoding. Check it out: http://www.x264bluray.com
Here’s the full three commands Dkottmair uses (third one needs to be launched in a separate terminal, sending it to the background using & will NOT work!):
mkfifo stream.y4m
mplayer -vo yuv4mpeg:file=stream.y4m -nosound (source-moviefile)
(separate terminal, same directory):
x264 –crf 22 –preset slow –tune film –weightp 0 –nal-hrd vbr –vbv-maxrate 15000 –vbv-bufsize 15000 –aud –keyint 24 –bframes 3 –slices 4 –level 4.1 –b-pyramid strict -o (output.264) stream.y4m
Dkottmair says: “Remember to add/change those parameters mentioned above as needed! And delete that y4m file afterwards, though it won’t eat much of your harddrive, since it always remains at 0 bytes! ;-)”
As a final tidbit, keep in mind that the y4m/x264 encoding won’t handle your audio. For real film/movie geeks, you won’t blink at this, but if you’re trying to burn home movies, you’ll still need to render out a seperate WAV file, which can still be done with the last ffmpeg command above.
Fire-up tsMuxerGUI, and import the files. Check the Blu-Ray radio in the output section, and specify a folder to generate the BD folder structure in. Double check the options under the “Blu-Ray” tab, for chapter creation, etc. When you’re ready, hit the “Start Muxing” button at the bottom of the window.
Before we dig into UDF creation in the next step, you should read this little comment, courtesy of Dkottmair:
“However, there’s one major issue with your usage of udftools: UDF 2.5. The problem is: UDFtools simply cannot create UDF 2.5 (or 2.6), which is a requirement for BD (or AVCHD) according to the standard. The maximum it does is UDF 2.01.
“This is also the reason why Brian gets that error on the PS3, it’s exactly what you see when the disc is not UDF 2.5, I know it, I’ve tried several times to burn working Blurays/AVCHDs with mkudffs-based OSS-tools such as Brasero or K3b! So when your discs play in your player, it is quite likely merely because your player doesn’t care what UDF-version the disc is in – Most players are built to be as compatible as possible, not to adhere 100% to the standard. Just like many DVD-players play DVDs that you burn in ISO instead of UDF…
“The only program I know (i also use it in my article) that can burn UDF 2.5 under Linux is Nero 4, which btw. works like a charm!) I’ve been bitching about these major shortcomings of mkudffs for ages now in several places (mailed lots of people, even filed a bug report for K3B: https://bugs.kde.org/show_bug.cgi?id=257602 ), but it appears nobody seems to care… Nonetheless, these tools claim in their changelog and release notes that they can “burn Bluray” – No. You can burn stuff onto BD-Rs using Bluray-Burners, but you CAN’T burn actual Blurays!”
With that out of the way, let’s look at this not-quite-solution that works for some players, but clearly not all… And remember, we need your help to drive continued support for UDF 2.5/2.6 in linux! Do your part! File a Bug!
Next, we need to make a UDF image file from the output of tsMuxeR. We have to create a UDF filesystem by hand, as a file on the regular system, mount it, copy the data into it, unmount, and then burn the resulting image.
Install udftools from the repos.
create a the file where the filesystem image will be stored:
mkudffs --vid="Blu-ray Movie" --media-type=hd --utf8 ./blu-ray.udf 11826176
Next, create, and mount that image as a file system. (PS, if you’re curious about the number ‘11826176’, its the number of 2k-blocks free on the BD-RE disc after formatting).
$ sudo mkdir /mnt/blu-ray
$ sudo mount ./blu-ray.udf /mnt/blu-ray -o loop
Copy the Blu-ray file structure created by tsMuxeR
$ sudo cp -R /path/to/bluray-content /mnt/blu-ray
Unmount:
$ sudo umount /mnt/blu-ray
This should create the contents of the Blu-Ray disc in the blu-ray.udf file. Now we can burn this to the BD-R/BD-RE disc.
Again, this shouldn’t take too long. Just be aware, throughout the muxing and mkudfiso steps, you are making copies of content which is probably very large. Expect hundreds of gigabytes to be consumed by this process, between the uncompressed MOV, the then compressed H.264 and WAV, the mux of those (which doesn’t recompress, only muxes them), and then a final copy of everything combined into the ISO. Essentially, you will have 3 copies of your film, NOT including the uncompressed version.
Once you’ve made the ISO, you should be able to burn it with growisofs. First use wodim to identify your BD burner:
$ wodim --devices
wodim: Overview of accessible drives (1 found) :
-------------------------------------------------------------------------
0 dev='/dev/scd0' rwrw-- : 'HL-DT-ST' 'BD-RE GBW-H20L'
-------------------------------------------------------------------------
In my case, my BD-RE is /dev/scd0.
Before we do the burn, we need to format. Supposedly, ‘growisofs’ will format for us on-the-fly, but its not perfect and we’ll likely see an error and the disk won’t finalize. So, we use dvd+rw-format to to the format first:
$ dvd+rw-format -ssa=default /dev/scd0
Then I’ll do the actual burn using ‘growisofs’ with the following command:
$ growisofs -dvd-compat -Z /dev/scd0=blu-ray.udf
Word on the street is, you can also use Nero to create/burn the UDF, though its not OFS, but it is relatively cheap (assuming it does the job).
There is room for improvement in this whole process…
The UDF image is created to a fixed size, based on the size of the BD-RE media. In fact, your movie (or other BD content) may be less than that. Ideally, we’d only like to burn the size of our data, which in this case, we won’t know until after we encode and mux. We could figure the number of blocks required (UDF’s default is 2048b-per-block), by dividing the resulting space required by the BD content after muxing/generating the folder structure by 2048, rounding up, and allocating the UDF filesystem to that number of blocks.
With the use of mkudffs and udftools, you should be able to create and mount the udf image file before running tsMuxeR, and then have tsMuxeR render directly into the UDF mount. Then unmount, and burn the image. The downside is going this route, you don’t know what size to make the image before doing the mux. You could make an educated guess: (size of WAV + size of H264 * 1.10), but regardless, you’ll end up with wasted space.
There is still a real need for a tool like mkudfiso, that can take existing data and stuff it into a UDF file. I would encourage anyone capable and knowledgeable, to consider submitting a patch to the linux-udf/udftools for a tool that provides the same functionality as mkudfiso, but that which actually works, and is part of the standard implementation.
udftools, and mkudffs appears to allow creating the UDF directly to the media, though the documentation is poor. In theory, you should be able to mkudffs to /dev/scd0, mount, copy and unmout, but its unclear in the documentation how to actually start and finalize the burn. I didn’t have the patience to continue investigating.
UPDATE: In the spirit of awesomeness, I spent the better part of today putting all of this into a nice, slick, and hopefully reliable script. Copy/paste the text below into a file “mkbdmovie”, put the file into /usr/local/bin (or /usr/bin), and chmod +x the file, it should mostly do the rest for you. It won’t install ffmpeg or udftools, or download tsMuxeR, but it will tell you when you don’t have them, and tell you how to get them. It even has some very basic usage if you run it without parameters. A couple of notes that aren’t mentioned in the usage:
If you don’t specify a movie with -m, you can specify -a and -v to pass the already encoded elements to the muxer, in which case it won’t re-encode the H264/PCM, only remux . If you like, you can specify -a and -v in addition to -m, and -a and -v will specify where to render the H.264 and PCM assets, so they can be saved and reused again if you have to remux everything again later.
In order for the script to mount and copy the BDMV folder structure into the UDF image, mount and cp have to be ran via sudo. It will prompt you for your password at that point. If anyone has a solution to that, I’d love to include it in the script.
The script figures the size of the blu-ray folder structure after muxing, and creates the UDF image just large enough to hold that data (per my “room for improvements” above).
The script creates a meta file used by tsMuxeR to determine how to mux everything, including specifying chapter locations. I used some stupid values. I meant to add another parameter to allow specifying them on the command line, but then I got bored. If you want different chapter locations (times) (which you most likely will) you’ll need to modify the script to reflect the new times. It should be pretty self-evident what needs to be changed. If you really want to mix things up, you can run the tsMuxerGUI to find/set all of the parameters you like, and then copy the metadata information into the script.
Finally, the script only creates an UDF file (specified as the last parameter on the command line) from an existing movie. You still need to render to movie out of your NLE/finishing tool, and also run the growisofs command above to do the actual burn.
#!/bin/bash
FFMPEG=`which ffmpeg`
TSMUXER=`which tsMuxeR`
MKUDFFS=`which mkudffs`
function usage {
echo "The arguments to use are"
echo "-m: The movie file to encode (ffmpeg compatible)"
echo "-v: The h.264 video asset to mux for BD content"
echo "-a: The WAV audio asset to mux for BD content"
echo "-b: The bitrate for h.264 video encoding (-m), default is 25M"
exit
}
function test_ffmpeg_x264 {
# Test whether ffmpeg has libx264 support compiled in
if [ `ffmpeg -formats 2>/dev/null| grep x264 | cut -c 3-4` != "EV" ]; then
echo "FFMPEG not compiled with libx264 support." | exit
fi
}
function test_ffmpeg {
if [ ! -x $FFMPEG ]; then
echo 'Could not find FFMPEG in the path. Try "sudo apt-get install ffmpeg".' | exit
fi
}
function test_tsmuxer {
if [ ! -x $TSMUXER ]; then
echo 'Could not find tsMuxeR in the path. Download from http://www.smlabs.net/tsmuxer_en.html' | exit
fi
}
function test_mkudffs {
if [ ! -x $MKUDFFS ]; then
echo 'Could not find mkudffs in the path. Try "sudo apt-get install udftools".' | exit
fi
}
function test_dependancies {
echo 'Verifying dependancies...'
test_ffmpeg
test_ffmpeg_x264
test_tsmuxer
test_mkudffs
}
function make_bluray_streams {
#convert the movie to HQ H.264 and WAV
echo "Encoding ${MOVIE} video to H.264, 1920x1080, ${BITRATE}bps - Pass 1"
$FFMPEG -i $MOVIE -s hd1080 -f rawvideo -an -pass 1 -vcodec libx264 -vpre slowfirstpass -b $BITRATE -bt $BITRATE -threads 0 -y /dev/null >>mkbdmovie.log 2>&1
echo "Encoding ${MOVIE} video to H.264, 1920x1080, ${BITRATE}bps - Pass 2"
$FFMPEG -i $MOVIE -s hd1080 -an -pass 2 -vcodec libx264 -vpre hq -b $BITRATE -bt $BITRATE -threads 0 $H264_FILE >>mkbdmovie.log 2>&1
echo "Encoding ${MOVIE} audio to PCM, 24bps, 48000 - Pass 1"
$FFMPEG -i $MOVIE -acodec pcm_s24le -ar 48000 $WAV_FILE >>mkbdmovie.log 2>&1
}
function mux_bluray_assets {
echo 'Muxing streams and generating BDMV file structure'
# create the metafile needed by tsMuxeR
echo "MUXOPT --no-pcr-on-video-pid --new-audio-pes --blu-ray --vbr --custom-chapters=00:00:00.000;00:05:00.000;00:10:00.000;00:15:00.000;00:20:00.000;00:25:00.000 --split-size=2GB --vbv-len=500" > $TSMUXER_META
echo "V_MPEG4/ISO/AVC, \"$H264_FILE\", fps=23.976, insertSEI, contSPS, ar=As source" >> $TSMUXER_META
echo "A_LPCM, \"$WAV_FILE\", lang=eng" >> $TSMUXER_META
# mux the two files and generate the BR-structure
$TSMUXER $TSMUXER_META $BDMV_PATH >>mkbdmovie.log 2>&1
}
function create_udf_image {
echo "Creating UDF Image: ${UDF_IMAGE}"
# calculate the UDF size necessary (as 2k blocks) to fit the data
UDFSIZE=`du -s -B 2K $BDMV_PATH | cut -f 1`
UDFWORKSPACE=`mktemp -d /tmp/UDF_XXXXXXXXX`
# make the udf filesystem, and mount it
if [ -e $UDF_IMAGE ]
then
rm $UDF_IMAGE
fi
#MKUDFCMD="/usr/bin/mkudffs --vid=\"$3\" --media-type=hd --utf8 \"$2\" $UDFSIZE"
$MKUDFFS --media-type=hd --utf8 $UDF_IMAGE $UDFSIZE >>mkbdmovie.log 2>&1
sudo mount $UDF_IMAGE $UDFWORKSPACE -o loop >>mkbdmovie.log 2>&1
# Copy the source into the UDF Workspace
sudo cp -Rv $BDMV_PATH/* $UDFWORKSPACE >>mkbdmovie.log 2>&1
#unmount, cleanup
sudo umount $UDFWORKSPACE >>mkbdmovie.log 2>&1
}
function cleanup {
sudo rm -rf $UDFWORKSPACE $TSMUXER_META >>mkbdmovie.log 2>&1
if [ $TEMP_H264 = 1 ]; then
sudo rm -rf $H264_FILE >>mkbdmovie.log 2>&1
fi
if [ $TEMP_WAV = 1 ]; then
sudo rm -rf $WAV_FILE >>mkbdmovie.log 2>&1
fi
}
MUX_ONLY=1
TEMP_H264=1
TEMP_WAV=1
BITRATE=25M
TMP_FILE=`mktemp -u /tmp/BDMV_XXXXXXXXXX`
TSMUXER_META=$TMP_FILE.meta
BDMV_PATH=`mktemp -d /tmp/BDMV_XXXXXXXXXX`
if [ $# -le 1 ]; then
echo "Invalid or insufficient parameters."
usage
fi
while [ $# -gt 1 ]
do
case $1
in
-m)
MOVIE=$2
MUX_ONLY=0
H264_FILE=$TMP_FILE.h264
WAV_FILE=$TMP_FILE.wav
shift 2
;;
-b)
BITRATE=$2
shift 2
;;
-v)
H264_FILE=$2
TEMP_H264=0
shift 2
echo "Using h.264 file: ${H264_FILE}"
;;
-a)
WAV_FILE=$2
TEMP_WAV=0
shift 2
echo "Using WAV file: ${WAV_FILE}"
;;
*)
usage;
;;
esac
done
UDF_IMAGE=$1
test_dependancies
if [ $MUX_ONLY = 0 ]; then
make_bluray_streams
fi
mux_bluray_assets
create_udf_image
cleanup
Posted in Film/Video, Geekdom, Softech
Tags: authoring, BD, BDMV, Blu-ray, burn, ffmpeg, growisofs, linux, tsmuvergui, tsmuxer, video, x264
