“21. Review and Comparison of FFV1 versus Other Lossless Video Codecs for Long-Term Preservation” in “Sustainable Audiovisual Collections Through Collaboration” | Open Indiana

Review and Comparison of FFV1 versus Other Lossless Video Codecs for Long-Term Preservation

Peter Bubestinger-Steindl

Abstract

For audio preservation, the codec question is resolved, but for video which lossless encoding to use is still in discussion. This paper shall give insight based on real-life practical experiences about what it is like to use FFV1, compared to other lossless codecs. Most of it is based on the work at the Austrian Mediathek, mainly due to its role as early adopter of FFV1.

Topics addressed are performance comparison (speed/compression), accessibility, interoperability, sustainability, standardization, transcoding and nonlinear editing, as well as current and future application/vendor support.

Keywords

lossless, video, encoding, codecs, compression ratio, FFV1, JPEG2000, Dirac, H.264, uncompressed, NLE, transcoding.

Introduction

At the end of 2009, the Austrian Mediathek (national audio/video archive)¹ was planning to start digitization of their tape-based video collection, consisting mainly of VHS and DigiBeta. The demand was not to use any lossy compression for the digital archival copy.²

At that time, the only two valid candidates for storing video in such a way were either uncompressed or JPEG2000-lossless. Uncompressed was not an option because the size of uncompressed material—even for PAL SD material (720 × 76, 8 bits-per-component)—has a noticeable impact on the requirements on hard-disk space and network-throughput on all machines that work with the material. This would have greatly exceeded the available budget.

We asked other institutions that were already using JPEG2000-lossless for preservation about their experiences. None of them were working with the JPEG2000-lossless (in MXF container) files directly—mainly because this format combination was supported by few applications or because there were severe interoperability issues.

Since eternal migration should be considered when choosing a format for long-term preservation, the effort of getting in and out of a format plays an important role. We continued searching, therefore, for alternatives and found FFV1 in a lossless-codec comparison from the Graphics and Media Lab at MSU.³ FFV1 was originally written by Michael Niedermayer as part of the FFmpeg project, leading to its name: FFmpeg Video codec 1.

The Mediathek then contacted Niedermayer, and hired him for implementing improvements such as multithreading and CRC checksums in the bitstream for additional error resilience. Dave Rice’s contributions were also very vital to the success of this undertaking.

The result was FFV1 version 3 (FFV1.3), which was extensively tested⁴ and officially released in August 2013.

Performance Comparison

In order to compare the performance of FFV1 in terms of speed and compression ratio, four lossless codecs were selected for a test setup⁵:

• JPEG2000-lossless

• H.264-lossless

• Dirac-lossless

• FFV1

Although more than these lossless codecs exist, these four were chosen due to their possible suitability for long-term preservation. Other codecs were omitted, due to certain properties:

• Proprietary licensing

• Limited colorspace options

• Limited subsampling options

• Insufficient compression

• Insufficient speed

The test-setup was to encode an uncompressed YCbCr video source to these lossless codecs, and then decode back. In order to confirm the mathematically losslessness of the encoding/decoding procedure, FFmpeg’s “framemd5”⁶ functionality was used.

The three codecs—JPEG2000, H.264, and Dirac—only have lossless as an option. The main use-case for these formats in practice is lossy. Attention must be paid to avoid accidentally running them in lossy mode; many applications that support these formats are usually set and tested for lossy mode. Due to the popularity of lossy encoding, many applications do not even implement the lossless mode, do it incorrectly, or are not tested, as well, as some of our other tests have shown.

JPEG2000-Lossless

The implementation used was “libopenjpeg.”⁷ At the time of testing, it did not support multithreading. There is also a JPEG2000 implementation available directly in FFmpeg, but it did not implement lossless mode correctly and could therefore not be used. In previous tests, Morgan “M-JPEG2000” codec package⁸ was used to create the JPEG2000 lossless files. With these files, the framemd5 validation was negative. Due to the fact that Morgan’s implementation is proprietary, we were unable to investigate what caused this mismatch. All we can say is that losslessness could not be verified. Therefore we chose libopenjpeg, which allows us to inspect and adapt the code if necessary.

H.264-Lossless

Nowadays, H.264 is omnipresent as well as widely supported by many applications and embedded devices. Yet, even though the H.264 standard contains a lossless mode, it is rarely implemented. The FFmpeg documentation explicitly states:

“Most non-FFmpeg based players will not be able to decode lossless …, so if compatibility is an issue you should not use lossless.”⁹

The reliability of this source can be assumed, because one of the developers of FFmpeg was Jason Garret-Glaser,¹⁰ the lead developer of “libx264,”¹¹ the Free Software implementation of H.264/MPEG-4 AVC. Most H.264 files currently existing are produced using “libx264”.

Dirac-Lossless

The BBC developed the wavelet codec “Dirac”¹² as open source from the beginning and standardized it as “VC-2” (SMPTE 2042–1:2009). The implementation used for these tests was “libschroedinger.”¹³ At the time of testing, it did not support multithreading. Although BBC did a great job with Dirac, its performance regarding speed as well as compression ratio is inferior to FFV1 and Dirac is less widely adopted than JPEG2000—both lossy as well as lossless.

Figure 1. Codec speed / size comparison chart.

Test Results

The environment these tests were run on was as follows:

• CPU: Intel(R) QuadCore(TM) i7–2600K CPU @ 3.40 GHz

• RAM: 8 GB DDR3 (1333 MHz)

• Disk: Intel SSDSA2CW080G3 (SSD)

• OS: GNU/Linux (Xubuntu 12.04.1, 64 bit)

• Tool: FFmpeg (version git N-59183-g3e62654)

The results of these tests show that the compression of FFV1 is almost equal to that ofJPEG2000-lossless, and noticeably smaller than H.264-lossless and Dirac-lossless. Here are two examples for common compression rates of FFV1 as observed in real-life usage andcompared to uncompressed. The size includes video (SD-PAL, 4:2:2, 8 bpc) and audio (PCM, 16 bit, stereo):

• Uncompressed: 1197,51 MB/min

• VHS: ca. 370 MB/min

• DigiBeta: ca. 650 MB/min

The speed improvements for encoding with FFV1.3 over FFV1.1 are also clearly visible in figure 1. On the ingest machines used at the Austrian Mediathek, FFV1.1 used around 70% CPU load, while the same material on the same machine had more than 90% CPU load on all four cores when running JPEG2000-lossless encoding.

Additional information, as well as the detailed parameters used for encoding/decoding of each codec can be found in the logfiles online.¹⁴ Impact of different FFV1 encoding parameters on size and speed can be seen in the results of the tests we ran for the development of FFV1.3.¹⁵

Accessibility and Interoperability

At this writing, the majority of applications and devices supporting FFV1 natively do so by using the FFmpeg/LibAV program libraries in their applications. Other applications that support system-codecs can handle FFV1 too, with appropriate codec-packages, such as ffdshow-tryouts¹⁶ or LAVFilters¹⁷ on Windows operating systems. Since FFV1 originated as part of the FFmpeg project, it has been available under a Free Software (open-source) license since the beginning.

The fact that the FFmpeg/LibAV libraries are very popular and used by a large number ofapplications, poses a different scenario that one usually encounters: with FFV1, any private developer, as well as companies, can use the exact same implementation of that codec. There is no necessity of writing one’s own.

With other formats, the paper-standard specifications might be open, but implementations are proprietary, possibly containing slight differences between vendors or program versions. The only times where interoperability issues with FFV1 were encountered, were if an application used an older version of FFmpeg/LibAV libraries that was written before August 2013 (the release date of FFV1.3). These issues usually resolved themselves automatically as soon as a new version of the application built for a more recent version of FFmpeg/LibAV libraries was released.

Sustainability

When first choosing FFV1, other institutions asked about sustainability concerns. The cases to be considered are:

• Decoding FFV1 in the future

• Different versions of FFV1

FFV1’s Free Software (open-source) license, guarantees that it is allowed to “use, study, share & improve” the code now and in the future. Given these rights, archiving the source code of FFmpeg/LibAV is the digital equivalent of archiving the schematics—including building-components—of an analogue record/replay machine. So even if FFV1 might become unsupported by applications in the future, any institution or private individual that wants to access or transcode files encoded with FFV1 can do so without artificial restrictions. This means that anyone could then hire a developer to adapt the code to run under the then current circumstances—including any technology not yet developed or known.

Basically, this applies to any format where the proper implementation is available under a Free Software license. Additional concerns regarding licensing or patenting issues were also raised by some. These were addressed as part of the PREFORMA project (see “Standardization” below). The conclusion was that there are no patents known that might be related to FFV1. Even if there were, they would only apply to distributed application binaries—not the source code. The situation is the same that applies to the popular player VLC, for example.

So even in such a worst case scenario, an institution could legally build and use FFmpeg, for example, to then transcode from FFV1 to any other format. As the speed comparison charts above show, the times and computing resources needed for such a process are considerably less than using any other comparable lossless codec.

Standardization

In 2014 the PREFORMA Project¹⁸ started. Its goal is to have conformance-checking tools for long-term preservation file formats implemented. A result is “MediaConch,”¹⁹ a tool for checking specification conformance of FFV1 and Matroska (MKV), as well as validating given format property policies (resolution, subsampling, etc.) of audiovisual files. MediaConch is being developed by “MediaArea,”²⁰ who are well known for “MediaInfo,”²¹ a tool commonly used in audiovisual archives. As part of MediaConch, efforts are currently underway for standardizing FFV1 over the “Internet Engineering Task Force” (IETF).

IETF was chosen as standardization entity for several reasons:

• The discussion process is completely transparent, and mailing lists are archived for public review later on.

• Anyone can be invited to join the process.

• Only individuals, no companies, are allowed within the standardization process, which removes negative influence due to any commercial conflict of interest.

• The resulting standards paper is then available to anyone free of charge.

In order to provide a full package for audiovisual preservation, the IETF standardization roadmap currently includes the following:

• Video: FFmpeg Video Codec 1 (FFV1)

• Audio: Free Lossless Audio Compression (FLAC)

• Container: Matroska (MKV)

The intention is to have standardization for every one of these three components, but without users being required to use them in that combination. Since the standardization is work in progress right now, interested parties are invited to join the “CELLAR” working group.²²

“CELLAR” stands for “Codec Encoding for LossLess Archiving and Real-Time Transmission” and was founded by MediaArea as part of the PREFORMA project.

Transcoding and Nonlinear Editing

The current status quo is that applications that support system-codecs, as well as any application that uses FFmpeg/LibAV libraries, can handle FFV1. The Austrian Mediathek is using “Kdenlive”²³ (Linux) and “VirtualDub”²⁴ (Windows) for minor editing purposes. This has shown that direct editing of several tracks of SD material over a 1 GBit ethernet can be done without problems.

Given a 1-GBit ethernet, assuming 80 MB/s data transfer rate and video material with around 650 MB/min, this allows at least seven concurrent streams. With FFV1 from analogue VHS (370 MB/min), twelve concurrent streams would be possible before saturating the network bandwidth. Although popular editing suites such as AVID or FinalCut do not support FFV1 yet, these experiences show that it is technically absolutely possible to do nonlinear editing on FFV1 over a local network with moderate hardware requirements.

Current and Future Application/Vendor Support

There is no technical restriction preventing professional vendors from including FFV1 support if they want to. For example, if AVID, FinalCut or Premiere included FFV1 support, one would be able to have a seamlessly lossless production workflow. It would omit the necessity for requiring a separate lossy copy for editing.

FFmpeg/LibAV program libraries are commonly used in hundreds of programs, open source, as well as proprietary ones. This can be seen as a proof of concept, showing that if programmers can implement FFV1 support so easily that they do it in their spare time, it can be assumed that it is easily possible for other vendors, too.

With FFV1 gaining more momentum in the video preservation domain, interest for vendor’s support may be increased. In terms of future improvements and additional features to FFV1, the intention is apply the basic principle of a minimalistic standard: “As complicated as necessary, but as simple as possible.”²⁵ This should make future support easier and keep interoperability issues down to a minimum, although most of them are only temporary by nature—as explained above.

Lossless Workflows

The Austrian Mediathek has used FFV1 productively as preservation format since mid-2011. The relevant steps in their workflow in terms of experience with lossless FFV1 encoded files are as follow:

• Ingest directly to FFV1

• Quality control (QC) on archive copy directly

• Automated mass transcodings for access copies, web, etc.

• Nonlinear video editing directly on FFV1

Except for ingest and QC, most of the automated transcodings are done using FFmpeg binaries on the command-line on GNU/Linux servers. For managing the whole workflow from tape-ingest to archive, the Mediathek uses “DVA-Profession”²⁶—an ingest workflow management and automation tool, designed mainly for preservation archive use cases, and released under a Free Software license (GPLv3). Although there is room for improvement, this system makes use of FFV1 from beginning to end—with very satisfying results.

For other use cases or institutions, it might be desirable to have more vendors supporting FFV1, as well as joining resources to have more end-user applications with graphical user interfaces created, or improving existing ones. It can be said, that the example of the Austrian Mediathek choosing and improving FFV1 in collaboration with other archives and companies has shown that open-source licenses allow users to influence and shape their software landscape in a very effective and cost-efficient way.

Conclusions

The experiences of more than five years of actively working with FFV1 on a daily basis confirms that using FFV1 was a good choice. It also allowed other institutions already to start digitization of their analogue material in a way that is accessible and highly interoperable. The properties of FFV1 also facilitate collaboration of institutions, as all improvements and experiences can be shared without artificial restrictions. The fact that the number of institutions choosing FFV1 as preservation codec, as well as their positive feedback when working with it, confirm the usefulness of this format.

Summing up the most important factors:

• FFV1 is more than four times faster than most J2K-lossless implementations.

• There is a wider choice of applications supporting FFV1 than J2K-lossless.

• Direct editing is currently not yet possible in most proprietary NLEs.

• There is increasing demand for GUI applications to handle FFV1.

• Sustainability of every version in existence is guaranteed by its Free Software license.

• There are no artificial restrictions getting in and out of FFV1.

PETER BUBESTINGER-STEINDL is an independent consultant and developer, specializing in professional open-source solutions for long-term preservation needs and lossless audiovisual archiving workflows. He studied Media Computer Science at the Technical University in Vienna and has worked as project leader and developer in the field of digital preservation since 2002. As employee of “NOA Archiving Solutions,” he set up and developed solutions for different broadcasting archives around the globe.

Notes

1. http://www.mediathek.at/.

2. http://dva-profession.mediathek.at/fileadmin/MEDIASERVER/dva-profession html/documentation/projekt-motivation/index.html.

3. http://compression.ru/video/codec_comparison/lossless_codecs_en.html.

4. http://download.das-werkstatt.com/pb/mthk/ffv1_stats/latest/.

5. http://download.daswerkstatt.com/pb/mthk/info/video/comparison_video_codecs_containers.html#codec_tests.

6. http://dericed.com/2012/reconsidering-checksums-published-in-iasa-journal/.

7. http://www.openjpeg.org/.

8. http://www.morgan-multimedia.com/morgan/php/products.php?sProductId=5.

9. https://trac.ffmpeg.org/wiki/Encode/H.264#LosslessH.264.

10. http://www.x264.nl/developers/Dark_Shikari/CV.pdf.

11. http://www.videolan.org/developers/x264.html.