Singularity on YouTube: HDR, SDR and lots of LUTs

Singularity, the 17th Blender Open Movie, was graded and mastered in HDR. When we were working on publishing the film, the goal was to have a single YouTube video that could be viewed on both SDR and HDR screens, and look breathtaking in both cases. And as you now know, this plan worked!

The rabbit hole begins!

Note

Grading and delivering Singularity in 4K HDR would not have been possible without the long list of HDR workflow improvements shipped in Blender 5.0. Read more about it in the dedicated article and Blender 5.0: Color Management release notes.

Singularity was test-uploaded on YouTube well in advance of the online premiere and the team were happy with the look: yes, there were differences between the different players and browsers compared to Blender, but an untrained eye would not have caught it. And then, a few days before the premiere our studio machines were updated to newer KDE, drivers, browsers, and maybe more. This was the moment when artists noticed the critical issues in the SDR version of upload: it looked vastly different across browsers, colours were desaturated, and black levels were off.

Washed-out look of the frames from the first YouTube upload

Note

Here and later in the article the frames are saved on a Windows laptop, Chrome web-browser, looking at fully processed HDR video upload. Hack: right-click on the video, then right-click again to bypass Youtube's context menu and get the browser's context menu which has the "Save Video Frame..." option. The screenshot corresponds to how the frame looked in the in-browser player.

After first noticing these issues on Linux, we compared the look on other operating systems. The results were all over the place, with most systems showing serious problems in the SDR version. It probably depends on the exact browser version, OS, and other factors. We had to fix the issue and act fast: the online release date was approaching, and we wanted to be sure Singularity would be presented in the best state possible.

YouTube HDR upload recipe

HDR video uploads to YouTube come with some restrictions on the frame rate, container, codec, etc. For Singularity we used the following:

• Container: MP4 (later Matroska)
• Codec: H.265
• Color depth: 12 bits
• Color primaries, EOTF: Rec.2020, Rec.2100 PQ.

The video is also expected to include proper metadata (explicit color primaries, white point, and so on). Fortunately, artists no longer need to worry about this: thanks to Blender 5.0's HDR pipeline and encoding improvements, Blender now outputs HDR videos with all required metadata automatically.

YouTube converts HDR to SDR using predefined algorithms that are neither clear nor explicitly documented. You still have some control through a custom lookup table (LUT) attachment, which maps colors from HDR to SDR. Check Youtube's Upload High Dynamic Range (HDR) videos documentation for more details.

Video color range

The one thing that, in foreshadowing, is important: the YUV color range. It is unclear whether Youtube has strong preference for it, but, in any case, Blender outputs what is called Full (interchangeably called "PC") range. There is also a so-called Limited (or "TV", sometimes also called "Broadcast") range.

These ranges relate to the raw values in the video that correspond to pure black and pure white. Without going too deep into details, the way to think about it is that Full/PC range encodes black as 0, and white as 4096 (for 12bit videos), while the Limited/TV range is somewhere around 256 to 3840. One might wonder: why this Limited/TV range thing? As usual in such topics, the answer is: legacy from the times TVs and broadcasting worked differently from "pure digital" world.

Providing custom LUT

Providing a custom LUT for SDR downconversion was the first thing we wanted to try. It felt like a good way to eliminate the unknowns on YouTube's side of the processing pipeline, since the original upload relied entirely on YouTube to do the right thing, with no guidance at all.

To set up a LUT for a quick test, we turned to the HDR-to-SDR-LUTs_YouTube repository. 2000nits_HDR-to-SDR.cube seemed the best fit for our setup, so we tried it first. The next step was to embed the LUT into the Matroska container. The command is based on Google's hdr_metadata repository instruction:

mkvmerge\
      -o singularity-final-HDR-stereo-with-cube.mkv\
      --colour-matrix 0:9\
      --colour-range 0:2\
      --colour-transfer-characteristics 0:16\
      --colour-primaries 0:9\
      --max-content-light 0:1000\
      --max-frame-light 0:300\
      --max-luminance 0:1000\
      --min-luminance 0:0.01\
      --chromaticity-coordinates 0:0.68,0.32,0.265,0.690,0.15,0.06\
      --white-colour-coordinates 0:0.3127,0.3290\
      --attachment-mime-type application/x-cube\
      --attach-file 2000nits_HDR-to-SDR.cube\
      singularity-final-HDR-stereo.mov

A full list of the rather cryptic parameters and their values can be found in the mkvmerge documentation. A quick breakdown (our commands used the British spelling for color, mainly due to being based on the instructions from Google that used this spelling):

• --colour-matrix 0:9 -- BT2020 non-constant luminance (in hindsight perhaps a value of 10 for BT2020 constant luminance would have been a better choice). The value of 0 means the first (video) stream in the container.
• --colour-range 0:2 -- full range (no clipping)
• Parameters to specify Rec.2100-PQ encoding:
  • --colour-transfer-characteristics 0:16 -- SMPTE ST 2084
  • --colour-primaries 0:9 -- ITU-R BT.2020

The rest of the parameters are more self-explanatory. The white point and chromaticity parameters match what Blender saves in files. The luminance parameters match the Google documentation. The main difference from the documentation is the use of the full/pc range instead of the broadcast/tv. One important observation: the command works the best with *.mov input. Using *.mp4 input seemed to work locally, but YouTube refused to process the video, by throwing an error saying something like "video processing has been aborted".

The conversion was done on Mac, so getting the mkvmerge command was easy using Homebrew: brew install mkvtoolnix. Here is the result:

The video started to look closer to the intended result: blacks were deeper, and colors were less washed out. However, it was still far from ideal.

Baking custom LUT

We continued with crafting our own lookup table for the SDR downconversion. The LUT is supposed to work as follows: the input is Rec.2020 colorspace, PQ-encoded, and the output is Rec.709 colorspace, sRGB-encoded. Since the lookup table is applied to the encoded colors from the input video (Rec.2100-PQ), it must output standard sRGB video. We only needed to get the right data from the movie's edit file, and luckily the grading of Singularity was done through a global VSE compositor node setup on top of all shots:

VSE compositor node setup in Blender

The top branch represents the grade for SDR version, and the bottom branch is for HDR grade, followed by a node to switch between the two. In order to go from HDR grade to SDR we need to "un-apply" HDR grade, and apply SDR grade. This conversion is not guaranteed to be invertible, since different input colors can map to the same output color. Did we try it anyway? Absolutely!

The plan was to:

• Create a lookup table for HDR grade
• Create a lookup table for SDR grade
• Invert the HDR grade
• Combine inverted HDR LUT and SDR LUT to get the final LUT for downconversion

The overall workflow of creating a LUT is:

• Create an EXR image with "unit" 3D LUT representation as pixels
  • Unit means that the LUT maps colors to themselves
  • The generate_unit_lut_image.py script was used for this.
• Load the EXR into Blender as a VSE image strip
• Apply color grading VSE setup
• Render the result, and it'll be an EXR representation of the LUT
• Convert EXR to Resolve Cube format using exr_to_cube.py script.

Keeping in mind the input/output colorspace requirements for the final LUT and the working color space of the grading setup, we ended up with the following baking setup in Blender:

• For the HDR grading LUT:
  • Assume input is Rec.2100-PQ (Rec.2020 colorspace, PQ encoded)
  • Convert from Rec.2100-PQ to Linear Rec.709
  • Apply HDR grade compositor network
  • Convert Linear Rec.709 back to Rec.2100-PQ
  • We called the result 3D LUT lut_hdr_in_2100pq_out_2100pq.cube (used later in commands)
• For the SDR grading LUT:
  • Assume input is Rec.2100-PQ (Rec.2020 colorspace, PQ encoded)
  • Convert from Rec.2100-PQ to Linear Rec.709
  • Apply SDR grade compositor network
  • Convert Linear Rec.709 to sRGB
  • We called the result lut_sdr_in_2100pq_out_srgb.cube.

Modified VSE compositor node setup in Blender

OpenColorIO not only provides a command-line tool for baking lookup tables, but can also invert 3D LUTs. It can even invert non-invertible lookup tables! The result might not be what one wants in terms of the final colors, but OpenColorIO can do it!

In order to piece the lookup tables together the following command was used:

ociobakelut --format resolve_cube\
  --invlut lut_hdr_in_2100pq_out_2100pq.cube\
  --lut    lut_sdr_in_2100pq_out_srgb.cube\
  "final_grade.cube"

After attaching the lookup table to a Matroska video and reuploading the video the result was... not great. The colors did not look right, way off from the intended grade.

By calculating the gradient of the final LUT we also confirmed that the HDR grading was indeed non-invertible. We've got the technology to bake lookup tables but as an attempt to match SDR grade exactly it didn't qualify. As for this video, we've fallen back to 2000nits_HDR-to-SDR.cube as it provided a closer color match.

Broadcast legalizer

The TV range was the next thing we looked into. We knew Singularity was outside of this range but still wanted FFmpeg to confirm. It can be done using FFmpeg's ffplay:

ffplay singularity-final-HDR-stereo.mp4 -vf signalstats=out='brng':color='Yellow'

The brng means: identify pixels that fall outside of legal broadcast range. As you can see from the amount of yellow pixels, all the blacks and highlights were outside of what is good for the broadcast.

According to the Internet, FFmpeg has a video filter that converts PC to TV range with minimal change to the look:

ffmpeg\
  -hide_banner\
  -i singularity-final-HDR-stereo.mp4\
  -filter:v lut,setparams=range='tv'\
  -codec:v libx265\
  -preset veryslow\
  -x265-params keyint=18:min-keyint=1:aq-mode=3:no-sao=1\
  -crf:v 20\
  -pix_fmt yuv420p12le\
  -color_primaries bt2020 -color_trc smpte2084 -colorspace bt2020nc\
  -color_range:v 'tv'\
  -tag:v hvc1\
  -movflags write_colr\
  singularity-final-HDR-stereo-tv.mov

This encoding was followed by the mkvmerge command to embed the lookup table. We reverted to 2000nits_HDR-to-SDR.cube and TV range:

mkvmerge\
      -o singularity-final-HDR-stereo-tv-with-cube.mkv\
      --colour-matrix 0:9\
      --colour-range 0:1\
      --colour-transfer-characteristics 0:16\
      --colour-primaries 0:9\
      --max-content-light 0:1000\
      --max-frame-light 0:300\
      --max-luminance 0:1000\
      --min-luminance 0:0.01\
      --chromaticity-coordinates 0:0.68,0.32,0.265,0.690,0.15,0.06\
      --white-colour-coordinates 0:0.3127,0.3290\
      --attachment-mime-type application/x-cube\
      --attach-file 2000nits_HDR-to-SDR.cube\
      singularity-final-HDR-stereo-tv.mov

That upload to YouTube looked very promising: the SDR version finally had true black levels! But Singularity now had more color contrast than we had initially intended, especially in the HDR version. This was likely caused by the filter that FFmpeg applies when converting video to the broadcast-friendly range.

 The easiest solution for us was to reduce the contrast in the grading setup and re-export the movie. It was as simple as setting the Power setting in the Color Balance node in the setup shown above to 1.0 (initially it was 1.18). After re-export, we applied the same process: converting the video to broadcast range, embedding the LUT, and re-uploading the video. And this is what resulted in the final version of the upload which you've all seen!

Final YouTube upload look

Conclusions

While we successfully uploaded Singularity to YouTube and achieved the intended look, a few things could make the process smoother next time. The first concerns how grading is done: instead of a branched SDR/HDR grade, the SDR grading setup can be applied on top of the HDR grade, making it possible to derive an exact LUT for SDR downconversion. We will remember this lesson for the next HDR production, but it's also worth thinking about establishing LUT generation tools as a built-in functionality in Blender.

To avoid external re-encoding, we need to have a way to output videos in TV range directly from Blender with no perceivable color change in the result. Or we could even go further and avoid the need for mkvmerge by somehow making Blender output a Matroska container with a LUT attached to it!

Beyond that, there is one thing outside our control that would make a big difference: proper YouTube support for Full range. Not much to ask.