Testing MrMC 3.3.1

[davilla]

I tried that sample, meh, it looked ok but since I'm not sure what I should be seeing, not a clue. Ask wesk05 what I should be looking for

EDIT: for hevc/hdr we use avfoundation, it does all the heavy lifting. To switch, we send the standard switch for hdr. I've verified the setting loading the file as an asset and asking for the range, same as what we are using.

That said, the setting to switch to hdr. There are actually two values, they seem to give the same effect so I'm not sure what is the difference. Which is why I wrote a test app to manually load up test files as assets and probe them for their reported range.

My trick to play using the native player fails because the file is an xxx.mp4. The native player wants xxx.mov. Little hesitant to just convert since these are hevc main 10/hdr and tools that do it might strip out something.

If wesk05 could convert the file to xxx.mov, then verify that it acts the same. Then I could look closer.

[davilla]

Also if he knows where the display luminance is found, I can trap for it to make sure that ffmpeg demux did not drop it.

[wesk05]

The point that I was making was ATV4K is "fabricating" HDR10 metadata. The metadata in the HDMI InfoFrame doesn't match the source metadata (HEVC SEI).

For all HDR10 clips played in MrMC, HDR10 metadata in the InfoFrame is:
Mastering Display Primaries: BT.2020
White Point: 0.3127, 0.329
Mastering Display Luminance (Max): 1000 nits
Mastering Display Luminance (Min): 0.005 nits
Maximum Content Light Level (MaxCLL): 4000 nits
Maximum Frame-average Light Level (MaxFALL): 1000 nits

By standard/technical design, displays are to use the Max. & Min. Luminance values for calculating the roll-off knee for display mapping. How many of the displays available in the market are actually using these parameters for display mapping is something that can be debated on another day

The Sony Camp clip was mentioned because it has incorrect metadata, but on playback it also has the metadata mentioned above in the HDMI InfoFrame.

The question that I have is: what controls the HDR10 metadata? Is it something that is set by AVFoundation or is it being set by MrMC?

I am not sure that I understand your question on display luminance. As for converting mp4 to mov, you can do that in XMedia Recode without any transcoding. Since the HEVC stream is untouched, you will not loose the metadata.

[davilla]

Hi wesk05

This is how we treat hevc/hdr.

1) Demux into packets by ffmpeg. This can via two paths, One for xxx.mov, the other for the rest. I bring this up because the two paths are treated slightly different inside. So far, I've not see any difference. The other potential item is ffmpeg packages sps, pps and vps into extra data which we feed into the decoder. If ffmpeg is not packaging sps, pps or vps for hdr fully and has missed propagating some fields, then this might be the point of error.

2) Decode by AVFoundation. We use something called CDVDVideoCodecAVFoundation, Kodi has no such decoder but Memphiz's YAB implementation does as he backported it from us. CDVDVideoCodecAVFoundation uses AVSampleBufferDisplayLayer for the decode. It is the other way to do hw decode on iOS/tvOS. CDVDVideoCodecAVFoundation is a BYPASS codec an works like a MediaCodec(Surface) on Android. We make the GLES layer transparent and add a layer under it for video. AVSampleBufferDisplayLayer take care of all the details regarding this layer and iOS/tvOS will blend everything correctly for display. Basically, we set up using sps, pps, vps passed up by demuxer in extradata. Then it gets opened and feed demux packets and the video magically appears on the layer. It bypasses the normal GLES renderer. Generally, ffmpeg demuxer will pass up all demuxed packets but it is possible that it eats something that it thinks is bogus. This is another potential place for error.

For SRD/HDR switch, We track some details that indicate SDR/HDR/Dolby Vision. When playback starts, we use the Apple API's to switch to the desired frame rate and range. This is where things get slightly interesting. SDR range accepts values of 0 or 1, HDR 2 or 3 and Dolby Vision is 4. Experimentally determined. But using a test app, and loading test files into an assets, then checking the range reported by the asset, SDR gives 1, HDR gives 3 and Dolby Vision gives 4. So inquiring minds ask, what are 0 and 2 ? These values are not documented yet.

That was long answer, short answer to 'HDR10 metadata' is if ffmpeg is handling it properly, we just pass everything to AVSampleBufferDisplayLayer and the rest is out of our hands.

What I have to dig into is where in demuxer are display luminance values kept. I suspect sps/pps/vps. Once I find where, then I can check ffmpeg to made sure that it is populating them from parsed values and not using some default.

EDIT: As I don't have the equipment that you have to see exactly what is display luminance values are going out to the display. Is there some test file that I can run where I can see the error ? Like some gray ramp that should show a but shows b ? For example, that sony test file. As I don't know what it should look like, it's rather useless to me as it looks fine.

[wesk05]

What I have to dig into is where in demuxer are display luminance values kept. I suspect sps/pps/vps. Once I find where, then I can check ffmpeg to made sure that it is populating them from parsed values and not using some default.

EDIT: As I don't have the equipment that you have to see exactly what is display luminance values are going out to the display. Is there some test file that I can run where I can see the error ? Like some gray ramp that should show a but shows b ? For example, that sony test file. As I don't know what it should look like, it's rather useless to me as it looks fine.

Display luminance and other parameters are in SEI that is part of SPS. See pages 297 and 301 of the HEVC spec. (https://www.itu.int/rec/recommendation. ... 5-201612-I) I believe MediaInfo software relies on FFmpeg and it is able to parse the mastering display color volume and content light level parameters in the SEI. HDRMaster (https://hdr.avtop.com/hdr_solutions_avtop) software by Florian also relies on FFmpeg. This would suggest that FFmpeg is working properly in parsing the metadata.

Is there some test pattern to check this out visually: this is where it gets tricky . As mentioned earlier, I can't say with much confidence how the displays available in market are handling this information. Case in point is the Sony Camp video. It shouldn't be displayed "properly" on any display because it has bogus metadata. The display mapping algorithm certainly differs between brands (https://www.youtube.com/watch?v=Lt6IflKAmWg). In the absence of a protocol analyzer, the cheapest option to monitor the HDR InfoFrame would be a HDFury Vertex or Integral. Integral only provides the hexadecimal values in the InoFrame which is not that difficult to decipher. Vertex can decipher it to human readable format.

[koying]

Just popping to tell that mediainfo does not use ffmpeg, last I check their sources.
They use their own parser.

[davilla]

Just popping to tell that mediainfo does not use ffmpeg, last I check their sources.
They use their own parser.

Correct and just to be clear, just because something uses ffmpeg and shows the Display luminance does not mean that extradata containing sps,pps and vps are setup correctly when demuxing for decode. This is under investigation.

[davilla]

Let's continue in the Testing MrMC 3.4.1 post. Testflight 3.4.1 will get posted shortly.

[wesk05]

Just popping to tell that mediainfo does not use ffmpeg, last I check their sources.
They use their own parser.

I see.

Correct and just to be clear, just because something uses ffmpeg and shows the Display luminance does not mean that extradata containing sps,pps and vps are setup correctly when demuxing for decode. This is under investigation.

Absolutely! Hopefully, you will find what is causing the problem and find a resolution for it.

[davilla]

waded into ffmpeg tonight. I need a shower

Display luminance is no kept in sps, it's part of a sei prefix nal. ffmpeg strips this off to what they call side data with a flag that indicates they type of side data. It's not clear if this side data gets merged back when fetching a frame's demux packet.

But all this points me into the right direction.