Latest post Sat, Jun 23 2018 3:53 PM by jef. 11 replies.
Page 1 of 1 (12 items)
Sort Posts: Previous Next
  • Fri, Jun 15 2018 4:36 AM

    • dreid
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • Toronto
    • Posts 226
    • Points 3,005

    Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    Hello all, I was wondering if you could help me with a vexing problem that perhaps one or more of you has encountered in the past.

    I am exporting my timeline: 2160p, 29.97 UHD. Codec, as per the AMA File Export tool is DNxHR HQX in an MXF OP1a wrapper.

    Problem is, when I send this file to the QC facility, they are reporting “frequency issues” (not level issues per se or how it looks (it looks great) but frequency issues in their terminology.) First of all, what does that mean?

    Well, they say that, when looking at my file using a Tektronix scope they see “black levels to around minus 30mV and video levels to about plus 730mV. Mostly from ringing around the fonts.”

    Well, I say that is impossible because I have applied the Avid Safe Color Limiter (rendered before export of course) which clips the levels at 0mV and 700mV. My scopes certainly reflect this: there is no excursions past 700 or below 0 when viewed with my built in Avid scopes and my external Blackmagic scopes. I realize the Blackmagic SmartScope Duo is no Tektronix, but a clip is a clip, it is either there or it is not. Blackmagic or Tektronix it shouldn’t matter: if I have applied a limiter, and my scopes show the limiter is working, then why is it not that way after I transfer the file to them?

    The QC facility says that, although the illegal excursions are completely invisible to the naked eye and are only visible on a high end scope, these -30 or +30 excursions could be enough for a broadcaster to reject my show.

    I say that I have rendered the limiter or clip before my export and that, on my scopes, all looks well, so how could the levels be out of whack when they reach their facility? The problem must be on their end, no?

    I am using Avid 8.5.3 on a PC. They are looking at my file (linking via AMA import) using Avid 8.4.2 on a Mac. Could that have anything to do with it?

    I know there are a lot of brilliant online editors in this forum who are very smart when it comes to file sharing and levels issues/incompatibilities. You have no idea how much this would mean to me to get this sorted out as we are at a standstill with my project right now.

    Essentially it seems to me to be a problem with Mac-Avid ability to understand the black levels of DNxHR HQX files exported by PC-Avid.

    Any advice at all would be EXTREMELY appreciated and rewarded.

    Thanks in advance everyone who reads this post!

    MC 8.5.3, DNxIO w/ 10.5.4, Win 7 Pro 64 bit SP1, Dell 7910, Dual 10 core Xeon 3.10, 128 GB DDR4 RAM, Quadro K5200, 1 TB SSD system drive, 1 TB SSD cache... [view my complete system specs]

    And back to a windowless room full of Avids he goes.

  • Fri, Jun 15 2018 10:50 PM In reply to

    • Lukas Boeck
    • Top 500 Contributor
    • Joined on Fri, Jul 31 2015
    • Austria
    • Posts 434
    • Points 5,075

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    hmm,

     

    any chance the "ringing around the fonts" correlates with external graphics that have been imported incorrectly with "do not modify" color levels?

    i'd get another opinion on the file in say after effects. measure with e.g. color picker.

     

     

    MC 8.4.4 Cusom bulid Supermicro X9SRL-F, 32 Gig Ram, 3ware 9750 6x6TB Seagate EC Drives, LSI 9207 + LTO-6, Mojo DX, Artist Color, Flanders CM240, GTX 1070... [view my complete system specs]
  • Sat, Jun 16 2018 9:45 AM In reply to

    • Bruno M
    • Top 500 Contributor
    • Joined on Thu, Feb 11 2010
    • London, UK
    • Posts 490
    • Points 6,200

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    Although I'm by no means a modern-day TV engineer, I did study television engineering in the late 1980's, and transient under/overshoots (or 'ringing') is a well known problem which is difficult to completely eradicate.

    The first thing you should do is bring the offending export back into MC and see if you can see any difference between it and your final sequence. If all looks OK to you using whatever internal /external scopes you have, we must assume that unless the the QC facility have some fault with their equipment, you are effectively blind to this problem.

    dreid:
    Blackmagic or Tektronix it shouldn’t matter

    Not all scopes are created equal. Proper signal measuring equipment that costs many thousands of dollars (like the Tektronic scopes) are designed to measure very high frequencies - after all, they're designed for accurate analysis of signals to identify the very problem we're talking about. Scopes costing a few hundred dollars are not going to have the circuitry to do this. In fact, if these cheaper scopes were showing any of these artifacts, it's likely they'd employ filtering circuits to 'clean up' the display.

    Knowing these limitations, I have a HD Leader scope which cost me several thousand dollars, and (with all the filters switched off) that shows various transient 'spikes' with SDI signals coming from my video hardware (blackmagic and Avid Mojo DX). These spikes are not visible in the software scopes in apps such as MC or Resolve.

    I'm not familiar with the technical specs in th US or Canada, but here in the UK this is a known problem that is mentioned in broadcaster's technical specifications.

    The BBC document specifically mentions the tolerances allowed - I've copeid and pasted them below...

    ---------------------------------------------------------------------------------

    1.2.1. Video Level Tolerance
    In practice, it is difficult to avoid generating signals slightly out of range, and it is considered reasonable to allow a small tolerance:
    • the RGB components and the corresponding Luminance (Y) signal, should not normally exceed the “Preferred Minimum/Maximum” range of digital sample levels in the table below,
    • measuring equipment should indicate an “Out-of-Gamut” occurrence only after the error exceeds 1% of an integrated area of the active image.
    For further details see the EBU Recommendation, EBU R103.
    Any signals outside the “Preferred Minimum/Maximum” range are described as having a gamut error (or as being out of gamut). Signals cannot exceed the “Total Video Signal Range” and will therefore be clipped.
    Full range video levels must not be used for delivered television programmes.
    Colour gamut "legalisers" should be used with caution as they may create artefacts in the picture that are more disturbing than the gamut errors they are attempting to correct. It is advisable not to “legalise” video signals before all signal processing has been carried out.

    -----------------------------------------------------------

    I can't seem to paste the chart referred to, but here's link the EBU document...

    https://tech.ebu.ch/docs/r/r103.pdf

    In respect of the 1% mentioned, I know there are Tektronic scopes that have this 1% 'button' specifically for the purpose of assisting QC sessions.

    So the question is, how bad is your out-of-gamut problem and do they fall within the technical specs demanded?

    It's difficult to know where your problem lies without looking at the material using a proper scope - certainly graphics and titles are obvious candidates.  In fact, Avid have for years used filtering on titles created with their title tool to help with this problem and when we all went HD, people noticed this softness compared to (say) titles brought in from other applications. I can't quite find the forum topics that covered this, but there was a console command to switch this off, with the possible problems this might cause.

    As far as the Avid legaliser is concerned, I certainly remember I never relied on this to corerectly legalise my exports as I discovered it wasn't properly catching everything - according to my separate hardware legaliser. This was in the days of outputting to tape. Today, with file based delivery, some facilities are using software specifically designed to look at all sorts of errors can manifest themselves in digital deliveries.

    At the end of the day, I guess you are going to have to comply with the conditions set by the specs and the QC facility, and you'll need to firstly be able to identify the problem, and secondly find ways to correct it, possibly with other software-based solutions or legalisers. You could of course just get a QC facility to do the legalising for you, and build this into the program costs.

     

     

    HP Z800,HP8600,HP8760WSonnetFusion RAID, Mediadock U320, Classic Nitris Mojo DXSymphony 5,6 & 8.5 JVC DTV1910 HD tube monitor, HP Dreamcolor, [view my complete system specs]
  • Sat, Jun 16 2018 11:40 AM In reply to

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    I always apply the Safe Color at 4:2:2 but only above video tracks and below title track (s).

    Desktop
    Corsair RM750X/Asus P8Z68 Deluxe(Gen3)/i7 2600K@4.4/Corsair 32G DDR3/PNY Quadro P2000D(390.77)/Samsung 850 Pro SSD(OS) 256 GB/ 2x2TB WD RE4@RAID 0/LG BH16NS40/W10 Pro 64bit/Media Composer Software 2018.7 with Symphony Option/2xHP 22'' Monitors/M-Audio AV30 Speakers.

     

    Larry as long as I live I will remember you.

     

    3 Heads Digital Films

  • Thu, Jun 21 2018 5:16 AM In reply to

    • dreid
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • Toronto
    • Posts 226
    • Points 3,005

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    Lukas Boeck:

    hmm,

     

    any chance the "ringing around the fonts" correlates with external graphics that have been imported incorrectly with "do not modify" color levels?

    i'd get another opinion on the file in say after effects. measure with e.g. color picker.

     

     

    Hi Lukas, thanks for taking the time to reply. Your idea to focus on the titles is a good one, thank you.

    So I looked again at the titles as they exist in After Effects. I measured the titles with the color picker and confirmed that the white level is not too hot (it is 235 on the RGB scale). Then I used the BCC videoscope plug in to look at those titles on a waveform in AE. All’s fine, the waveform shows the levels at 235 (well below the maximum of 255). Then I looked at the codec I used for rendering out that title: Grass Valley HQX (10 bit) with alpha. Next I re-imported that file into Avid. 

    With the new title edited into the timeline, I adjusted the drop shadow slightly so there is not as much contrast between white and black. 

    I sent a new file to QC to see if that helps... will keep you posted. Thanks again!

    MC 8.5.3, DNxIO w/ 10.5.4, Win 7 Pro 64 bit SP1, Dell 7910, Dual 10 core Xeon 3.10, 128 GB DDR4 RAM, Quadro K5200, 1 TB SSD system drive, 1 TB SSD cache... [view my complete system specs]

    And back to a windowless room full of Avids he goes.

  • Thu, Jun 21 2018 5:35 AM In reply to

    • dreid
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • Toronto
    • Posts 226
    • Points 3,005

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    Bruno M:

    The first thing you should do is bring the offending export back into MC and see if you can see any difference between it and your final sequence. If all looks OK to you using whatever internal /external scopes you have, we must assume that unless the the QC facility have some fault with their equipment, you are effectively blind to this problem.

    Hello Bruno, thank you very much for your thoughful reply, much appreciated!

    Yes, I brought the offending export back into MC (via Link to Media) and yes it looks the same under the scopes, meaning that the clipping is intact and levels are legal. It seems that somehow, some way, when I send the encoded MXF file to the QC facility, using the AMA Export feature, they are not seeing the clipping or legalizing. 

    I keep coming back to the same issue in my head: if I have applied a legaliser before I export, why is it not showing up on their system?

    The reason must be one of 3 things: 1) my legaliser is not working, 2) my legaliser settings are not right, or 3) somewhere along the line, the DNx codec is broken.

    I checked my Safe Colour Limiter settings. Here are my settings. 4:2:2 Safe is checked. Composite/Luma levels set to a low of 0 and a high of 700 mV. RGB Levels set to a low of 16 and a high of 234.

    Safe Colour Settings are all the same. When I apply the Safe Colour Limiter, I look at the YC Wavform in the Avid, they seem to be doiing a perfect job of clamping the levels so that no excurisions below 0 and none above 700. Like I said however, when QC gets this export, thay're not seeing the same results. They are linking to the MXF file using their MAC- Avid (I am on Windows-Avid) and reporting, as stated above, significant excursions below 0 and above 700. Like you said, I am essentially blind to the problem.

    Bruno M:

    As far as the Avid legaliser is concerned, I certainly remember I never relied on this to corerectly legalise my exports as I discovered it wasn't properly catching everything - according to my separate hardware legaliser. This was in the days of outputting to tape. Today, with file based delivery, some facilities are using software specifically designed to look at all sorts of errors can manifest themselves in digital deliveries.

    I keep coming back to one overriding thought: if the clamp or legaliser is applied before export, then why is there any ringing or overshoot at all when the file gets to their location? Is the Avid legaliser not working/not trustworthy? This is from Avid when talking about the use of their Safe Color Limiter: When 422 Safe is selected, effect processing is slower, but every pixel in every frame is guaranteed to be within your safe color ranges … Guaranteed accuracy.

    Bruno M:

    At the end of the day, I guess you are going to have to comply with the conditions set by the specs and the QC facility, and you'll need to firstly be able to identify the problem, and secondly find ways to correct it, possibly with other software-based solutions or legalisers. You could of course just get a QC facility to do the legalising for you, and build this into the program costs.

    Yes, that is wher I am at, getting the QC facility to encode for me. Coincidentally, they say that when they take my file and use their hardware legalizer as part of the encoding process to make a ProRes file, the clamping works. To me, though, this is like a band-aid solution. The idea that my show is not legal when it leaves my studio is not OK with me. Even the slightest hint that my show will be rejected by broadcasters whether in Europe or in North America is not OK with me. Note that I am not being bull-headed about all of this, I am fully open to the theory that all of this is my own errors, my issues. I am not blaming anyone, I just would like to find the right workflow and see this fixed.

    Anything that you can do to help me understand what is going on here (and specifically to help me understand what my Avid Safe Color Limiter settings should be) would be most appreciated!

    PS, thank you for the EBU standards document, very usefull!

    MC 8.5.3, DNxIO w/ 10.5.4, Win 7 Pro 64 bit SP1, Dell 7910, Dual 10 core Xeon 3.10, 128 GB DDR4 RAM, Quadro K5200, 1 TB SSD system drive, 1 TB SSD cache... [view my complete system specs]

    And back to a windowless room full of Avids he goes.

  • Thu, Jun 21 2018 5:37 AM In reply to

    • dreid
    • Top 500 Contributor
    • Joined on Thu, Oct 13 2005
    • Toronto
    • Posts 226
    • Points 3,005

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    George Angeludis:

    I always apply the Safe Color at 4:2:2 but only above video tracks and below title track (s).

    Hi George, thank you for your response. Question: why do you place the Safe Colour effect above the video but below the titles? What is the advantage of that?

    MC 8.5.3, DNxIO w/ 10.5.4, Win 7 Pro 64 bit SP1, Dell 7910, Dual 10 core Xeon 3.10, 128 GB DDR4 RAM, Quadro K5200, 1 TB SSD system drive, 1 TB SSD cache... [view my complete system specs]

    And back to a windowless room full of Avids he goes.

  • Thu, Jun 21 2018 5:52 AM In reply to

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    Do I have to measure/correct the title levels?

    Desktop
    Corsair RM750X/Asus P8Z68 Deluxe(Gen3)/i7 2600K@4.4/Corsair 32G DDR3/PNY Quadro P2000D(390.77)/Samsung 850 Pro SSD(OS) 256 GB/ 2x2TB WD RE4@RAID 0/LG BH16NS40/W10 Pro 64bit/Media Composer Software 2018.7 with Symphony Option/2xHP 22'' Monitors/M-Audio AV30 Speakers.

     

    Larry as long as I live I will remember you.

     

    3 Heads Digital Films

  • Fri, Jun 22 2018 8:07 AM In reply to

    • frediko
    • Not Ranked
    • Joined on Sat, Sep 27 2008
    • Norway
    • Posts 71
    • Points 950

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    Interesting thread.

    This shows that even with a safe color limiter you risk the contrast of hard edges like text to trigger QC issues. In playing back a video signal which goes from edge of limiter to the other edge of limiter in one pixel it might overshoot that limit on playback somehow. The solution might be to not have such a hard contrast of say full white super sharp text on dark background, either by no going full white, or softening the title edges a little.

    I've previously discussed this a bit when figuring out wheter to turn off HDTitleFilter in console for sharper titles. Media composer has had this setting at "on" since HD projects were introduced exactly to deal with the issue of hard contrast titles triggering QC issues like you describe.

    We decided to go for it and have had it disabled for years and years without QC giving us trouble. We've only delivered HD files to broacasters though. The quality of the titles are so much better with HDTitleFilter diabled, both with title tool and marquee. This might be a problem coming back in UHD projects I guess.

    I don't know how you make your titles, but i suggest you either make them anything but 100% white on 100% black at least.

    Some background on the issue with HDTitleFilter and sharp titles:
    https://community.avid.com/forums/p/182774/849897.aspx

    https://community.avid.com/forums/p/92610/644722.aspx

    Also adding screengrabs for reference about turning HDTitleFilter on or off with example.

    I really hope Avid nails title quality with good antialiasing and no blurry edges for the next title tool.

    fred


    Mostly hp z420/440 on Media Composer 8.x and Symphony 8.x Also Dell Precision m3800 | Win 10 | 16GB memory | Quadro k1100 [view my complete system specs]
    Filed under:
  • Sat, Jun 23 2018 10:45 AM In reply to

    • Lukas Boeck
    • Top 500 Contributor
    • Joined on Fri, Jul 31 2015
    • Austria
    • Posts 434
    • Points 5,075

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    Hi George, thank you for your response. Question: why do you place the Safe Colour effect above the video but below the titles? What is the advantage of that?

    maybe i can answer that.

    Titles have more reviews during production and should be safe by default.

    when he has the Safe Color Limiter rendered, he's in for a speedy export for most of his timeline.

    i do a mixdown of with colof limiter instead of render.

     

    @ frediko: cool tekst.

    sidenote: min hoovercraft er full av ål.

     

    MC 8.4.4 Cusom bulid Supermicro X9SRL-F, 32 Gig Ram, 3ware 9750 6x6TB Seagate EC Drives, LSI 9207 + LTO-6, Mojo DX, Artist Color, Flanders CM240, GTX 1070... [view my complete system specs]
  • Sat, Jun 23 2018 3:24 PM In reply to

    • Bruno M
    • Top 500 Contributor
    • Joined on Thu, Feb 11 2010
    • London, UK
    • Posts 490
    • Points 6,200

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    Lukas Boeck:
    Titles have more reviews during production and should be safe by default.

    That's a pretty dangerous asumption to make - given that in Dried's original post, it was exactly the fonts that QC were blaming for the illegal levels! At all the places I've worked, we've never assumed anything coming out of MC is broadcast safe, whatever Avid (or the Editor) may tell us!

    I think there's some things that need clarifying. Whilst safe colour effects (or external legalisers) normally deal with illegal levels and gamut errors, it's perfectly possible they may not trap these high frequency overshoots or ringing. In fact, in the process of clipping/rolling-off illegal levels, these very same legalisers can introduce their own overshoots, and just make the problem worse.

    In the days of SD, these problems didn't seem to really happen, but since the introduction of the higher frequencies that exist in HD, software and hardware designers have had to take more care to avoid the problem. This is the thrust of Frediko's post about sharp titles and Avid's HDTitleFilter solution.

    I'm willing to bet that working at UHD is just making this problem even worse and Avid's legaliser plugin has not been updated in a few years, and hasn't got adequete filtering. This is why many top-end facilities use hardware legalisers and scopes - they've seen the problem and have invested in the hardware to inspect/correct the issues and pass the broadcaster's specifications.

     

    HP Z800,HP8600,HP8760WSonnetFusion RAID, Mediadock U320, Classic Nitris Mojo DXSymphony 5,6 & 8.5 JVC DTV1910 HD tube monitor, HP Dreamcolor, [view my complete system specs]
  • Sat, Jun 23 2018 3:53 PM In reply to

    • jef
    • Top 50 Contributor
    • Joined on Sun, Feb 26 2006
    • Maryland
    • Posts 3,225
    • Points 39,125

    Re: Problem with black levels (frequency) of DNxHR HQX when exporting UHD timeline

    Will take what Bruno said and add an example.

    We had a spot just fail our internal QC for a disclaimer having high freq. illegal levels. So I investigated.

    Spot was finished in After Effects.  When the After Effect project was open and the disclaimer scene was viewed on external scopes via an AJA IO, everything was good.

    But the moment we made a file - either DNx HD or ProRes (both at highest quality) and viewed via the same path the issue was present.

    The disclaimer was over a bright scene with a high complexity moving background.  All we could surmise is that this was to complex for either of the compression schemes and the error was being created in that step.

    Our only solution was to add an overall levels reduction of the last scene until the still produced "spikes" were within spec.

    The point here is that the compression tech we use can in certain cases cause problems on their own.  And a legalizer before the compressor may not save you.

    Jef

    Avid DS 11.0.2 R.I.P | MC 8.5.3 / 8.9.4 | OS 10.11.x - various MacPro Towers - home system MacPro Dual 2.8 8core GTX680 "Harpertown" QT 10.0... [view my complete system specs]

    _____________________________________________

    Jef Huey

    Senior Editor

    Old Stuff  http://vimeo.com/album/3037796

Page 1 of 1 (12 items)

© Copyright 2011 Avid Technology, Inc.  Terms of Use |  Privacy Policy |  Site Map |  Find a Reseller