Topics

Filmlook or TV?

Geoff Boyle
 

I've been asking for people’s opinions about what needs to be included in the next camera evaluations but the answers are highly contradictory.

It comes back to the old which do you prefer, the look of film or the look of TV. The majority of people will say film.  Or at least digital images which emulate the look of film.

They then insist that camera evaluations are done using TV charts and to TV levels. 

Given that the original rushes will be made available and also 16 bit EXR files the only people looking at the processed results are either those who can't use the original files for some reason or other and those who can't be arsed.

The second group can... The first group don't need to look at something designed to make the cameras look like TV cameras. They need to know that a workflow to get the best film look has been followed. Let's not get into the pointless argument of what the film or TV looks are, we all know what we mean by those terms.

We then get to the question of where do we put the levels, there are all kinds of answers varying from 30% to 50% but let’s make this clear, I'm working within ACES so log levels on a waveform don't apply. Once an IDT has been applied and we are in ACES space we are working with an environment that has an inherent toe and shoulder and straight middle gamma.

So I've gone back to basics. All the years I sot commercials on film and scanned the neg into a video or later digital environment I used a simple reference. The Kodak Gray Scale Plus. They tell you where to put the mid grey part of the chart in very simple terms. I'm looking at the back of one now and it clearly says 122, 122, 122 in 8 bit values. In the past I've used 128, 128, 128 and I've had a lot of grief about it. It turns out I was bloody close to the recommended Kodak level and far closer than any of the corrections I was being given.

I will not use TV limited charts.

 

 

Cheers

 

Geoff Boyle NSC FBKS

Cinematographer

Netherlands

www.gboyle.nl

 

Art Adams
 

I will not use TV limited charts.

I guess I'm going to be "that guy" again.

What does a "non-TV limited" chart look like? I'm not sure I know what that is.

Is this the chart you're going to use?


I guess there's no reason not to, except that your tests will never be viewed on film and those colors don't mean anything in the digital world. There's no way for anyone viewing it to know if the chart is properly reproduced.

This is especially problematic given that camera color matrices can distort colors without affecting neutrals. At least "TV" charts have references that can be easily examined in Resolve, because every vectorscope in the world has Rec 709 targets. Short of printing your references on film and checking the results with a densitometer I'm not sure how that Kodak chart will be useful.

The Kodak TAF setup paper has this to say:

TAF color bars are not intended to match electronically generated color bars. Rather, they provide a method to obtain a repeatable telecine setup. Once you obtain a setup that generally provides pleasing results, the position of the color bars on a vectorscope can be noted and used as a reference for future setups.


Those of us without recent TAF film strips will be at a bit of a disadvantage.

Also, starting on page 23 of that document, it actually gives color reference values—for 17 different film stocks. They differ for every film stock. And you're not shooting on film anyway.

Given that the original rushes will be made available and also 16 bit EXR files the only people looking at the processed results are either those who can't use the original files for some reason or other and those who can't be arsed.

Correct. And 99% will be looking at them on computer displays or Rec 709 monitors. So what's the problem with "TV limited" charts, especially when they contain more information than film charts ever did, and are relevant to the display medium where a film chart isn't?

The film look has nothing to do with whether "TV limited" charts are used or not.

We then get to the question of where do we put the levels, there are all kinds of answers varying from 30% to 50%

No, there's not. Each display format has known reference values. I'm sure people here can tell you what they are. If you want what you shoot to look roughly the same way on each kind of display then you should expose the your reference chart using that value. You only really need to know one (Rec 709 is the easiest) and then the others should track well enough.

Saying "122 is close enough to 128" is like saying "The gray card was overexposed by a half stop, but that's close enough." If your lab printed your gray card a half stop too bright, would you be happy? That defeats the purpose of shooting it in the first place.

Besides, on that website, Kodak gives a waveform reference value for their card's gray patch: 45%.

Once an IDT has been applied and we are in ACES space we are working with an environment that has an inherent toe and shoulder and straight middle gamma.

Sure. So what? You don't look at it in ACES space when you expose it, or do you? And even in ACES space, there's a target for an 18% gray card. Why not hit it properly?

I know I'm pissing into the wind, but camera tests are hugely complex things. It's easy to make a product look bad. I'd rather they look good or bad on their own. To do that, I need to know what I'm looking at, with references in the frame that I can analyze properly without having to get a film lab involved. There aren't many of those around at the moment.

--
Art Adams
Director of Photography
San Francisco Bay Area

Geoff Boyle
 

Art, In the dime and distant past I used TAF to make sure that telecines and scanners were correctly set up, I’m not doing anything like that here. I do still have a roll of TAF right next to me on my desk.

I am using a Kodak Gray Scale Plus chart, yes the one you link to, on the back of it it gives values to set the various levels at. 488 is not half a stop from 512, stop exaggerating for effect.

I don’t know about most people but my eyes aren’t good enough to see on a Resolve waveform the difference between the two, I know that there is a nice glowing reference at 512 that I can easily see and line up to.

OK, maybe I should line up the width of a gnats todger less than 512 but can I be consistent then and consistency of the point I use is far more important than that gnats todger.

I have no problem at all with you taking the raw material and setting it up however you think it should be displayed. I’ll add a whole section dedicated to your way of doing it.

I will be using CML stress tests charts, Macbeth charts and Kodak charts, I’ll be throwing in a few odd elements as well.

In recent years I’ve allowed myself to be distracted from what I want to see from these tests.

The discussions this year have helped me to clarify exactly what I want to know and it’s not what the cameras look like on charts.

I want to know what their comparative sensitivities are, the Kodak Gray Chart for this

I want to know how they show different colours, Stress & Macbeth for this.

I want to know how they change colour and saturation with exposure, Stress for this.

I want to know how they cope with an HDR scene, multiple Stress charts at different levels for this.

I know that you’re going to say that using the charts that I am you can’t do it you’re way from my rushes, fine do a complete test your way. I’ll happily host that for people who want to know what charts look like.

 

Cheers

 

Geoff Boyle NSC FBKS

Cinematographer

Netherlands

www.gboyle.nl

 

From: cml-general@... <cml-general@...> On Behalf Of Art Adams
Sent: 24 June 2018 19:02
To: Cml-General <cml-general@...>
Subject: Re: [cml-general] Filmlook or TV?

 

I will not use TV limited charts.

I guess I'm going to be "that guy" again.

 

What does a "non-TV limited" chart look like? I'm not sure I know what that is.

 

Is this the chart you're going to use?

 

 

I guess there's no reason not to, except that your tests will never be viewed on film and those colors don't mean anything in the digital world. There's no way for anyone viewing it to know if the chart is properly reproduced.

 

This is especially problematic given that camera color matrices can distort colors without affecting neutrals. At least "TV" charts have references that can be easily examined in Resolve, because every vectorscope in the world has Rec 709 targets. Short of printing your references on film and checking the results with a densitometer I'm not sure how that Kodak chart will be useful.

 

The Kodak TAF setup paper has this to say:

 

TAF color bars are not intended to match electronically generated color bars. Rather, they provide a method to obtain a repeatable telecine setup. Once you obtain a setup that generally provides pleasing results, the position of the color bars on a vectorscope can be noted and used as a reference for future setups.

 

 

Those of us without recent TAF film strips will be at a bit of a disadvantage.

 

Also, starting on page 23 of that document, it actually gives color reference values—for 17 different film stocks. They differ for every film stock. And you're not shooting on film anyway.

 

Given that the original rushes will be made available and also 16 bit EXR files the only people looking at the processed results are either those who can't use the original files for some reason or other and those who can't be arsed.

 

Correct. And 99% will be looking at them on computer displays or Rec 709 monitors. So what's the problem with "TV limited" charts, especially when they contain more information than film charts ever did, and are relevant to the display medium where a film chart isn't?

 

The film look has nothing to do with whether "TV limited" charts are used or not.

 

We then get to the question of where do we put the levels, there are all kinds of answers varying from 30% to 50%

 

No, there's not. Each display format has known reference values. I'm sure people here can tell you what they are. If you want what you shoot to look roughly the same way on each kind of display then you should expose the your reference chart using that value. You only really need to know one (Rec 709 is the easiest) and then the others should track well enough.

 

Saying "122 is close enough to 128" is like saying "The gray card was overexposed by a half stop, but that's close enough." If your lab printed your gray card a half stop too bright, would you be happy? That defeats the purpose of shooting it in the first place.

 

Besides, on that website, Kodak gives a waveform reference value for their card's gray patch: 45%.

 

Once an IDT has been applied and we are in ACES space we are working with an environment that has an inherent toe and shoulder and straight middle gamma.

 

Sure. So what? You don't look at it in ACES space when you expose it, or do you? And even in ACES space, there's a target for an 18% gray card. Why not hit it properly?

 

I know I'm pissing into the wind, but camera tests are hugely complex things. It's easy to make a product look bad. I'd rather they look good or bad on their own. To do that, I need to know what I'm looking at, with references in the frame that I can analyze properly without having to get a film lab involved. There aren't many of those around at the moment.

 

--

Art Adams

Director of Photography

San Francisco Bay Area

 

Adam Wilt
 

I will not use TV limited charts.

Ah, now I see the problem. Geoff thinks we’re asking for “TV charts” and trying to grade his tests like TV camera tests. No wonder he’s upset!

Geoff, it’s not about TV charts; that’s not the point at all. We don’t care a fig about “TV charts” or “film charts”. We’re not trying to turn you into Alan Roberts. All we care about is useful information

A chart like the Cine ChromaDuMonde has a logarithmic ramp, a stepped gamma-coded grayscale, and colours that ramp between known primaries and secondaries, and it’s laid out so that those patches display clear patterns on the ’scopes. (The colours and grayscale are the same as on the ChromaMatch, BTW, just laid out differently.)

These patterns — regardless of their ranges or nominally “correct” levels — do recognizable and comprehensible things on WFMs and vectorscopes, things that provide direct, graphical evidence of what’s going on in an image that might otherwise be difficult to discern or elucidate: “oh, that’s what happening to my highlights”; “I now understand why this camera makes nice magentas and blues even though bluish purples* look weird”; that sort of thing. 

Granted, looking at ’scopes is a crutch. But just as a crutch helps an ambulatorily-challenged person walk, ’scopes help some of  us see.

And yes: we can window way down on the ChromaMatch to see roughly what the gamma-coded grayscale is doing, even if it’s missing the log ramp and the clearer WFM display of a rectilinear grayscale. And both the ChromaMatch and the stress test do useful things on the vectorscope. So it’s not the end of the world if there’s no CCDM; with a bit of added work those of us who care can winkle out 96% of the same info from the charts on offer.

That’s all: I just wanted to clarify that we were asking for certain charts not because they’re “TV charts” or we’re aiming for “TV look”, but because they present the same data in a more readily ’scope-viewable format and/or provide more data (like a log ramp) that can make it easier to understand what’s happening along the tonal scale.

But again: if you’ve got the ChromaMatch and the stress test, we data-driven geeks can deal with it.

Thanks for doing these tests.

Adam Wilt
technical services: consulting / coding / camerawork
Vancouver WA USA (no, not that Vancouver, the other one)

*Sorry, Art: “bluish minus greens”!   ;-)

Art Adams
 

What Adam said. It's not about making cameras look like TV by religiously adhering to their intended levels. It's having something known in the shot for comparison.

I would never actually set up a camera to a Chroma did Monde. They're made for a different era. But they still provide useful standardized info: mid tone gamma info, color rendition, diffuse white, black and gray levels... I just expose for middle gray, white balance, and see where things land. That alone is hugely valuable info.

--
Art Adams
DP
San Francisco Bay Area



On Jun 24, 2018 at 3:26 PM, <Adam Wilt> wrote:

I will not use TV limited charts.

Ah, now I see the problem. Geoff thinks we’re asking for “TV charts” and trying to grade his tests like TV camera tests. No wonder he’s upset!

Geoff, it’s not about TV charts; that’s not the point at all. We don’t care a fig about “TV charts” or “film charts”. We’re not trying to turn you into Alan Roberts. All we care about is useful information

A chart like the Cine ChromaDuMonde has a logarithmic ramp, a stepped gamma-coded grayscale, and colours that ramp between known primaries and secondaries, and it’s laid out so that those patches display clear patterns on the ’scopes. (The colours and grayscale are the same as on the ChromaMatch, BTW, just laid out differently.)

These patterns — regardless of their ranges or nominally “correct” levels — do recognizable and comprehensible things on WFMs and vectorscopes, things that provide direct, graphical evidence of what’s going on in an image that might otherwise be difficult to discern or elucidate: “oh, that’s what happening to my highlights”; “I now understand why this camera makes nice magentas and blues even though bluish purples* look weird”; that sort of thing. 

Granted, looking at ’scopes is a crutch. But just as a crutch helps an ambulatorily-challenged person walk, ’scopes help some of  us see.

And yes: we can window way down on the ChromaMatch to see roughly what the gamma-coded grayscale is doing, even if it’s missing the log ramp and the clearer WFM display of a rectilinear grayscale. And both the ChromaMatch and the stress test do useful things on the vectorscope. So it’s not the end of the world if there’s no CCDM; with a bit of added work those of us who care can winkle out 96% of the same info from the charts on offer.

That’s all: I just wanted to clarify that we were asking for certain charts not because they’re “TV charts” or we’re aiming for “TV look”, but because they present the same data in a more readily ’scope-viewable format and/or provide more data (like a log ramp) that can make it easier to understand what’s happening along the tonal scale.

But again: if you’ve got the ChromaMatch and the stress test, we data-driven geeks can deal with it.

Thanks for doing these tests.

Adam Wilt
technical services: consulting / coding / camerawork
Vancouver WA USA (no, not that Vancouver, the other one)

*Sorry, Art: “bluish minus greens”!   ;-)

Mark Sasahara
 

I'm going to do something really stupid and open my mouth.

Please forgive me if I am being dense and there are already answers to my questions.

Do we need new testing methods and measurement tools? Do we need a new Gray card? Would it even be gray?

What I've gleaned from this thread is that the present test charts and measurement tools seem to struggle to give us the critical information needed to measure and evaluate our latest image making tools (cameras and more).

Does a gray card, Chroma Du Monde, or other charts deliver completely useful information today, as they did the day they were released? Do waveform and vector scopes give us exact and useful information as they did four decades ago? Are they not able to measure critical information and give useful, accurate info? How do you quantify the "jello effect", or image aliasing, or "tearing"?

Does it make sense to point to Pantone colors that correspond to primary, secondary, tertiary colors and skin tones? Skin tones are very difficult, since people's skin can vary with ethnic heritage, temperature, emotion and other variables. Could you refer to a certain color # and reproduce that color in any medium, digitally, on a screen, or in print and have them match? If it's paint, probably. It seems like a good idea. Skin is the hardest, since it is many translucent layers of tissue, liquid and many other things that are nothing like a pigment. Skin tone is one of the most important aspects to our art and craft, yet the most nebulous. 

Do we need a new tool akin to the waveform and vector scope? How do LED emitters, daylight, tungsten, HMI and other lighting sources affect the sensor and how do we measure that? Is there a neutral standard that can be formulated? Do we want one? It seems that for something like a multi cam shoot, it would be ideal to have a dead neutral camera setting that can match across manufacturers. Any warming, cooling, color shifting can be added after, but all the cams would match: B, C, I, P, S. In reality, each manufacturer's color science will skew the colors, dynamic range, contrast, etc. into ways that each mfr finds "pleasing". Another arbitrary factor.

There are a lot of things that cameras do that we want them to do and there are a lot of things we don't want them to do.

Are there tools to measure these different characteristics?

This may need to be moved to a separate thread, but I just want to ask if there need to be new testing and measurement methods.

Thanks,

-Mark Sasahara, DP, NYC

Mark Sasahara
  marksasahara@...
   718-440-1013
    http://msasahara.com


On Sun, Jun 24, 2018 at 6:44 PM, Art Adams <art.cml.only@...> wrote:
What Adam said. It's not about making cameras look like TV by religiously adhering to their intended levels. It's having something known in the shot for comparison.

I would never actually set up a camera to a Chroma did Monde. They're made for a different era. But they still provide useful standardized info: mid tone gamma info, color rendition, diffuse white, black and gray levels... I just expose for middle gray, white balance, and see where things land. That alone is hugely valuable info.

--
Art Adams
DP
San Francisco Bay Area



On Jun 24, 2018 at 3:26 PM, <Adam Wilt> wrote:

I will not use TV limited charts.

Ah, now I see the problem. Geoff thinks we’re asking for “TV charts” and trying to grade his tests like TV camera tests. No wonder he’s upset!

Geoff, it’s not about TV charts; that’s not the point at all. We don’t care a fig about “TV charts” or “film charts”. We’re not trying to turn you into Alan Roberts. All we care about is useful information

A chart like the Cine ChromaDuMonde has a logarithmic ramp, a stepped gamma-coded grayscale, and colours that ramp between known primaries and secondaries, and it’s laid out so that those patches display clear patterns on the ’scopes. (The colours and grayscale are the same as on the ChromaMatch, BTW, just laid out differently.)

These patterns — regardless of their ranges or nominally “correct” levels — do recognizable and comprehensible things on WFMs and vectorscopes, things that provide direct, graphical evidence of what’s going on in an image that might otherwise be difficult to discern or elucidate: “oh, that’s what happening to my highlights”; “I now understand why this camera makes nice magentas and blues even though bluish purples* look weird”; that sort of thing. 

Granted, looking at ’scopes is a crutch. But just as a crutch helps an ambulatorily-challenged person walk, ’scopes help some of  us see.

And yes: we can window way down on the ChromaMatch to see roughly what the gamma-coded grayscale is doing, even if it’s missing the log ramp and the clearer WFM display of a rectilinear grayscale. And both the ChromaMatch and the stress test do useful things on the vectorscope. So it’s not the end of the world if there’s no CCDM; with a bit of added work those of us who care can winkle out 96% of the same info from the charts on offer.

That’s all: I just wanted to clarify that we were asking for certain charts not because they’re “TV charts” or we’re aiming for “TV look”, but because they present the same data in a more readily ’scope-viewable format and/or provide more data (like a log ramp) that can make it easier to understand what’s happening along the tonal scale.

But again: if you’ve got the ChromaMatch and the stress test, we data-driven geeks can deal with it.

Thanks for doing these tests.

Adam Wilt
technical services: consulting / coding / camerawork
Vancouver WA USA (no, not that Vancouver, the other one)

*Sorry, Art: “bluish minus greens”!   ;-)

Geoff Boyle
 

Thank you Mark,

 

You have summed up the problem.

 

18 years ago as we transitioned from SD to HD we could pretty much use the techniques that had been used in the past. As we moved from Film to digital for high end work, starting 14 years or so ago, we could still pretty much use the old test gear but the cameras were improving rapidly and were already at that point of transition producing images that the charts wouldn’t, well, chart.

We went in 18 years from under 1K to 2K to 4K to 8K.

Displays have improved, we have HD, UHD and HDR to contend with.

We still use the old charts and measurements.

 

I have to make it clear at this point that I’m not interested in the numbers, you know what phrase comes next, I want to take a really good calibrated monitor and compare cameras. I want to shoot them in a neutral environment and assemble in a neutral post environment.

 

Anything I do will be stated clearly and will be done in exactly the same way to every camera.

 

I think a large part of the problem is that people want numbers that they can use to compare cameras, this is a powerful marketing tool for manufacturers “mine’s bigger than yours”. I just want to look at the pictures and form an opinion from what the images look like.

 

Cheers

 

Geoff Boyle NSC FBKS

Cinematographer

Netherlands

www.gboyle.nl

 

From: cml-general@... <cml-general@...> On Behalf Of Mark Sasahara

 

Do we need new testing methods and measurement tools? Do we need a new Gray card? Would it even be gray?

 

What I've gleaned from this thread is that the present test charts and measurement tools seem to struggle to give us the critical information needed to measure and evaluate our latest image making tools (cameras and more).

 

 

Mark Sasahara
 

Thank you Geoff!

I am very grateful that you are doing this type of test regularly and so thoroughly.

My question leaps from "What Chart"? to "What chart needs to be invented"? (to measure these sensors?). What testing methodology needs to be invented to measure 21st century sensors, displays and broadcast performance?

How close are we to having a 25 stop REC2020 chart, or something similar? Seems like a simple chart ain't gonna cut it anymore.

Heh, who do I call @ SMPTE?

- Mark Sasahara, DP, NYC


 



Mark Sasahara
  marksasahara@...
   718-440-1013
    http://msasahara.com


On Sun, Jun 24, 2018 at 11:57 PM, Geoff Boyle <geoff.cml@...> wrote:

Thank you Mark,

 

You have summed up the problem.

 

18 years ago as we transitioned from SD to HD we could pretty much use the techniques that had been used in the past. As we moved from Film to digital for high end work, starting 14 years or so ago, we could still pretty much use the old test gear but the cameras were improving rapidly and were already at that point of transition producing images that the charts wouldn’t, well, chart.

We went in 18 years from under 1K to 2K to 4K to 8K.

Displays have improved, we have HD, UHD and HDR to contend with.

We still use the old charts and measurements.

 

I have to make it clear at this point that I’m not interested in the numbers, you know what phrase comes next, I want to take a really good calibrated monitor and compare cameras. I want to shoot them in a neutral environment and assemble in a neutral post environment.

 

Anything I do will be stated clearly and will be done in exactly the same way to every camera.

 

I think a large part of the problem is that people want numbers that they can use to compare cameras, this is a powerful marketing tool for manufacturers “mine’s bigger than yours”. I just want to look at the pictures and form an opinion from what the images look like.

 

Cheers

 

Geoff Boyle NSC FBKS

Cinematographer

Netherlands

www.gboyle.nl

 

From: cml-general@... <cml-general@...> On Behalf Of Mark Sasahara

 

Do we need new testing methods and measurement tools? Do we need a new Gray card? Would it even be gray?

 

What I've gleaned from this thread is that the present test charts and measurement tools seem to struggle to give us the critical information needed to measure and evaluate our latest image making tools (cameras and more).

 

 

Art Adams
 

Hi Mark-

I'm going to do something really stupid and open my mouth.

I wish more people would. :) Well, open their own mouths, not just yours.
 
Do we need new testing methods and measurement tools? Do we need a new Gray card? Would it even be gray?

We do need new testing methods, although I don't know what they would be.

We can only print about six stops of dynamic range with ink and white paper, maybe a bit more with a glossy black. We have cameras that capture 14+ stops of dynamic range. It's probably time for a back-lit chart that covers the same dynamic range and shows a variety of colors across that range. This would not be a cheap chart, though. The DSC Labs Xyla covers 20 stops but only in one color, and there are those that say it isn't accurate enough. It's handmade and costs $3,000.

18% gray is a fairly well established standard, so I think it's a good way to go for exposure. When compared to diffuse white, 2% black, the noise floor and white clip, it's the only thing that doesn't move around much. For each manner of displaying an image there's a target for how bright middle gray should be.
 
Does a gray card, Chroma Du Monde, or other charts deliver completely useful information today, as they did the day they were released?

No. When the Chroma Du Monde came out it was meant to be exposed a specific way. The 90%(ish) white chip was meant to be placed at 100% on a waveform. The black chip (or black cavity, depending on the chart) was meant to be placed at 0%. The crossover was meant to fall at something like 56.5% or something like that. (I have no idea why other than "SMPTE says so.") None of those values apply anymore, at least not with cinema style cameras. Chroma Du Mondes are now available with an 18% gray background, which helps with exposure because that's the only solid exposure reference we can use across cameras. The white chip falls in different places depending on how a log curve or Rec 709-compliant curve is constructed. The black chip often isn't black enough to fall at 0% on a waveform. Still, it's helpful to see how these references fall when exposed to the one reference that doesn't move (18% gray).

The color chips are useful because they represent known colors in the digital world. It used to be that, at 2x gain, they fell into their boxes on a vectorscope, but that's no longer the case as we don't like hues to be that saturated. Often the cameras don't place them in the boxes anyway due to differences in "secret sauce" or because they simply don't saturate hues equally. No camera seems to do a good job at saturating cyan to the extent that Rec 709 expects. Canon tends to skew reds toward orange and greens toward blue. Sony used to pump yellows like crazy, and Arri's original 1D Arri Classic 709 LUT punched red pretty hard. The patterns those colors make on a vectorscope are instructive, even if they don't land where they were originally supposed to fall. (And even if they don't saturate to where they are "supposed to," the vectors still count for color accuracy.)

The design of that chart is interesting because one can see where color channels contaminate each other due to the patterns it creates in a parade RGB waveform. This is an unintended side effect that I discovered years ago when trying to figure out what the original RED ONE was doing under tungsten light.

Ultimately, pretty pictures are in the eye of the beholder, but when the beholder wants to know why the pictures are pretty—or aren't—a chart is good for isolating elements of the image and clarifying what is happening. We know what the chart is supposed to look like, and we have tools that show us how close or far values are from a standard. We can use that information to compare and contrast images that we like and that we don't like to try to figure out what's going on, and whether we can make those images "better." We can also push those images around in specific ways because we can see exactly what we are doing. If I want to make reds more pure, then I can shoot a chart and quickly see how a reference red deviates from where the display expects it to be. I can then push it in the direction that I think I want it to go and see what happens.
 
Do waveform and vector scopes give us exact and useful information as they did four decades ago?

Sure. The charts have fallen behind because the dynamic range isn't there, but the tools still work just fine. They're very instructive.
 
How do you quantify the "jello effect", or image aliasing, or "tearing"?

Ah. That's a different question. That's like quantifying noise. I don't know that anyone has been able to quantify either of those well yet.
 
Does it make sense to point to Pantone colors that correspond to primary, secondary, tertiary colors and skin tones?

What primary, secondary and tertiary colors? Those aren't universal standards. There is no perfect red, green, blue, cyan, magenta or yellow. Those have to be defined in some way. In Rec 709, those colors are defined based on certain specs which differ in P3 and Rec 2020.

I suppose you could define other hues as standards, but I'm not sure what those would be based on.

There are other obstacles. The Chroma Du Monde does something clever in that every primary color falls at 80% in its color channel and 40% in the others: for example, the red patch will be 80% in the red channel and 40% in the blue and green channels. This is because it's impossible to print a hue that's so saturated that it will be 80% red and 0% in green and blue. It means that the chart's colors aren't perfect hues, so visually it may not be the best reference, but one can tune up a camera's color pretty quickly using only a waveform monitor.

Skin tones are very difficult, since people's skin can vary with ethnic heritage, temperature, emotion and other variables.

Actually they all fall within a narrow range that differs primarily in brightness alone.
 
Skin is the hardest, since it is many translucent layers of tissue, liquid and many other things that are nothing like a pigment.

True, but a chart doesn't have to incorporate subsurface scattering to get that hue right.
 
Do we need a new tool akin to the waveform and vector scope?

Sure! I don't know what that would be, though. Technically the vectorscope was never meant to be used the way we use it. It was originally designed to verify color over broadcast TV. That doesn't really matter, though, because it's a very handy visual reference even when used "incorrectly."
 
How do LED emitters, daylight, tungsten, HMI and other lighting sources affect the sensor and how do we measure that?

(1) Shoot a known reference (white, black, gray, primary and secondary colors) under a known and common light source (tungsten, and some form of daylight that can be widely matched.)
(2) Shoot that reference with your LED or HMI light.
(3) Compare the differences visually, and also using tools that allow for more objective information as eyes are easily fooled.

I did this years ago for Cineo products back when they were still working under the PRG banner.
 
It seems that for something like a multi cam shoot, it would be ideal to have a dead neutral camera setting that can match across manufacturers.

The most you can hope for is that manufacturers make their own cameras look as similar as possible. Otherwise, there's no making them neutral across manufacturers. There's no point: each of them approaches color differently, both physically, mathematically and artistically. It not as if sensors are generic things that are shaped into a branded look. Their design, their CFAs, their spectral cut filters and OLPF, how they handle noise... all those things impact color long before before the math kicks in.

There are a lot of things that cameras do that we want them to do and there are a lot of things we don't want them to do.

Yes. Camera color is a massive series of compromises, and each manufacturer compromises in different ways... or not. For example, Arri made the Alexa LF by increasing the size of the sensor instead of making the photo sites smaller because they wanted the look of the new camera to match all of their other cameras. Smaller photo sites means a lower signal-to-noise ratio, which can affect color reproduction dramatically, so making a higher resolution sensor in an S35 form factor wouldn't meet their standards.

Are there tools to measure these different characteristics?

Well... yes. Many of the printed charts we have now do a decent job. There is no one chart that maps out every difference across a camera's full dynamic range. That's what is missing.

--
Art Adams
Director of Photography
San Francisco Bay Area

Geoff Boyle
 

I tried using 3 charts many years ago, keeping the centre on at “Normal” and increasing and decreasing the outer 2.

 

I’m kinda going back to that but differently 😊

 

https://www.cinematography.net/edited-pages/Film-Digi-comparison.htm

 

It was horrendously time consuming and to do it +/- 6 stops would be very difficult.

 

But maybe next year 😊

 

 

Cheers

 

Geoff Boyle NSC FBKS

Cinematographer

Netherlands

www.gboyle.nl

 

 
Edited

On 25 Jun 2018, at 08:23, Art Adams <art.cml.only@...> wrote:

The color chips are useful because they represent known colors in the digital world. It used to be that, at 2x gain, they fell into their boxes on a vectorscope, but that's no longer the case as we don't like hues to be that saturated. 
 
That’s what a racks engineer looking at that chart placed under a studio camera needs to see when matching cameras. 
 
When I started in TV and was daily racking cameras on network TV, Day to day we used a standard chart with no colour so you can make sure the colour balance is neutral.  You need white chips and black cavity to make sure all these are aligned,  steps tell you the gamma curves match.  Colour charts were bought in every month to check the cameras colour alignment. (Well we had one in the studio we to keep an eye on the BVP5s).
 
A camera line up chart like the Chroma Du Monde might not tell you what a camera is doing artistically, but they perform the job they were designed to do brilliantly.  Namely if  the black/white/gamma etc is wrong on the WFM then its wrong in the camera, because the chart and WFM are right.
 
Around the world studio engineers are doing this job daily, along with those charts, Waveform monitors and Vectorscopes.   
 
A WFM is just voltage meter showing you the levels on an SDI signal chain between 0 and 1V.  And that’s the signal that’s being delivered to Transmisson or your streaming server etc.  Its not voodoo!
 
So the question is are you using tools designed to test something different - in which case the results might not be what your expecting, or are you performing the wrong tests?

Michael

Michael J Sanders: Director of Photography 
  

Mobile: +44 (0) 7976 269818   
Linkline Diary: +44 (0)20 8426 2200


Paul Curtis
 

On 25 Jun 2018, at 08:23, Art Adams <art.cml.only@...> wrote:
We do need new testing methods, although I don't know what they would be.
I agree, and here are my thoughts and an idea...

We are trying to understand digital cinema cameras which means we want to quantify what they see, compared to each other. We are setting up test subjects and changing exposure to see how the act over the course, then we're eyeballing them.

But as we move into HDR and as the display outputs evolve, a lot of the cameras are able to capture more and a lot of us aren't running the right displays to be in a position to compare visually. How we are viewing the results is affecting the results.

We are used to seeing gamut triangles but those familiar triangles are just a slice through the gamut and so IMHO are of limited real world use. I don't know what the current attitude is to attachments on the list so i've linked an image

https://inventome.blob.core.windows.net/media/3dgamuts.jpg

On the left is looking down on a gamut where you see the familiar triangle, it's just the 709 primaries. In the middle is how the volume looks from the side and on the right is an image plotted into that volume. You can see from the side that it's quite easy to have an out of gamut colour that from the top (our usual view of a gamut) looks like it is in gamut. So it's quite easy to see how a camera (who's spectral response isn't a clean set of primaries) can create colours that are out of gamut for a 709 space and then we're into how software/firmware handles that - remapping/clipping and so on. Now with wider spaces it gets even more complicated.

I think what Geoff does with ACES exr's makes sense so long as all the transforms into ACES space don't map/destroy the cameras original gamut. Then we could take the exr's and map them into a volume and perhaps compare the results visually like this.

But does a reflective scene really test a camera? if you ramp up and down exposure does that work in the same way that if you were shooting brightly coloured highly saturated lights and scenes at night for example? I can't work that one out.

To my mind you need a single frame that encompasses all that the camera can see, range, saturation which means a single scene that holds more intensity and more saturation than any of the cameras out there. And i don't know what that is or even if it's possible.

So here's an idea. I wonder whether pointing a camera at some super high end HDR display would actually be the best choice, that display showing an CG generated chart of colour and intensity ramps. IMHO i think that would be really interesting to see but i'm not in a position to test this (not having a suitable display) - does anyone else think that's an approach with any merit? What is the highest dynamic range monitor out there? It's easy to generate an EXR with RGB chips from 0 to full saturation and grey scale ramps as well. Even secondary chips. Then a single frame could show more than any of the tested cameras can see in one go. Then you have a simple comparison.

(If not then maybe a series of reflective exposures could be merged together to get a single HDR frame resulting in a snapshot of everything the camera can see (so take a few exposures, adjust the under and over to match the 18% chip) and average them together. So you get one exposure that has under and over in it showing the full range of the camera but you don't get to test noise then. i think this might be a bit too artificial though and not real world...)

cheers
Paul

Paul Curtis, VFX & Post | Canterbury, UK

Art Adams
 

I wonder whether pointing a camera at some super high end HDR display would actually be the best choice, that display showing an CG generated chart of colour and intensity ramps.

I suggested this to a prominent chart company and there was no interest, but I think it's an excellent idea. I'm sure there would be some issues of display resolution beating against the sensor resolution, but it seems to be the only way to get all that info in one place at one time.

Remember how Apple Color (originally something else, I can't remember the name) had a 3D vectorscope? That was a really useful tool. I don't know why someone doesn't bring that back. I'd think it would do similar things to the graphics you created.

There's another thing that can be done with Geoff's tests:


I shot a Chroma du Monde on an Alexa through four different lenses and then used a color picker to build a spreadsheet of how the hues shifted. I wonder if this would be useful to do for camera test results.

The problem is that this is simply one more way to visualize information, and such visualizations—while instructive—only focus on one or two aspects of the image at a time. Looking at actual images is the best way to compare cameras and lights for artistic reasons, but this kind of graphic data helps us dissect what we're seeing to understand it. There's a lot to understand, and a lot of ways to dissect. It's hard to make one chart that could do all this.

I do think the OLED technique could be useful. It seems one could do all sorts of things: dynamic range ramps, saturation ramps... maybe not resolution, but anything related to color and contrast should work well this way. With the proper math it might be the best way to examine a camera's daylight response, because otherwise where do you find consistent daylight? I've used HMIs in the past because they are common, but they are not consistent—but then neither is daylight.

Actually, one could possibly construct a 3200K and 5600K chart all in one. That would be really interesting, but would require someone who's much better at math than I am.

The OLED would not tell you what kind of environment the cameras are in, but it would be a great way to compare them.

--
Art Adams
Director of Photography
San Francisco Bay Area

Bob Kertesz
 

I would never actually set up a camera to a Chroma did Monde. They're
made for a different era.
They're made for a different type of production than perhaps you're used
to doing - live television, or 'live-to-tape' (I hate that term) shows
like variety, sitcoms, sports, news, and content-free derivative reality
garbage. That does not make them from a different era, just fulfilling a
different need than that which many here have.

And to keep things cinematography related, some people have found them
to be useful for creating their own decent REC709 LUTs from a live
camera image, or to see why some manufacturers' built in REC709 LUTs are
so unfuckingbelievably awful.

-Bob

Bob Kertesz
BlueScreen LLC
Hollywood, California

DIT, Video Controller, and live compositor extraordinaire.

High quality images for more than four decades - whether you've wanted
them or not.©

* * * * * * * * * *

Mark Sasahara
 

Dear Art,

This is great, Thank you! Both your post and Adam's have been incredibly helpful.

It is weird that it seems like some of the charts were never exactly correct. But it that's the limitations of paper and ink. As someone who started in stills, shooting a Kodak gray card and chart with a decent gray scale, was great for B&W. If you were shooting color, you'd stick a Macbeth color chart in there, with the first two. It's nice that things like the X-Rite Video Passport exist. I have the mini and front box versions. It's not a C du M, but they get me where I need to go. Now I'm wondering If I should spend the $$$ on a Chroma du Monde, or not.

Yeah, I have to say with the Alexa LF, using three ALEV sensors was a smart move.

So, who is working on the next generation of test chart, if it is in fact a paper chart?

Best,

-Mark Sasahara, DP, NYC

Mark Sasahara
  marksasahara@...
   718-440-1013
    http://msasahara.com


On Mon, Jun 25, 2018 at 3:23 AM, Art Adams <art.cml.only@...> wrote:
Hi Mark-

I'm going to do something really stupid and open my mouth.

I wish more people would. :) Well, open their own mouths, not just yours.
 
Do we need new testing methods and measurement tools? Do we need a new Gray card? Would it even be gray?

We do need new testing methods, although I don't know what they would be.

We can only print about six stops of dynamic range with ink and white paper, maybe a bit more with a glossy black. We have cameras that capture 14+ stops of dynamic range. It's probably time for a back-lit chart that covers the same dynamic range and shows a variety of colors across that range. This would not be a cheap chart, though. The DSC Labs Xyla covers 20 stops but only in one color, and there are those that say it isn't accurate enough. It's handmade and costs $3,000.

18% gray is a fairly well established standard, so I think it's a good way to go for exposure. When compared to diffuse white, 2% black, the noise floor and white clip, it's the only thing that doesn't move around much. For each manner of displaying an image there's a target for how bright middle gray should be.


--
Art Adams
Director of Photography
San Francisco Bay Area

Art Adams
 

If you're just shooting a chart for grading, I'd stick with what you have. A Chroma Du Monde is overkill. A DSC Labs Oneshot would be enough, but I'm thinking of picking up a Passport to see what the quality is like. I feel like X-Rite is in this for the long haul.

The next generation of test chart should be a large OLED display, at least for color and dynamic range. I suggested this to a company I work with and there wasn't any interest, but I think it's an interesting idea. This wouldn't work for a grading chart, but no reason why it shouldn't work for camera and color comparisons. I don't know who's working on anything new.

The LF is two sensors wide. The Alexa 65 is three.

--
Art Adams
Director of Photography
San Francisco Bay Area

Mark Sasahara
 

Thanks for the correction on the sensors.

Yeah, I like the idea of the OLED display. Makes sense, but, how do you put the Cavi-Black in the middle though? :~) I assume cost was the main factor. Go to China, or find a way to do it yourself, here. Or sell the idea.

Which One Shot? Glossy, or Matte? According to Adam, there is a difference. From what I remember from M&P (Materials and Processes of Photography) classes, I think I would go glossy. That would show dynamic range and colors better, like a glossy print/display.

Cheers,

-Mark Sasahara, DP, NYC

Mark Sasahara
  marksasahara@...
   718-440-1013
    http://msasahara.com


On Tue, Jun 26, 2018 at 9:35 PM, Art Adams <art.cml.only@...> wrote:
If you're just shooting a chart for grading, I'd stick with what you have. A Chroma Du Monde is overkill. A DSC Labs Oneshot would be enough, but I'm thinking of picking up a Passport to see what the quality is like. I feel like X-Rite is in this for the long haul.

The next generation of test chart should be a large OLED display, at least for color and dynamic range. I suggested this to a company I work with and there wasn't any interest, but I think it's an interesting idea. This wouldn't work for a grading chart, but no reason why it shouldn't work for camera and color comparisons. I don't know who's working on anything new.

The LF is two sensors wide. The Alexa 65 is three.

--
Art Adams
Director of Photography
San Francisco Bay Area

Art Adams
 

Yeah, I like the idea of the OLED display. Makes sense, but, how do you put the Cavi-Black in the middle though? :~)

Hacksaw. :) OLED displays are glossy, so it'll be a pretty deep black. Deep enough. It would be an interesting way to test a camera—against the display upon which its footage will appear. Seriously, though, it's the only way I know of right now to create a wide color gamut high dynamic range chart. You can't print it... you could do a back-lit version, but it'd be pretty expensive. Instead, write some code, do some math, and put it up on a really good OLED HDR display.

When it comes to charts, glossy is always better for technical reasons. It's difficult to work with because of the reflections, but matte charts have reflections too. They're not as obvious as they are very soft, but they do desaturate color and reduce dynamic range. It's a bit like what happens when you flare a lens. A glossy chart will always be more accurate, because if you can see the reflection you can get rid of it. Can't do that on a matte chart.

They also have a version that's half matte and half glossy, although they haven't sent me one of those to play with.
--
Art Adams
Director of Photography
San Francisco Bay Area