Web Analytics

Curves and Bit Depth - was Sigma fp


Ben Allan ACS
 

Of course, Linear is different to Rec.709.  Current display technology can’t reproduce a true linear image - ACES is an example of true linear encoding and how it looks.  I think the confusion about this was born with Kodak calling their original LUTs a LOG to LIN conversion meaning that it was - “more linear than LOG” ;-)  Kodak though, also accurately described Cineon LOG as a "log-like” exposure response - ie. not strictly logarithmic.

The other thing is that there is also a difference between mathematical log for storing data and visual LOG for creating a more grade-able image.  Most RAW formats use mathematical log to store the data but this needs to be converted into either a visual LOG format or something else like Rec.709 for playback. These visual LOG curves are much more subjective and often go wrong when the designer tries to makes them too mathematically correct. This subjective element is also why each manufacturer has their own LOG curve with it’s own LUT’s.  If they were all mathematically logarithmic then you could use the different manufacturers LUT’s interchangeably.

Because non-RAW recordings require in-camera image processing, many cameras do this at least at 14-bits because processing at higher bit depth than the input or output will produce more accurate results if any conversion is taking place.

Another thing worth remembering is that LOG recordings are called “compressed” in the sense that the compress a wider dynamic range into the same recording dynamics than a more linear format would.  This is a completely separate issue to data compression.

_______________________________
Ben Allan ACS CSI

Producer | Cinematographer | Colorist
ACS National Secretary
Host of "The T-Stop Inn” Podcast




On 27 Dec 2020, at 6:47 pm, Pawel Achtel ACS <pawel.achtel@...> wrote:

Most high-end sensors perform either 10-bit or 12-bit AD quantization. There is no useful signal below 12-bits in any sensor that I know. It is just noise.  


Pawel Achtel, ACS
 

Ø  [Ben Allan] I think the confusion about this was born with Kodak calling their original LUTs a LOG to LIN conversion meaning that it was - “more linear than LOG” ;-)  Kodak though, also accurately described Cineon LOG as a "log-like” exposure response - ie. not strictly logarithmic.

 

Thanks Ben for pointing this all out – all relevant and all correct!

Indeed, uncompressed RAW still requires a curve, whether it is for display, “linearization” or one that “looks more the way our eyes see”. But, in my view, camera is not the place to apply it. There are many advantages of applying it downstream (in post). The reason I would not want any curve applied in-camera is because there are other processing steps that are more beneficial to be applied upstream and which should not be applied in-camera because such processing is either too expensive in terms of GPU required or there is not enough control to do so in-camera, or both.

 

As I said, using log encoding instead of linear saves maybe 20% (12-bit -> 10-bit), give or take depending on the actual implementation. I would call this saving insignificant. And yes, absolutely, the actual processing is usually done at higher bit depth: usually 14-bit or 16-bit to avoid adding quantization noise during the conversion process. Some people, incorrectly, assume that there is actually 14-bit or 16-bit sensor data there; nope! There isn’t!

 

Ø  [Mitch Gross] DPs who don’t have to care about workflow want uncompressed RAW.

Huh? Uncompressed RAW gives you more, not less, control and image quality during post workflow.

Compressed RAW was revolutionary (and beneficial) back in 2009 when storage was expensive (and slow). This is not the case now. Solid state storage is counted in Terabytes these days and offers speeds sufficient for real-time uncompressed 4K+ transfer and playback.

This storage can be purchased in office supplies at low cost. If storage media cost is an issue, perhaps a better cinematographer would be a better way to fix the problem. J

 

Uncompressed RAW is most often quicker (and cheaper) in post than compressed RAW because compression uses GPU to decompress. And, the better the compression the more GPU it will use. At higher resolutions it is very difficult to implement real-time compression that wouldn’t require a $10,000 GPU to compress/decompress in real time.

 

Ø  [Geoff Boyle] The thing is, we're back at the my number is bigger than your number game, and it's bullshit.

Ø  The only thing that matters is what the image looks like.

Sorry Geoff, but I strongly disagree. You can’t compare anything unless you can quantify it. And, in order to quantify it, you need to measure it and put a number on it. It’s called an objective test.

And yes, one number will be bigger than the other number: this is the whole point of testing. It’s called science, not “bullshit”, and this is how equipment is professionally tested and evaluated. Otherwise it is just unsupported, meaningless personal opinion, which is fine, but also completely useless when it comes to making an informed decision J

 

Ø  I don't know of any cinematographer who has refused to shoot with an Alexa because the Res was too low. It's still the camera that is used as a reference.

I agree. Alexa LF produces an amazing amount of detail and contrast. And, indeed, I had an opportunity to measure MTF of Alexa LF (actually put a number on it) and it was the highest I have measured on any other mass-produced digital cinema camera and exceeding that of other “6K’, “8K” and “12K” cameras. A very interesting result. And, this is exactly why we need to measure and quantify things instead of making assumptions (or trust marketing bullshit).

 

More than happy to lend you Sigma fp to test, but it would be probably easier and quicker for me to source BMD 12K in Sydney. Or, we can shoot the same, repeatable test remotely and compare notes. Either way sounds like fun J

 

Kind Regards,

 

Pawel Achtel ACS B.Eng.(Hons) M.Sc.

“Sharp to the Edge”

 

ACHTEL PTY LIMITED, ABN 52 134 895 417

Website: www.achtel.com

Mobile: 040 747 2747 (overseas: +61 4 0747 2747)

Mail: PO BOX 557, Rockdale, NSW 2216, Australia

Address: RA 913 Coles Bay Rd., Coles Bay, TAS 7215, Australia

Location: S 42° 0'14.40"S, E 148°14'47.13"

Email: Pawel.Achtel@...

Facebook: facebook.com/PawelAchtel

Twitter: twitter.com/PawelAchtel

Skype: Pawel.Achtel

_._,_._,_


Paul Curtis
 

I've had one of these since it first came out and it is an amazing camera for it's size and cost. I've spent a lot of time ripping apart the RAW and working out where it's limitation are (for me), this is from memory:

- The sensor appears to be one of the Sony ones and like most sensors in this range, movie capture is limited to 12 bit off the sensor. You will never get more than that. Stills are 14 bit. Log is not needed, RAW is better in every way so long as you've got the workflow and pipeline to support it, you'd never use this cam as a wedding videographer. As a B or C cam it's great.

- Remember that in full frame mode it is downsampling 6K to UHD and with that comes softness. There is no OLPF as such and they rely on that downsampling to hide moire. Personally it's not something that has raised it's head for me but i have seen it. If you do DC mode (crop mode) then you get a UHD crop from the centre of the sensor and that is 1:1 pixel, it's noticeably sharper (and more susceptible)

- Like all of these cameras the workflow is paramount - highlight recovery is essential to extending highlight range and soft rolloffs, also careful transforms of colour are needed to roll off highlight colour too. I've been able to match this to Red footage under IPP2 remarkably well.

- But the range really is around 11/12 stops and for me that is the bigger issue because it means as a B cam side by side with a Red for example, that i cannot use the same lighting as sigma can't hold the highlights outside a window for example, where as Red can. But if you want to tape this camera to the ceiling or put it on some weird rig - it's brilliant.

- Rolling shutter is there and it's very average - better than Sony a7sII but nowhere near the sIII. Longer lenses can really show up the issues too and because it is so light that handholding means micro jitters and this can show up easily.

- As a form factor as a hand held camera the lack of EVF is a pain. The OVF is lovely and offers a great 3 points of contact with the eye but it's position is fixed and it adds so much size to what is a tiny package.

- The 8 bit RAW has a custom log curve in it, well an S curve that keeps the full range in just 8 bits - it's amazing quality. The 12 bit holds the shadows better though. In all cases extreme shadows go green. There's no point in having log as 12 bit RAW gets you all you're going to get off the sensor and it's uncompressed so pulling all the details out works well.

- You can whack a tentacle sync into the audio for TC. Audio is ok overall but i assume we don't care about that.

- The native colourspace is neither 709 or P3, it's whatever is native to the camera. There seems to be a slight advantage to working in a P3 project in Resolve but not really enough to make a huge change. I like the colour of it, it seems natural and malleable.

Having said this i'm hoping a Komodo will fit the bill for me, i have one on the way and will see, but that solves the rolling shutter and the range issue... But i will keep the fp as it's also my stills camera at the moment.

If anyone has any specific questions happy to help if i can..

cheers
Paul

Paul Curtis, VFX & Post | Canterbury, UK


Geoff Boyle
 

I was referring to the standard Alexa not the LF.
Of course we need objective measurements but ultimately we work in a world where taste is more important than fact.
Another story I've told many times is of being dragged by engineers to look at an Olympus 4k video camera, this was in the days when HD was new, 4k was unheard of.
Of course the camera was 4 k but 4k of no dynamic range and dreadful colour response. Unuseable for making movies but the 4k impressed the hell out of the engineers.

Cheers
Geoff Boyle NSC FBKS
EU based cinematographer


Mitch Gross
 

On Dec 27, 2020, at 6:24 AM, Pawel Achtel ACS <pawel.achtel@...> wrote:

Mitch Gross] DPs who don’t have to care about workflow want uncompressed RAW.

Huh? Uncompressed RAW gives you more, not less, control and image quality during post workflow.




Yes, but you are misunderstanding what I mean by “workflow.” Clients don’t want uncompressed RAW. They don’t want to deal with it. It is something different and requires different workflow. When the day’s shooting is done they need three (often more) copies of the footage, including one that they can view on their laptop or is quickly uploaded to a cloud service. They also need a copy of the footage available in a video format with a standard LUT applied for viewing in a Rec709 environment. The files need to load into an Avid, or Premiere, or whatever. And they do not want to wait a day or even a few hours to get the footage like this — they want it when they leave set. Frankly it better be uploading during the course of the day. 

I know that uncompressed RAW is technically the best, purest form to get the signal from the camera to post. I’m the guy who helped introduce the production world to the Phantom cameras, which were the first uncompressed RAW systems generally available to most people (not a lot of Dalsa shooters out there). Uncompressed RAW is the best quality — so what? It is a more burdensome workflow because it does not deliver an immediate result in the form most clients want and it requires more memory which means more time in transfers. I will again state that a good codec such as ProRes with a standardized Log curve applied (C-Log/Log-C/V-Log/etc. - choose your camera’s flavor) is more than enough for 99% of professional production needs and it is easier for clients to use because it is what they expect. Is it technically as good? No. Does that matter? No. 



Mitch Gross
New York


 

This is especially true in the corporate world, a lot of whom now have in house post prod departments.  

There are lots of factors at work there such as availability of storage (when getting IT to sign off on a big server is huge ask) to how to reduce billable hours to one person or a specific machine (seriously one of my clients has a couple of top spec workstations that they internally Bill this way), let alone the person doing the edit has five things on the go and another step is going to impact on that.

So often now I work on the concept of acceptable losses!

Michael

Michael J Sanders

London based Cinematographer/DP 
Mobile: +44 (0) 7976 269818   
Web: www.mjsanders.co.uk


On 27 Dec 2020, at 15:35, Mitch Gross <mitchgrosscml@...> wrote:

They don’t want to deal with it. It is something different and requires different workflow.


Pawel Achtel, ACS
 

Ø  [Mitch Gross] Uncompressed RAW is the best quality — so what? It is a more burdensome workflow because it does not deliver an immediate result in the form most clients want and it requires more memory which means more time in transfers.

You are confusing two things: Log curve (which saves very little space) and compression (which requires additional compression in camera and decompression in post).

 

Log, or let’s call it in general: input curve, can easily travel as metadata or be applied in edit/viewing environment. There is negligible overhead of doing so. It doesn’t need to be “backed-in”. The size benefit of log raw vs. linear raw is relatively minor.

 

Compressed raw requires more powerful hardware (GPU), not less. In particular on a notebook. But, Sigma fp offers both compressed formats and uncompressed DNG. You can chose. Compatibility wise: uncompressed Cinema DNG can be readily used in any NLE and workflow tools and it is more compatible than any compressed raw. For example, BRAW is not widely supported on anything but Resolve and even Resolve requires expensive GPU. RED RAW requires decent GPU too.

Ø  [Geoff Boyle] Of course we need objective measurements but ultimately we work in a world where taste is more important than fact.

Yes, but if we don’t change it no one else will. It amazes me how many DOPs do not know how to produce or read MTF graph or measure dynamic range or measure colour accuracy or gamut of a camera. As part of the industry we need to stick to the fact and objectivity. It is important.

 

Kind Regards,

 

Pawel Achtel ACS B.Eng.(Hons) M.Sc.

“Sharp to the Edge”

 

ACHTEL PTY LIMITED, ABN 52 134 895 417

Website: www.achtel.com

Mobile: 040 747 2747 (overseas: +61 4 0747 2747)

Mail: PO BOX 557, Rockdale, NSW 2216, Australia

Address: RA 913 Coles Bay Rd., Coles Bay, TAS 7215, Australia

Location: S 42° 0'14.40"S, E 148°14'47.13"

Email: Pawel.Achtel@...

Facebook: facebook.com/PawelAchtel

Twitter: twitter.com/PawelAchtel

Skype: Pawel.Achtel

_._,_._,_


John Brawley
 


On Dec 27, 2020, at 2:59 PM, Pawel Achtel ACS <pawel.achtel@...> wrote:

Ø  [Mitch Gross] Uncompressed RAW is the best quality — so what? It is a more burdensome workflow because it does not deliver an immediate result in the form most clients want and it requires more memory which means more time in transfers.

You are confusing two things: Log curve (which saves very little space) and compression (which requires additional compression in camera and decompression in post).

In my many years of experience with DNG as a DP, simply because it is such an open format, it’s also the most likely to get screwed up and mangled in the post pipeline.  Every system opens it a different way despite the metadata and loaded tables.  PS, Google, apple finder, chrome, Resolve.  You open a single frame in any of those platforms and they all look totally different.  No one can work that way for a "lay” hand off.  It requires more oversight from the DP, which these days is typically less possible.

With increasing camera resolution the DNG file sizes also shoot up commensurately.  Even though hard drive space costs are lowering, the rate of file size increase with 4K+ files in DNG negates those cost savings in my view.   

Tell me the data rate for an 8K uncompressed DNG recording in-camera and then let’s run the numbers on what that is like for post compared to REDCODE @ 8K.  What kind of media can even do 8K uncompressed DNG ?  

Uncompressed is just as unwieldily for even competent post facilities.   My last two shows are very large budget for TV (10 million + per ep).  

On one everyone LOVED the Alexa 65 (which only shoots Arri RAW) in my testing, in combination with DNA primes.  It was the chosen camera package until post found out about it.

The Arri RAW data offload went from a single person on a 12 hour shift to three people across 17 hours to get it turned around each day even with HDE. Not only that, the delay for any VFX pulls went up astronomically. They also hit us for a lot of extra hard drive space charge.  

The main reason we didn’t go Alexa 65 was because POST said they couldn’t offload, process and turn those size Arri RAW files around fast enough.  And that meant it ALSO cost more.   Remembering that many shows do post in another country and “the internet” is how files are moved around, you are truly DREAMING if you think DNG is genuinely just as manageable as compressed RAW formats.  Sure at a local hardware level maybe, but not once you have multiple instances and you need to move those files around...

While the computational power to process a DNG is less, the bandwidth required is a heck of a lot more onerous on balance.  No not for an individual system, but when you have to move all these files around to multiple places, bandwidth is a problem.  Computing horse power isn’t so much of a problem to overcome.  No one wants to wait for DNG’s to be moved over the internet.  And that’s how post is done.

So instead we went to Arriraw from a regular SXT, (GASP less than 4K for a major streamer) because the streamer AND post didn’t want the “overhead" of 4K+ uncompressed.  And this was a 10 million per ep show.

Save for a giant budget film maybe, or special application where less compression actually makes a difference, no one is going to willingly pay for uncompressed DNG.  That’s my actual real world experience in the last two years of bigger budget TV,



BRAW is not widely supported on anything but Resolve and even Resolve requires expensive GPU. RED RAW requires decent GPU too. 


Pretty sure you’re quite wrong here. BRAW is actually widely supported.

Resolve obviously, but also

Adobe Premiere
Avid Media Composer
Silverstack (Used by many for dailies ingest / transcode)
On-Set Dailies (Also used for dailies processing / transcode)
Shot Put Pro (used for ingest)
Scratch
Baselight
Lightworks
Edius
PrimeTranscoder
Kyno
Syntheyes (match moving post software)

Basically the only thing BRAW isn't currently supported in is FCP. 


Ø  [Geoff Boyle] Of course we need objective measurements but ultimately we work in a world where taste is more important than fact.

Yes, but if we don’t change it no one else will. It amazes me how many DOPs do not know how to produce or read MTF graph or measure dynamic range or measure colour accuracy or gamut of a camera. As part of the industry we need to stick to the fact and objectivity. It is important. 
 

Pawel, I think you’re projecting (and judging) what “creative” people want.  How would you know ?  I personally don’t think MTF’s matter much when everyone is detuning lenses these days.  Resolution isn’t the only property of a lens and in my view, for most cinematographers and creative users of lenses, it’s way down the list of qualities that are desired.

Just like the argument about “accurate” colour and “nice” colour.  

While MTFs are useful to some, they are totally irrelevant to others.  A metric that’s important in the work that you do, isn’t always what’s important to another image maker and frankly I think that’s why most don’t care much for MTFs.  What does it “tell” you how it looks on a human face or how the lens performs with lots of backlight.  

Isn’t the MTF just one of many things that “could” matter to the individual creative choice of an image maker ?


John Brawley ACS
Cinematographer
Los Angeles


Adrian Jebef
 

Yo all I’m gonna say is it’s 2021 so if any of youz still think shooting uncompressed or raw or best quality whatever  is “hard” or “not worth the effort” or “Post says...” y’all need to stop listening to the Crybabies out there and just shoot full rez, full quality every time. Unless of course y’all shooting weddings and reality tv. Truth is the kid down the street has a faster workflow with his laptop than any big box Post House these days. No need to subsidize their old tech by being afraid of using the new tech.

When in doubt ask your DIT to run the Post numbers if you’re concerned about Budget and camera format.  Usually they’re the smartest tool in the shed.  Plus it’s always nice to get a third opinion before someone makes you do something ya don’t need to do.

Adrian Jebef 
DIT










On Dec 27, 2020, at 5:27 PM, John Brawley <john@...> wrote:


On Dec 27, 2020, at 2:59 PM, Pawel Achtel ACS <pawel.achtel@...> wrote:

Ø  [Mitch Gross] Uncompressed RAW is the best quality — so what? It is a more burdensome workflow because it does not deliver an immediate result in the form most clients want and it requires more memory which means more time in transfers.

You are confusing two things: Log curve (which saves very little space) and compression (which requires additional compression in camera and decompression in post).

In my many years of experience with DNG as a DP, simply because it is such an open format, it’s also the most likely to get screwed up and mangled in the post pipeline.  Every system opens it a different way despite the metadata and loaded tables.  PS, Google, apple finder, chrome, Resolve.  You open a single frame in any of those platforms and they all look totally different.  No one can work that way for a "lay” hand off.  It requires more oversight from the DP, which these days is typically less possible.

With increasing camera resolution the DNG file sizes also shoot up commensurately.  Even though hard drive space costs are lowering, the rate of file size increase with 4K+ files in DNG negates those cost savings in my view.   

Tell me the data rate for an 8K uncompressed DNG recording in-camera and then let’s run the numbers on what that is like for post compared to REDCODE @ 8K.  What kind of media can even do 8K uncompressed DNG ?  

Uncompressed is just as unwieldily for even competent post facilities.   My last two shows are very large budget for TV (10 million + per ep).  

On one everyone LOVED the Alexa 65 (which only shoots Arri RAW) in my testing, in combination with DNA primes.  It was the chosen camera package until post found out about it.

The Arri RAW data offload went from a single person on a 12 hour shift to three people across 17 hours to get it turned around each day even with HDE. Not only that, the delay for any VFX pulls went up astronomically. They also hit us for a lot of extra hard drive space charge.  

The main reason we didn’t go Alexa 65 was because POST said they couldn’t offload, process and turn those size Arri RAW files around fast enough.  And that meant it ALSO cost more.   Remembering that many shows do post in another country and “the internet” is how files are moved around, you are truly DREAMING if you think DNG is genuinely just as manageable as compressed RAW formats.  Sure at a local hardware level maybe, but not once you have multiple instances and you need to move those files around...

While the computational power to process a DNG is less, the bandwidth required is a heck of a lot more onerous on balance.  No not for an individual system, but when you have to move all these files around to multiple places, bandwidth is a problem.  Computing horse power isn’t so much of a problem to overcome.  No one wants to wait for DNG’s to be moved over the internet.  And that’s how post is done.

So instead we went to Arriraw from a regular SXT, (GASP less than 4K for a major streamer) because the streamer AND post didn’t want the “overhead" of 4K+ uncompressed.  And this was a 10 million per ep show.

Save for a giant budget film maybe, or special application where less compression actually makes a difference, no one is going to willingly pay for uncompressed DNG.  That’s my actual real world experience in the last two years of bigger budget TV,



BRAW is not widely supported on anything but Resolve and even Resolve requires expensive GPU. RED RAW requires decent GPU too. 


Pretty sure you’re quite wrong here. BRAW is actually widely supported.

Resolve obviously, but also

Adobe Premiere
Avid Media Composer
Silverstack (Used by many for dailies ingest / transcode)
On-Set Dailies (Also used for dailies processing / transcode)
Shot Put Pro (used for ingest)
Scratch
Baselight
Lightworks
Edius
PrimeTranscoder
Kyno
Syntheyes (match moving post software)

Basically the only thing BRAW isn't currently supported in is FCP. 


Ø  [Geoff Boyle] Of course we need objective measurements but ultimately we work in a world where taste is more important than fact.

Yes, but if we don’t change it no one else will. It amazes me how many DOPs do not know how to produce or read MTF graph or measure dynamic range or measure colour accuracy or gamut of a camera. As part of the industry we need to stick to the fact and objectivity. It is important. 
 

Pawel, I think you’re projecting (and judging) what “creative” people want.  How would you know ?  I personally don’t think MTF’s matter much when everyone is detuning lenses these days.  Resolution isn’t the only property of a lens and in my view, for most cinematographers and creative users of lenses, it’s way down the list of qualities that are desired.

Just like the argument about “accurate” colour and “nice” colour.  

While MTFs are useful to some, they are totally irrelevant to others.  A metric that’s important in the work that you do, isn’t always what’s important to another image maker and frankly I think that’s why most don’t care much for MTFs.  What does it “tell” you how it looks on a human face or how the lens performs with lots of backlight.  

Isn’t the MTF just one of many things that “could” matter to the individual creative choice of an image maker ?


John Brawley ACS
Cinematographer
Los Angeles


Jeff Kreines
 

John Brawley makes a lot of sense. His section re MTFs was music to my ears. My old mentor Ricky Leacock, when he found out what the early 16mm Mk I Zeiss Superspeeds cost, said “Baltars are good enough.”  (He used to shoot with a BNC.)

My partner Jo DeMott, who is much smarter than I am, hated the 9.5 Zeiss that I was trying on my new 1980 Aaton.  Once I saw dailies, I concurred.  Too sharp and the colors were weirdly over saturated.  No character.  So I adapted a 10mm Switar to my LTR.  Looked silly on the camera but great on the screen.  But I digress. 

Cinema DNG is another format from hell.  We had one scanner user insist on uncompressed storage, so we added Cinema DNG.  

Image sequences (like DXF) made sense for 1990s effects work, where a “shot” might be 800 frames long and worked on by several people. 

But for for capture, editing, and archival work, they are dumb.  They are slow and require fast storage to play back in real time. 

As I recall, Resolve couldn’t handle the MXF-wrapped C-DNG, so 1440 files for a 24fps minute were needed, which adds overhead to play back. 

Resolve is great with Cineform RAW... ;-)

Jeff “one-note” Kreines
Kinetta
jeff@...
kinetta.com

Sent from iPhone. 

On Dec 27, 2020, at 7:27 PM, John Brawley <john@...> wrote:


On Dec 27, 2020, at 2:59 PM, Pawel Achtel ACS <pawel.achtel@...> wrote:

Ø  [Mitch Gross] Uncompressed RAW is the best quality — so what? It is a more burdensome workflow because it does not deliver an immediate result in the form most clients want and it requires more memory which means more time in transfers.

You are confusing two things: Log curve (which saves very little space) and compression (which requires additional compression in camera and decompression in post).

In my many years of experience with DNG as a DP, simply because it is such an open format, it’s also the most likely to get screwed up and mangled in the post pipeline.  Every system opens it a different way despite the metadata and loaded tables.  PS, Google, apple finder, chrome, Resolve.  You open a single frame in any of those platforms and they all look totally different.  No one can work that way for a "lay” hand off.  It requires more oversight from the DP, which these days is typically less possible.

With increasing camera resolution the DNG file sizes also shoot up commensurately.  Even though hard drive space costs are lowering, the rate of file size increase with 4K+ files in DNG negates those cost savings in my view.   

Tell me the data rate for an 8K uncompressed DNG recording in-camera and then let’s run the numbers on what that is like for post compared to REDCODE @ 8K.  What kind of media can even do 8K uncompressed DNG ?  

Uncompressed is just as unwieldily for even competent post facilities.   My last two shows are very large budget for TV (10 million + per ep).  

On one everyone LOVED the Alexa 65 (which only shoots Arri RAW) in my testing, in combination with DNA primes.  It was the chosen camera package until post found out about it.

The Arri RAW data offload went from a single person on a 12 hour shift to three people across 17 hours to get it turned around each day even with HDE. Not only that, the delay for any VFX pulls went up astronomically. They also hit us for a lot of extra hard drive space charge.  

The main reason we didn’t go Alexa 65 was because POST said they couldn’t offload, process and turn those size Arri RAW files around fast enough.  And that meant it ALSO cost more.   Remembering that many shows do post in another country and “the internet” is how files are moved around, you are truly DREAMING if you think DNG is genuinely just as manageable as compressed RAW formats.  Sure at a local hardware level maybe, but not once you have multiple instances and you need to move those files around...

While the computational power to process a DNG is less, the bandwidth required is a heck of a lot more onerous on balance.  No not for an individual system, but when you have to move all these files around to multiple places, bandwidth is a problem.  Computing horse power isn’t so much of a problem to overcome.  No one wants to wait for DNG’s to be moved over the internet.  And that’s how post is done.

So instead we went to Arriraw from a regular SXT, (GASP less than 4K for a major streamer) because the streamer AND post didn’t want the “overhead" of 4K+ uncompressed.  And this was a 10 million per ep show.

Save for a giant budget film maybe, or special application where less compression actually makes a difference, no one is going to willingly pay for uncompressed DNG.  That’s my actual real world experience in the last two years of bigger budget TV,



BRAW is not widely supported on anything but Resolve and even Resolve requires expensive GPU. RED RAW requires decent GPU too. 


Pretty sure you’re quite wrong here. BRAW is actually widely supported.

Resolve obviously, but also

Adobe Premiere
Avid Media Composer
Silverstack (Used by many for dailies ingest / transcode)
On-Set Dailies (Also used for dailies processing / transcode)
Shot Put Pro (used for ingest)
Scratch
Baselight
Lightworks
Edius
PrimeTranscoder
Kyno
Syntheyes (match moving post software)

Basically the only thing BRAW isn't currently supported in is FCP. 


Ø  [Geoff Boyle] Of course we need objective measurements but ultimately we work in a world where taste is more important than fact.

Yes, but if we don’t change it no one else will. It amazes me how many DOPs do not know how to produce or read MTF graph or measure dynamic range or measure colour accuracy or gamut of a camera. As part of the industry we need to stick to the fact and objectivity. It is important. 
 

Pawel, I think you’re projecting (and judging) what “creative” people want.  How would you know ?  I personally don’t think MTF’s matter much when everyone is detuning lenses these days.  Resolution isn’t the only property of a lens and in my view, for most cinematographers and creative users of lenses, it’s way down the list of qualities that are desired.

Just like the argument about “accurate” colour and “nice” colour.  

While MTFs are useful to some, they are totally irrelevant to others.  A metric that’s important in the work that you do, isn’t always what’s important to another image maker and frankly I think that’s why most don’t care much for MTFs.  What does it “tell” you how it looks on a human face or how the lens performs with lots of backlight.  

Isn’t the MTF just one of many things that “could” matter to the individual creative choice of an image maker ?


John Brawley ACS
Cinematographer
Los Angeles


John Brawley
 




On Dec 27, 2020, at 6:18 PM, Adrian Jebef via cml.news <adrianjebef=yahoo.com@...> wrote:
Yo all I’m gonna say is it’s 2021 so if any of youz still think shooting uncompressed or raw or best quality whatever  is “hard” or “not worth the effort” or “Post says...” y’all need to stop listening to the Crybabies out there 


Thanks for DITsplaining that to me man, but there’s a bunch of studio execs, post production execs and producers that are in front of you my friend that get to arbitrate these things. 

I’m all for shaking the tree, but money talks.  

JB

John Brawley ACS
Cinematographer
Currently shooting compressed RAW in Los Angeles




Ben Allan ACS
 

The other thing about MTF is that there’s no inherent loss of MTF in the image chain and with digital grading we have so much more control of the ways that contrast behaves in different parts of the image, that it is possible to change the contrast/resolution perception of the audience quite dramatically in ways that are fairly independent of the camera, lenses etc. While MTF still matters to some extent, it’s not nearly the massive issue it was with the photochemical process where you would lose MTF at every stage from the lens to the print.

Also, as John alludes to, the scale of a big production can actually make issues like data rate more of a problem than they are on a small production. Just like I can easily do 4K on my laptop but for a VFX driven blockbuster - 3.4K is a big enough stretch!

_______________________________
Ben Allan ACS CSI

Producer | Cinematographer | Colorist
ACS National Secretary
Host of "The T-Stop Inn” Podcast

On 28 Dec 2020, at 1:26 pm, Jeff Kreines <jeff@...> wrote:

John Brawley makes a lot of sense. His section re MTFs was music to my ears. My old mentor Ricky Leacock, when he found out what the early 16mm Mk I Zeiss Superspeeds cost, said “Baltars are good enough.” (He used to shoot with a BNC.)


Adrian Jebef
 

Oh well. I tried.

-AJ








On Dec 27, 2020, at 7:51 PM, Ben Allan ACS <benallan@...> wrote:

The other thing about MTF is that there’s no inherent loss of MTF in the image chain and with digital grading we have so much more control of the ways that contrast behaves in different parts of the image, that it is possible to change the contrast/resolution perception of the audience quite dramatically in ways that are fairly independent of the camera, lenses etc. While MTF still matters to some extent, it’s not nearly the massive issue it was with the photochemical process where you would lose MTF at every stage from the lens to the print.

Also, as John alludes to, the scale of a big production can actually make issues like data rate more of a problem than they are on a small production. Just like I can easily do 4K on my laptop but for a VFX driven blockbuster - 3.4K is a big enough stretch!

_______________________________
Ben Allan ACS CSI

Producer | Cinematographer | Colorist
ACS National Secretary
Host of "The T-Stop Inn” Podcast

On 28 Dec 2020, at 1:26 pm, Jeff Kreines <jeff@...> wrote:

John Brawley makes a lot of sense. His section re MTFs was music to my ears. My old mentor Ricky Leacock, when he found out what the early 16mm Mk I Zeiss Superspeeds cost, said “Baltars are good enough.” (He used to shoot with a BNC.)


Mitch Gross
 

On Dec 27, 2020, at 6:00 PM, Pawel Achtel ACS <pawel.achtel@...> wrote:

The size benefit of log raw vs. linear raw is relatively minor.
Perhaps you should once again reread what I wrote. I did not reference Log RAW v Linear RAW (although I do not fully agree with your point anyway, but I digress).

I state once again that 99% of production can do just fine recording a VIDEO CODEC such as ProRes and encoding using a Log curve as provided by just about every manufacturer in their cameras.

Except that the Sigma FP does not offer a Log curve, which was my freakin’ point from the get-go. Given a choice of how I expect most people will utilize a US$1800 camera the size of a deck of cards, I believe that the VAST majority would find a high quality video codec with a Log curve more desirable than an uncompressed RAW recording. If it’s a B-cam than Log video is most likely to match the workflow your A-cam is probably using. If the Sigma FP is your A-cam then I suspect that your scale of production is probably one that would prefer a video workflow over an uncompressed RAW data one. I could be wrong, but I’m betting that the odds are damn well in my favor.

I don’t care if the RAW isn’t hard. I don’t care if you can work with the RAW on your laptop. I don’t care if hard drives are cheap. I don’t care.

I do care that my clients and post will have various issues with it that are not nonsensical. They have issues of time and turnaround and access and established functional workflow, and all of those and more are completely real and justified.

Who gives a shit if it’s better if it’s not what the guy signing the checks wants? If a high quality video format and workflow can do the job nicely than why not fit within that system? Give most people a Log curve on a fat video recording so that it can be graded in post and everyone is happy. Again, for 99% of jobs the results will be great and absolutely more than sufficient for the task at hand.

I really hope my point has been made. I don’t know how more clearly I can state it.


Mitch Gross
New York


Mitch Gross
 

On Dec 27, 2020, at 9:18 PM, Adrian Jebef via cml.news <adrianjebef=yahoo.com@...> wrote:

y’all need to stop listening to the Crybabies out there and just shoot full rez, full quality every time. Unless of course y’all shooting weddings and reality tv.
I’m sure you were trying to be colloquial and casually humorous in your post, but with all due respect I have to say that if a DIT said that to a DP on most sets I’ve been around than that person would likely not be returning the next day. We are all professionals here and I know that Mr. Brawley knows of what he speaks and frankly we would all benefit to listen.

Mitch Gross
New York


Geoff Boyle
 

ask your DIT to run the Post numbers if you’re concerned about Budget and camera format.  Usually they’re the smartest tool in the shed.

 

Not my personal experience.

 

I’ll stick with JB’s experiences.

 

I was lucky enough to work mainly with RAW or film scans at much higher than needed resolutions but I generally worked on productions where 45 seconds was a long project and ones that had huge budgets. Regularly $1.5M in todays figures.

 

cheers
Geoff Boyle NSC FBKS
EU based cinematographer
+31 637155076

www.gboyle.nl

www.cinematography.net

Become a CML Patron https://www.patreon.com/bePatron?u=43292735

 

._,_


Maxwell Geoffrey
 

Pawel,

     While do agree with most of what you’re saying from a technical standpoint, I think you’re missing the point that Mitch and Geoff are making.
     Firstly, while uncompressed RAW is certainly a gold standard in terms of getting the absolute highest quality recording from a given camera sensor, the files are bigger and do require more attention to be paid to them in post to ensure that the images shot on set are what actually gets delivered in a final deliverable.  Compressed (non-RAW) recordings will work seamlessly in a REC 709 workflow with almost any NLE and grading application, simply requiring a show LUT or power grade to achieve a finished image.  No attention needs to be paid to ensuring that the proper metadata for ISO, white balance, and gamma curve are being used for each shot, and unlike RAW, the shots don’t require any post production noise reduction.  All of this is completely unfeasible on shoots with faster turnarounds, regardless of budget, but especially on lower budget productions.
     Moreover, the space savings of not shooting RAW, no matter how insignificant they may seem on paper or how cheap storage media becomes, ultimately translate to more shooting time on set, less money spent on drives for backups, or both.  Personally, I’d much rather work with uncompressed RAW that was encoded as 12 bit log, because the 25% smaller files compared to 16 bit linear mean that I get a fourth take on a card if I need it or more later if I don’t, and frankly, I can’t think of a single time when I haven’t been able to roll off the highlights in a grade to my liking because a few extra pixels were encoded at (for example) code value 4071 instead of whatever the 16 bit linear equivalent of 4071.25 may have been.
     In a realistic situation, shooting a short turnaround project on a camera like the C300 Mark III and choosing to shoot 10 bit XF-AVC at 410Mbps over the 12 bit 1Gbps compressed RAW nets me 144% more recording time and a 59% savings in media costs, and if I want higher quality, I can shoot at 810Mbps and still net 12% more recording time and offer a space savings of 19%.  Both of these options save the headache of shooting RAW and then having to oversee the entire post production color management pipeline, which is frankly nonexistent on this kind of project.
     To wrap up on less of a technical context, while I do agree with you that DPs should definitely understand the technicalities of the entire image creation and finishing pipelines, those technicalities often don’t matter nearly as much as we sometimes like to think they do.  As much as technical testing may be a part of our job in pre-production, what ultimately gets seen and gets us hired again are the images we shoot on set, and if they don’t move people or serve the story well, then who cares what bit depth or codec the source files were?  Just my thoughts.

Best Wishes,

Maxwell Geoffrey
DP | Colorist
New York

On Dec 27, 2020, at 18:00, Pawel Achtel ACS <pawel.achtel@...> wrote:



Ø  [Mitch Gross] Uncompressed RAW is the best quality — so what? It is a more burdensome workflow because it does not deliver an immediate result in the form most clients want and it requires more memory which means more time in transfers.

You are confusing two things: Log curve (which saves very little space) and compression (which requires additional compression in camera and decompression in post).

 

Log, or let’s call it in general: input curve, can easily travel as metadata or be applied in edit/viewing environment. There is negligible overhead of doing so. It doesn’t need to be “backed-in”. The size benefit of log raw vs. linear raw is relatively minor.

 

Compressed raw requires more powerful hardware (GPU), not less. In particular on a notebook. But, Sigma fp offers both compressed formats and uncompressed DNG. You can chose. Compatibility wise: uncompressed Cinema DNG can be readily used in any NLE and workflow tools and it is more compatible than any compressed raw. For example, BRAW is not widely supported on anything but Resolve and even Resolve requires expensive GPU. RED RAW requires decent GPU too.

Ø  [Geoff Boyle] Of course we need objective measurements but ultimately we work in a world where taste is more important than fact.

Yes, but if we don’t change it no one else will. It amazes me how many DOPs do not know how to produce or read MTF graph or measure dynamic range or measure colour accuracy or gamut of a camera. As part of the industry we need to stick to the fact and objectivity. It is important.

 

Kind Regards,

 

Pawel Achtel ACS B.Eng.(Hons) M.Sc.

“Sharp to the Edge”

 

ACHTEL PTY LIMITED, ABN 52 134 895 417

Website: www.achtel.com

Mobile: 040 747 2747 (overseas: +61 4 0747 2747)

Mail: PO BOX 557, Rockdale, NSW 2216, Australia

Address: RA 913 Coles Bay Rd., Coles Bay, TAS 7215, Australia

Location: S 42° 0'14.40"S, E 148°14'47.13"

Email: Pawel.Achtel@...

Facebook: facebook.com/PawelAchtel

Twitter: twitter.com/PawelAchtel

Skype: Pawel.Achtel


Adrian Jebef
 

lol

I swear these threads always derail into a bunch of old farts yelling at each other to slow down. Like, “Don’t push the Camery past 65 or Momma gonna find out!” It seems y’all know best: let off the gas.

Sorry some of you don’t care. Sorry Sigma released a camera that is too powerful. Sorry Geoff doesn’t have any DIT friends with a new computer. And sorry if I offend some fragile DP ego. Oh well that’s the film biz for ya. Ya can’t have it all. Maybe Sigma will add HD ProRes Log recording in the mark ii.

Adrian Jebef
DIT







On Dec 27, 2020, at 10:46 PM, Maxwell Geoffrey <maxwell@...> wrote:


Pawel,

     While do agree with most of what you’re saying from a technical standpoint, I think you’re missing the point that Mitch and Geoff are making.
     Firstly, while uncompressed RAW is certainly a gold standard in terms of getting the absolute highest quality recording from a given camera sensor, the files are bigger and do require more attention to be paid to them in post to ensure that the images shot on set are what actually gets delivered in a final deliverable.  Compressed (non-RAW) recordings will work seamlessly in a REC 709 workflow with almost any NLE and grading application, simply requiring a show LUT or power grade to achieve a finished image.  No attention needs to be paid to ensuring that the proper metadata for ISO, white balance, and gamma curve are being used for each shot, and unlike RAW, the shots don’t require any post production noise reduction.  All of this is completely unfeasible on shoots with faster turnarounds, regardless of budget, but especially on lower budget productions.
     Moreover, the space savings of not shooting RAW, no matter how insignificant they may seem on paper or how cheap storage media becomes, ultimately translate to more shooting time on set, less money spent on drives for backups, or both.  Personally, I’d much rather work with uncompressed RAW that was encoded as 12 bit log, because the 25% smaller files compared to 16 bit linear mean that I get a fourth take on a card if I need it or more later if I don’t, and frankly, I can’t think of a single time when I haven’t been able to roll off the highlights in a grade to my liking because a few extra pixels were encoded at (for example) code value 4071 instead of whatever the 16 bit linear equivalent of 4071.25 may have been.
     In a realistic situation, shooting a short turnaround project on a camera like the C300 Mark III and choosing to shoot 10 bit XF-AVC at 410Mbps over the 12 bit 1Gbps compressed RAW nets me 144% more recording time and a 59% savings in media costs, and if I want higher quality, I can shoot at 810Mbps and still net 12% more recording time and offer a space savings of 19%.  Both of these options save the headache of shooting RAW and then having to oversee the entire post production color management pipeline, which is frankly nonexistent on this kind of project.
     To wrap up on less of a technical context, while I do agree with you that DPs should definitely understand the technicalities of the entire image creation and finishing pipelines, those technicalities often don’t matter nearly as much as we sometimes like to think they do.  As much as technical testing may be a part of our job in pre-production, what ultimately gets seen and gets us hired again are the images we shoot on set, and if they don’t move people or serve the story well, then who cares what bit depth or codec the source files were?  Just my thoughts.

Best Wishes,

Maxwell Geoffrey
DP | Colorist
New York

On Dec 27, 2020, at 18:00, Pawel Achtel ACS <pawel.achtel@...> wrote:



Ø  [Mitch Gross] Uncompressed RAW is the best quality — so what? It is a more burdensome workflow because it does not deliver an immediate result in the form most clients want and it requires more memory which means more time in transfers.

You are confusing two things: Log curve (which saves very little space) and compression (which requires additional compression in camera and decompression in post).

 

Log, or let’s call it in general: input curve, can easily travel as metadata or be applied in edit/viewing environment. There is negligible overhead of doing so. It doesn’t need to be “backed-in”. The size benefit of log raw vs. linear raw is relatively minor.

 

Compressed raw requires more powerful hardware (GPU), not less. In particular on a notebook. But, Sigma fp offers both compressed formats and uncompressed DNG. You can chose. Compatibility wise: uncompressed Cinema DNG can be readily used in any NLE and workflow tools and it is more compatible than any compressed raw. For example, BRAW is not widely supported on anything but Resolve and even Resolve requires expensive GPU. RED RAW requires decent GPU too.

Ø  [Geoff Boyle] Of course we need objective measurements but ultimately we work in a world where taste is more important than fact.

Yes, but if we don’t change it no one else will. It amazes me how many DOPs do not know how to produce or read MTF graph or measure dynamic range or measure colour accuracy or gamut of a camera. As part of the industry we need to stick to the fact and objectivity. It is important.

 

Kind Regards,

 

Pawel Achtel ACS B.Eng.(Hons) M.Sc.

“Sharp to the Edge”

 

ACHTEL PTY LIMITED, ABN 52 134 895 417

Website: www.achtel.com

Mobile: 040 747 2747 (overseas: +61 4 0747 2747)

Mail: PO BOX 557, Rockdale, NSW 2216, Australia

Address: RA 913 Coles Bay Rd., Coles Bay, TAS 7215, Australia

Location: S 42° 0'14.40"S, E 148°14'47.13"

Email: Pawel.Achtel@...

Facebook: facebook.com/PawelAchtel

Twitter: twitter.com/PawelAchtel

Skype: Pawel.Achtel


Geoff Boyle
 

I swear these threads always derail into a bunch of old farts yelling at each other to slow down

 

Oh dear, once again, these old farts have been pushing the technology to its limits for far longer than you’ve been alive.

The thing is, we’ve learnt from that experience.

We know what the technology is capable of but we also know that to survive in this business we have to give the clients what they want.

I shot the first NAB promo for the Quantel DPE 5000 at MPC in 1978/9, I could list the history but it’s a waste of time. I’ll stop there.

What am I shooting with at the moment? Err, a 12K compressed RAW camera, usually shooting at Q0 the highest data rate.

The thing is, that works for me but I also understand the realities of production, something you clearly don’t.

 

cheers
Geoff Boyle NSC FBKS
EU based cinematographer
+31 637155076

www.gboyle.nl

www.cinematography.net

Become a CML Patron https://www.patreon.com/bePatron?u=43292735

 

 


Bruce Allen
 

RE the Sigma fp... I hope they do a mk.2 with less rolling shutter (using an updated Sony sensor) and full sensor readout mode (the whole 3:2 area, pixel for pixel, not a weird 6K to 4K “Raw” downsample). I would buy that!

Next request would be both good autofocus AND a manual remote follow focus accessory that works for Sigma’s line of lenses with built-in motors (surely the Sigma camera can talk to the Sigma lenses and tell them to rack smoothly when you move a knob!). That would be a nice tiny setup that pros could tolerate as C camera.

RE: data rates in general - I don’t think it’s fair to call DPs asking for reasonable data rates “old farts”.

It’s NOT yet standard for producers to buy a stack of ultra-fast M.2 SSD drives (in triplicate). And that’s what you need for a proper backup strategy with very high data rates.

We can all push towards helping producers understand the benefits of less compression and faster storage - while still acknowledging reality. I wouldn’t want to use anything with a higher datarate than Alexa LF RAW for the promos I do right now.

I am personally not a fan of REDCODE at high compression rates (noise goes haywire and blotchy).

But I am ALSO not a fan of spending the night after a long shoot trying to get everything backed up so you can make your flight with an intact backup set.

Pre-COVID I co-directed some promos for Netflix for Bridgerton (one is playing on Times Square now a year later)... awesome but my evening after the shoot consisted of DIT handing me a two giant G-RAIDs, then me setting up my extra-fast SSDs (which I bought at my own expense) to backup, then doing a nervous dance until dawn when I headed straight to Heathrow to try to get to Sundance to make my screening. Backup finished with maybe 40 minutes to spare.

I do actually remember thinking “glad this was Alexa LF raw... not Alexa 65 raw!”)

You want crew to be able to get backups done fast. I do not want the DIT (or me!) to miss the fun part of the camaraderie of filmmaking and chatting to everyone... just because someone wants them to spend every night backing up uncompressed footage that will end up heavily compressed and downsampled to below the resolution it was shot at anyway.

So my vote for the Sigma fp’s next version would be light raw compression onboard please. Even if it makes it a bit bigger.

Bruce Allen
Director & VFX supervisor
Los Angeles