Raw, Log and Uncompressed Explained
By Andy Shipsides, Thursday, 17 May 2012
At AbelCine, we offer a class on video formats that covers a variety of different terms and compression formats. One of the more common questions I’m asked is, “What’s the difference between uncompressed video, Raw and Log recording?” With so many cameras these days offering different recording options, combined with the popularity of external recorders, it’s no wonder there are a lot of questions about this topic.
Recently, I was asked if shooting Log was like shooting Raw. A short answer is, “Well, yes and no,” which I know isn’t terribly clear. Raw recording is very different than Log, but they have similar applications. To really answer the question, and to understand the difference between all of these formats, we need a little bit of background. ARRI’s ALEXA camera is unique in that it can output Raw, uncompressed and record in a Log format, so I’ll use that camera as an example throughout this discussion. Let’s start with Raw, which comes first for many reasons.
Raw Like Sushi
The idea of Raw recording for motion pictures wasn’t popular until the release of the RED ONE camera a few years ago. RED brought the idea of Raw recording to the masses, though they weren’t the first. Both ARRI and DALSA had cameras that could output Raw sensor data. Raw recorders weren’t exactly common on-set gear though, so we have to hand it to RED for getting the Raw party started in the motion picture business.
So what is Raw anyway? Simply put, it’s just sensor data before any image processing. In a single-sensor camera, like the ALEXA, color is produced by filtering each photosite (or pixel) to produce either red, green or blue values. The color pattern of the photosites most often used is the Bayer pattern, invented by Dr. Bryce E. Bayer at Kodak. The Raw data in a camera like this represents the value of each photosite. Because each pixel contains only one color value, Raw isn’t viewable on a monitor in any discernible way. In a video signal that we can see on a monitor, each pixel contains full color and brightness information; video can tell each pixel on a monitor how bright to be and what color. This means that Raw isn’t video. Raw has to be converted to video for viewing and use. This is usually done through a de-Bayer process, which determines both color and brightness for each finished pixel in your image. The converting of Raw information into video can be time-consuming for post-production, though great tools are available to make it fairly painless. The upside to Raw recording is that no typical video processing has been baked in. The sensor outputs exactly what it sees—no white balance, ISO or other color adjustments are there. This, along with a high bit rate, allows for a huge amount of adjustment in post.
Every camera has a “Raw” step in the image-capturing process. The sensor information is always gathered before converting to a video output, but not all cameras let you record or output that data. The cameras from RED record exclusively Raw data. Sony’s new F65 can record in Raw or HD video, and the ARRI ALEXA can output Raw data over an SDI connection, while recording video internally at the same time. These cameras are converting the Raw data into video for monitoring; the ALEXA adds the ability to output the Raw data at the same time.
So if the Raw data is the real information coming off the sensor, does that mean that it’s uncompressed? This is where things get a bit trickier.
How Uncompressed is it?
Raw data isn’t necessarily uncompressed. In fact, it’s usually compressed. The RED cameras shoot in REDCODE, which has compression options from 3:1 to 18:1. Likewise, Sony’s F65 has 3:1 and 6:1 compression options in F65RAW mode. The Raw data is compressed in much the same ways that traditional video is compressed, and the process does have some effect on image quality. How it shows up in the finished video output can be hard to detect, and the less compressed options in both of these cameras are often considered fairly lossless (limited quality loss). On the other hand, ALEXA outputs uncompressed Raw data, which can be recorded externally. This would be the closest thing to a true uncompressed signal.
So what about uncompressed video? The term “uncompressed” obviously implies a lack of compression, but it isn’t exactly clear what this means in terms of video. As we said before, video is derived from Raw data off a masked single sensor, or something like a 3-CCD imaging block, and usually something is lost along the way. Raw data is usually at high bit depth, between 12- and 16-bit, but video is usually around 8- or 10-bit. In RGB (4:4:4) video, each pixel contains color and brightness information, which would be rather large with 16-bit depth. So, video is generally reduced in bit depth. Additionally, color information is generally reduced as well, from 4:4:4 to 4:2:2. These are both forms of compression that happen in the camera, even before recording. A standard for HD-SDI output on a professional camera is considered to be uncompressed; however, the specification for a single HD-SDI output in a 1920×1080 resolution is 10-bit 4:2:2. Is HD-SDI not uncompressed then? Well, it’s still uncompressed in the sense that there has been no block, wavelet or temporal-based compression applied. Therefore, we can call it uncompressed 10-bit 4:2:2 video—sounds better than color and bit subsampled video, doesn’t it?
The ALEXA can output uncompressed video over its HD-SDI outputs, either in 10-bit 4:2:2 (over Single-Link HD-SDI) or 10-bit 4:4:4 (over Dual-Link HD-SDI). Oddly enough, the camera can record in 12-bit 4:4:4 ProRes internally, but a standard HD-SDI only supports 10-bit. In this case, the uncompressed output is more limited in bit depth because ARRI is conforming to the standards of the signal.
But if Raw is Raw and video is video, then what the heck is Log recording?
Examples of Raw, Log and Rec. 709 video formats.
Bump on a Log
The new Sony, Canon and ARRI cameras all have a Log recording mode. When the Log modes are activated, the image becomes flat and desaturated, but you can still see it on a monitor. This should clue you in that Log recording is just standard video recording in the sense that all pixels display color and brightness information. Log isn’t Raw; it’s video. However, it’s a special way of capturing that maximizes the tonal range of a sensor.
The idea of Log recording came about with Kodak’s Cineon system for scanning film. The system scanned film into a Log format that corresponded to the density of the original film. This maximized the information from the film that could be stored in the video format. Because this information has many shades of gray—very low contrast—it needs to be corrected for proper viewing on a monitor.
Sony, Canon and ARRI have all taken the idea of Log film scanning and applied it to their sensors. They map a “Log” gamma curve that pulls the most information off of their sensors. Sony calls their map S-Log, Canon’s is Canon Log, and ARRI’s is LogC. Each is designed for a specific camera, but all have a similar result. Because Log is a video image, manipulations like white balance and ISO are baked in. A transform of this video data, known as a lookup table (LUT), is required for proper viewing, which makes the video look more “normal” to us. A standard LUT converts the Log video to standard (Rec. 709) HD video.
The ALEXA converts its Raw data into LogC video; this information can be recorded or sent out over HD-SDI. A LUT can be applied for viewing and also can be recorded if desired. Because of this, any step in the chain—Raw, LogC video or standard video—can be recorded.
Putting it all Together
So, going back to our original question, “Is shooting Log like shooting Raw?” The answer is still “yes and no.” Hopefully, now you can see why. Raw is not Log because Log is in a video format, and Raw is not video. Raw data has no video processing baked in and has to be converted into video for viewing. Log is video and has things like white balance baked into it. They’re very much not the same; however, they’re both designed to get the most information out of the sensor. Raw is getting everything the sensor has to offer; likewise, Log curves are designed to get the most tonal range out of the sensor. While they’re very different formats, they have the same general application. Both Raw and Log can be uncompressed, but that depends on the recording device. These terms, and many others, have all become part of our vocabulary in this digital cinema world. Hopefully, this article has helped you navigate them just a little bit better.
This resource is based on an article written by Andy Shipsides for HDVideoPro Magazine. To learn more about topics such as video formats, check out our Understanding HD Series offered by AbelCine’s Training Department.