Questions About Color Depth and File Compression

[Krocket]

Bit Depth: The number of bits(1s or 0s) to indicate the color of a single pixel.

Typical Monitor: 1920x1080=2,073,600 PIXELS

     2^24=16.7 Million DISPLAY COLORS (24 bits per pixel)

     (2,073,600)*(24)=(6,220,800 Bytes) or (49,766,400 bits)

Image One: 1920x1080=(268KB) or (2,195,456 bits)

Image Two: 1920x1080=(212KB) or (1,736,704 bits)

 

SO, is 6.2 million Bytes the MAX size of a 1920x1080 image or is it always that big and file compression makes it smaller? (Don't know much about file compression) OR is it if a pixel is black it may only use one 0 to define it as such so it ultimately cuts down on the size and is the cause for the different file sizes of the two images? Not exactly about programming or software but I didn't think it belonged in general discussion.

[Ciccioo]

the 6MB image is a normal bitmap image, it has no compression, so we can say it's the worst case scenario, or the "maximum size" as you say

 

the two images you linked are JPEG, so they're compressed, and basically the more rich the picture is, the bigger the file becomes

if an image is black, it can even be 300 megapixels, but the information is one: black. a single pattern that can be quickly encoded in a minimum amount of space

compression algorithms try to find patterns: the more patterns they find, the more effective the compression

 

the first picture apparently has more data in it, the compressing algorithm could find less patterns to encode in a lighter way

[Krocket]

the 6MB image is a normal bitmap image, it has no compression, so we can say it's the worst case scenario, or the "maximum size" as you say

 

the two images you linked are JPEG, so they're compressed, and basically the more rich the picture is, the bigger the file becomes

if an image is black, it can even be 300 megapixels, but the information is one: black. a single pattern that can be quickly encoded in a minimum amount of space

compression algorithms try to find patterns: the more patterns they find, the more effective the compression

 

the first picture apparently has more data in it, the compressing algorithm could find less patterns to encode in a lighter way

That is so cool. Would love to know how algorithms like that work.

[Ciccioo]

That is so cool. Would love to know how algorithms like that work.

lol me too

 

well if your will is strong you can just google it, the algorithm itself shouldn't be excessively complex

[WanderingFool]

That is so cool. Would love to know how algorithms like that work.

 

 

lol me too

 

well if your will is strong you can just google it, the algorithm itself shouldn't be excessively complex

 

haha, you beat me to the punch while I was still writing my post.  I am writing a bit more of an overview of the compressions...but it is going to take a bit longer...but I will explain at least jpg a bit more in depth....got to put my computer image courses to use

[CornOnJacob]

Look up Computerphile's vids on compression. I'm on a tablet and I don't have the time to find and link them right now.

[WanderingFool]

I think I had a post which explained this in more depth...but that have been on another forum since I can't find it here

 

Anyways, @Krocket there are a few things that need to be talked about in order to properly cover this subject....I will go over the summary first so that it is easier for quick reference...then I will go into more depth of talking about them.

 

Summary:

So a typical monitor will display the 1080p 24 bit color as you said.  Now uncompressed it will alway send that 6.2million bytes of data (per image update, or roughly 5.932MB), at 60 fps that is ~356MB/second.  4K at 60fps is 1.39GB/second (11.123Gb/second), which is why HDMI has trouble since HDMI 1.4 only has 1.02GB/second (8.16Gb/second) capacity (not 2.0..which hardly any devices support).  It should actually be noted that monitors receive the 24 bit signal, but your video card has to actually process information in 32 bit (8 bits for transparency)

 

So to answer your question, for monitors they are normally that big.  Now onto those images.  Images are a bit tougher to explain, but ultimately most images are compressed (notable exceptions are BMP files).  With images you have 2 types of compression, lossless and lossy.  A lossless compression will give you back the image you originally gave (every single pixel exactly the same), while a lossy compression will toss away data (or lose it during processing) so you won't get back the same image...but you will get back one that hopefully the human eye can't tell a difference between.  Since lossy tosses away data you can actually save a lot more room.  More details explained in the full description...but basically trust me, it does save room by tossing data

 

To address your zero just putting a single 0, there is actually a lot longer explanation...but for simplicity sake I will just say using that method means you could end up having a lot bigger file than intended (There are compression algorithms that do this, but in the worst case scenarios they could balloon the original file size to 10% of the size )

 

 

More detailed Explanation:

So for the monitor bit, it is actually fairly accurate and simple, so I won't really go into more details about that.  This section will be handing lossy (jpg types) and lossless compression (png) (I actually will deal with this first because jpg uses lossless compression after the lossy portion).

 

Lossless:

So as mentioned above, lossless compression will retain the data.  There is no loss in data...go figure   This is much like zipping a text file, you trade processing power for file size (which is the case with most compressions).  So a classical lossless compression is RLE (This is actually similar to what png uses)...basically the most simple approach to seeing how this works is through example:

 

Imagine you have a data stream that only contains numbers (0-9)...so 00000111111222111  and you know that most data coming in will come in clumps of the same number.  So instead of storing all the number's over and over again you just store the current number and the amount of times it appears....so 00000111111222111 becomes 0(5)1(6)2(3)1(3)...since you have five 0's then six 1's, etc.  This does have it's faults though imagine 0101...it becomes 0(1)1(1)0(1)1(1), effectively doubling the size.

 

 

Lossy:

Now I will deal more with what I know of jpg, because I have had way too many classes that taught me about it.

Images are shown in RGB, because that is what our eyes see...but jpg doesn't store it's data in RGB...instead it stores it in YUV.  YUV is basically storing intensity (Y), and a blend of the RGB color into two colors (UV).  This is only very slightly lossy because you get bits of fractions you have to chop off (If you initially got info as YUV though it would actually be lossless).  The neat thing about our eyes is we can tell intensity a lot better than subtle colors...so what jpg does is something very agressive...it simply chops of every other pixel in the UV sections.

 

So a 100x100 image has a Y of 100x100, but the U and V portions are only images of 50x50.  So it is 1:1:1 down to 1:1/4/:1/4 or in other words you cut the file size down by half.

 

The next part of jpg uses DCT ( It is a mathematical principle that basically clumps similar data together (and you can reverse it).  With 2D DCT, the amount of process can be huge when increasing the size...so the image is chopped up to 8x8 blocks.  This allows many DCT's to be run without too much extra power (This is why you get horrible square blocks when you compress an image too much).  Now I just recommend reading wiki on this, and looking at this image http://en.wikipedia.org/wiki/File:Dctjpeg.png

Each square on the image is actually what it is "looking" for, and the closer it matches the larger the number.  Now the thing about jpg is that they figure that you can see the large changes, and not the very small changes (so topleft of the image from wiki is important, bottom right is least important).  This means you can toss away a lot of the numbers if they are small (ie don't match well).  And if you do that you end up with a lot of 0's as you approach the bottom right.  So jpg only has to store the amount of information that is visually important.  This saves a lot of room.

 

So you have half the image size by the first bit, but then chopping away the less important data means you reduce it even further.

 

With that said I am out of time for typing, I might come back to this later and fill it in if I get the time and have the brain capacity to do it still.

[Avratz]

Where I work, we compress images by first running a 32bit TIFF through a color quantizer to generate a 256 color (8bit) optimized palette (Photoshop in the past. I've since implemented an in house tool to do this). Then the image is compressed with basic RLE (run length encoding). A combination of lossy and lossless compression.

 

Instead of each pixel storing the 32bit RGBA color value, it holds an 8bit palette index. For example, if a pixel is red, and red is the 20th color in the palette then that pixel will hold the value 20. This gets us to 25% of the original file size, and that's before RLE.

 

It's a simple technique to generate much smaller images. The palette size does have it's limitations, but with the right art there's little to no quality loss. Algorithm's like Wu's quantizer and NeuQuant are quite good at generating palettes.

 

http://en.wikipedia.org/wiki/Palette_(computing)

 

http://en.wikipedia.org/wiki/Indexed_color

 

http://en.wikipedia.org/wiki/Color_quantization

 

http://en.wikipedia.org/wiki/Run-length_encoding

[Krocket]

Well, I've learned a heck of a lot. Read and skimmed most of the articles presented by everyone. @WanderingFool glad I was somewhat close to the concept behind lossy compression and thank you for your usual indepth replies @Avratz hopefully I can start learning more about all the different types of compressions and the science behind them.

 

Look up Computerphile's vids on compression. I'm on a tablet and I don't have the time to find and link them right now.

Haha that sent me on a computerphile watching spree ^-^

[LukeTim]

That is so cool. Would love to know how algorithms like that work.

At its most basic, JPEG splits the image into 8x8 blocks.

 

Each 8x8 block is - after each value is shifted so that it runs from -127 to 128, making 0 the middle number - run through a 2 dimensional DCT algorithm (if you've done anything with Audio and know what a FFT is, a DCT is a similar idea... it takes an image signal and converts it from spacial domain to frequency domain)...

Once you have done this, you are left with a block of frequency components. Because in any image you can largely do away with much of the very highest frequency detail without losing too much quality (the human eye can only distinguish so much detail, especially if you're not looking really close up!), you then "quantize" each frequency component by doing some Matrix maths. Your quantization matrix is careully chosen so that lower frequency components are maintained and higher frequency components are mostly done away with (ie, all quantized to zero) - this is the part that makes it lossy because you have lost high frequency components, it's also the point in the algorithm that has the most impact upon final quality. Aggressive quantization can compress massively (lots of components go to 0), but the quality is worse because you've effectively lost those components... JPEG lossless I believe is the same as usual but skips this quantization step. Since so many of the values in the block are now 0, you can do a simple Huffman code, Run-length Encoding or some other lossless arithmetic/variable-length coding technique to shrink it down to a much smaller size...

 

If you want the original image back, you just do everything up above but in reverse order... and use an inverse DCT...

It's actually not as complicated as you might think.

If you want to do video, you more or less do the same thing but compress further still by only encoding the differences between frames (since two consecutive frames will be very similar, you can encode the first as I described above and then only encode the parts that are signififcantly different in the second frame) this sacrifices quality, however. Most video standards have a profile where all frames are compressed similar to as above, without looking at differences between frames, as this maintains quality and has the benefit of being easier/quicker to decode, you might have seen this if you have a video camera with an "All I" option (most often as opposed to "IPB").

Hope this helps... Video (de)Compression is something I do as a day job so if you want to know more I would be happy to maybe help answer questions.

[WanderingFool]

Oh also, look into wavelet's....they are used in JPG 2000, the funny thing is it was meant to replace jpg but it really hasn't (although it is far superior to jpg).  I think the wikipedia article actually does a good job at explaining it, and the image used for it shows the concept quite well.

 

*edit nearly forgot the link http://en.wikipedia.org/wiki/Wavelet_transform

[LukeTim]

Yeah, the Wavelet transform would simply replace the DCT in my explanation...