Regarding lossy, lossless, compressed, and uncompressed file formats

[Hyydrah]

There is a widespread misunderstanding of file formats that I thought should be cleared up

 

There are different types of file formats, each with their own purposes. They are often known as being lossy or lossless, and compressed or uncompressed

 

A compressed file is a file that is made smaller in size, and may or may not be lossy. An uncompressed file is, obviously, uncompressed

 

A lossy file is a file that loses potential quality compared to its original. A lossless file, on the other hand, does not lose quality since it is either unprocessed (intentionally made to retain all of the original data) or can be reconstructed to precisely form its original after being compressed

 

It should be noted that saving a lossy file as a lossless file later on will not increase its quality, but rather it will preserve the data from the lossy file (which already lost quality from being saved as a lossy file)

 

 

 

Here are some common file formats

 

Audio

 

.flac: lossless, compressed

 

.mp3: lossy, compressed

 

.ogg: lossy, compressed

 

.wmv: lossy, compressed

 

.wav: lossless, uncompressed

 

Images

 

.png: lossless, compressed

 

.jpg: lossy, compressed (typically lossy)

 

Video

 

Lagarith Lossless Codec: lossless, compressed

 

.mp4: lossy, compressed (typically lossy)

 

Despite some people believing that there is no noticeable difference between a lossy and lossless file, in actuality there is, even on low-mid range hardware. For instance, on my S23A700D (a TN panel), I could distinguish a quality difference between screenshots using the JPG file format versus the PNG file format. Even with headphones as cheap as $100, there is often a drastic difference between a MP3 and FLAC/WAV file (in this case with Klipsch Image S4's). The main message that I want to propagate is that files should be saved in lossless formats if you value the preservation of their quality. It is understandable that saving hundreds of songs as FLACs may not be feasible for the average person (at least for the time being), but the next time you decide to rip a CD or take a photo that you care about, please consider the above

[joelthezombie15]

Cool, I always did wonder exactly what that meant, i knew the rough meaning but now i know fully :p

[Vitalius]

I assume converting/saving in a format that is both lossless and compressed requires exponentially more time and computing power than lossy or uncompressed?

[Hyydrah]

I assume converting/saving in a format that is both lossless and compressed requires exponentially more time and computing power than lossy or uncompressed?

I would imagine, but it's worth it due to the decreased file size and lossless quality that comes with it. From my experience, WAV files (lossless, uncompressed) are oftentimes twice the size of FLAC files (lossless, compressed) of the same song, which sound the same otherwise.

[KaareKanin]

I assume converting/saving in a format that is both lossless and compressed requires exponentially more time and computing power than lossy or uncompressed?

For anything else than video or huge batch operations i don't think you'll notice too much on a modern computer. Actually coming to think of it, I think FLAC takes less time to encode than mp3

[WanderingFool]

I assume converting/saving in a format that is both lossless and compressed requires exponentially more time and computing power than lossy or uncompressed?

 

Actually computing a lossless compression can take less time than a lossy compression.  The encoding for png, uses something like run length encoding (or I think it was that...been a while since my image processing classes).

 

For run length encoding it works similar to this: write down the color and then how many times to repeat it

e.g. imagine having an image with 55 55 55 55 55 as colors.  you would write 55 5. ie. 55 appears 5 times in a row.

 

Run length encoding is very easy to do.  Compare this to the lossy compression of jpeg

1. Split the image into 8x8 blocks
2. For each block run an 2d dct (ie. do roughly 120 calculations)
3. Quantize (another 64 operations)
4. Perform lossless compression on the results (Note this means using the same compression techniques it would require at least the same time)

Of course this is now heavily supported by CPU's/GPU's so you won't actually see too much of an performance issue.

 

 

The main reason lossy compression is used over lossless compression is the compression ratio.

lossy jpeg roughly 16:1 (depends on your quality though)

png 3:1

tiff 2:1

The size difference is staggering and depending on the types of images it can be near impossible to tell.  jpeg of course sucks at edges (that is where faults can usually be seen).

 

For audio, I would argue that a properly formed mp3 using a high bitrate one could not tell the difference (with that said your compression ratios start to near lossless compression ratios).

 

 

 

For anything else than video or huge batch operations i don't think you'll notice too much on a modern computer. Actually coming to think of it, I think FLAC takes less time to encode than mp3

 

Actually this makes me want to discuss a key feature of video compression.

 

Typically the time it takes to encode something is only the second thought of an compression algorithm.  The first is decoding, and the best example of this is video encoding/decoding

Encoding a video takes a really long time compared to video decoding which takes a fraction of the time.

[Hyydrah]

Typically the time it takes to encode something is only the second thought of an compression algorithm.  The first is decoding, and the best example of this is video encoding/decoding

Encoding a video takes a really long time compared to video decoding which takes a fraction of the time.

I think a good analogy for that might be the comparison between reading and writing.

[Hyydrah]

Added some content to the original post.

[TheProfosist]

standard JPEG only has lossy compression. 

 

Another lossless video compression is huffyuv its faster than lagarith but doesnt have anywhere near as good compression and isnt multicore aware. Another thing to take into consideration when dealing with lossless video is throughput of the storage device you can run into issues with this with standard hard drives at times which can make real time lossless capture not possible.

 

And ive never had a video file that can be viewed in realtime that was losslessly compressed. And from my experiences it is not possible to put any of those into a mp4. Now days mp4 usually refers to H.264 video with AAC audio. 

[grayperview]

FLAC rips sound great if done properly, especially compared to MP3's.

 

You kind of get used to MP3 over time...when you hear a song using lossless, the difference is astounding.

[TheProfosist]

FLAC rips sound great if done properly, especially compared to MP3's.

 

You kind of get used to MP3 over time...when you hear a song using lossless, the difference is astounding.

 

Agreed especially when you have a good revealing setup.

[grayperview]

maybe CD's and vinyl are the future??? ;)  

[TheProfosist]

maybe CD's and vinyl are the future??? ;)

 

CDs are holding us back and have been for many years :(

[OddballMV]

CDs are holding us back and have been for many years :(

 

How exactly? Digital formats certainly haven't caught up in the very long time they have been around.

[TheProfosist]

How exactly? Digital formats certainly haven't caught up in the very long time they have been around.

 

Sure they have DVD supported up to 24bit 96khz on 2 channels and BD supports far greater than that, at least 24/192 7.1. This is via LPCM which is uncompressed lossless. FLAC, WavPAK, and maybe more can encode these streams easily. I mean heck the Beatles Discography USB flash drive which came out years ago had 24bit 44.1khz FLAC files on it.

[OddballMV]

Sure they have DVD supported up to 24bit 96khz on 2 channels and BD supports far greater than that, at least 24/192 7.1. This is via LPCM which is uncompressed lossless. FLAC, WavPAK, and maybe more can encode these streams easily. I mean heck the Beatles Discography USB flash drive which came out years ago had 24bit 44.1khz FLAC files on it.

 

Ah i thought you meant CDs as a whole catergory of physical media versus digital download. Yes disks have moved on since CD, even Sony with their SACD years and years ago. I certainly don't think standard CDs have that long left in them though, which is sad because it's how i buy all my music at the moment.

[Lyons]

I don't see why FLACs aren't feasible for the average person now, a FLAC song averages at about 40mb, meaning 5000 songs is only 195gb. That's over 400 albums (@ 12 songs/album), taking up only 20% of a 1tb drive, which these days costs under $100.

[TheProfosist]

I don't see why FLACs aren't feasible for the average person now, a FLAC song averages at about 40mb, meaning 5000 songs is only 195gb. That's over 400 albums (@ 12 songs/album), taking up only 20% of a 1tb drive, which these days costs under $100.

 

Who ever said they werent feasible? The only thing I can think of is that most mp3 players that I can think of dont support it. But most smartphones with 3rd party apps do.

[OddballMV]

I don't see why FLACs aren't feasible for the average person now, a FLAC song averages at about 40mb, meaning 5000 songs is only 195gb. That's over 400 albums (@ 12 songs/album), taking up only 20% of a 1tb drive, which these days costs under $100.

 

Who ever said they werent feasible? The only thing I can think of is that most mp3 players that I can think of dont support it. But most smartphones with 3rd party apps do.

 

When you are talking about smartphones / tablets / ultrabooks / ssd laptops there just isn't the storage there to keep even mid-sized FLAC libraries without using a huge portion of your storage versus mp3.

Edited July 6, 2013 by OddballMV

[Lyons]

Who ever said they werent feasible? The only thing I can think of is that most mp3 players that I can think of dont support it. But most smartphones with 3rd party apps do.

 

The OP said they aren't.

 

When you are talking about smartphones / tablets / ultrabooks / ssd laptops there just isn't the storage there to keep even mid-sized FLAC libraries without using a huge portion of your storage versus mp3.

 

Modern quad core CPU's are more than capable of converting to MP3/v0 quickly for lower storage devices, and people generally don't keep their entire libraries on their portable devices. Anyone serious enough about audio to be using FLACs will have good organisation of their storage facilities and device coordination.

[TheProfosist]

When you are talking about smartphones / tablets / ultrabooks / ssd laptops there just isn't the storage there to keep even mid-sized FLAC libraries without using a huge portion of your storage versus mp3.

 

get a nice big fat SD card for them then? my previous 16gb had a fair amount of space and my new 64gb should be more than enough for casual listening. I mean how big of a collection do you need to carry?

[TheProfosist]

The OP said they aren't.

 

o didnt read that part apparently, my bad

Edited July 6, 2013 by TheProfosist

[OddballMV]

Modern quad core CPU's are more than capable of converting to MP3/v0 quickly for lower storage devices, and people generally don't keep their entire libraries on their portable devices. Anyone serious enough about audio to be using FLACs will have good organisation of their storage facilities and device coordination.

 

I use ALAC at home and on my iPod Classic and refuse to put any music on my phone so i don't bother organising anything, just keep it all on the classic  :D. Then again i'm probably not the 'average' user. This is why i'm hoping CDs carry on for a good while longer because the original lossless version has to come from somewhere and i can't see the big music download sites starting to sell in lossless formats, it's a niche market at the moment.

 

get a nice big fat SD card for them then? my previous 16gb had a fair amount of space and my new 64gb should be more than enough for casual listening. I mean how big of a collection do you need to carry?

 

It's not that i want to listen to it all, It's more a time thing, i can't be arsed keep putting things on my player, then taking it off to make more room all the time. So i keep my iPod Classic wrapped up in cotton wool because there isn't much else on the market were it to die.

[TheProfosist]

I use ALAC at home and on my iPod Classic and refuse to put any music on my phone so i don't bother organising anything, just keep it all on the classic  :D. Then again i'm probably not the 'average' user. This is why i'm hoping CDs carry on for a good while longer because the original lossless version has to come from somewhere and i can't see the big music download sites starting to sell in lossless formats, it's a niche market at the moment.

 

 

It's not that i want to listen to it all, It's more a time thing, i can't be arsed keep putting things on my player, then taking it off to make more room all the time. So i keep my iPod Classic wrapped up in cotton wool because there isn't much else on the market were it to die.

 

You can actually get lossless stuff from a few places now but its mostly DJ stuff. An example is Beatport. I doubt this is going to spread into the consumer market any time soon as most people are just fine with their streaming music or music or iTunes or comparable services. CDs are definitely a niche market now days and even though there have been some different versions they were only written more accurately the  audio data was still 16bit 44.1khz

 

I also have a iPod Classic (6th gen sadly, so no rockbox or nice headphone amp) that I have my entire collection on in my car. Thats really the only time I need my whole collection. Otherwise I only need the latest albums and my favorites on my phone as im rarely listening long enough to listen to 1 or 2 albums at a time at most and if that.

[h264]

I assume converting/saving in a format that is both lossless and compressed requires exponentially more time and computing power than lossy or uncompressed?

 

practically, in terms of modern computing terms, the power required is negligible, and so is the extra storage space for lossless.

 

CDs are holding us back and have been for many years :(

 

CDs are already very close to the ideal of audible transparency, but the way they are mastered is definitely holding us back.

 

 

When you are talking about smartphones / tablets / ultrabooks / ssd laptops there just isn't the storage there to keep even mid-sized FLAC libraries without using a huge portion of your storage versus mp3.

 

all the music on my smartphone is in FLAC.  I just rotate it out when I'm done listening to it, just like I'd have to do with MP3s anyways since there's not enough space either way.

 

 

There is a widespread misunderstanding of file formats that I thought should be cleared up.

 

There are different types of file formats, each with their own purposes. They are often known as being lossy or lossless, and compressed or uncompressed.

 

A compressed file is a file that is made smaller in size, and may or may not be lossy. An uncompressed file is, obviously, uncompressed.

 

A lossy file is a file that loses potential quality compared to its original. A lossless file, on the other hand, does not lose quality since it is either unprocessed (intentionally made to retain all of the original data) or can be reconstructed to precisely form its original after being compressed.

 

It should be noted that saving a lossy file as a lossless file later on will not increase its quality, but rather it will preserve the data from the lossy file (which already lost quality from being saved as a lossy file).

 

 

 

Here are some common file formats.

 

Audio

 

FLAC: lossless, compressed

 

MP3: lossy, compressed

 

WAV: lossless, uncompressed

 

Images

 

PNG: lossless, compressed

 

JPG: lossy, compressed (typically lossy)

 

Video

 

Lagarith Lossless Codec: lossless, compressed

 

MP4: lossy, compressed (typically lossy)

 

Despite some people believing that there is no noticeable difference between a lossy and lossless file, in actuality there is, even on low-mid range hardware. For instance, on my S23A700D (a TN panel), I could distinguish a quality difference between screenshots using the JPG file format versus the PNG file format. Even with headphones as cheap as $100, there is often a drastic difference between a MP3 and FLAC/WAV file (in this case with Klipsch Image S4's). The main message that I want to propagate is that files should be saved in lossless formats if you value the preservation of their quality. It is understandable that saving hundreds of songs as FLACs isn't feasible for the average person (at least for the time being), but the next time you decide to rip a CD or take a photo that you care about, please consider the above.

 

 

I'll see if I can make the difference a bit more clear in the FAQ.  Thanks for this.  You forgot h.264/h.265 as a compressed lossy video codec though ^_^

.