comparison 160Kbps vs 320Kbps vs FLAC Blindtest (aka Is Spotify Premium and/or Hi-Fi Worth It?)

[LAwLz]

TL:DR:

Download one or more of these archives. Listen to the different songs. Pick which one out of A, B and C you think sounds the best.

One is 160Kbps Vorbis.

One is 320Kbps Vorbis.

One is FLAC (around 1000Kbps).

 

Download links at the bottom.

 

 

 

 

Why?

A few years ago I made a comparison between AAC, Vorbis and FLAC. It was pretty fun but not really anything scientific. I used lots of different codecs and at different settings. I've decided to redo the test but with more songs, a wider range of songs, and now only two different codecs. FLAC and Vorbis.

The reason why I chose those two, and the reason why I chose those bitrates is because of Spotify. They recently announced "Spotify Hi-Fi" which costs 5 dollars more each month, but lets you stream lossless audio.

 

The regular Spotify subscription has three different qualities.

96Kbps

The normal quality setting on mobile.

 

160Kbps

Standard quality for the desktop and web player.

High quality setting on mobile.

 

320Kbps (this requires Spotify Premium)

High quality setting on desktop.

Extreme quality setting on mobile.

 

 

So I thought I'd be interesting to see if the people on the forum can hear the difference between the free 160Kbps, the premium 320Kbps or this new Hi-Fi lossless settings.

 

So the test...

Here is what I have done:

1) Picked out 40 different songs from my collection of lossless music.

2) Cut out a 30 second portion of each song.

3) Added a fade in and fade out effect which each lasts 2 seconds. Just to make it less jarring when listening.

4) Made three separate version of each song. Two with Vorbis with a target bitrate of 160 and 320 Kbps, and one kept in the original lossless quality.

5) Converted all of them back to FLAC so that you can't just look at the file format. I also used different compression settings so that you can't just look at the file size.

 

So the result are three identical looking files for each song. One named A, one B and one C. One of them contains ~160Kbps Vorbis audio. One contains ~320Kbps Vorbis audio, and one contains ~1000Kbps FLAC audio.

The challenge is to figure out which one is which.

Please bear in mind that the naming does not change between songs. All songs that ends with A has the same encoding settings. All songs that ends with B has the same. All songs that ends with C is the same.

 

 

Download links:

Here is the download link for the answers. The file is password protected though because I don't want people cheating. I will release the password at some point in time.

Each one of these download links contains 6 different songs (except the last one which only has 4).

 

800 - Aimee - Ayumi - Bella - Cocola - Colleen

Continent - Daisy - Dance - Death - Elizabeth - Eru

Ginger - Green - Horo - Illya - Iselin - Jenkees

Joanne - Kumin - Kuuko - Lotte - Lucy - Maggie

Maria - Merica - Misaka - One - PenPen - Rem

Sahono - Sayaka - Suffle - Silica - Stefani - Sun

Tony - Uiharu - Yui - Yuri

 

I also want to apologize in advance for my music taste. I tried to pick a variety of songs but since I mostly listen to weeb music that's the majority...

[LAwLz]

Password: C\qSj{!n$nqaDz8@

 

Fun fact.

The ~160Kbps collection takes up 22.7MB

The ~320Kbps collection takes up 47.7MB

The FLAC collection takes up 150MB.

 

You can see why Spotify use the settings they use. If everyone was streaming lossless quality they would need more than 6 times the bandwidth from their servers.

[manikyath]

i've actually did some testing between AAC and vorbis for my phone a while ago, and my conclusion pretty much was that at the standard 320kbps AAC is slightly better doing blind A/B testing, but at lower bitrates vorbis truly shines, so if your use case has limited storage space, and the fine details arent as important (for example listening music in an already noisy enviroment) you can save a lot of space using vorbis over AAC, otherwise, just go 320Kbps AAC.

 

should also add that i dont really care for FLAC, i dont really have the headphones or speakers to hear the difference it brings either way.

[WereCat]

I can hear subtle differences but I can't tell which sounds better. All tracks sound good to me. I tried the 1st and 5th album.

I can easily tell apart the differences between 320Kbps and FLAC in the music I listen to (orchestral music), but I was never able to tell much of a difference in other genres (of course, there are some exceptions).

[Volbet]

Edit: Added spoiler as to not affect the results.

 

Hmm, my best guess would be A is the FLAC file, C is the 320kbps Vorbis and B is the 160kbps Vorbis.
This is where I'm horribly wrong and the Elder Council revokes my audiophool license. 

[RatedBlam]

This is really damn tricky. If I had to say something, one of them sounds softer. 

I've only ever notice a big difference if the particular record was recorded really long ago, or if it was done badly. But both of those go without saying.

I hope this will crush my view on the world in a positive way.

 

[SSL]

Lossless streaming is weird ploy. It's obviously meant to appeal to audiophiles, who I would expect make up only a small portion of the target market.

[SSL]

Conclusion hidden so as not to influence results:

 

Spoiler

A couple things to note. First; the construction of this test is flawed by having the same encoding used for all samples of the same letter. I only need to identify the encoding for two samples in order to correctly identify all of them. Second; I can easily cheat by examining the spectrogram of the file.

 

In "Death", sample A is very obviously the FLAC. Transients and treble attack in particular sound much more defined. For the remaining tracks, I found the samples difficult or impossible to differentiate. Can't even begin to say which is 320 vs 160; even where FLAC is identifiable, I'd say both the lossy samples are about equally compromised.

 

I didn't bother taking the time to download and listen to the other sets. This kind of thing is boring and it reveals nothing that we didn't know before - lossless makes a difference in isolated cases, and newer lossy compression schemes are better than mp3, to the point where they are comparable to lossless for most people with most gear. I certainly wouldn't pay for lossless streaming, but then I don't use any streaming services whatsoever to begin with. The streaming model doesn't fit with how I like to own music, much less listen to it.

[LAwLz]

13 minutes ago, SSL said:

Conclusion hidden so as not to influence results:

 

  Hide contents

A couple things to note. First; the construction of this test is flawed by having the same encoding used for all samples of the same letter. I only need to identify the encoding for two samples in order to correctly identify all of them. Second; I can easily cheat by examining the spectrogram of the file.

 

In "Death", sample A is very obviously the FLAC. Transients and treble attack in particular sound much more defined. For the remaining tracks, I found the samples difficult or impossible to differentiate. Can't even begin to say which is 320 vs 160; even where FLAC is identifiable, I'd say both the lossy samples are about equally compromised.

 

I didn't bother taking the time to download and listen to the other sets. This kind of thing is boring and it reveals nothing that we didn't know before - lossless makes a difference in isolated cases, and newer lossy compression schemes are better than mp3, to the point where they are comparable to lossless for most people with most gear. I certainly wouldn't pay for lossless streaming, but then I don't use any streaming services whatsoever to begin with. The streaming model doesn't fit with how I like to own music, much less listen to it.

I don't see how keeping the letters the same makes the test flawed. What I want to test is if people can hear the difference for any type of song (not all kinds), which is why I posted a somewhat wide variety. It might have been better to not reused the letters, but I don't see it as that big of a deal.

About the second point. I expect people to not be assholes. I would have to go to every single person's home to monitor them to avoid the possibility of them cheating that way. Or maybe I could have made a website where there's A/B blind testing, but then there would be no way of knowing if the results I posted afterwards were actually true. I might just have gone "lol nobody got it right!" even if some people did.

 

 

The test is more about letting people test if they can hear it. I can't. Not even in the sample you suggested. That might be because my hearing is messed up though (got tinnitus a few years ago and it has gotten worse). So if you are cheating, then you're only lying to yourself.

[SSL]

3 minutes ago, LAwLz said:

I don't see how keeping the letters the same makes the test flawed. What I want to test is if people can hear the difference for any type of song (not all kinds), which is why I posted a somewhat wide variety. It might have been better to not reused the letters, but I don't see it as that big of a deal.

 

It could influence perception of the other tracks. I might start to think I can hear a difference once I know which is which. Bias can be a funny thing. It doesn't completely invalidate the results, but it's just something to be aware of.

 

4 minutes ago, LAwLz said:

So if you are cheating, then you're only lying to yourself.

 

Ha.

[Dackzy]

36 minutes ago, SSL said:

I certainly wouldn't pay for lossless streaming, but then I don't use any streaming services whatsoever to begin with. The streaming model doesn't fit with how I like to own music, much less listen to it.

 

heh who says you need to pay for it? Just use Tidal free month and then go back to Vinyls and CDs  

You mention that you have less music with a streaming program, why is that? 

[SSL]

Just now, Dackzy said:

heh who says you need to pay for it? Just use Tidal free month and then go back to Vinyls and CDs  

You mention that you have less music with a streaming program, why is that? 

 

No, I said:

 

40 minutes ago, SSL said:

The streaming model doesn't fit with how I like to own music, much less listen to it.

 

Which means that I like to own physical music and listen to it from a local device that I own and control. I also like to listen to tracks as original presented in their albums, not the "playlists" that tweens are so fond of these days.

[Dackzy]

Just now, SSL said:

 

No, I said:

 

 

Which means that I like to own physical music and listen to it from a local device that I own and control. I also like to listen to tracks as original presented in their albums, not the "playlists" that tweens are so fond of these days.

 

I kinda feel the same way as you, but I also use streaming services where I just add albums and then play each album for itself. I have two playlists that are a mix of different artists and both are for when I work out  

[themctipers]

My listening

Spoiler

for 'Colleen', I find that C sounds the best. There's better highs, and the bass is a tiny bit louder and deeper.

A - Lossless

B - 160kbps

C - 320kbps

 

i regret trying to listen more,

 

[LAwLz]

Password: C\qSj{!n$nqaDz8@

 

Answers:

A - FLAC
B - Vorbis 160kbps
C - Vorbis 320kbps

 

[SSL]

So what's the conclusion?

[LAwLz]

11 minutes ago, SSL said:

So what's the conclusion?

Not as big sample size as I had hoped, but it seems like people can tell the difference in some songs, and they can tell the difference (somewhat) between 320Kbps and 160Kbps.

 

So for a few types of songs, Spotify's lossless service is not completely useless, and it seems like the regular subscription definitely has a noticeable difference in audio quality.

[Dackzy]

14 minutes ago, LAwLz said:

So for a few types of songs, Spotify's lossless service is not completely useless, and it seems like the regular subscription definitely has a noticeable difference in audio quality.

 

 

wait does spotify have a lossless version?

[kirashi]

On 4/10/2017 at 2:58 PM, Dackzy said:

wait does spotify have a lossless version?

According to The Verge and some users on Reddit, it's coming soon. 
http://www.theverge.com/2017/3/1/14776780/spotify-hi-fi-preparing-launch-lossless-audio-tier

[Volbet]

7 hours ago, kirashi said:

According to The Verge and some users on Reddit, it's coming soon. 
http://www.theverge.com/2017/3/1/14776780/spotify-hi-fi-preparing-launch-lossless-audio-tier

This baffles me even more than Tidal. 

Atleast Tidal marketed their lossless streaming as an added bonus to their high-pay tier. 

It was mainly marketed as a way to pay the artist more.

 

Now, Spotify doesn't seem to be doing this. They seems to just go "Pay more and get lossless", which is a really strange move.

The demographic for Spotify doesn't seem like the market for lossless audio. 

Hell, 2/3 Spotify users are still on the free tier. And the most prevailing argument for paying (that I've heard, mind you) is offline play and ad free shuffle, not an upgrade in sound quality.

Not to mention that a lot of users use Spotify on-the-go, and lossless streaming is going to rape your data plan. 

 

Lastly, the audiophiles that Spotify might try to market towards probably aren't intersted. 

The people that care about audio quality to that degree are more than likely still buying physical media and are avoiding streaming like the plague. 

[kirashi]

6 minutes ago, Volbet said:

This baffles me even more than Tidal. 

Atleast Tidal marketed their lossless streaming as an added bonus to their high-pay tier. 

It was mainly marketed as a way to pay the artist more.

 

Now, Spotify doesn't seem to be doing this. They seems to just go "Pay more and get lossless", which is a really strange move.

The demographic for Spotify doesn't seem like the market for lossless audio. 

Hell, 2/3 Spotify users are still on the free tier. And the most prevailing argument for paying (that I've heard, mind you) is offline play and ad free shuffle, not an upgrade in sound quality.

Not to mention that a lot of users use Spotify on-the-go, and lossless streaming is going to rape your data plan. 

 

Lastly, the audiophiles that Spotify might try to market towards probably aren't intersted. 

The people that care about audio quality to that degree are more than likely still buying physical media and are avoiding streaming like the plague. 

Oh I cannot disagree - I mean, it's great that even Spotify is going to jump on the "lossless" streaming bandwagon, but it doesn't make sense to me. Most audiophiles are still going to prefer the warmth and more realistic sound that physical media provides, and usually will have the means to afford a high-end stereo setup required to appreciate such a setup. Because of this, I don't see audiophiles as being the target market here.

 

On the other hand, regular "everyday peons" shouldn't even be paying for TIDAL "lossless" since it's being marketed as "better" without pushing for expensive $600 Headphones and a DAC to power them on the go. I might be blind, but I've just seen TIDAL pushing "lossless" quality without actually educating the general public on the expensive equipment, cabling, and configuration required to actually experience "lossless" tracks.

 

However, I believe if Spotify plays its' cards right, and succeeds in the testing that the article claims they're doing, they might decide to partner with WiFi connected stereo manufacturers like SONOS or B&O:Play when pushing their "lossless" streaming tier. While offerings from SONOS still don't match some crazy audiophile setups where there's a DAC, pre-amp, amp, expensive speaker wiring, speakers, and power conditioning at play, their single unit speakers blow other compact stereo systems out of the water, so offering "lossless" streaming to those users might make sense, especially considering the price of some of their speakers.

 

I've put the word "lossless" in quotes because you actually cannot achieve bit-perfect replication of an analogue source. The closest thing to a true lossless recording would involve being in the recording studio during a bands' recording sessions, which is not attainable for obvious reasons, so we'll just have to settle with "lossless" quality.

[SSL]

1 hour ago, Volbet said:

snippsters

 

Audiophiles care, the problem is the total audiophile market segment is small. This is partly why margins on hi-fi crap are so high. And also why lossless streaming seems like a strange move.

 

1 hour ago, kirashi said:

I've put the word "lossless" in quotes because you actually cannot achieve bit-perfect replication of an analogue source. The closest thing to a true lossless recording would involve being in the recording studio during a bands' recording sessions, which is not attainable for obvious reasons, so we'll just have to settle with "lossless" quality.

 

Lossless implicitly refers to the encoding, not the fact that the encoded data is (or is not) itself lossless.

[MasterChromatica]

Audio codecs

 

MP3

By now everybody should be familiar with the MP3 (MPEG2 layer 3) audio codec. It is the nearly-universal standard format for audio compression. It gives a very good compression ratio and is playable by almost all devices. Its only down side is that it is limited to two channels (stereo).
 

AC3

Also known as Dolby Digital (DD), AC3 is a multichannel audio codec used on most DVDs. It is not as compressed as MP3 (which leads to larger file sizes), has of slightly higher quality and is capable of 5.1 channel surround sound. Higher quality rips will often keep the original 5.1 AC3 track intact if it is available on the disc.
 

AAC

Advanced Audio Coding is a successor to MP3. It's Apple's default audio codec for their iPod and iPhone devices. Though technically (slightly) superior to MP3, it is usually only found in MPEG4 files that have been encoded specifically for the iPod.
 

FLAC

Free Lossless Audio Codec is exactly what it says. It's an open source lossless standard for delivering an exact duplicate of CD audio. It is almost never used in a video file container, but you will see high quality soundtrack rips in this format.
 

Ogg

Ogg Vorbis is a free and open source, lossy audio codec project headed by the Xiph.Org Foundation and intended to serve as a replacement for MP3. The format has proven to be popular among supporters of free software. They argue it has higher fidelity and is completely free by nature, unencumbered by patents.
 

DTS

DTS audio codec is a competitor to AC3. It is technically superior to AC3, though only someone with good ears and very good home theater equipment would notice a difference. DTS tracks are mostly included on big-budget DVD releases of major films.

 

 

 

 

 

Transcode FAQ

 

What is a transcode?

Wikipedia says that "Transcoding is the direct digital-to-digital conversion from one (usually lossy) codec to another." A transcode is any conversion of format.
 

Which transcodes are bad?

The rules generally allow only a single lossy stage in the encoding process, and it must be the final stage. So FLAC->MP3 would be allowed, but MP3->Ogg Vorbis and Ogg Vorbis->FLAC would not be allowed.
 

What is a lossy encoder?

Most lossy encoders use a low-pass filter (LPF) when encoding. The filter is set to cut frequencies above a certain point and leave those below. Encoders operate in this manner because high frequencies are more difficult to encode and a person's hearing is less sensitive to higher frequencies. MP3 encoders at 128 kbps will typically use a LPF at 16 kHz. As you raise the bit rate, the frequency threshold also rises. At 192 kbps the LPF is usually set at 18 kHz or higher. Conversely, lossless encoders do not remove any frequencies from the original file.
 

Why is lossy transcoding bad?

Whenever you encode a file to a lossy format (such as MP3, M4A (AAC), Ogg Vorbis, AC3, or DTS) information is permanently lost. It doesn't matter what you do, it's impossible to get this information back without making a new rip from the original lossless source. If you re-encode it, you are reducing the quality. This applies to any lossy to lossy conversion, so even if you convert from 320 kbps to 192 kbps, the final file will still sound worse than if you had just ripped to 192 kbps from the lossless source in the first place.

It's also important to remember to verify that lossless rips actually came from an original source. People that download lossless expect it to be identical to the original. There's no point in people downloading a bigger file just to get another lossy rip.
 

So how do I verify that my upload isn't a transcode?

The simplest way is to rip and encode it from the original source yourself. That way, you know that there has been only one lossy step (or that the rip is truly lossless, if you decided to do a lossless rip).

The recommended software are Audacity, Audition, and SoX (Spectro and Spek have potential for analyzing transcodes too, but there are still a few missing features that the three preferred programs possess.)
 

What is the difference between FhG and LAME?

FhG and LAME are simply two different MP3 encoders. Both operate in a similar way - using the low pass filter to remove higher frequencies and compress the file. Each encoder uses a mathematical algorithm to determine which frequencies to disregard in order to produce the final file, and it is this algorithm that differs between the two (and, in fact, all) MP3 encoders. Most people will tell you that the LAME algorithm is better than the FhG algorithm in that it removes fewer frequencies for the same file size and produces a "cleaner" encode. I strongly recommend using LAME over FhG. (For more information, read the MP3 Specific Dupe Rules.)
 

 

 

How to interpret the frequency spectrogram of songs

I've seen a lot of discussion about how to spot transcodes. Many people have suggested using a spectral analysis from programs like Cool Edit / Adobe Audition and looking at the "cut-off" point. There is some disagreement about how effective this is, but those who recommend it suggest looking for cut-offs between 16 kHz (as the signature of a 128 kbps MP3 source) and 22 kHz (i.e. no cut-off at all) as the signature of a lossless source.

One counter argument to this "cut-off" level method is that the same cut-off that characterizes lossy encodes may also be the result of a poor quality recording - a bootleg of a live show or a "third world" vinyl master.

A number of spectral views have been posted and linked to but nearly all of these have been analyses of entire tracks... which IMHO is NOT the most effective way to use spectral analysis to detect transcodes.

What I haven't seen anyone discuss is the "blocky" appearance of the frequency spectrogram of lossy rips, which is noticeable only when you zoom in close enough. IMHO this is a more reliable way to detect whether a file, which purports to be lossless, has in fact been transcoded from a lossy rip, and may even be a useful way to detect re-encodes from lower to higher bit rate MP3s (although this is much harder whatever method you use). However, some consideration of the source material is necessary here as well. Electronic music, for example, frequently makes use of instruments that use the same technology as lossy audio encoding, so a blocky appearance in the spectrogram might be normal for a lossless source under certain circumstances.

The image below illustrates what I mean. The track (from an album by Philip Glass) was ripped from CD to FLAC and a 1-second sample was saved to 320 kbps LAME MP3 and 128 kbps FhG MP3; and then in each case saved again to FLAC. The spectral analysis was done at full screen on a monitor with a resolution of 1920 x 1080. Each of the three strips below is of the same 0.15 second interval in all three audio files.

 

 

 

And here are bigger strips of the three spectral analyses. The zoom level is the same - bigger simply means that what is shown here is around 0.5 seconds - and NOT the whole track!

And here are bigger strips of the three spectral analyses. The zoom level is the same - bigger simply means that what is shown here is around 0.5 seconds - and NOT the whole track!

FLAC

320 kbps LAME MP3


128kbps FhG MP3

[Majestic]

Subjectively I couldn't tell the difference. Which again proves that audio mastering > bitrate. I can easily tell the shit mix from the good one though

[MasterChromatica]

1 hour ago, Majestic said:

Subjectively I couldn't tell the difference. Which again proves that audio mastering > bitrate. I can easily tell the shit mix from the good one though

If you look a the side to side in my aforementioned post you will see that Flac is the highest then 320 and so forth, that is because flac also has a much wider range of frequency's. So in turn if you have good ears, and a very good sound system/ headphone's Flac is king, then 320.