AAC vs Vorbis vs FLAC blind test (can you hear the difference)

[LAwLz]

Hello everyone!
I was looking through the thread about favorite codecs and though it would be fun to see if people can actually hear the difference between let's say AAC and FLAC.
Now for those of you who don't know, FLAC is a lossless compression algorithm. It does not remove any sound, so if you rip a CD to FLAC, then the FLAC file will sound the same as the CD. If you rip it to a lossy format such as AAC or MP3, then some of the audio will be lost (usually notes humans can't hear anyway).

I have prepared some ~30 seconds audio clips from different songs. I have encoded them to AAC with various bitrate at ~320Kbps, Vorbis/OGG at ~224Kbps, and I have also threw in a lossless FLAC file.
All 3 files are originally from lossless FLAC files with a bitrate of ~1000Kbps. So, let's see if you guys can hear the difference between ~224Kbps, ~320Kbps and 1000Kbps.
I have picked a quite variety of music. Jazz, techno-ish, classic rock, pop, folk music-ish and so on.

 

All you guys need to do is listen to some or all of the songs, decide which one sounds worst, second best and best, and then say which number is which codec and bitrate. The number and codec are the same on all by the way, so if you for example say that 1 is FLAC, then 1 is FLAC on all songs. I have also fiddled with the file sizes, so don't try to cheat by looking at which one is biggest or smallest, and try to figure it out that way because it won't work. Also, all the files might be FLAC now, but that's because I have transcoded them back and forth. I started with a FLAC file, turned it into an AAC file, and then transcoded it back again. Since FLAC is a lossless codec, the new FLAC file will sound just like the AAC file did. I only did this so that you couldn't cheat and look at the file format.
 

TL;DR: Download the songs, listen to them and then tell me which one out of 1, 2 and 3 sounds the best, and which one sounds the worst.

List of songs (don't mind the names):

Africa

Boombox

Champion

Dango

DeskCar

D_Rock_City

Genesis

God

Gonna_Be

Haruhi

Hello

Intro

Island

Kanon

Mami

Moon

Platinum

Sooner

Squid

Tsukihi

Wolf

Working

 

Answers

The answers file tells you which file is which codec, but it is password protected and I won't release the key for a little while. It's just here so that I can't lie and change the answers.
Good luck and have fun.

 

 

The password can now be found on page 4.

List with full songs and song names can be found on page 5.

[Ilumic]

i cheated ;D but, i can hear a clear difference between all 3 , with flac sounding the best, i dont know why but the bass is just deeper with flac on everything 

[Godlygamer23]

The post needs fixing.

[Matsy]

This happens all the time when you edit the post....no idea why, its very annoying.

[Worth]

Dat html

[LAwLz]

Well damn. Looks like my post shat itself. It should be fixed now.

[h264]

Stupid edit bug... use the full reply mode.

As for your test, that's nice, but I apparently need to do a guide on doing a proper double-blind ABX test in foobar2000. Been a while since I bothered to do that myself.

EDIT: oh, here we go:

[mr moose]

You should have left the answers out until enough people had put down what they think. It would have been interesting. I'll listen later, bit busy right now

[LAwLz]

Stupid edit bug... use the full reply mode.

As for your test, that's nice, but I apparently need to do a guide on doing a proper double-blind ABX test in foobar2000. Been a while since I bothered to do that myself.

 

EDIT: oh, here we go:

Yes I know of that plugin. It's really good. Not really suitable for this test though since people could just lie and go "yes I heard the difference".

 

 

You should have left the answers out until enough people had put down what they think. It would have been interesting. I'll listen later, bit busy right now

The answers are password protected. I only linked them because that way, I can't cheat and change the answers.

If I hadn't uploaded the answers, I could just say whatever I wanted. For example if I wanted to "prove that 244Kbps Vorbis is as good as FLAC" then I could just check which answer most people pick as best sounding, and say that was the Vorbis file, even if it was the FLAC file.

 

I'll release the password in a day or so.

[mr moose]

Yes I know of that plugin. It's really good. Not really suitable for this test though since people could just lie and go "yes I heard the difference".

 

 

The answers are password protected. I only linked them because that way, I can't cheat and change the answers.

If I hadn't uploaded the answers, I could just say whatever I wanted. For example if I wanted to "prove that 244Kbps Vorbis is as good as FLAC" then I could just check which answer most people pick as best sounding, and say that was the Vorbis file, even if it was the FLAC file.

 

I'll release the password in a day or so.

Well done, proof I haven't looked otherwise I would have known

 

I am pissed because I lent my only decent head phones to the missus who took them to Vanuatu for the week. I don't think my Microsoft head set has the definition to tell an mp3 from a banana.

 

EDIT: looks like I didn't even read your post in full  seeing as you said in plain text it was protected. :wacko:

[h264]

well, shucks ....

 

 

just kidding.  That's exactly what I expected from a FLAC vs MP3 320 CBR comparison.  Their waveforms are 93.97% identical to begin with.

[h264]

Yes I know of that plugin. It's really good. Not really suitable for this test though since people could just lie and go "yes I heard the difference".

 

If they did that I'd be like "pics of your test results and logs or it didn't happen!"

[mr moose]

I had a quick listen to Africa and boombox and I could not tell a difference. Maybe a slight one but I could easily be fooling myself because I want to hear a difference.

[stevv]

no wav?

[mr moose]

no wav?

wav is raw data, i.e files on a cd, so it should be the same as flac.

[h264]

wav is raw data, i.e files on a cd, so it should be the same as flac.

 

>exactly the same waveform as a FLAC

 

fify

[stevv]

wav is raw data, i.e files on a cd, so it should be the same as flac.

 

I think decoding and software/hardware might be a variable with flac vs wav even if the end product are equal.  I guess that's a topic for another thread, but could possible kill two bird with one stone on this.  

[mr moose]

>exactly the same waveform as a FLAC

 

fify

I was originally going to say identical to flac, but I felt it wasn't entirely accurate even though the end result is.

 

I think decoding and software/hardware might be a variable with flac vs wav even if the end product are equal.  I guess that's a topic for another thread, but could possible kill two bird with one stone on this.  

Not too sure about that, there really is only one way to decode a compressed file, regardless of the program.  because it is an algorithm the end result should be identical every time.

[Kuzma]

Just did a few songs of my own (converted them to 320Kbps Mp3s) and cbb to do 10 so only did 8 and I got 6/8; I can tell which is the FLAC due to the way the guitar sounds (and bear in mind I'm using speakers so with headphones it would be even more evident)

[mr moose]

Just did a few songs of my own (converted them to 320Kbps Mp3s) and cbb to do 10 so only did 8 and I got 6/8; I can tell which is the FLAC due to the way the guitar sounds (and bear in mind I'm using speakers so with headphones it would be even more evident)

not with mine it wouldn't,  fear the Microsoft headset, for it is in a league of its own. :ph34r:

[Kuzma]

not with mine it wouldn't,  fear the Microsoft headset, for it is in a league of its own. :ph34r:

I got a Lifechat Lx-3000; that headset is pretty damn good for its price; I like it ^_^ and I use it; although I can say, apple don't cheap out on displays or sound when it comes to their laptops.

[mr moose]

I got a Lifechat Lx-3000; that headset is pretty damn good for its price; I like it ^_^ and I use it; although I can say, apple don't cheap out on displays or sound when it comes to their laptops.

try using them for Lawlz's comparison samples. :unsure:  I couldn't tell them apart, I reckon the freq, rolls of somewhere after 7Khz making it difficult.

[kuddlesworth9419]

I only just bought a sound card and after listening to a FLAC song for the first time ever I can clearly tell there is a huge difference. Ace Combat - The Unsung War sounds particularly good. 

[smjpl]

Great thread. I can't wait to go home and listen to "Hello" on my ATH-AD700's. The speakers I have with me are just god awful. This is something I am really interested in though. Will post back later, 

[LAwLz]

Lots of people claiming to hear the difference yet no one is guessing which one is which. A bit suspicious if you ask me...