The Lab Just Bought This…

[Needfuldoer]

8 minutes ago, porina said:

This might be ok for internal uses, but if you want to be taken seriously externally then it isn't really an option.

A desk made out of radiators isn't to be taken seriously either, but the New Janky Workshop still made one of those!

 

It doesn't have to be a serious attempt at replicating its functionality 1:1, but coming up with a homegrown knockoff would help drive home what makes expensive testing equipment so special.

[porina]

5 minutes ago, Needfuldoer said:

A desk made out of radiators isn't to be taken seriously either, but the New Janky Workshop still made one of those!

The difference here is we're talking about test and measurement equipment, nothing at all like what is in essence an extreme PC build.

 

I'll give you the 2nd part, but it is probably a bit too niche for the audience outside of the labs. And it would have to be carefully segregated to not dilute the intent of what labs will become.

 

 

 

Random unrelated thought: I wonder what software they're using. I didn't recognise what was shown. Software licencing in this area can be really expensive too. In the long run, hardware is not so significant.

[quater]

There is not enough shitposting in this thread so i gonna risk getting bonked bonk to horny jail....

We need some proper ASMR test from this Head!

Someone has to lick its ears clean!

 

I don’t make the rules. but it’s the internet....

[spartaman64]

18 hours ago, AnonymousGuy said:

It's like Linus doesn't know Rtings exists.  Cable testing was something new...headphone testing at best you're a shittier copy of Rtings.

afaik rtings use the ear microphone measuring which has its uses to see how much deviations in headphone placement affect the FR curve but overall an inferior and less accurate system than rigs like gras or B&K. if you want accurate measurements then you should look at sources like oratory1990, crinacle, ASR etc that have a proper measuring rig like linus is getting. and its always nice to be able to cross-compare data between different sources though as someone else mentioned most people use gras and linus using B&K affects how directly comparable their data is. and if by cable testing you mean measuring a signal through a cable thats not new at all ASR has been doing it for a while.

 

also nobody says only one review/test of a CPU is enough so gamersnexus, jayztwocents etc should shut down because LTT exists and the same thing applies here for audio. and LTT as a bigger channel might be able to get access to stuff the other reviewers cant

[Spuriae]

9 hours ago, porina said:

My background is telecoms side so I'm not so familiar with the audiophile side or Harman curve, and a quick search is providing plenty of results that don't say an awful lot useful. Is it just a target frequency response curve? Where I used to work we standardised worldwide on B&K, but did look at a stand alone Gras ear simulator at one point as a cheaper alternative. I got one on loan and compared it. We didn't go ahead with it in the end because the response was slightly different to B&K. Not massively so, but enough that it would add complications when comparing and sharing results. While it could be compensated in processing, it was an extra step. Ended up buying another from B&K.

While most audio companies have their own in-house target curve, the Harman curve is the standard for public comparisons because no higher-quality curve is publicly available. The basic shape of the curve is based on the expected measurement of a dummy head in the ideal listening position of an ideal speaker room. The curve was then adjusted based on blind tests in which listeners were given access to shelf filters and asked to adjust them based on preference. Multiple iterations of the curve are described in AES papers.

 

The problem is similar to the one you describe: most of the audiophile headphone industry and media has standardized around GRAS, because the curve was developed with GRAS and Harman's publicly available compensation is only valid for GRAS. Harman has compensations for other simulators which they (supposedly) also use internally, but those are confidential, and they say that those aren't completely comparable using the target.

 

Even with a valid compensation curve for the 5128, the B&K's impedance is slightly different, so there will always be some variation that cannot be corrected for. Even if the frequency response results on the B&K can be technically more accurate to human hearing, this variation (which cannot be completely fixed in post) means that the results are not directly comparable with most existing headphone measurements.

 

9 hours ago, porina said:

Random unrelated thought: I wonder what software they're using. I didn't recognise what was shown. Software licencing in this area can be really expensive too. In the long run, hardware is not so significant.

Looks like REW. Despite the name (and it being free), it's excellent general-purpose audio measurement software.

 

8 hours ago, spartaman64 said:

afaik rtings use the ear microphone measuring which has its uses to see how much deviations in headphone placement affect the FR curve but overall an inferior and less accurate system than rigs like gras or B&K. if you want accurate measurements then you should look at sources like oratory1990, crinacle, ASR etc that have a proper measuring rig like linus is getting. and its always nice to be able to cross-compare data between different sources though as someone else mentioned most people use gras and linus using B&K affects how directly comparable their data is.

Rtings uses the HEAD Acoustics HMS II.3, which is the other HATS shown in the video. Innerfidelity was another notable HMS user. Oratory, Crinacle, and ASR use GRAS. Head-Fi and Soundguys use B&K.

[my2dumbledores]

5 hours ago, Nimrodor said:

While most audio companies have their own in-house target curve, the Harman curve is the standard for public comparisons because no higher-quality curve is publicly available. The basic shape of the curve is based on the expected measurement of a dummy head in the ideal listening position of an ideal speaker room. The curve was then adjusted based on blind tests in which listeners were given access to shelf filters and asked to adjust them based on preference. Multiple iterations of the curve are described in AES papers.

 

The problem is similar to the one you describe: most of the audiophile headphone industry and media has standardized around GRAS, because the curve was developed with GRAS and Harman's publicly available compensation is only valid for GRAS. Harman has compensations for other simulators which they (supposedly) also use internally, but those are confidential, and they say that those aren't completely comparable using the target.

 

Even with a valid compensation curve for the 5128, the B&K's impedance is slightly different, so there will always be some variation that cannot be corrected for. Even if the frequency response results on the B&K can be technically more accurate to human hearing, this variation (which cannot be completely fixed in post) means that the results are not directly comparable with most existing headphone measurements.

Goodness me, what a perfectly succinct explanation. I'm going to refer people to this post in future when it comes up on Reddit/Head-Fi/ASR etc.

But yes, it seems Linus has spent a lot more than he needed to here, and the result will be information which is less useful. I'm certainly far less interested in Head-Fi or Soundguys measurements than I am anything which comes from Dr Olive, Amir, Oratory1990, Crinacle or Resolve.

Bit of a shame, but we'll see. Maybe their new reviewer is good enough to compensate for the B&K. With enough data it will start to make sense. Either way, it'll be a huge improvement over what they've been doing up until now.

[edmazing]

I'm a little confused.

So the audio engineer is being anonymized, the explanation for that is being placed behind a paywall and he's directly influencing decisions on testing hardware. 

I mean you paid to employ him so I guess it's ultimately your call but that seems sketchy. Like maybe there's at least a better way to communicate things.
Like he's wanted or something so we blurred him.

[porina]

6 hours ago, Nimrodor said:

The curve was then adjusted based on blind tests in which listeners were given access to shelf filters and asked to adjust them based on preference. Multiple iterations of the curve are described in AES papers.

Interesting. What people think they want to hear vs speakers. Using the above I've found some better results to read up on it further.

 

6 hours ago, Nimrodor said:

Even with a valid compensation curve for the 5128, the B&K's impedance is slightly different, so there will always be some variation that cannot be corrected for.

Impedance can be measured and compensated, but again it is more additional work. And the more adjustments you have to make, the more uncertainty you add.

 

6 hours ago, Nimrodor said:

Even if the frequency response results on the B&K can be technically more accurate to human hearing, this variation (which cannot be completely fixed in post) means that the results are not directly comparable with most existing headphone measurements.

I don't have access to the data any more, but again I don't recall that difference being that big.

 

6 hours ago, Nimrodor said:

Looks like REW. Despite the name (and it being free), it's excellent general-purpose audio measurement software.

Looks interesting, will have to play with it later.

 

Part of my original question was due to seeing what looked like a Head Acoustics MFE VI on the bench, and their software support contracts can really rack up once you start going on the options.

 

4 minutes ago, edmazing said:

So the audio engineer is being anonymized

Believe they had a case in the past where they featured a new starter who didn't stick around. Now they don't do that until the person has worked for them for some time.

[Needfuldoer]

3 hours ago, edmazing said:

I'm a little confused.

So the audio engineer is being anonymized, the explanation for that is being placed behind a paywall and he's directly influencing decisions on testing hardware. 

I mean you paid to employ him so I guess it's ultimately your call but that seems sketchy. Like maybe there's at least a better way to communicate things.
Like he's wanted or something so we blurred him.

Linus talked about new hires on WAN Show before. There's a law (either province-specific or at the national level) where employers and employees can break off an employment agreement within the first 90 days for any reason with no fault to either party. LMG implements this as a probationary period where they're employed, but don't make on-screen appearances.

 

The audio engineer isn't out of that probation period yet. We'll "meet" them once they are, should they stick around.

 

As for putting the results on Floatplane, I don't think the reasoning has been discussed yet. (If I had to guess, it's because there's no Labs website yet. I hope the Labs findings will be free for open consumption at some point though.)

[LAwLz]

2 hours ago, Needfuldoer said:

As for putting the results on Floatplane, I don't think the reasoning has been discussed yet. (If I had to guess, it's because there's no Labs website yet. I hope the Labs findings will be free for open consumption at some point though.)

I hope they put them on the forum for everyone to see, although I can totally see them putting it behind paywalls.

[Kilrah]

AFAIK Linus has mentioned on previous WAN shows that the intent was for findings to be public, but with the possibility of a paid tier for industry style detailed reports.

[sub68]

On 5/30/2022 at 3:48 PM, AnonymousGuy said:

It's like Linus doesn't know Rtings exists.  Cable testing was something new...headphone testing at best you're a shittier copy of Rtings.

Its nice to have another third party doing testing.

They probably do know but third party with cross references is a nice testing ground.

[Spuriae]

15 hours ago, porina said:

Impedance can be measured and compensated, but again it is more additional work. And the more adjustments you have to make, the more uncertainty you add.

I don't have access to the data any more, but again I don't recall that difference being that big.

You're right that we may be overreacting and it won't be as much of an issue in practice as it's being made out to be right now. There are valid reasons to choose to go for the 5128 (one being that it's not unlikely that someone will release a newer, better target curve using the 5128 at some point in the future). The choice to go with the 5128 without evaluating the GRAS at all is just unexpected given the current state of the hobby, since whether or not the differences can be corrected, the measurements still cannot be compared to existing ones with certainty since we have no good way of verifying whether they can be (short of comparing a GRAS 45C).

 

Dr. Sean Olive (Harman) gave a talk on the 5128's applicability to the Harman curve at a B&K conference last year. The talk itself is unrecorded (as far as I can tell), but a discussion of it is in this ASR thread. Of particular interest are the slides Olive posted on Twitter, linked in post 11.

 

Also (potentially) relevant is ASR's measurements with the 5128. The headphones tested in this thread have also been tested on the GRAS 45C in ASR's official review threads, so you can compare one-to-one. With the relatively small sample size, however, it's difficult to tell how much of the error is correlated and could have been fixed with a higher quality compensation.

 

Other random thoughts:

• The bass difference in Olive's presentation probably isn't as big of a deal as it's made out to be.
  • As shown in the LTT video, the 5128's seal appears accurate to the average human.
  • HATS seal less well than flat fixtures, so GRAS's equivalent HATS would also likely have similar issues in the bass.
  • On the other hand, having a rig that seals well, then breaching the seal manually using glasses (like DIYAH does), may be a better approach than splitting the difference and only measuring an average seal.
• The maker of AutoEQ put out an informal study comparing different headphone test rigs' compliance with the Harman curve (but not the 5128). Average variance can be fixed with compensation; however the uncorrelated variance cannot. Uncorrelated variance is typically ±2.5dB in the pinna gain region, meaning non-GRAS simulators will be off by at least that much in ~1/3 of all headphones tested, even after an ideal compensation.
  • The study lacks a control, so it's hard to say how much of the random error is not due to the measurement rigs themselves. A control could be created by comparing Crinacle's new measurement database with Oratory's data. However, Crinacle's data is behind a $15/month paywall, and I don't have time at the moment to write the code to check it myself.

[spartaman64]

if you guys are interested what a more established person in the measurement community thinks of this lol also he gives a lot of historical/technical information behind the hats

[perseides]

 

For those who do not know who he is, he is the guy behind in-ear fidelity who hosts largest database on headphones, especially iem measurements. 

 

Also he goes into the history of headphone measurements in daily detail and offer some (unsolicited) advice at the end. 

[Kilrah]

@perseides Merged into the video's discussion thread where it was already posted.

[Bkindmonk]

I have searched the forums and floatplane and have been unable to locate the write up. Has anyone else have any better luck?