Labs Transparency Thread

[GoStormPlays]

Question about Labs:

 

When the site is in full throttle, how long can we be expecting after a product releases before we get labs results on the lttlabs site? I know this might vary a little for when you guys might get a product before launch, but I'd like to know if there's an average turnaround or deadline that's going to be instituted when this thing is fully launched.

[ChaosghoulJD]

Hey team,
posted this under previous videos, posting it here now as well:

I would really like to see battery size being added to the charging graphs of the phone videos you guys are putting out.

The newest one (Pixel 8a) is now at least comparing "to full charge" which is a bit better than the previous seemingly random x minutes to y percent of charge, I have to admit I'm still disappointed nonetheless.

 

In this newest graph for example it makes it look like the Pixel 7a charges a bit faster than the Pixel 8a.

The 7a charges to full about 2% faster, but it's also charging a battery which is about 2.4% smaller, making the Pixel 8a (45,3mAh/min) actually charge faster than the 7a (44,29mAh/min).

 

 

I know the difference in this case is not as big (and basically in the realm of "it's the same") as in previous videos where you compared phones with multiple hundreds of mAh of difference, I would still like this being improved upon. Being overly critical and unfair I'd say these graphs are misleading at best.

Yes, I know mAh isn't the be-all, end-all when it comes to battery size comparisons, but implying a VW beetle is better than a Lamborghini because it reaches its maximum velocity faster, is just... not great.
(I know the beetle doesn't do that, I know this isn't the best analogy one can think of and I also know you didn't say which phone is better at charging, you're still putting these numbers out there and you are ranking the chart).

[GoStormPlays]

1 hour ago, ChaosghoulJD said:

Hey team,
posted this under previous videos, posting it here now as well:

I would really like to see battery size being added to the charging graphs of the phone videos you guys are putting out.

The newest one (Pixel 8a) is now at least comparing "to full charge" which is a bit better than the previous seemingly random x minutes to y percent of charge, I have to admit I'm still disappointed nonetheless.

 

In this newest graph for example it makes it look like the Pixel 7a charges a bit faster than the Pixel 8a.

The 7a charges to full about 2% faster, but it's also charging a battery which is about 2.4% smaller, making the Pixel 8a (45,3mAh/min) actually charge faster than the 7a (44,29mAh/min).

 

 

I know the difference in this case is not as big (and basically in the realm of "it's the same") as in previous videos where you compared phones with multiple hundreds of mAh of difference, I would still like this being improved upon. Being overly critical and unfair I'd say these graphs are misleading at best.

Yes, I know mAh isn't the be-all, end-all when it comes to battery size comparisons, but implying a VW beetle is better than a Lamborghini because it reaches its maximum velocity faster, is just... not great.
(I know the beetle doesn't do that, I know this isn't the best analogy one can think of and I also know you didn't say which phone is better at charging, you're still putting these numbers out there and you are ranking the chart).

For the people actually using these devices, the mhp isn’t going to matter as much as a unit of time would.

[Chaosghoul]

10 hours ago, GoStormPlays said:

For the people actually using these devices, the mhp isn’t going to matter as much as a unit of time would.

I don't think I agree and I would argue the opposite actually.

 

With a reasonably big enough battery (which you'd need to show that it is), most people will charge their phone over night, so the charging speed is actually almost completely irrelevant, as long as it charges fast enough to charge to full in 4-7 hours (which they all do at this point).
Battery size and longevity actually do start to matter more where you can even get two days out of a high end phone on light use (I charge my S24 Ultra every other night instead of every night).

 

I am also not arguing for exchanging the time measurement with a "charged to x mAh" metric, I'm saying I'd like to see both numbers so you can make an educated decision about your phone. Just one of these alone without the other is basically meaningless is what I'm trying to say,

[GoStormPlays]

5 hours ago, Chaosghoul said:

I don't think I agree and I would argue the opposite actually.

 

With a reasonably big enough battery (which you'd need to show that it is), most people will charge their phone over night, so the charging speed is actually almost completely irrelevant, as long as it charges fast enough to charge to full in 4-7 hours (which they all do at this point).
Battery size and longevity actually do start to matter more where you can even get two days out of a high end phone on light use (I charge my S24 Ultra every other night instead of every night).

 

I am also not arguing for exchanging the time measurement with a "charged to x mAh" metric, I'm saying I'd like to see both numbers so you can make an educated decision about your phone. Just one of these alone without the other is basically meaningless is what I'm trying to say,

There are systems in place to purposely slow charging on phones, so charging overnight doesn’t matter.

[Chaosghoul]

2 hours ago, GoStormPlays said:

There are systems in place to purposely slow charging on phones, so charging overnight doesn’t matter.

Exactly.

 

Which is why charging speed does not matter in the vast majority of cases and is also a (imo) useless statistic if you don't know how big the battery is/how long you can use your phone before needing to charge.

[seon123]

Not sure if you're checking for feedback here too, but maybe a ping will help @LMGcommunity

 

For the RM750e review. https://www.lttlabs.com/articles/psu/corsair-rm750e

 

Product Overview

• No one I've seen calls it "ATX 3", as that can be ambiguous. Typically one would refer to ATX 3.0 or 3.1. It's also a strange thing to highlight here, imo, as I doubt you would not refer to e.g. the RMx as an "ATX 2.52" PSU or whatever version.
• Efficient cooling...? Any PSU cools itself adequately temperature wise.
• "Stable and reliable performance" sounds like some generic crap that a clueless mUh 80+ rAtInG type person would say. What does "stable performance" mean? Do you mean voltages being within spec? Reliable performance? How did you assess the reliability? EDIT: Seems like you didn't. Or are you referring to the hold up time? Either way, it's not clear. Imo, something like "good overall performance" would be adequate, without saying more things than intended.
• "Resilient to a variety of input voltages". It's full range...? Just like pretty much all PSUs sold in 120V countries...?
• You mention the cables, but not their wire gauge, or whether or not they use HCS terminals. Or is "premium" for you just individually sleeved cables...? Imo, 16 AWG PCIe cables with Mini-Fit Plus HCS terminals is premium, while individually sleeved 18 AWG PCIe cables with regular Mini-Fit Jr. is not.

 

Hardware

• You seem very hung up on the fan size in the summary. The fan and PSU size is a choice, and a larger fan does not mean it runs quieter. Mentioning the fan bearing instead of the "what if 140mm fan" would be more relevant, as that gives some idea about the reliability. EDIT: In the Computex Noctua video, it was correctly explained why a larger fan is not better, due to the static pressure requirements for a PSU fan. The RMe is known for being quite loud, but the fan size on its own is not why. A factor that would actually contribute towards that is the stamped fan grill, instead of a wire fan grill (though again, not the sole reason for its loudness)
• Mentioning the 80+ rating in the summary seems quite random, and not necessary. We're reading a PSU review, why would we need to know whether or not it meets some arbitrary efficiency targets by a shady organisation? The efficiency is a small part of choosing a PSU, and the 80+ rating in particular should absolutely not be a major factor when choosing a PSU, imo.
• For some reason, you are perpetuating that the PSU side of the 12VHPWR connector matters - it doesn't. Just like how the PSU side of the SATA cables also doesn't matter. The PSU side of the cables is absolutely not PCIe/CPU, saying that is dangerous, and could lead to people plugging in incompatible cables.
• I would expect you to mention the cable specs here. Even the length, which is so basic. And whether or not the cables have capacitors, which can be bulky. Tell us about how bulky they are. E.g. the 24 pin cable on the Whisper M is ridiculously bulky, due to these, and the rubber boot around them. This is easy to overlook when choosing a PSU, and an example of exactly why you'd want to read a review, and not just look at the specs.

Spoiler

https://www.tomshardware.com/reviews/bitfenix-whisper-series-550w-psu,4805-2.html

• The Case Effective Length is useful, as just the PSU length itself might not be enough to estimate whether or not it will fit in some cases.
• I couldn't find the operating temperature from Corsair either, but HWBusters claims 0-40C. Curious where they found that.
• OEM and anything about the internals could be nice

 

Features & Software

• Lacks multi rail OCP on 12V, thus does not have all protections For higher power PSUs, this really starts to matter. A 1600W PSU that lacks multi rail OCP would be fully happy sending all 1600W through a single SATA cable. That does not sound like a PSU with all standard protections.
• Seems a bit strange to lump lighting modes into Modes, next to the fan modes?
• You mention the protections here, but only test them under Productivity & Efficiency? Wouldn't it make more sense to have them together?

 

Performance

• You mention "electrical noise levels", but also ripple? "Electrical noise levels" is nto mentioned anywhere else, so seems like you just mentioned ripple twice?
• You mention the audio noise level not being ready yet, which is understandable, but you could have the fan RPM?
• I would like to have tables to read off data.
• What loads were used for the load regulation tests? Not testing crossloads would hide the main problem of group regulated PSUs.
• What is the load during the Power excursion tests? 20%? 50%? 100%? 120%? Intel's PSU design guide has a table for these, depending on the duration of the power excursion, in Table 3-3.
• The graphs for the power excursion lack units or scale? What do you want us to do with literally just the shapes of the waveforms?

 

Productivity & Efficiency

• Not sure why you're so obsessed with verifying only the 80+ certification, but not Cybenetics Lambda, ETA or ATX version (all of which you did note under the specifications)
• "Sustaining brownouts at the expected level". What does that mean? What do you expect? Intel's design guide for ATX 3.0 PSUs recommends >16ms for AC loss to PWR_OK hold up time at 80% load, and requires 11ms at all loads. You tested for 16ms power loss at 100% load (which is what I would personally expect from a mid-high end PSU), but you never stated any of this.
• "The RM750e tends to recover and activate again after power has been restored, rather than staying off like some other PSUs." ??? If there is a fault somewhere, surely you would want the PSU to shut down, and need to be manually reset. You don't want the PSU to keep turning on??? Unless I am misunderstanding what you're trying to say?
• As mentioned earlier, strange to separate the listing of protections from the actual protections testing.
• Efficiency graphs usually don't start at 0% for a reason. But knowing a lot of LTT's fan base's reading comprehension, maybe this is for the better, to get rid of people spouting mUh EfFiCiEnCy CuRvE.
• In the efficiency graphs, the colour in the graph seems to not quite match up with the legend? Especially the Gold one. Is that just me?
• I would like to see a number for the PSU's hold up time at 100% load, rather than just a highlighted area of 16ms.
• More tick marks for the time in the hold up test would be useful, so we can actually read the time for each event. And also mentioning that the power good signal is off before the other rails lose power.
• Some horizontal lines in the OCP graph could help us read off of it. There's also not much point starting it at 0%. Having it go from e.g. 100% - 200% would be much more useful, with steps every 10%. Also mention what the current at 100% is for each rail somewhere nearby, so we don't have to scroll all the way up to find it.

 

 

Just from a quick read through. Hopefully at least some of it will be useful, so we can get another reputable source for PSU information. Despite all of my notes, this is far better than I was worried it'd be. Good luck with the work!

Edited June 5, 2024 by seon123
Something something

[TheLANguy]

1 hour ago, seon123 said:

<snip>

You might also want to submit that in the "LTT Labs Website Beta Feedback" form (if you have not already) to make sure it goes to the Labs team.

https://forms.gle/gpoXDeyEHFK5M5iH8

[silentdragon95]

I'd like to add that an analysis of the internal components of the PSU would also be extremely helpful.

[iLikeBananas]

I don't know if this has been asked before, but is Labs going to test headphones? I assumed we would see headphone tests soon because of this head you guys bought for headphone testing. I would love to see charts on those expensive headphones

[PearProgrammer]

Hi Team,

 

I have a suggestion/question for the Handbrake chart of recent CPU reviews. I'd like to run the exact same benchmark on my CPU since you don't include the 5800X3D in all reviews. The chart is just labeled "Big Buck Bunny - 1080P". On my machine there is a 23% difference if I use the 1080P30 input vs the Native resolution 60fps version and output to 1080P. Its unclear if the title refers to the input, output, or both.

 

Example at 12:07 : 

 

 

I believe input resolution, input frame rate, and output resolution should be specified in the chart.

 

Thanks for coming to my Ted Talk.

[LMGcommunity]

On 3/30/2025 at 12:56 PM, PearProgrammer said:

Hi Team,

 

I have a suggestion/question for the Handbrake chart of recent CPU reviews. I'd like to run the exact same benchmark on my CPU since you don't include the 5800X3D in all reviews. The chart is just labeled "Big Buck Bunny - 1080P". On my machine there is a 23% difference if I use the 1080P30 input vs the Native resolution 60fps version and output to 1080P. Its unclear if the title refers to the input, output, or both.

Hey there, you can find our Handbrake settings here:

https://github.com/LTTLabsOSS/markbench-tests/blob/main/handbrake/presets/h264_bigbuckbunny_1080p_cpu_test.json

 

https://github.com/LTTLabsOSS/markbench-tests/blob/main/handbrake/presets/av1-svt_bigbuckbunny_1080p_cpu_test.json

 

Also, we use a lossless version of Big Buck Bunny from https://media.xiph.org/

[The K-Man89]

On 8/24/2023 at 6:12 PM, LMGcommunity said:

As promised, here is the thread where you can post your transparency suggestions for Labs. Here are some examples of things we'd like to see feedback about:

• Ideas in our benchmarking.
• What you'd like to see regarding test system information.
• Benchmark details.
• Testbed variations.

And, of course, any other transparency-related feedback is welcome, but similar to the Sponsor Complaints subforum, any off-topic comments will be subject to moderation.

 

Thank you for wanting to help us improve the way we do things here!

I think something interesting to add would be Power Efficiency Benchmarking. Basically the best performance to power efficiency ratio whether it's hardware combinations or overclocking stats, it would give the end user a good reference for getting the most bang for their buck while still being able to afford food because power is expensive these days like everything else and PC hardware is only getting more and more power demanding.