Statistical evidence Intel most likely cherrypicks cpus for reviewers

[Syryquil]

While not a smoking gun, the barrel is definitely hot.

 

 

Linus made a video on this topic, but only used a sample size of two. This uses a sample size of thousands, using the data gathered by Silliconlottery and the overclock results by reviewers. It shows that all cpus were at least Above average, and many were in the top 3% of all cpus. Take this as you will, but it seems like almost proof.

[Enderman]

Lol it's this guy again with his conspiracy theories based on other people's data instead of his own...

 

I'm sure reviewers that do this as their literal day-to-day job have far more experience overclocking than the average 15 year old gamer.

The data from siliconlottery is pretty much useless.

Motherboard, ram, and cooling also have an effect on how far you can push your CPU.

[Energycore]

2 minutes ago, Enderman said:

Lol it's this guy again with his conspiracy theories based on other people's data instead of his own...

How is using other people's data bad in any way? Institutions such as the UN constantly use data from outside sources....

 

3 minutes ago, Enderman said:

The data from siliconlottery is pretty much useless.

Why? Can you find a sound argument as to why that data is useless? Other than "I don't like it"

[Blademaster91]

Because that guy isn't anyone but a biased conspiracy theorist, they're like the tabloid writer of tech youtubers. Taking data from silicon lottery and a few overclockers doesn't mean much when the actual "silicon lottery" is thousands upon thousands of chips.

[Syryquil]

1 minute ago, Blademaster91 said:

Because that guy isn't anyone but a biased conspiracy theorist, they're like the tabloid writer of tech youtubers. Taking data from silicon lottery and a few overclockers doesn't mean much when the actual "silicon lottery" is thousands upon thousands of chips.

Silliconlotterys database is 1000s

[dizmo]

Even if they do, I don't see a problem with that. If it was true that only that chip could perform like that, then that's wrong. But if it's feasible that a decent percentage of chips can perform like that, then go ham. It's overclocking we're talking about here. It's not like base clocks, which are what's being sold to you is different.

[Enderman]

8 minutes ago, Energycore said:

How is using other people's data bad in any way? Institutions such as the UN constantly use data from outside sources....

It's not bad, but he never seems to do any testing on his own.

Probably because he doesn't want to spend any money.

All this guy does is make videos about "binned GPUs" or other stuff intended to go viral.

He's been doing this for years.

His entire channel is just "intel sucks" or "nvidia sucks" and then taking data that supports his argument.

 

8 minutes ago, Energycore said:

Why? Can you find a sound argument as to why that data is useless? Other than "I don't like it"

Are you talking about the siliconlottery forums or the actual site?

If the data is from random internet users, the "maximum overclock" obtained by a teenager who just enabled auto-OC on his motherboard is going to have a much lower frequency and far higher voltage than that which an experienced overclocker will obtain by manually overclocking. Which is what all reviewers do.

So of course the reviewers seem to get "excellent" chips when compared to the other 99% of people who overclock.

[Syryquil]

4 minutes ago, Enderman said:

It's not bad, but he never seems to do any testing on his own.

Probably because he doesn't want to spend any money.

All this guy does is make videos about "binned GPUs" or other stuff intended to go viral.

He's been doing this for years.

His entire channel is just "intel sucks" or "nvidia sucks" and then taking data that supports his argument.

 

Are you talking about the siliconlottery forums or the actual site?

If the data is from random internet users, the "maximum overclock" obtained by a teenager who just enabled auto-OC on his motherboard is going to have a much lower frequency and far higher voltage than that which an experienced overclocker will obtain by manually overclocking. Which is what all reviewers do.

So of course the reviewers seem to get "excellent" chips when compared to the other 99% of people who overclock.

He's not comparing to random people though. He uses the silliconlottery actual data. This is pretty obvious you didn't watch.

[Energycore]

Just now, Enderman said:

It's not bad, but he never seems to do any testing on his own.

Probably because he doesn't want to spend any money.

All this guy does is make videos about "binned GPUs" or other stuff intended to go viral.

He does a different kind of journalism and I understand that it's unconventional. I don't think we should look at AdoredTV as a reviewer, he is not even though sometimes he does reviews. You're allowed to dislike the man, but don't attack the procedure (using other people's data is perfectly fine in journalism as long as you give credit) if what you dislike is the man.

 

2 minutes ago, Enderman said:

Are you talking about the siliconlottery forums or the actual site?

If the data is from random internet users, the "maximum overclock" obtained by a teenager who just enabled auto-OC on his motherboard is going to have a lower frequency and higher voltage than that which an experienced overclocker will obtain by manually overclocking. Which is what all reviewers do.

So of course the reviewers seem to get "excellent" chips when compared to the other 99% of people who overclock.

You should know which siliconlottery data you're talking about, you're the one who stated

Quote

The data from siliconlottery is pretty much useless.

So same question right back atcha.

 

The data Adored used seemed to only be from their shop, not the forums, if that's what you're asking.

 

I would like to believe that the data SL publishes in their own shop is based on manual overclocking from someone with technical expertise.

[Syryquil]

10 minutes ago, dizmo said:

Even if they do, I don't see a problem with that. If it was true that only that chip could perform like that, then that's wrong. But if it's feasible that a decent percentage of chips can perform like that, then go ham. It's overclocking we're talking about here. It's not like base clocks, which are what's being sold to you is different.

The problem is those top 3% chips, of which there were 8 IIRC.

[Enderman]

7 minutes ago, Energycore said:

He does a different kind of journalism and I understand that it's unconventional. I don't think we should look at AdoredTV as a reviewer, he is not even though sometimes he does reviews. You're allowed to dislike the man, but don't attack the procedure (using other people's data is perfectly fine in journalism as long as you give credit) if what you dislike is the man.

It's not fine if there is no standard procedure for taking the data.

If everyone had the same CPU, motherboard, ram, and exact same method of overclocking to find the max frequency then yeah it would be ok.

 

7 minutes ago, Energycore said:

The data Adored used seemed to only be from their shop, not the forums, if that's what you're asking.

 

I would like to believe that the data SL publishes in their own shop is based on manual overclocking from someone with technical expertise.

I would believe that a shop that makes profit from binned CPUs would like to make it seem rare and exclusive for CPUs to reach those high frequencies in order to sell those CPUs at higher prices while in reality having a large amount of stock that can reach it.

But hey if you want to think that retailers never lie, I won't try to stop you.

[Syryquil]

2 minutes ago, Enderman said:

I would believe that a shop that makes profit from binned CPUs would like to make it seem rare and exclusive for CPUs to reach those high frequencies in order to sell those CPUs at higher prices while in reality having a large amount of stock that can reach it.

But hey if you want to think that retailers never lie, I won't try to stop you.

Very good point I hadn't thought of.

[Blademaster91]

48 minutes ago, Energycore said:

He does a different kind of journalism and I understand that it's unconventional. I don't think we should look at AdoredTV as a reviewer, he is not even though sometimes he does reviews. You're allowed to dislike the man, but don't attack the procedure (using other people's data is perfectly fine in journalism as long as you give credit) if what you dislike is the man.

 

You should know which siliconlottery data you're talking about, you're the one who stated

So same question right back atcha.

 

The data Adored used seemed to only be from their shop, not the forums, if that's what you're asking.

 

I would like to believe that the data SL publishes in their own shop is based on manual overclocking from someone with technical expertise.

What that person makes is hardly "journalism".  Stirring up a story that easily goes viral to people that don't know any better is just pure clickbait BS.

They're making the usual edgy clickbait title then sucking viewers in to waste 20mins+ of their time with grossly exaggerated claims on

something that isn't even really news. Is some company that profits from pre-testing chips while binning some for overclocks suddenly a bunch of evil liars?  As mentioned we're talking about overclocks, the chips run at specified base clocks so there isn't anything "unethical" like this youtuber always likes to play here.

Edit-  I mean hardware companies sending out better than retail samples is kind a given with things besides CPUs.

I can't blame them too much for "clickbaiting" lots of youtubers do it, but you sure wouldn't see them investigating if AMD is sending out ryzen samples that overclock better. Most of their popular topics appeal to an audience that only follow his biased arguments instead of forming their own informed opinions.

[Energycore]

Just now, Enderman said:

It's not fine if there is no standard procedure for taking the data.

If everyone had the same CPU, motherboard, ram, and same method of overclocking to find the max frequency then yeah it would be ok.

 

I would believe that a shop that makes profit from binned CPUs would like to make it seem rare and exclusive for CPUs to reach those high frequencies in order to sell those CPUs at higher prices.

But hey if you want to think that retailers never lie, I won't try to stop you.

You have shifted from "getting data from outside is never good" to "ok it's sometimes good but this time it's not", you have a right to believe this but I will not continue arguing this point. I don't think it was perfect but I do think it's significant enough to point a finger at.

 

You have a point, SL does stand to make a profit off this data, so it's a non-zero chance that they have inflated these numbers. I don't think it's enough grounds to completely dismiss the data but something to point at. Keep in mind we should never assume guilt of someone, the legal system is built around assuming innocence until proven otherwise.

 

The problem I see here is that there's a low chance that anyone else has data of this kind so SL is likely the best we can do.

[dizmo]

16 minutes ago, Syryquil said:

The problem is those top 3% chips, of which there were 8 IIRC.

Doesn't matter. There's still 3% of chips that can hit that performance. So you do have a chance of getting one. If there's an issue, it's with reviewers saying you'll hit that. If they do their due diligence and state that overclocking is a lottery and your results may vary, it still shows the utmost performance the chip can hit.

 

Again, they're not selling you overclocking results. It's a value add. They're selling you higher base clock, which is the same across all chips.

[Enderman]

9 minutes ago, Energycore said:

You have shifted from "getting data from outside is never good" to "ok it's sometimes good but this time it's not",

 

I thought it was obvious why taking user data on overclocks was bad.

Anything I said in this thread is specifically about THIS thread only, not any other journalism, data collection, statistics, etc.

 

In my first post I mentioned "other people's data instead of his own" because that's what he's always done in the past, it's a pattern I noticed, not because I was trying to point out that it is bad (even though in this case it is).

 

My point is that this guy is known for finding anything possible he can hate on nvidia/intel about and then form an argument supporting his position. Just throw away all the points and data that say otherwise. Just look at all his most viewed videos for examples.

 

 

17 minutes ago, Syryquil said:

Very good point I hadn't thought of.

Which is why taking data from individual users is often more unbiased than taking results from a single retailer that profits from the results.

However in this case as I mentioned earlier, there are a billion variables that affect the results from each individual user, making the resulting data unreliable to draw a conclusion from.

 

For example, if you take one of the 'gold' CPUs and give it to some random gamer, they click "auto oc" in their motherboard settings, and then that CPU gets 4.8GHz instead of 5.2, and uses 1.4v instead of 1.35, or even manually overclocking it will obtain different results based on motherboard and ram and cooling.

 

A proper test would require

-obtaining a large sample of CPUs from multiple retailers and from multiple locations

-testing all the CPUs on the same motherboard, ram, and cooling as the review samples

-testing multiple review sample CPUs from several reviewers, not just one, again on the same motherboard, ram, and cooling

-finding the normal distribution and variance of the performance results

-finding the normal distribution and variance of the review sample CPU results

-run a t.test to compare the normal distributions and find if they are statistically different within a 95% CI

[Toysrme]

I buy CPU's 20 at a time, bin them (delid & liquid metal the i7's); sell the best to friends & keep near the best.
I've also long bought ES and QS Xeons for my home server farm.

I can say these two things. First, there are basically three tiers of chips. Taking the current i7-7700k and i7-8700k, of which I've bought, binned, liquid metal & resold more than 40 in 20-lot batches in the last year. Binning to the top Intel specced voltage (1.52v) & working down to 1.35v. (Also, NOT using avx offset.)

3/20 (15%) chips are head and shoulders above the average chip.  They'll BTD at 5.6ghz and run any software at 5.4ghz.

13/20 (65%) are average. BTD at 5.4ghz, run all software at 5.2-5.3ghz.

4/20 (20%) are not great, but WILL run any software 5.2ghz stable.
Every chip will run 5ghz without touching voltage beyond LLC-1, every chip will run 4.9ghz out of the box touching nothing.

Chip makers got busted sending out Golden Samples to early internet reviewers in the 1990's and it really hurt their credibility. Frankly, most of them get late production QS chips that are not even in the same ballpark as the second retail stepping that will happen a quarter after a new chip launches.
 

======

Regarding ES/QS samples. They are no where near as good as the retail samples. Typically, the paste applied in the IHS is sloppy (For instance, why so many previews/day-1 reviews claim major "thermal issues" with Haswell, Sky/Kaby/Coffeelake parts, but those don't exist in retail chips), and the yields are not great. They require a fraction more voltage to run the same (or much lesser frequency if an ES chip).

======

So no, sending out "golden samples"; that 1/1000 chip is not a thing on the CPU review side. Now that nvidia is hiding ASIC scores; I wouldn't put it past the video card OEM's sending out higher ASIC scoring cards to reviewers tho (but not the outright golden samples). Look at the fiasco nearly EVERY video card OEM was guilty of with Maxwell 2.0 launches. The vbios's before launch day were faster, with higher power targets & more voltage applied than day-1 retail vbios were.
So you do see attempts at better sampling in "recent times" on the video card end of reviewing. (Tho one can argue if you mail ALL cards with the "hotter" vbios, it's not cheating)

 

[TahoeDust]

Intel does not need to cheat.  For people without budget constraints, they are clearly the better product.  For others, value is in the eye of the beholder. 

[DarkSmith2]

I mean by the test metholodgy of silicon lottery alone, not even mentioning the very weak AiO they are using,  the data they can provide is more then limited and therefore also cant reflect or being compaired to reviewers results.

 

well, put it this way, aslong as your are delidding, have decent cooling, good motherboard and PSU the chances are very very low that you wont hit 5GHz+ on the 8700k. Which is more than reasonable.

 

[leadeater]

3 hours ago, Toysrme said:

So no, sending out "golden samples"; that 1/1000 chip is not a thing on the CPU review side

Realistically Intel isn't going to waste their time finding enough golden CPUs to send out to all the reviewers anyway, better to focus on getting all CPUs as close to each other as possible.

 

For people that do believe this is actually a thing how is Intel actually binning the CPUs, like exactly how. That's kind of extremely important to answer for this to even be a thing. Binning for a product SKU and binning for OC is wildly different and frankly most don't care about OC in reviews, stock is the important thing and every CPU will be in margin of error at stock.

[mr moose]

Frankly I don't know what is more embarrassing, merely posting an adored video or defending him as something other than clickbait conspiracy nut. 

 

 

[Cookybiscuit]

Whoa manufacturers cherry pick their shit to send to reviewers? No fucking way?

1 hour ago, leadeater said:

Realistically Intel isn't going to waste their time finding enough golden CPUs to send out to all the reviewers anyway, better to focus on getting all CPUs as close to each other as possible.

 

For people that do believe this is actually a thing how is Intel actually binning the CPUs, like exactly how. That's kind of extremely important to answer for this to even be a thing. Binning for a product SKU and binning for OC is wildly different and frankly most don't care about OC in reviews, stock is the important thing and every CPU will be in margin of error at stock.

I disagree with the wasting time argument, Intel has more money than God, it can't be that hard for them to pay a guy to sit in a room for 12 hours with 1000 CPUs and pick out 100 of the best ones. I think the whole "8700K gets 5GHz ez" idea that everyone seems to have is largely down to the reviews suggesting this and isn't actually reflected in practice. This is of course keeping in mind that consumers who actually read CPU reviews are a drop in the very small bucket that is the desktop hardware market, so it's not like Intel lives or dies by what the reviews say.

Edited March 31, 2018 by Cookybiscuit

[Stefan Payne]

1 hour ago, leadeater said:

Realistically Intel isn't going to waste their time finding enough golden CPUs to send out to all the reviewers anyway, better to focus on getting all CPUs as close to each other as possible.

It isn't that hard to do, if you do the binning anyways, you can get the best of the best and put them in a special storage for reviewers. 

 

So you should expect golden samples if the sample comes from a manufacturer!

ONLY when the sample is bought, its safer to assume its not a Golden Sample.

 

Oh and another thing:
With Intel the initial production seems to be the better ones and gets worse over time. That's how it was with other stuff most of the times...

[leadeater]

19 minutes ago, Stefan Payne said:

It isn't that hard to do, if you do the binning anyways, you can get the best of the best and put them in a special storage for reviewers. 

And exactly how are they doing this? I know how I would do this on retail purchased CPUs in limited numbers but Intel for hundreds of reviewers?

 

To get these golden CPUs you are pushing them way out of spec, something that the production line process is not testing for and you can't test for it during stock specification testing. So is Intel grabbing CPUs after production then? Well that seems pointless since you can't really find the true golden CPUs without delidding.

 

Interrupting mass production to pull CPUs off the line before lidding and paying employees to OC them to find golden CPUs is a huge burden and interruption, far more costly than the supposed benefit to sending golden CPUs to reviewers. 

 

All this hassle to get a CPU to a reviewer that clocks 200 Mhz higher under OC and runs slightly cooler, makes no financial sense. I don't even remember the last time I really took much notice of the OC results in a review either, inconsequential and variable to take any meaning from it.

 

Anyway if you want something better than my ramblings then here is Gamers Nexus explanation on the binning process.

 

Quote

Another part of it, though, is that Intel fabricates millions of dies for use in all manner of computing platforms. When you deal with those sorts of numbers, the manufactured components are going to have a wide spectrum of performance tolerance, at which point (to increase yield per wafer / ROI) off-spec components are binned into higher/lower spec products. If Intel has a bin filled with underperforming dies that "didn't make the cut" for a high-end product, they can increase yield-per-wafer by shipping those in lower-end products that don't require the same level of performance.

 

Intel's spec for K-SKU processors demands higher thermal, frequency, and voltage tolerance than—for instance—their power-saving S-line. In general, this means that consumers purchasing K-SKU products will have an objectively "better" die housed under their IHS, being that it's been tested and qualified for more abusive computing environments. Intel knows their K-SKU devices will be marketed to overclockers, and thus selectively uses higher-quality components for them.

https://www.gamersnexus.net/guides/1140-silicon-die-bin-out-process-explained

 

It's important to realize the spec of K SKUs is a fixed not variable metric and if a die meets it then it passes that spec, it does not tell you how well above that spec it can go and Intel will not test dies to breaking point, they will not sell potentially damaged goods.

 

[Stefan Payne]

2 minutes ago, leadeater said:

something that the production line process is not testing for and you can't test for it during stock specification testing. So is Intel grabbing CPUs after production then?

Do they need to test for it?

Intel knows their CPUs better than anyone, so its possible that they don't need no "special testing" for that and are able to run a script and calculate the best of the best, sir.

 

So no, they don't grab them after production, there is no need for that.