Should reviewers accept early product samples or purchase their own?

[harryk]

Currently many manufacturers provide early product samples to various publishers so they can write and publish reviews on launch day. The manufacturers receive essentially free advertising and the publishers receive more views by riding the launch day hype train. 

 

However in response to the recent controversy with NVIDIA and Hardware Unboxed (see main thread in News for details), many are writing that NVIDIA's actions create inherent distrust for any reviewer who receives early product samples since NVIDIA appears to be selecting reviewers which "align with their view".

 

What should be the standard practice (for any manufacturer, not just NVIDIA)? Should reviewers accept early product samples or purchase their own after public release?

[PlayStation 2]

I've never had a problem with reviewers accepting samples so long as they keep consistency with what they do otherwise. I don't particularly like when reviewers are restricted to what they're supposed to show.

[SansVarnic]

Never had an issue with samples. As long as the company giving the sample understands that the review will be unbiased regardless.

 

*edit

It should be fairly obvious that samples are meant for honest reviews. Honest reviews produce better sales. Paid, pre-conditioned reviews make for bad sales and the reviewer losing their jobs especially since the majority now are YT channels and they will lose viewers over it and being called shills and sellouts.

Edited December 12, 2020 by SansVarnic

[Levent]

That would eliminate almost %95 of all reviews on the internet. Plus what do you expect them to do after they are done with the product? sell it on ebay for a loss? It should be up to consumer to make up their own mind, there is always a chance of getting a lemon whatever you buy, especially when product is just released.

[WereCatf]

6 minutes ago, harryk said:

Should reviewers accept early product samples or purchase their own after public release?

Why would just simply accepting early product samples matter? The product is still the product and can be reviewed. It only matters, if the samples come with strings attached, but your question doesn't specify such a condition.

[Dedayog]

14 minutes ago, WereCatf said:

Why would just simply accepting early product samples matter? The product is still the product and can be reviewed. It only matters, if the samples come with strings attached, but your question doesn't specify such a condition.

I think the OP is asking that if the Company provided sample is binned, tweaked, tested, etc.

 

I would prefer if the reviewers  used the products that we as consumers would run out and buy rather than a super special top of the line performance tested one.

 

It's like reviewing an 8700k that's binned at 5.2GHz by Silicon Lottery vs one you ran and got at Microcenter.  Showing 8700k 5.2GHz performance is misleading for most consumers since they cannot achieve that.  

 

Why do you think Car and Driver buys a car from a dealer and tests it for their performance and year-long testings?

[WereCatf]

3 minutes ago, Dedayog said:

I think the OP is asking that if the Company provided sample is binned, tweaked, tested, etc.

OP never mentioned any such conditions, either. Also, there is no evidence of any of the major manufacturers doing that, so I do not see it as a concern.

[harryk]

17 minutes ago, SansVarnic said:

Never had an issue with samples. As long as the company giving the sample understands that the review will be unbiased regardless.

 

*edit

It should be fairly obvious that samples are meant for honest reviews. Honest reviews produce better sales. Paid, pre-conditioned reviews make for bad sales and the reviewer losing their jobs especially since the majority now are YT channels and they will lose viewers over it and being called shills and sellouts.

My opinion is that it is always biased. Simply by having the manufacturer select who receives an early product sample and who doesn't. Even if the reviewer pledges to be independent and unbiased, every reviewer will test things with their method and put their spin on it. So why would the manufacturer select reviewers whose methods and general opinions are not going to produce a positive outcome.

 

16 minutes ago, WereCatf said:

Why would just simply accepting early product samples matter? The product is still the product and can be reviewed. It only matters, if the samples come with strings attached, but your question doesn't specify such a condition.

My opinion is there are always strings attached when the reviewer receives a privileged position and receives an early product sample. I want to know what other people's opinions are or if I'm the weird one.

 

[harryk]

7 minutes ago, Dedayog said:

I think the OP is asking that if the Company provided sample is binned, tweaked, tested, etc.

That is another layer of the same cake but not the focus of my question

[WereCatf]

Just now, harryk said:

My opinion is there are always strings attached when the reviewer receives a privileged position and receives an early product sample.

That's not an opinion, that's a projection. Go on, do tell what these strings are?

[harryk]

6 minutes ago, WereCatf said:

That's not an opinion, that's a projection. Go on, do tell what these strings are?

NVIDIA sends LTT graphics cards for review. That puts LTT in a privileged position; there is no contract, no commitment to receiving future products. NVIDIA can decide for any reason to stop sending LTT products. Thus it is in LTT's best interest to appease NVIDIA or else lose their privileged position. In other terms, NVIDIA holds power over LTT in this relationship. The only card LTT can play is the ethics one, but capitalism stomps on ethics more often than not.

[WereCatf]

2 minutes ago, harryk said:

NVIDIA sends LTT graphics cards for review. That puts LTT in a privileged position; there is no contract, no commitment to receiving future products. NVIDIA can decide for any reason to stop sending LTT products. Thus it is in LTT's best interest to appease NVIDIA or else lose their privileged position. In other terms, NVIDIA holds power over LTT in this relationship. The only card LTT can play is the ethics one, but capitalism stomps on ethics more often than not.

That's not an attached string. Attached strings are stipulations set by the manufacturer, but, in your example, there are no such things and LTT is free to do as they please, even if that did mean LTT wasn't going to be receiving early samples anymore.

[mdk777]

This has been debated for the last 20 years.

 

The conclusion is that you should always take reviewers results with a grain of salt.

Everyone who follows the industry knows that some sites have obvious bias.

 

You will read reviews of an obviously piss poor product launch and discover that the reviewer has managed to find the most arcane test metric to matke that product shine.

Or a careful review of the test bed will note a crippling of the system to favor the opposition, etc. etc. etc.

 

Considering this has been the year of PAPER launches, it really doesn't matter.

People were buying products sight unseen, unreviewed at all.

 

Yeah, I will wait for extensive and unbiased reviews from reputable sites that have demonstrated years of reasonable even handedness.

 

But even those sites need to maintain their revenus. Buying samples is not likely in the cards (ha ha) for them.

 

[Senzelian]

It's fine how it is.

[unsorted]

This is an age-old thing.

 

Put it this way: there are very few reviewers that don't put in a more lenient and forgiving review when receiving free hardware samples, whether they are conscious of it or not. They simply won't mention the things that someone who has spent their own money on the item will likely mention. There is always a concern about breaking links with your supplier beyond repair. Plus there is always the entertainment angle.

 

That is why, personally, I only consider reviews very rudimentary guides (at best).

 

Reviewers are in essense just spending a short time with the hardware and in that sense are the least-likely to give you an accurate picture of what it's like to live with the hardware...quirks and all. There are no long-term reviews here. Very few use what they review. It's even debatable that the bulk of the audience even cares. So there's that angle too.

 

As for what should be standard practice: we all know what brings in the eyeballs. Early access to hardware brings in the eyeballs. Being an early reviewer does. The standard practice is up to the reviewer but 99.999% of the time it is this. We essentially have a relationship where both parties benefit from doing it this way. The audience might get a glossed-over review, but there are no victims here. Right? Right?

 

 

 

 

 

 

 

 

 

 

[Kisai]

5 hours ago, harryk said:

Currently many manufacturers provide early product samples to various publishers so they can write and publish reviews on launch day. The manufacturers receive essentially free advertising and the publishers receive more views by riding the launch day hype train. 

That's called influencing. And it has two issues:

a) influencers and reviewers getting "different", sometimes "beta quality" versions that might not perform like the release model

b) sometimes the contractual agreement for it, says you can't say anything negative about it, you must adhere to a contract, or they must review your review before you publish it.

 

 

Quote

However in response to the recent controversy with NVIDIA and Hardware Unboxed (see main thread in News for details), many are writing that NVIDIA's actions create inherent distrust for any reviewer who receives early product samples since NVIDIA appears to be selecting reviewers which "align with their view".

 

Yes. The only time you should accept product demo/sample/beta hardware is if you are testing it in the context you agreed to. It's not a free product, and violating that contract means you need to send it back.

 

 

Quote

What should be the standard practice (for any manufacturer, not just NVIDIA)? Should reviewers accept early product samples or purchase their own after public release?

 

Depends on the purpose:

a) You receive something to review it, and it alone, then you are bound to that contract, and you should make your review prolog say so "This is a paid promotion for (product)"

b) You are reviewing something without the manufacturer's influence, then go nuts, as long as it's honest. Purchase it yourself.

c) You are reviewing something for Quality/Purchasing decisions (eg consumer reports), in which case you have to not only purchase it yourself, but you have to purchase it from consumer retail sources (eg bestbuy) not dedicated enthusiast retail channels (like newegg.)

 

This is why when you purchase a dell or hp machine, if you are reviewing "the machine" you buy it from bestbuy, but if you're reviewing "Dell or HP" you purchase it from their first party sales channel through a third party, because you're not just reviewing the purchase, but the purchasing and customer support process.

 

Computer parts, are ultimately enthusiast purchases, so at best, you have someone like Steve from GN who will do B, and do everything possible to evaluate the performance and build quality of the part, which may in turn influence people to purchase a specific part or manufacturer's version of a part. Yet Steve also gets parts under A, early review parts, and he doesn't accept arm-twisting.

 

 

Like LTT and GN have their priorities. Linus likes cool things, Steve likes to see how much performance can be squeezed out of things. Neither of them I would say are beyond reviewing something for money, but would trust that neither would review rubbish while praising it.

 

[IRMacGuyver]

It's up to the reviewer if they succumb to pressure.  It'll be obvious over time who gives real honest reviews and who turns out to be a paid shill. 

I like to point to LGR.  That dude started getting review copies of The Sims games, didn't change his review style at all, continued to rip EA apart for destroying the series, and eventually they stopped sending him review copies since it wasn't buying the good press they hoped for. 

 

The bigger issue to me is game and tech journalists that accept advertising money from the companies they review products for.  Linus seems to have somehow struck a decent balance where they know they can't buy a review and yet still partner with him for advertising.  That's pretty unheard of most of the time.  I've seen many magazines and websites submit to giving a good review to a shitty product just because they paid for massive advertising.  IGN comes to mind. 

 

I think you should be looking for advertisers that are in a different industry than what you're reviewing. That's why I like manscaping and vacuum ads.  Linus isn't gonna review that sort of stuff so they can't influence his reviews.  Though I would still like to see Linus give actual rating numbers or some buy don't buy system... more importantly though to me is the consumer reports approach of listing what you should buy instead next to crappy products. 

[Arika]

I would prefer they buy their own so there is no question about them being sent a cherry picked product. 

[TetraSky]

They should be able to accept early access products... As long as they aren't obligated to only say positive things about it and can give their actual take on it, good or bad.

[SpaceGhostC2C]

If you were designing a whole system for the entire industry, maximizing consumer welfare, and could fully enforce it around the world, in my opinion the answer is no, review samples would not be part of that system.

 

However, in any realistic (and desirable? I mean, who would have such power, and it having such power be a good thing, outside of a pure thought experiment?) setting, review samples will exist, and fully honest and benevolent reviewers will accept them. I guess technically an alternative equilibrium where everyone big and honest agrees to boycott review samples could exist, and those doing launch-day reviews would hence be labelled dishonest shills by "the community"; however, it would hardly be sustainable or even effective, as the market is much bigger than "the community".

[WkdPaul]

The issue isn't the sample themselves, but rather how nVidia decided to cut HUB from FE samples AND sent them an email explaining it was because they didn't like how GPUs were covered.

 

nVidia could've just cut them off, and not say anything, and come next FE release, simply not send anything appart from an excuse that they couldn't supply one in time.

 

The fact that they sent an email explaining HUB was cut off because of editorial differences is the real issue here.

[A51UK]

I think accept early product samples is a good things for the customer as will get review and information about the product on launch or some days before. If YouTuber have to buy them it would make a very big delay in the reviews, there a different problem that most people do not think about that a lot products are very expensive. It could be anything from £1000 to over £10,000+. The YouTuber may not be able to keep financing video for many of them most expensive products (e.g. high end CPU, server CPU, high end GPU). We could lose out any many products review not just early product samples but others. 

[harryk]

3 hours ago, wkdpaul said:

The issue isn't the sample themselves, but rather how nVidia decided to cut HUB from FE samples AND sent them an email explaining it was because they didn't like how GPUs were covered.

 

nVidia could've just cut them off, and not say anything, and come next FE release, simply not send anything appart from an excuse that they couldn't supply one in time.

 

The fact that they sent an email explaining HUB was cut off because of editorial differences is the real issue here.

So everything would be fine if NVIDIA denied to blacklist HUB for the same reason ("editorial differences") but didn't tell anyone? To me that is worse. Either way NVIDIA is selecting their preferred reviewers that will publish what they want.

[WkdPaul]

13 minutes ago, harryk said:

So everything would be fine if NVIDIA denied to blacklist HUB for the same reason ("editorial differences") but didn't tell anyone? To me that is worse. Either way NVIDIA is selecting their preferred reviewers that will publish what they want.

They did in the past though ... IIRC, they didn't sent them GPUs for a while after they exposed the GT 1030 DDR4 swap.

 

That letter is a huge problem, it's openly threatening reviewers to play ball "or else...", and it's going to be a huge issue for future reviews, if someone thinks a review is a bit too favorable, they'll bring this issue back asking if it's what happened in that review.

Edited December 13, 2020 by wkdpaul

[RejZoR]

They can be supplied for as long as there are no strings attached in form of mandatory nice talk about the product. Reviews are suppose to be the ones spotting problems or flaws and presenting a realistic image about the product and you make sure you take good feedback from that and address the negatives or problems. So next product will have favorable review as standard because you fix or address those flaws. Another thing is not fixing things and demanding things from reviewers. Which is odd given we're talking about NVIDIA. They've had great products for many years, good extra features, a lot of firsts. And they still feel the need to pull shit like this. It's weird. Why would one who's on top to behave like this?

 

It's also funny how self absorbed they are when it comes to RTX and DLSS. They are great features, but given how limited their availability is in the actual games, making grandiose speeches about them is pretty lame. I've also seen how they want to push DLSS usage in most games used by reviewers and present GeForce cards with DLSS as if that's their default performance over competition. Which is just an absolute misrepresentation of facts. DLSS is game specific thing and a such, cannot be used as a global representation of performance. If users could use it in ANY game, then yes. But not now.