Using badly made games as benchmarks?

[Humbug]

I was just reading an article at Hardocp, and I realized that they have started using Watchdogs as a benchmark.

http://www.hardocp.com/article/2014/08/01/sapphire_vaporx_r9_290x_trix_oc_video_card_review/4#.U9yHnGPxvzw

A game which is universally accepted that the devs screwed up, has stuttering issues and graphics which do not come close to justifing the performance, and it cannot mange it's VRAM budget.

 

Apart from that in the past I have also seen a lot of websites use games like Crysis 2... Even though it is well known that crytek used over-tessellation on some surfaces and forced the game engine to render things which are not shown on screen, thereby artificially slowing down performance.

 

It's time the gaming press, and hardware reviewers became a bit smarter. Such badly optimized or artificially stunted games should be shunned, we should hold devs to high standards.

The last thing we should be doing is using these games as benchmarks to judge video cards, and there is no need for it since 95% of AAA PC games don't have these issues.

[Mooshi]

Reading that title, I immediately thought of Minecraft.

[Excon]

Or or or, maybe someone wants to know what fps they will get in that game because they enjoy it? Just because it's not an accurate representation doesnt mean its pointless to benchmark

[Vacsol]

Watch_dogs is a popular game and people looking to buy a new GPU might want to know how it will perform with that game.

it's not rocket surgery

[LinusTech]

Real games (however you might feel about their optimization) that real people play are a much more relevant measurement of graphics card performance than techincally excellent games that aren't fun or synthetic benchmarks.

 

Totally disagree...

 

This is why a variety of games is used for testing graphics cards.

[Vacsol]

Real games (however you might feel about their optimization) that real people play are a much more relevant measurement of graphics card performance than techincally excellent games that aren't fun or synthetic benchmarks.

 

Totally disagree...

 

This is why a variety of games is used for testing graphics cards.

go to bed linus. although I should probably do the same

[Humbug]

Or or or, maybe someone wants to know what fps they will get in that game because they enjoy it? Just because it's not an accurate representation doesnt mean its pointless to benchmark

It's valid in terms of benchmarking the game. To see how it performs across a range of GPUs.

It's not valid to test CPUs and GPUs.

 

Anyway I see your point that some gamers buy these games, but the purchase of them should not be encouraged by the gaming press or the hardware reviewers by giving them more coverage.

[helping]

You mean like Crysis? We've done this for a long time, might as well keep doing it.

 

It still gives an accurate representation of real world performance, despite the game being bad (opinion) or poorly optimized (fact)

If it's popular enough it will still yield useful data.

[Humbug]

Real games (however you might feel about their optimization) that real people play are a much more relevant measurement of graphics card performance than techincally excellent games that aren't fun or synthetic benchmarks.

I see your point and I don't like synthetic benchmarks either.

But don't you think the gaming press and reviewers should try to steer the devs in the right direction by holding them accountable for their work. If reviewers recognize that something is handicapped either due to incompetence or malice then they can avoid using it to judge hardware. I realize that people play these games. But this strategy would put more pressure on devs, rather than giving them more coverage. There are plenty of other games like Crysis 3, Tomb Raider etc which are demanding and have advanced graphics.

 

I am not against running a new game across a range of GPUs to see how it scales.

[Humbug]

You mean like Crysis? We've done this for a long time,

crysis was demanding because it was more graphically advanced than every other game at the time.

[helping]

crysis was demanding because it was more graphically advanced than every other game at the time.

it's also horribly optimized

[Humbug]

it's also horribly optimized

was it? I'm not so sure. Similarly I wouldn't say star citizen is badly optimized. It's demanding but it has the graphics to show for it.

[juretrn]

was it? I'm not so sure. Similarly I wouldn't say star citizen is badly optimized. It's demanding but it has the graphics to show for it.

Even modern hardware struggles to get decent fps in Crysis. In a 7 year old game that is much more likely to be bad optimization.

[LinusTech]

I see your point and I don't like synthetic benchmarks either.

But don't you think the gaming press and reviewers should try to steer the devs in the right direction by holding them accountable for their work. If reviewers recognize that something is handicapped either due to incompetence or malice then they can avoid using it to judge hardware. I realize that people play these games. But this strategy would put more pressure on devs, rather than giving them more coverage. There are plenty of other games like Crysis 3, Tomb Raider etc which are demanding and have advanced graphics.

 

I am not against running a new game across a range of GPUs to see how it scales.

 

I suspect you overestimate how much power the PC enthusiast press has over game developers...

 

We can't even get BASIC crap like FPS locks removed so we can benchmark the games AT ALL sometimes...

[Jumper118]

I was just reading an article at Hardocp, and I realized that they have started using Watchdogs as a benchmark.

http://www.hardocp.com/article/2014/08/01/sapphire_vaporx_r9_290x_trix_oc_video_card_review/4#.U9yHnGPxvzw

A game which is universally accepted that the devs screwed up, has stuttering issues and graphics which do not come close to justifing the performance, and it cannot mange it's VRAM budget.

 

Apart from that in the past I have also seen a lot of websites use games like Crysis 2... Even though it is well known that crytek used over-tessellation on some surfaces and forced the game engine to render things which are not shown on screen, thereby artificially slowing down performance.

 

It's time the gaming press, and hardware reviewers became a bit smarter. Such badly optimized or artificially stunted games should be shunned, we should hold devs to high standards.

The last thing we should be doing is using these games as benchmarks to judge video cards, and there is no need for it since 95% of AAA PC games don't have these issues.

the funny thing is that watchdogs looks much worse than crysis 2, but you get way lower fps. watchdogs also looks like something from 2008 and has no tesselation whatsoever.

[Tygron]

Some people just want to know "what graphics card do I have to buy to run x game that I want to play" Think from a "console peasant" point of view or someone who doesn't really know or doesn't want to figure out what the graphics cards they're looking at really are. Most consumers want to go "this is what I need" and plug and play. That's why consoles are big, that's why I was a console only gamer for the longest time. I still get really angry when I have dumb issues with games, like I can't play Mirrors Edge because of some older PhisX technology that even with legacy drivers the game still crashes. I can't play Sonic Adventure 2 through on my PC because it just won't load anymore even though i've played through most of the game and at the time it stopped working absolutely no updates software, driver, or hardware wise happened. Was just over night "nope not gonna work anymore". The computer hadn't even been shut down, and restarting didn't fix it. Mass Effect has the same problem. Sure I could be playing "better games" But I wanna finally play through all of Mass Effect cause I just want to, and it's horrible to play on the 360. Sonic Adventure 2 was fun for me on the Gamecube (wasn't a Dreamcast guy back in the day so I didn't know of it until the re-release) and getting it on Steam gave me a reason to play through it again. And I never got to finish Mirrors Edge, and I find it fun enough to entertain me when I just want to sit here with a controller and I have nothing better to do.

 

Long explanation for a simple answer. But yea. I could be playing AAA titles, and stuff. I do. But I also enjoy the more simple or unfortunately badly ported/made/whatever games on PC. And I think it's cool that there's reviewers out there who give people that just want to see numbers the numbers. In fact while typing this out my bf asked me to look for parts and put together a PC for him. He could very easily google it himself and do a little research cause he's good at doing that sort of thing. But he's playing Diablo 3 and know's I enjoy this sort of thing so instead of him putting the work into it, I can. It's a little different of a situation but in the case he went and looked it up himself, he'd be looking at benchmarks and stuff to figure out what would be the parts he needs. And unfortunately, Watch_Dogs is the type of game that, while bad, he'd be looking at.

[Briggsy]

Unless you are planning to use your gaming PC to play optimized benchmarks all day, reviewers/testers will test what the hardware was designed to run, even if its a bunch of spaghetti.

[Nicolas]

Well honestly it's not too bad of an idea to benchmark on poorly optimized software because if you have powerful enough hardware that will runs smoothly on the real virtuality engine then you can be sure it will run fine on most things

[Tacitus]

Hardware reviewers would look like they're out of the loop if they aren't benchmarking modern games. As one of the biggest launches this year so far it is a sensible choice since so many will want to know how cards perform with it.

 

At the end of the day, badly optimised or not, it is what it is and people should see how poorly it runs on hardware they are considering buying.

[killcomic]

Get a game that runs like crap for most people and benchmark it with new hardware. Makes sense to me! It gives you an idea how new GPUs perform against older ones in worst case scenarios.

[Pomfinator]

Such irony, people complain about watch dogs yet don't complain when Crysis is still in most GPU benchmarks to this date.

[killcomic]

Such irony, people complain about watch dogs yet don't complain when Crysis is still in most GPU benchmarks to this date.

Probably because Crysis still looks phenomenal with playable settings (and apart from the textures, it still looks great today). On the other hand, Watch Dogs looks like the dog's breakfast if you want to keep to at least 30 fps.

[YoloSwag]

Well I'm thinking you guys forgot something about Crysis. Sometime ago, the devs I think, said that Crysis: Warhead would run better on the same hardware while still looking good. It is like they implied that Crysis really wasn't that well-optimized.

 

So Crysis: Warhead would do a better example, I think, in this situation.

 

AND! I've ran Crysis on its max in-game settings, 1080p, 53 avg FPS, on a GTX 660, someone said something about modern hardware not running it well, so I don't know about that though...

 

I've finished it before on just the crappy Intel HD 2000 Graphics, just saying I know what its like

[kuddlesworth9419]

You are better off finding a game that does run well and just uncapping the frame rate. There will be less problems other then the power of your hardware. SOme low end GPU's perform better in some games then high end GPU's.

[YoloSwag]

You are better off finding a game that does run well and just uncapping the frame rate. There will be less problems other then the power of your hardware. SOme low end GPU's perform better in some games then high end GPU's.

 

I remember something like that happened to CPUs once, where the top of the line from Intel actually performed not so great compared to its half-price brothers. It might still be happening, and as for GPUs, it may explain a lot of things I've encountered during my gaming time.