Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
Humbug

Using badly made games as benchmarks?

Recommended Posts

Posted · Original PosterOP

I was just reading an article at Hardocp, and I realized that they have started using Watchdogs as a benchmark.

http://www.hardocp.com/article/2014/08/01/sapphire_vaporx_r9_290x_trix_oc_video_card_review/4#.U9yHnGPxvzw

A game which is universally accepted that the devs screwed up, has stuttering issues and graphics which do not come close to justifing the performance, and it cannot mange it's VRAM budget.

 

Apart from that in the past I have also seen a lot of websites use games like Crysis 2... Even though it is well known that crytek used over-tessellation on some surfaces and forced the game engine to render things which are not shown on screen, thereby artificially slowing down performance.

 

It's time the gaming press, and hardware reviewers became a bit smarter. Such badly optimized or artificially stunted games should be shunned, we should hold devs to high standards.

The last thing we should be doing is using these games as benchmarks to judge video cards, and there is no need for it since 95% of AAA PC games don't have these issues.

Link to post
Share on other sites

Reading that title, I immediately thought of Minecraft.


SNOWHEART - laptop

Model: TRACER III 17R XTREME VR 800 || CPU: Intel i7-8750H  || RAM: HyperX Impact 16GB || GPU: Nvidia RTX 2070 || Storage: Intel 660P 512GB +  AData SX8200 1TB + Samsung 850 Evo 250GB || Display: 1920x1080p 144hz

 

 

PRISIMHEART 2.0 - desktop

Case: FD Meshify C Mini || PSU: EVGA Supernova P2 750w || MB: Asrock Fata1ity AB350 Gaming-ITX/ac || CPU: AMD Ryzen R5 1600 || CPU Cooler: Scythe Choten TUF edition || RAM: G.Skill Flare X 16GB || GPU: Galax GTX 1070 EXOC-SNPR || Storage: Samsung 860 Evo 1TB + Crucial MX500 1TB + SG Firecuda 2TB

 

PERIPHERALS / DISPLAY

Keyboard: Razer Blackwidow Elite || Mouse: Logitech G502 Proteus Spectrum + Steelseries RIval 650 || Monitor: HP Omen 32

 

Link to post
Share on other sites

Or or or, maybe someone wants to know what fps they will get in that game because they enjoy it? Just because it's not an accurate representation doesnt mean its pointless to benchmark


i7 4790k | Asus Z97 Maximus VII Hero | 16Gb Dominator Platinum 2133mhz | Gigabyte Windforce 970 1563/1928


H100i w/ Swiftech Helix fans | Evga G2 850w | 6x SSDs | 6x Seagate Barracudas


Fractal Define R4 Titanium| HD558 w/ Fiio E10 | AT2020 | G710+ | Logitech G700s |  DP2710 Asus PA248Q

Link to post
Share on other sites

Watch_dogs is a popular game and people looking to buy a new GPU might want to know how it will perform with that game.

it's not rocket surgery


INTEL CORE I5 4670K | NVIDIA GTX 980 | NOCTUA NH-L9i | GIGABYTE GA-Z97X-SLI | KINGSTON 120GB V300

CM STORM QUICKFIRE TK | BENQ XL2420TE | ROCCAT SAVU | FRACTAL DEFINE R4

 

Link to post
Share on other sites

Real games (however you might feel about their optimization) that real people play are a much more relevant measurement of graphics card performance than techincally excellent games that aren't fun or synthetic benchmarks.

 

Totally disagree...

 

This is why a variety of games is used for testing graphics cards.

Link to post
Share on other sites

Real games (however you might feel about their optimization) that real people play are a much more relevant measurement of graphics card performance than techincally excellent games that aren't fun or synthetic benchmarks.

 

Totally disagree...

 

This is why a variety of games is used for testing graphics cards.

go to bed linus. although I should probably do the same


INTEL CORE I5 4670K | NVIDIA GTX 980 | NOCTUA NH-L9i | GIGABYTE GA-Z97X-SLI | KINGSTON 120GB V300

CM STORM QUICKFIRE TK | BENQ XL2420TE | ROCCAT SAVU | FRACTAL DEFINE R4

 

Link to post
Share on other sites
Posted · Original PosterOP

Or or or, maybe someone wants to know what fps they will get in that game because they enjoy it? Just because it's not an accurate representation doesnt mean its pointless to benchmark

It's valid in terms of benchmarking the game. To see how it performs across a range of GPUs.

It's not valid to test CPUs and GPUs.

 

Anyway I see your point that some gamers buy these games, but the purchase of them should not be encouraged by the gaming press or the hardware reviewers by giving them more coverage.

Link to post
Share on other sites

You mean like Crysis? We've done this for a long time, might as well keep doing it.

 

It still gives an accurate representation of real world performance, despite the game being bad (opinion) or poorly optimized (fact)

If it's popular enough it will still yield useful data.


Error: 410

Link to post
Share on other sites

 

Apart from that in the past I have also seen a lot of websites use games like Crysis 2... Even though it is well known that crytek used over-tessellation on some surfaces and forced the game engine to render things which are not shown on screen, thereby artificially slowing down performance.

 

Ok, acutally that would not be representative for most of the titles and i agree that devs should invest more time into optimizing, but i guess the thought behind using hardware-devouring games is just that they wont bring so many graphics cards to their limit current and next generations, so you have better comparability over a longer time. Taking watchdogs as standard is crap, i guess we all agree there^^


- intentionally left blank -

Link to post
Share on other sites
Posted · Original PosterOP

Real games (however you might feel about their optimization) that real people play are a much more relevant measurement of graphics card performance than techincally excellent games that aren't fun or synthetic benchmarks.

I see your point and I don't like synthetic benchmarks either.

But don't you think the gaming press and reviewers should try to steer the devs in the right direction by holding them accountable for their work. If reviewers recognize that something is handicapped either due to incompetence or malice then they can avoid using it to judge hardware. I realize that people play these games. But this strategy would put more pressure on devs, rather than giving them more coverage. There are plenty of other games like Crysis 3, Tomb Raider etc which are demanding and have advanced graphics.

 

I am not against running a new game across a range of GPUs to see how it scales.

Link to post
Share on other sites
Posted · Original PosterOP

You mean like Crysis? We've done this for a long time,

crysis was demanding because it was more graphically advanced than every other game at the time.
Link to post
Share on other sites

crysis was demanding because it was more graphically advanced than every other game at the time.

it's also horribly optimized


Error: 410

Link to post
Share on other sites
Posted · Original PosterOP

it's also horribly optimized

was it? I'm not so sure. Similarly I wouldn't say star citizen is badly optimized. It's demanding but it has the graphics to show for it.
Link to post
Share on other sites

was it? I'm not so sure. Similarly I wouldn't say star citizen is badly optimized. It's demanding but it has the graphics to show for it.

Even modern hardware struggles to get decent fps in Crysis. In a 7 year old game that is much more likely to be bad optimization.

 

Spoiler

CPU:Intel Xeon X5660 @ 4.2 GHz RAM:6x2 GB 1600MHz DDR3 MB:Asus P6T Deluxe GPU:Asus GTX 660 TI OC Cooler:Akasa Nero 3


SSD:OCZ Vertex 3 120 GB HDD:2x640 GB WD Black Fans:2xCorsair AF 120 PSU:Seasonic 450 W 80+ Case:Thermaltake Xaser VI MX OS:Windows 10
Speakers:Altec Lansing MX5021 Keyboard:Razer Blackwidow 2013 Mouse:Logitech MX Master Monitor:Dell U2412M Headphones: Logitech G430

Big thanks to Damikiller37 for making me an awesome Intel 4004 out of trixels!

Link to post
Share on other sites

I see your point and I don't like synthetic benchmarks either.

But don't you think the gaming press and reviewers should try to steer the devs in the right direction by holding them accountable for their work. If reviewers recognize that something is handicapped either due to incompetence or malice then they can avoid using it to judge hardware. I realize that people play these games. But this strategy would put more pressure on devs, rather than giving them more coverage. There are plenty of other games like Crysis 3, Tomb Raider etc which are demanding and have advanced graphics.

 

I am not against running a new game across a range of GPUs to see how it scales.

 

I suspect you overestimate how much power the PC enthusiast press has over game developers...

 

We can't even get BASIC crap like FPS locks removed so we can benchmark the games AT ALL sometimes...

Link to post
Share on other sites

I was just reading an article at Hardocp, and I realized that they have started using Watchdogs as a benchmark.

http://www.hardocp.com/article/2014/08/01/sapphire_vaporx_r9_290x_trix_oc_video_card_review/4#.U9yHnGPxvzw

A game which is universally accepted that the devs screwed up, has stuttering issues and graphics which do not come close to justifing the performance, and it cannot mange it's VRAM budget.

 

Apart from that in the past I have also seen a lot of websites use games like Crysis 2... Even though it is well known that crytek used over-tessellation on some surfaces and forced the game engine to render things which are not shown on screen, thereby artificially slowing down performance.

 

It's time the gaming press, and hardware reviewers became a bit smarter. Such badly optimized or artificially stunted games should be shunned, we should hold devs to high standards.

The last thing we should be doing is using these games as benchmarks to judge video cards, and there is no need for it since 95% of AAA PC games don't have these issues.

the funny thing is that watchdogs looks much worse than crysis 2, but you get way lower fps. watchdogs also looks like something from 2008 and has no tesselation whatsoever.


Rig Specs:

AMD Threadripper 4999WX@4.8Ghz

Asus Zenith III Extreme

Sapphire 6900XXXT Quadfire

G.Skill Ripheartout X OC 5000Mhz C12 DDR4 4X8GB  

Super Flower Power Leadex 2000W Psu's X2

Harrynowl's 775/771 OC and mod guide: http://linustechtips.com/main/topic/232325-lga775-core2duo-core2quad-overclocking-guide/ http://linustechtips.com/main/topic/365998-mod-lga771-to-lga775-cpu-modification-tutorial/

ProKoN haswell/DC OC guide: http://linustechtips.com/main/topic/41234-intel-haswell-4670k-4770k-overclocking-guide/

 

"desperate for just a bit more money to watercool, the titan x would be thankful" Carter -2016

Link to post
Share on other sites

Some people just want to know "what graphics card do I have to buy to run x game that I want to play" Think from a "console peasant" point of view or someone who doesn't really know or doesn't want to figure out what the graphics cards they're looking at really are. Most consumers want to go "this is what I need" and plug and play. That's why consoles are big, that's why I was a console only gamer for the longest time. I still get really angry when I have dumb issues with games, like I can't play Mirrors Edge because of some older PhisX technology that even with legacy drivers the game still crashes. I can't play Sonic Adventure 2 through on my PC because it just won't load anymore even though i've played through most of the game and at the time it stopped working absolutely no updates software, driver, or hardware wise happened. Was just over night "nope not gonna work anymore". The computer hadn't even been shut down, and restarting didn't fix it. Mass Effect has the same problem. Sure I could be playing "better games" But I wanna finally play through all of Mass Effect cause I just want to, and it's horrible to play on the 360. Sonic Adventure 2 was fun for me on the Gamecube (wasn't a Dreamcast guy back in the day so I didn't know of it until the re-release) and getting it on Steam gave me a reason to play through it again. And I never got to finish Mirrors Edge, and I find it fun enough to entertain me when I just want to sit here with a controller and I have nothing better to do.

 

Long explanation for a simple answer. But yea. I could be playing AAA titles, and stuff. I do. But I also enjoy the more simple or unfortunately badly ported/made/whatever games on PC. And I think it's cool that there's reviewers out there who give people that just want to see numbers the numbers. In fact while typing this out my bf asked me to look for parts and put together a PC for him. He could very easily google it himself and do a little research cause he's good at doing that sort of thing. But he's playing Diablo 3 and know's I enjoy this sort of thing so instead of him putting the work into it, I can. It's a little different of a situation but in the case he went and looked it up himself, he'd be looking at benchmarks and stuff to figure out what would be the parts he needs. And unfortunately, Watch_Dogs is the type of game that, while bad, he'd be looking at.

Link to post
Share on other sites

Unless you are planning to use your gaming PC to play optimized benchmarks all day, reviewers/testers will test what the hardware was designed to run, even if its a bunch of spaghetti.


R7 2700X | EVGA RTX 2080 | Corsair 1000i | 32GB DDR4 @ 3000Mhz | CM Scout 2 | Vizio 4K 50'' HDR

Link to post
Share on other sites

Well honestly it's not too bad of an idea to benchmark on poorly optimized software because if you have powerful enough hardware that will runs smoothly on the real virtuality engine then you can be sure it will run fine on most things


MSI Z87-GD65 - GTX 760 DirectCU II - i5 4670k @4.0GHz - 16GB Corsair Vengeance @1866 MHz

Link to post
Share on other sites

Hardware reviewers would look like they're out of the loop if they aren't benchmarking modern games. As one of the biggest launches this year so far it is a sensible choice since so many will want to know how cards perform with it.

 

At the end of the day, badly optimised or not, it is what it is and people should see how poorly it runs on hardware they are considering buying.

Link to post
Share on other sites

Get a game that runs like crap for most people and benchmark it with new hardware. Makes sense to me! It gives you an idea how new GPUs perform against older ones in worst case scenarios.


"Fighting for peace is like screwing for virginity"

- George Carlin (1937-2008)

Link to post
Share on other sites

Such irony, people complain about watch dogs yet don't complain when Crysis is still in most GPU benchmarks to this date.

Probably because Crysis still looks phenomenal with playable settings (and apart from the textures, it still looks great today). On the other hand, Watch Dogs looks like the dog's breakfast if you want to keep to at least 30 fps.


"Fighting for peace is like screwing for virginity"

- George Carlin (1937-2008)

Link to post
Share on other sites

Well I'm thinking you guys forgot something about Crysis. Sometime ago, the devs I think, said that Crysis: Warhead would run better on the same hardware while still looking good. It is like they implied that Crysis really wasn't that well-optimized.

 

So Crysis: Warhead would do a better example, I think, in this situation.

 

AND! I've ran Crysis on its max in-game settings, 1080p, 53 avg FPS, on a GTX 660, someone said something about modern hardware not running it well, so I don't know about that though...

 

I've finished it before on just the crappy Intel HD 2000 Graphics, just saying I know what its like :P


You can bark like a dog, but that won't make you a dog.

You can act like someone you're not, but that won't change who you are.

 

Finished Crysis without a discrete GPU,15 FPS average, and a lot of heart

 

How I plan my builds -

Spoiler

For me I start with the "There's no way I'm not gonna spend $1,000 on a system."

Followed by the "Wow I need to buy the OS for a $100!?"

Then "Let's start with the 'best budget GPU' and 'best budget CPU' that actually fits what I think is my budget."

Realizing my budget is a lot less, I work my way to "I think these new games will run on a cheap ass CPU."

Then end with "The new parts launching next year is probably gonna be better and faster for the same price so I'll just buy next year."

 

Link to post
Share on other sites

You are better off finding a game that does run well and just uncapping the frame rate. There will be less problems other then the power of your hardware. SOme low end GPU's perform better in some games then high end GPU's.


 (\__/)

 (='.'=)

(")_(")  GTX 1070 5820K 500GB Samsung EVO SSD 1TB WD Green 16GB of RAM Corsair 540 Air Black EVGA Supernova 750W Gold  Logitech G502 Fiio E10 Wharfedale Diamond 220 Yamaha A-S501 Lian Li Fan Controller NHD-15 KBTalking Keyboard

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×