Jump to content
  • entries
    7
  • comments
    5
  • views
    2,977

Benchmarking Prodecures

LogicalDrm

940 views

I haven't done anything that would require benchmarking in ages. Last change made to my old log file was 2014 after I changed some case fans and wanted to check if they did something to temps. This July I changed graphics card which is something I would say anyone should run their own sequence after. So after any bigger change (CPU/GPU mainly) anyone should run same benchmarks they have run before change. I will come back to this in a moment. In my situation need was really big. I went from 2011 midrange card to 2013 top end card. Something that in paper is bit better than 280X, card that I was eyeing for upgrade year ago. Yes, talking about used cards. Upgrade was from GTX 560Ti factory OC'd to reference GTX 780. Benchmarks would look awesome. In numbers if not actually, but thats more about nature of tests themselves.

 

So why this post? Well, so I can refer to it when someone asks advice about benchmarking. Since my last run was 3 years ago and I haven't really played any new games, all my game benchmarks are pretty bad choices. Like NFS Shift and Battlefield Bad Company 2. There might be Battlefield 3, but thats it. This new batch won't have any real gameplay. Because of two reasons. 1. I still lack good games for realistic benchmarking, Battlefield 4 is probably on the heavier side. So nah on games. And 2. I don't want to buy games just because I could get good benchmark out of them. Even games with benchmark tools would be just for it since I don't play 3rd person single player adventure/action games (Metro 2033, Tomb Raider, Witcher 3). This will be list of benchmarks which are free and provide good base for anyone looking to create their own sequence.

 

1. What is my "sequence"?

 

I use term "sequence" to describe procedure where I run multiple tests, check temps and mark down scores. Marking down scores and temps is good practice in general. Not just for bragging, but to check how much you spending money actually improves the systems performance. So my sequence involves having few monitoring software open, taking numbers in notepad and running several benchmarks one by one. Yes, it will take some time, some 1.5h for me. But you need to do it once and then you can just refer notes later if someone asks something. I've used temp readings many times to advice on high temps under stress tests and idles on my older hardware.

 

To actual point. I have 10 software, 12 tests, 1 main monitoring software with 2 others running, notepad with template for scores and temps and FRAPS for one odd out fps reading. I cover tests later. I happen to have 2 monitors, but all this can be done on one monitor. 2nd is good for having all monitoring software there. Like in this manor:

2nd_monitoring.thumb.png.1795b7d076ff4785661f9d97cdad8fb1.png

 

So MSI Afterburner is present because of habit of looking at graphs. I actually don't use it for GPU temp monitoring anymore, but habit of looking at fan and temp graphs remains. Main screen is where I look at GPU temp when test is running. Mainly since its new piece and I want to see how my fan settings are holding. Main software here is RealTemp with GPU temp monitoring open. I reset Maximum readings after every test to get reliable readings for all tests individually. Under them all is my normal main monitoring software, OpenHardwareMonitor. Notepad is on main monitor since I don't need it until after test finishes.

 

Things I do and would recommend. I have habit of doing this after cold boot. I would recommend booting before running sequence. Its easiest way, since there's least amount of extra software running on background. For this sequence I added way of closing all extra stuff I have open. Skype, several driver software's, basically anything except multimonitor manager, fan controls and virus scanner. Some might close virus scanner too, but I don't mind it. Then another boot after all testing is done. To get everything working as normal. I would recommend setting any fan profiles like you are going to use them before running test on new components. Makes more realistic comparison. As for temps, I have idle's. Which I will take after first test has run. Reason being that idle's coming directly from boot will be lower than what you are going to see any other time. For testing temps I use max temp. Its most relevant. Since RealTemp shows temps for all cores, I use average with easy count. Take highest and lowest and split difference with upward rounding. Like 66C and 58C would be 62C (8/2=4, 58+4=62C) or 67C and 60C would be 64C (7/2=3.5, round up 4).

 

2. Tests and score keeping

 

Let's start with synthetic benchmarks. I will give some background on why I use software listed here, where to get it and what settings to use (if needed).

 

3DMark https://www.futuremark.com/downloads/3dmark.zip

If you have had gaming PC for some time, you know this software already. Futuremark's (Yay! for Finnish company) 3DMark has been industry standard for almost 20 years. Its combined benchmark, meaning that it tests both GPU and CPU within single run. First 3DMark I've used is 3DMark03. I've had 05, 06. Vantage and 11. 3DMark06 was used for long time because of its DX9 support. Until Vantage with DX10.1 it was only thing to test new hardware reliably. Thats important. Because of the way benchmarks work, if you change something the score will also differ. So comparing two results between different versions of software can cause issues. I first noticed this fact with FurMark.

 

But back to 3DMark. Free version has 3-4 tests. Most common is FireStrike which is for current gen gaming PCs. You can run it with lower end hardware if you are like me and want before upgrade scores to compare. I also used tad lighter SkyDiver. Mainly because I couldn't get 06 running anymore. So let both tests run, mark score to notepad, mark max temps for GPU and CPU, remember to let temps go back to "idle" in between and reset max readings before running the test. Thats it. Free version doesn't have any settings to toggle. Only thing I would like to toggle is Demo.

 

CINEBENCH R15

Yes, they actually have product name in caps. Maxon makes professional 3D modeling and animation software as their main source of income. But Cinebench has become one reason for their homepage to get constant traffic. It has 2 tests, individually for CPU and GPU. GPU uses OpenCL, CPU renders image using all available threads. Cinebench gives some comparison for similar systems, but I wouldn't look that graph too much. I also don't think it as very taxing software. Run tests for both CPU and GPU, with temp normalization in between.

 

This is among those software with this batch that has single part focus on testing. I don't have just for CPU, but I might look into that part more. Intel Extreme Tuning Utility has CPU testing, but I don't know if it works with AMD. Anyway, having whole benchmark just for single component has some advantages. Like if you'd like to test air cooling myth about radiating GPU heat. So running GPU only test would raise only GPUs temp notably and do something to CPUs temp also. I don't look at utilization when I run these tests, but it could be one thing to check also if you want to gather more data.

 

Catzilla http://www.catzilla.com/download

ALLBenchmark's test is different from the two above because it  has very noticeable sound effects and music. Otherwise its just another combined benchmark. I've used it since I heard about it from OC3D's TinyTomLogan. TTL is someone who's opinion on OC and CPU performance means a lot to me. I've picked other go-to software from him with OCCT, a stress test software for CPU. But back to Catzilla. Basic version only has 540p benchmark, but you get 720p one by creating account to their site. Easy thing with Google, Twitter or Facebook linking. Rest of the stuff is like before.

 

RealBench https://rog.asus.com/rog-pro/realbench-v2-leaderboard/

ASUS' RealBench is combined test which uses real life tasks for benchmarking. Image manipulation, rendering, video encoding and multitasking. Besides giving total score, you get score per test. I mark all of them down. I got to see how much GPU did bottleneck CPU on CPU heavy tasks. Result? Not that much. But some.

 

UserBenchmark

This is new to my lineup. I haven't looked much into what it actually measures. But looks to be lighter side combined test. I would say it replaces Novabench I had on earlier lineup. Results are given in three categories, for Gaming, Desktop and Workstation in percentage. I marked those percentages as results.

 

Heaven and Valley

These two from Unigine are pure GPU tests. Heaven is what is commonly used for GPU OC testing, temp testing and benchmarking by reviewers. Valley is bit heavier, so I'd say running them both is good practice. Like with 3DMark, there are more tests you can use. I used highest presets for both, Extreme for Heaven and ExtremeHD for Valley.

 

FFXIV: Stormblood

Like said earlier, I don't have real gameplay benchmarks in this set. So this game from 2013 is lower end gaming benchmark for me. It has preset for Mazximum settings, but I pumped those bit more. You can check my settings from PDF attached. I use score as scoring, but you could have also given FPS.

(FFXIV_Stormbloo_benchmark_LoGiCalDrm_config.pdf)

 

Star Swarm http://store.steampowered.com/app/267130/Star_Swarm_Stress_Test/

This game engine benchmark is free on Steam. Released 2014 by Oxide Games, guys behind Ashes of Singularity, its has capability of simulating space battle game. There are few options to simulate different style of games. I used Attract with Extreme settings. Score will be given as average FPS. You can select some other combination, important part being that same settings are used before and after any upgrade made to keep score consistent.

 

demo2 https://files.scene.org/view/parties/2015/assembly15/demo/demo2_by_ekspert.zip

This one is something I cooked up. Its newer than both other gaming style benchmarks, its done with Unreal Engine 4 and I have no clue if it has any relation to real world or not. Its Demo made by group called Ekspert for Assembly LAN Demo Compo 2015, in which it placed 2nd. There are few remarks I want to make about demo's and demoscene before going actual benchmarking part. Demoscene is all about digital art. Animation, coding, graphics, music, indie game development. At least in Europe, many software and game dev companies have their roots deep into demoscene. If you are doing the things I mentioned above and want to show off your skills by competing, maybe look if there are parties/compo's held in your area. The two known Finnish companies with demoscene background are Futuremark (surprise, surprise) and Rovio. One for making first PC demo at the time when Amiga and Commodore 64 were main platforms, other for making mobile game back when those were played on Nokia N-Cage's.

 

Now back to benchmarking part. Demo2 doesn't have built-in scoring system. So I've used FRAPS to calculate average FPS. Running 1080p version gives warning about using Fraps for recording purposes, but loads just fine after that. I start benchmark counter as soon as demo starts and check scores afterwards. Nothing more to it.

 

Others?

As I said along the lines, I would like to have more modern game benchmarks, as well as CPU only benchmark. So I will be looking around for those and adding them here. If someone reading this has ideas about free or cheap games with included benchmarks, please let me know.

 

3. Scores and comparison

 

At this point you should have raw data text file. Something like this one I'm using:

benchmsrk.txt

Which includes system specs for each test cycle. This would be the file you are updating during tests. Feel free to use it as template or comparison. But what now? Well, you can just compare by eye results, use it to quickly refer temps and so on. But what if you want know how much better system performance is after upgrade. Thats where Excel (or Sheets, Calc and so on) comes in. Copying results to Excel (&co) can be annoying, but do it once to get template correct and maybe adjust .txt file to help in future. Here's my .xlsx for reference. Also free to be used as template or comparison:

benchmark_LoGiCalDrm.xlsx

 

In file I've got some extended system info and notes about tests. Which are pretty much same as in here. But main thing is +- column (Excel hint: add < ' > in front of symbol to exclude any automatic formula). It calculates how many percentage better new score is compared to last. Works best when score changes less than 100% or new score is over twice the value of old. Formula used is pretty simple:

=(<new>/<old>)-1

Shown in percentage. You can add color coding and such if needed. If new value are over twice bigger, remove <-1> to get accurate score.

 

There's some oddness in that file in CPU temps. I had issue with Speedfan missing fan profiles for 2 front intakes. Which I fixed after I had changed to new GPU. So those are something to ignore.

 

4. Conclusion

 

I hope this helps those who are new to benchmarking. Note that this is just how I do things and you should be taking it as guide or advice. Make it your own. I will be doing some fixes along the lines, but as I will not be getting major upgrade in few years, its quite possible I'll be making another batch of tests when next upgrade is on me.

0 Comments

There are no comments to display.

×