MarkBench Development and Feature Requests

[AdamFromLTT]

Presenting... MarkBench!

This is the home of MarkBench discussion for the time being! While this post is relatively empty right now, you can look forward to occasional development updates, answers to common questions and (eventually) a link to the software!

 

What the heck is MarkBench?
Wouldn't you like to know? And wouldn't we like to tell you! Check out our announcement video!

 

When will it be released?

When it's ready!
 

Is it free?

When it launches, MarkBench will be completely free. 

 

Can I contribute?

In the future? Maybe! In the now? Development will be done in-house. BUT, you can contribute by providing feedback and feature suggestions in this thread!

 

 

We want to hear from you! What do you think of MarkBench? What features would you like to see?

 

[4611tdude]

Love it! Will there be an easy way to tell if you thermal throttled and by how much?

[AlphaS]

Great tool! I really want it for Linux.

[Rainman875]

If you need a Beta Tester, I'm in!!

[sergiogd112]

 

It might be a good idea to include non-gaming related test. Such as Blender rendering, Ai training, encoding and decoding, etc. That way you may be able to test other components, such as CPUs

[littledarkpoet]

Simply Awesome

[Saccaed]

Very interested in this project. Will contribute where possible.

[Ridix786]

Free big W

[Rbtmn21A]

Thank you guys so much!

Making it open source would be a huge bonus

[Greyspectre]

It would be nice if it also stored/uploaded a snapshot of system specs. It would be neat if in the future when the database is more mature if you could filter results via things like CPU. For example what is the performance of this set of tests on these different cards when using a 12900k

[Sousuke]

This looks super interesting and will most certainly give your team more time to do some amazing things, I was ... pessimistic over what Labs would add to a youtube channel but it looks to be working out for you guys and you've managed to recruit some passionate people over there so glad that you guys are continuing to add and improve! 

 

I can't think of any features you should add as I am not someone who benchmarks systems as much as I used to do. 

[rcmaehl]

55 minutes ago, AdamFromLTT said:

-snip-

Feature Request:
 

Multi-GPU testing support for those with Motherboards with multiple 8x slots directly connected to the CPU? Automatically disable GPUs other than the one being tested to simplify other parts of testing.

If you need any of this automated, I will gladly create an LGPL module that you all can call from python.

[mkhornet]

Seems like a nice piece of software  If you want to talk some ideas and solutions, hit me up. I'm an ex automotive application engineer, that handled software for engine/battery test benches and test centers, so I might be able to provide you with some insight

[Drakonio]

VERY INTERESTING PRJECT!!! 

Ok, no help wanted for now... ;( and I thought this was a great opportunity to collaborate on open source project ... 

So I would love to help test too.... but I only have one hardware and not particularly powerful one (Dell "gaming" notebook from 2015). 

But if there is a way to help I am in! 

Cheers and good luck with this amazing project. 

[ElongatedMusk]

Feature Request:

 

Benchmarks that test the capabilities of ones specific silicon and then using that information to apply the best possible overclock.

[RTX 3071]

There should be user-made scripts/configs for games that aren't officially in the app. That was the first thing which got in my mind.

 

Another thing which would be cool is a dotted FPS map (I don't know what these are called in English, I mean the maps which use dots to show stuff) which will show the average, 1% and 0.1% FPS on the same picture. It'd make the jumps and stutters more noticeable than a line graph, especially their frequencies. Python already has good libraries for that so shouldn't be a lot of workload as well.

 

This one might not be helpful for everyone but what about a local server which will use more than 1 machine for the same tests at the same time? For example imagine if 6 PCs are connected to the local server, they'll run the benchmarks and once they are complete their datas will be in the same dataset. Tho I'm not sure how professional this approach is when there's the option of data merging but I'm sure labs team knows it better

 

Edit: Overclock profile testing with connecting to MSI afterburner would be great as well. For example it can test all of my 5 profiles and output the results.

[bluedot55]

Interesting. One potential use case is attempting to test some of the more difficult to test, but extremely valuable games. Probably the single most stressful popular game for a system to run right now, and also a pc exclusive that is often causing people to build a computer, is escape from tarkov. This thing is brutal to run, often being cited as using more then the typical 16gb of ram, and easily crushing modern CPUs and GPUs. 

 

Another one is path of exile. This game can drag an 8 core cpu at full all core load to under 60 fps in stressful situations, and actually scales up to a full 16 threads ATM. It's another one of those games that many people may be defining what hardware they need by, but it's essentially impossible to test. It might be possible to directly work with the devs to get a version that's stable and consistent, in which case this would be invaluable. 

 

Other potentially fun and overlooked things is testing heavily modded setups, for things like city skylines, minecraft, possibly future bethesda games, etc. Those can easily blow out the requirements, and people may well be looking to get custom hardware to just let them add a few more mods at acceptable performance. 

[NikitaCartes]

Maybe consider about probability distribution of FPS. This graph can show even very small differences between runs. Example shows fps with difference between driver version.

 

This    program can create them and it supports FrameView log.

[Justin04]

Great tool! Will it compare your results with other users like Userbenchmark?

[vkAndy]

An exciting project that will definitely work to level the playing field for consumers, and empower them to make better choices on new products. :) 

 

Just wanted to drop a link that may be helpful, as Linus mentioned some Grafana pain in the video (which I know only too well.)

 

There's a library which you can use to build out Grafana dashboards in Python which is honestly not half bad. May help with some of the pain points, as your need to make more Dashboards: https://github.com/weaveworks/grafanalib

Much nicer than hand-rolling JSON config files, or throwing a templating language like Jinja into the mix.

 

 

[Shadowalker125]

I would be very interested in helping with the opensource project when It is available. I noticed that your planning on just in house dev for now though.

[Spook1es]

It would be nice if at some point it could expose some dylib/dll interfaces for other software to hook into it or if it's already doing a lot of html and stuff then maybe it would be easier to provide some http endpoint or maintain a versioned API for stuff to hook into. Either way, if that's something you'd eventually want to add, you'd want to keep it in mind early on.

I'm already thinking how it could be useful for automated provisioning of new systems to catch hardware anomalies before you hand a computer off to someone else. 

[GorujoCY]

Not gonna lie but everything is looking very promising and I'm all for it, im willing to contribute if a little and especially beta test it or some sort!

My suggestion is adding a way to submit the data online for the community to see, so if LTT for example doesn't wanna test the Intel Arc A380 for example, well maybe another youtuber or an individual does those automation benchmarks and submits them onto the website for users to see, this will generalize the way we see gaming and productivity data while finally not worrying about bloody userbenchmark and their manipulations or something like that but since the data would be community contributed, it really does help to give consumers realistic expectations going forward... 

An optional one would also be trying to automate VR testing too, with what Headset is connected and what way etc. (eg. Direct SteamVR: Valve Index, SteamVR via Oculus Link/Air link/Virtual Desktop: Oculus Quest 2, SteamVR via iVry: phonevr, SteamVR via Oculus:  Oculus Rift S, SteamVR via WMR: HP Reverb G2 and etc. and for "other" headsets: SteamVR via OpenXR: unknown) and I can also see the appeal for VR youtubers to use this if it gets implemented, genuinely guys you got this!


I hope this gets implemented and this is definitely going to be the perfect benchmark application by far!

[rcmaehl]

Feature Request:

4 minutes ago, ChrissyGG said:

would be nice for public release to have email notification so you get a notification so you actually know when its done instead of having to conviently look at it.

 

That ^, additionally SMS notifications. This can be done without any other work once email notifications are implemented.

AT&T: <10 digit phone #>@txt.att.net

T-Mobile: <10 digit phone #>@tmomail.net

Verizon: <10 digit phone #>@vtext.com

All others can be looked up by "<carrier> sms gateway"

[ToboRobot]

A database of system scores so users can compare and validate their system performance.