Coool

I can't wait for it to be open-source so I can add tests for some of my personal favorite games.

Having it open source would be nice to test hardware on VR and see how it stacks up. Once it’s public you can show users what to upgrade to have a better experience playing at desired settings. 

I probably will have a lot more to say in the future, but for now I'll mostly stick with my thoughts on publishing the "mother of all testing databases". In the screenshots so far and in the video this is shown to be Grafana. Which makes a lot of sense as it is a powerful that can present data in a lot of neat ways. 
 
It also is a tool that (sometimes depending on the source) can wildly misrepresent data if you are not careful. For example it is often combined with Prometheus as an underlying data source in IT environments for monitoring applications. However the way Prometheus works means that averages can be misleading if the dataset is over a too short period of time. It is a bit too much tl;dr to explain in depth and I fully expect the folks in the lab to be aware of this as well. 
 
However when making it a public dataset a lot of people will not be aware of these sorts of caveats. So my suggestion there would be to think about what to expose and how. For example if you are giving grafana dashboards to everyone make sure to also put a big fat explanation (or link to it) next to it that goes into how to read the data. That way you can somewhat reduce changes of people wildly misinterpreting your presented data in internet slapfights. 
 
Having said all that, I am a big fan of using what is already there for exposing and applying all this data. It is a sensible approach that hopefully will allow a lot more than would be possible if you tried to do it all with custom solutions.  
 
In that regard, I am curious to how things are set up in the background. Is the labs team leaning on the floatplane development infrastructure to make sure things they build are maintainable and such?  If not, what is the approach there. To me it seems that a lot of what the labs does is effectively very similar to modern software development and specifically very similar to test automation and performance testing. So I'd expect that internally there could be a lot of benefit from setting up things in a same way. So for example making use of gitlab (self hosted) for the development, hosting and possibly even triggering of MarkBench through pipelines.  Because to keep things consistent you of course want to make sure things are run with similar version of MarkBench and the same configs for games. In a similar sense I can see a benefit in keeping a repository with sets of game configs for various types of benchmarking. In fact, with a little bit of effort I can even see a future where you don't use a GUI or where the GUI doesn't trigger local MarkBench. But, instead triggers a pipeline which fires up the benchmarks on a few test benches in parallel. 
 
As someone who works as a test automation engineer I see a lot of potential here. Needless to say that I am excited about this 😄 In fact, if it wasn't for me being located in Europe I honestly would have considered applying for a ltt job at some point. 

Host project on github/gitlab/bitbucket etc., so we can also actively participate in its development 

You should definitely open source this! It would be cool to have a "My Hardware Specs" portion of your portfolio on this forum that would automatically upload your hardware specs & performance metrics from Mark Bench.

From what I have been told this is just a way to automate benchmarks that were developed by others. 
Could you please clarify what this "MarkBench" is and how it functions so that we can better be able to address your request for feedback. 
Is this actually coming with its own benchmarking software? Or is it an automation tool for running other benchmark/stress test software? 
 
If it is the latter I would like to see stress test type software along with the benchmarks that give scores/results. When dealing with used hardware I find it is just as important to stress test components as much as it is to test in benchmarks and verify/validate the scores/results. 
 
I also find it useful to do comprehensive testing rather than relying on only a few tests. 
 
The bundle of software I would like to see included to run with automation (but not limited to):
 
3DMark
PCMark
Unigine - Heaven, Valley, Superposition
PassMark Performance Test
Novabench
Kombustor
Basemark
Geekbench
GFXbench
UserBenchmark
Cinebench - R15 and beyond
OCCT
Furmark
BurnInTest
AIDA64
Sandra SiSoftware
Prime95
LinX
Crystal DiskMark
Crystal DiskInfo
HWiNFO
CPU-Z
GPU-Z
HWMonitor
Speccy
Fraps
Afterburner
EVGA Precision
 
And easy to automate software that can log the data, screenshot results and produce graphs and comparisons would be ideal. "Set it, forget it and walk away until it's finished" is the goal. Run, log, move onto the next and repeat. 
 
 

I'm a serial early adopter of OSS projects in enterprise environments and I strongly suggest at the least exposing your git repo and some build instructions to the public as soon as possible. Getting your tool into the hands of developers, engineers and tinkerers is a great way to improve the product. So long as the builds are reasonably stable don't be bashful about going public with a long list of features not yet implemented. This is coming from a guy who started using Hashicorp Vault in prod at 0.6 and Terraform at 0.10.
 
Avoid feature creep, Mark Bench should be all about running the benchmarks in a reliable and repeatable manner, it should not be provisioning/building infrastructure as well as there are already tools and stacks that do that far better. Having an accessible API so those other tools and stacks can orchestrate Mark Bench would be more valuable than trying to re-invent several wheels.

Releasing the source code under some type of GPL license on GitHub would be cool.

Feature suggestion:
add a section where people can share their results from their own hardware since the result can change from a pc to another so that they can compare their rigs to others

Here are some things I would like:
Command Line Interface Integration with or into Phoronix Test Suite GPL License (to make it so no other company can take it proprietary)

As someone who worked on similar systems and also had to use them at a hardware manufacturer, here's some tips/features that'll make everyone's lives better:
A proper cli, and NOT an interactive one. You should be able to kick it off just with flags/args and have it run, then exit when finished Ofc the automation for games is gonna be jenk, terrible, and platform specific. Just how it goes. But the tooling around it can be made cross platform, and individual tests should be classes which can be defined for different apps and OSes/arches Rely as much as possible on OS and vendor tools/APIs vs random python/go packages to get metrics. They die, get unmaintained, etc all the time. Unless you plan to pick them up as well Don't try and make metrics in your automation tool. Just collect raw data and let the data people use it in the rawest form Doing things the "intense" way where you have a controller system running something that hooks into a server on the DUT is a lot of work, but rewards you with power cycling abilities, handling crashes, hooking into computer vis tools you're developing to analyze the screen, etc. Others do this, for a reason. Idk how big your team is or what your timelines are like, but long-term this enables a lot more data collection than otherwise available running locally only Devs probably don't get a say, but time spent making a UI is time wasted. A good cli is all you need for automation. It is better, in fact. But since your stuff is also used by a media company and maybe the public idk how much you can afford to ignore that. All I know is I have wasted many days trying to automate proprietary tools who have fancy UIs but not 1000s of people to interact with them every day to run all the tests they're used for, and I always wished their time was better spent

A database of system scores so users can compare and validate their system performance.

Thank you guys so much!
Making it open source would be a huge bonus

It might be a good idea to include non-gaming related test. Such as Blender rendering, Ai training, encoding and decoding, etc. That way you may be able to test other components, such as CPUs

Games are good, but I'm not much of a gamer. Here's what I'd like to see
1) multi-platform. 
2) data I/O, SQL queries, massive copies etc. 
3) Video encode/decode
4) Compilation. The Linux Kernel would be a suitable benchmark.
5) The core being open source so that it can be ported to other systems. How does that new Intel chip compare to IBM big iron?

Will this Benchmark tool at some point either be open source or partially open source. I can see a lot of potential for community members to build "harnesses" for games that may be not as popular but still interesting.
That way someone could ask the Lab to please benchmark their favourite game while also providing an easy way for that to happen.

https://docs.google.com/spreadsheets/d/1AddRvGWJ_f4B6UC7_IftDiVudVc8CJ8sxLUqlxVsCz4/edit#gid=1849517480
 
Welcome to the forum.

For all of you not on Floatplane here is G Drive link to full size images. Thank me later.
 
https://drive.google.com/drive/folders/1gBFhVlE3FXMhe_t76ZuAhrHsQ6GRi6dL

The power supply unit inside the PC is a actually a 12-volt only PSU following the new ATX12VO standard, which explains why it needs a breakout board for SATA power (no 3.3/5 volt output) and the seemingly low 360-watt rated output. It also means that the power supply is more efficient and offer super low standby power.
 
Consider this: they use a mere 460-watt version of the same 12VO PSU, even if you manually configure the machine to the highest possible config (9900K+2080).
 
Calling it "basement tier" clearly shows that you don't even follow the latest development in tech or understand the legal basis for Dell to make such decision.
 
Maybe it's a good time for you to make a Techquickie for the new PSU standard.
 
Edit note: Insert screenshot with highlight

I bought one myself (4980HQ) here's to hoping it works and is actually a 4980HQ. as for the board I bought a cheap Z87 from sapphire ($30 shipped). I doubt a Z97 board would work better vs a Z87 board, as this is still technically Haswell/Crystalwell
 
As for iGPU overclocking, I could oc my HD 4600 on my Alienware 17 R1, unlocked chip (4930MX) or not (4700MQ) but I preferred the dGPU (GTX 780M/980M/1070)
I wish I still had it so I could get a PGA version of the 4980HQ
https://item.taobao.com/item.htm?id=561463114198

He was on some form of maximus vi, however he had all kinds of random issues like it failing to post every other reboot. Not sure how much is his board and how much is cpu. His board was a bit rough when he got it from ebay. And the cpu further flattened out the pins. Plus I wonder how well a z87 board with a pre-broadwell bios supports the l4 cache. Basically as long as you don't use a gigabyte board these cpus should work to some degree out of the box. Based on his experience I can't recomend it as a daily chip however I've yet to test it on a proper z97 board. Need to convince him to send it to me for testing?
 
As for the IGP all intel HD graphics pre-lake chips can be overclocked. They've always had fully unlocked multiplier on z series boards until lga 1151 where the IGP is also locked on locked chips. (no bclk past 102.99 on locked skylake igp either as disabling the IME takes avx and igp with it). My friend managed this on water https://hwbot.org/submission/3978281_ 

I have a friend who has been messing with this chip recently, at least on his board it was semi unlocked. He had multiplier control up to 44x, however it also supported bclk straps (locked chips aren't supposed to have that) so he was able to set the 125 bclk strap and run 38x multiplier to get his chip to 4.8 on water for benching. Also got some "strong" igpu scores on it as it has a gpu just a bit slower than what the 5775c has as well as l4 cache. The difference is haswell clocks significantly better than broadwell cpus side and probably gpu side as well.

The iGPU can eat 30w.
About 28w at idle
 

an example please. 4700hq and 4700mq have same turbo clocks, same base clock, same iGPU, only different in socket (BGA1364 vs PGA946)

There isnt HK back then, but there are MX and MQ. Only MX is unlocked, MQ is pretty much HQ in PGA form factor.