Valve release 'dynamic image quality' tech for Unity game engine VR

[Humbug]

With traditional PC gaming we are used to dialing in our graphics settings, and then experiencing a very dynamic frame-rate as the scene complexity changes.
A different more complicated approach would be to have a target frame-rate and dynamically adjust the rendering quality of the next frame based on how much time the last frame took to render. The tech is actually part of Valve's Source 2 engine. However the SDK for that engine is still not out. Instead we are getting this tech because Valve has released it as a rendering plugin for the hugely popular unity game engine. Theoretically there is nothing to stop this from being used in non-VR games but that's not expected to be the primary use case. The idea is to use the highest possible image quality while staying above target framerate. So scene complexity and midrange GPUs are accomodated without downgrading high end guys. Don't expect this to make things like Star Citizen VR playable on a 750ti. But it will be beneficial to people who are howering just under the minimum spec of R9 290 or GTX 970. Unity developers will get the tech at no extra cost.

 

The Unity game engine supports both the HTC Vive and the Oculus Rift.

Quote

VR engines can't just slow to a crawl when scenes get crowded or complicated. Remember, dipping below 90 fps can be literally sickening. In the past, VR engines have dealt with this problem using tricks like reprojection. If a complex scene meant a frame wasn't ready in time, the engine could run a quick transformation on the previous frame, reprojecting the old image to estimate what the scene might look like at a new head position and angle. That gives the engine some time to catch up with itself, but it's an imperfect solution that can lead to juddery images that look out-of-focus (especially if the trick is used frequently). Valve's Alex Vlachos urged developers only to use this as "a last resort safety net" and not to rely on that technique unless the game is running on hardware well below the minimum specifications.

 

A better solution, as Vlachos laid out in detail in his GDC talk, is adaptive quality adjustment. This basically means that the system monitors the processing time for each rendered frame and immediately lowers or raises the quality of the rendering based on how much headroom is available for that scene.

 

If the user is looking at a relatively plain and detail-free wall, for instance, the engine can temporarily jack up the image quality to take advantage of all the extra available processing time. When looking at the complicated innards of an Aperture robot, though, the engine would need to quickly dial back the quality to maintain that 90 frames-per-second performance. The goal is to use 70 to 90 percent of the GPU's power at all times while always pushing out frames in that tight 11.1 ms window (though Vlachos says operating within 10 ms is better to leave some GPU resources for the rest of the system).

 

Certain graphical features can't really be comfortably altered using this adaptive method; the user would probably notice if specular reflections or shadows suddenly popped on and off as they moved around in VR. But the engine can comfortably scale its rendering resolution and viewport size slowly up and down without any jarring effects. The level of detail in edge-smoothing features like multisample anti-aliasing (MSAA) can also be tweaked on the fly relatively easily.

 

 

There is also an algorithm known as fixed foveated rendering which can be used to dial back the image quality starting from the edges of the HMDs 110 degree FOV in order to prioritize high quality rendering in the center of the displays. For this particular feature most devs will probably consider it safer to use only for outer edges of the FOV until eye-tracking in HMDs become commonplace. The HTC Vive and Oculus Rift are head tracking HMDs without eye tracking. However it's worth noting that people normally do not move their eyes to observe the edges of their FOV (although they physically can), people generally move their eyes in a quite limited range and use it in combination with  head movements. So the technique can still be useful...

Quote

At the very far edges of the performance curve, the engine can even pull off some VR-specific graphics tricks to ensure rendered frames are ready on time. Using fixed foveated rendering, for instance, the engine can lower the apparent resolution in a ring around the edge of each eye's circular VR view. Radial density masking similarly skips rendering alternating pixels near the edge of the player's vision, filling them in based on their neighbors instead. These tricks save a lot of processing time—enough for a 10- to 25-percent performance improvement on their own, Vlachos said—while also mirroring our eyes' natural tendency to see objects in more detail near the center of our vision.

 

 

Quote

 

Lowering the VR bar

With all this in mind, Vlachos was able to set a number of stair-step quality levels for Valve's Aperture Science demo. At the high end (achievable reliably only on a top-of-the-line GTX 980Ti card), the demo can run at 140 percent of the default resolution and at enhanced 8x MSAA. At the low end, the resolution drops to 65 percent of the default, MSAA scales back to 4x, and Radial Density Masking is turned on.

With those changes, a demo that was originally designed for a GTX 970 can run with a consistent frame rate on a four-year-old GTX 680 (even if details like text are hard to read at the heavily reduced resolutions required). While that doesn't mean Vlachos is comfortable recommending such old hardware as the new "minimum spec" for SteamVR, providing room for such a harsh test gives artists more room to manage the trade-offs between highly detailed in-game objects and lower rendering resolutions. It also automatically improves how good VR games can look on high-end hardware without the need to spend time writing new shader code or drawing new art assets.

Rather than hoarding this adaptive quality scaling for its own Source 2 engine, Valve will be making it available as a free Unity Engine rendering plug-in in the next few weeks. The hope is that the indie studios making many of the first VR games will be able to make those games look and perform better instantly without much additional work. Technical details aside, it's nice to see Valve doing this kind of low-level work to make sure virtual reality games perform as well as possible right out of the gate.

 

source

http://arstechnica.com/gaming/2016/03/how-valve-got-passable-vr-running-on-a-four-year-old-graphics-card/

 

************

general interest

 

 

[CommandMan7]

I would love to have this as an option in non-vr games, perhaps even setting manual FPS targets much in the same way you can make custom fan curves. I just hope this doesn't replace old-fashioned settings menus, only augment them, as some games I prefer the higher quality visuals over framerate. Overall, I hope this will make the PC as compelling a platform for users of lower end hardware, much the same way as PSVR and PlayStation.

[zMeul]

this is how you bring VR to the masses, not by releasing 800+ USD VR headsets that require a ~1000$ PC to run it

[dmegatool]

That would be very cool for non VR games. Personnaly I don't really care about every single settings. Just gimme what my machine can at 60 fps and I'll be fine. You should be able to set your target though... Maybe even a list a priority. Like I don't care about AA or shadows so tune them down first. I don't know... 

 

The video in the OP is quite interessting. It's technical but not TOO much. Good stuff. 

[alamox]

1 hour ago, zMeul said:

this is you bring VR to the masses, not by releasing 800+ USD VR headsets that require a ~1000$ PC to run it

 

the minimum spec is BS, because you dont have to keep the current level of fidelity of normal AAA games into a VR game, Devs could scale down the fidelity then go up as mainstream performance goes up, and this is what is likely to happen, the vast majority of games will run on much lower spec PCs than what the HMD set as a minimum, today the only real barrier to VR is the price of the headsets

[zMeul]

4 hours ago, alamox said:

the minimum spec is BS, because you dont have to keep the current level of fidelity of normal AAA games into a VR game, Devs could scale down the fidelity then go up as mainstream performance goes up, and this is what is likely to happen, the vast majority of games will run on much lower spec PCs than what the HMD set as a minimum, today the only real barrier to VR is the price of the headsets

so you'd buy a 800$+ HMD to look at blocks .. ok 

[Humbug]

5 hours ago, zMeul said:

so you'd buy a 800$+ HMD to look at blocks .. ok 

I think what he's saying is that there is a huge variety of graphical quality in VR games. On one hand we have Elite dangerous and Project Cars etc. But you also get a lot of games with simpler graphics like fantastic contraption and budget cuts...  I'm sure something like a GTX 960 or R9 380 can push those comfortably in VR. They don't have AAA quality assets/detail but they are very fun and immersive in VR.

 

Still I guess people who spend $800 on HMDs will probably already have strong GPUs. And that's the target market for first gen VR.

[zMeul]

9 minutes ago, Humbug said:

first gen VR.

pfff

 

SONY built the "1st gen VR", not HTC and not FaceBook - their whole system costs as much as Rift or Vive

SONY is the only one that understood what needed to be done, and you know what's funny ... console owners are more interested in VR than PC players: http://www.gamesindustry.biz/articles/2016-03-21-survey-playstation-xbox-gamers-more-interested-in-vr-than-pc-players

 

[Sky Daddy]

1 hour ago, zMeul said:

pfff

 

SONY built the "1st gen VR", not HTC and not FaceBook - their whole system costs as much as Rift or Vive

SONY is the only one that understood what needed to be done, and you know what's funny ... console owners are more interested in VR than PC players: http://www.gamesindustry.biz/articles/2016-03-21-survey-playstation-xbox-gamers-more-interested-in-vr-than-pc-players

 

These surveys are always missing one very important question, "Have you tried VR?" (not google cardboard either)

 

I guarantee the people who answered yes to that would be interested regardless of platform. 

 

[zMeul]

Just now, Sky Daddy said:

These surveys are always missing one very important question, "Have you tried VR?" (not google cardboard either)

excuse me!?

where do you try VR outside of buying it!? NOWHERE, except trade show and similar events where the press has primary access, not the general public

recently, if I recall right, HTC made some deal with a shop to have a demo unit(s) available for the public

[alamox]

49 minutes ago, zMeul said:

excuse me!?

where do you try VR outside of buying it!? NOWHERE, except trade show and similar events where the press has primary access, not the general public

recently, if I recall right, HTC made some deal with a shop to have a demo unit(s) available for the public

they are not out yet, but once they are you will find demo stands in almost every electronics store, retailer, etc, educating ppl about VR is a major part of seling it, so in a year or so, everyone will have the ability to go out and test it.

[zMeul]

1 minute ago, alamox said:

they are not out yet, but once they are you will find demo stands in almost every electronics store, retailer, etc, educating ppl about VR is a major part of seling it, so in a year or so, everyone will have the ability to go out and test it.

what are you talking about!? they're taking pre-orders

 

VR isn't something you buy on a whim or by seeing someone else play, you need to test it in person because you might find out it makes you sick

[ShadowCaptain]

its nice to see how serious valve is with VR

[Doobeedoo]

Neat stuff indeed.

[Sky Daddy]

9 hours ago, zMeul said:

excuse me!?

where do you try VR outside of buying it!? NOWHERE, except trade show and similar events where the press has primary access, not the general public

recently, if I recall right, HTC made some deal with a shop to have a demo unit(s) available for the public

Oculus has in store demo's planned. I imagine PSVR will be demoed as well. http://www.roadtovr.com/uk-retailer-currys-lets-you-go-hands-on-with-the-htc-vive-right-now/

 

Still, people who haven't tried VR may not fully understand how immersive it really is. It really makes their opinion invalid imo.

[Humbug]

19 minutes ago, Sky Daddy said:

 

Still, people who haven't tried VR may not fully understand how immersive it really is. It really makes their opinion invalid imo.

ya it's very obvious from the posts on this forum that most people haven't experienced it and are just trying to relate to it based on their experience with traditional AAA gaming which doesn't work..

[MEC-777]

In other words; expect the image to get super pixelated when the action gets heavy.  

 

/jk

[dragosudeki]

1 hour ago, Humbug said:

ya it's very obvious from the posts on this forum that most people haven't experienced it and are just trying to relate to it based on their experience with traditional AAA gaming which doesn't work..

Not really. VR is definitely an interesting technology, but I'm not going to spend anything for it even if it was priced lower. It would definitely hinder my multitasking (having multiple monitors is better than having a virtual viewing space in my opinion, especially when wearing the headset for prolonged use will likely tire you out or be more worse than you than looking at monitors). I really don't want more clutter around my desk and I'd rather not spend time plugging in/out all those cables whenever I want to put it away. To top it all off, I personally prefer AR over VR.

[alamox]

1 hour ago, MEC-777 said:

In other words; expect the image to get super pixelated when the action gets heavy.  

 

/jk

super pixelated games can still be alot more fun than most AAA games

ok i give up didnt find how to embed youtube video

 

[MEC-777]

9 minutes ago, alamox said:

super pixelated games can still be alot more fun than most AAA games

 

XjK9rK8qLg8

In some cases, yes.

[Belgarathian]

Halo 5 uses similar tech to reach 60FPS on the Xbox One. I really enjoyed it and only did I notice it was happening one or two times coming our of a large battle with lots of effects. 

 

This would be great in all games, not just VR. 

[Humbug]

7 hours ago, dragosudeki said:

Not really. VR is definitely an interesting technology, but I'm not going to spend anything for it even if it was priced lower. It would definitely hinder my multitasking (having multiple monitors is better than having a virtual viewing space in my opinion, especially when wearing the headset for prolonged use will likely tire you out or be more worse than you than looking at monitors). I really don't want more clutter around my desk and I'd rather not spend time plugging in/out all those cables whenever I want to put it away. To top it all off, I personally prefer AR over VR.

I was not touting VR for productivity. although the possibilities are there for rendering large work spaces people probably don't want to wear headsets for such prolonged periods of time until they become smaller.

We were merely talking about immersive gaming.