Oxide Developer Confirms Latest Nvidia Drivers Support Asynchronous Compute

[Parhelia]

Source: http://www.overclock.net/t/1590939/wccf-hitman-to-feature-best-implementation-of-dx12-async-compute-yet-says-amd/370#post_24898074

 

Quote

I can confirm that the latest shipping DX12 drivers from NV do support async compute. You'd have to ask NV how specifically it is implemented.

 

 

So, apparently that's happening. I'm not really surprised since Maxwell is an advanced/modular enough design to support running different workloads simultaneously; it obviously just needs the scheduler/drivers/CPU to be able to, well, schedule it. Don't know if it matter much for Maxwell (since it already feeds it shaders very nicely) except to maybe reduce performance overhead from context switches, but there it is. It's doing it.

 

TLDR: Nvidia's driver team are wizards. 

 

 

 

 

 

 

To the mods, sorry if this isn't considered a good enough source (although it is a confirmed Oxide developer). I've been lurking here for a while and saw no websites or people posting this, so I just signed up and did it.

[wyattzx]

I'm confused. Software can't add hardware, can it? I thought the cards themselves hadn't implemented asynchronous compute?

[Castdeath97]

Fix that font! Set it to automatic.

[Guest]

Fixed Formatting:

Quote

I can confirm that the latest shipping DX12 drivers from NV do support async compute. You'd have to ask NV how specifically it is implemented.

Source: http://www.overclock.net/t/1590939/wccf-hitman-to-feature-best-implementation-of-dx12-async-compute-yet-says-amd/370#post_24898074

 

So, apparently that's happening. I'm not really surprised since Maxwell is an advanced/modular enough design to support running different workloads simultaneously; it obviously just needs the scheduler/drivers/CPU to be able to, well, schedule it. Don't know if it matter much for Maxwell (since it already feeds it shaders very nicely) except to maybe reduce performance overhead from context switches, but there it is. AMD's Robert Hallock was just proven wrong for stating that Maxwell was utterly incapable of doing it. It's doing it.

 

TLDR: Nvidia's driver team are wizards.

[TechGod]

You can't add a hardware feature with software...

[Parhelia]

8 minutes ago, NonaHexa said:

I'm confused. Software can't add hardware, can it? I thought the cards themselves hadn't implemented asynchronous compute?

The hardware people seemed to be talking about, if I'm not mistaken, is on-GPU scheduling hardware which GCN has and Maxwell doesn't. Maxwell relies on Nvidia designing drivers (running on the CPU) capable of telling the shaders what to do whether it's in DX11 or DX12.

 

Nvidia simply not getting around to doing that yet would explain why async compute wasn't working on Maxwell, but I don't see why it can't do it other than that. The hardware for Maxwell to do graphics and compute both is already there (I mean, the GPU does have shaders organized in separate units or SMMs capable of doing either), but Nvidia's scheduler just didn't support the feature allowing them simultaneously yet.

 

I'm no expert on the subject and am just giving my thoughts, but it makes sense to me.

 

7 minutes ago, Castdeath97 said:

Fix that font! Set it to automatic.

Sorry, new here. I think I fixed it. Is that better?

[Castdeath97]

Just now, Parhelia said:

The hardware people seemed to be talking about, if I'm not mistaken, is on-GPU scheduling hardware which GCN has and Maxwell doesn't. Maxwell relies on Nvidia designing drivers (running on the CPU) capable of telling the shaders what to do whether it's in DX11 or DX12.

 

Nvidia simply not getting around to doing that yet would explain why async compute wasn't working on Maxwell, but I don't see why it can't do it other than that. The hardware for Maxwell to do graphics and compute both is already there (I mean, the GPU does have shaders organized in separate units or SMMs), but Nvidia's scheduler just didn't support the feature allowing them separately yet.

 

I'm no expert on the subject and am just giving my thoughts, but it makes sense to me.

 

Sorry, new here. I think I fixed it. Is that better?

yes

[zMeul]

Huddy and couple of million asshats should eat crow by now

 

there wasn't even in question if Maxwell v2 can do Async Compute

the question was it if it can do it inside the pipeline at the same time as regular graphics - my guess is nVidia added a layer of software to the scheduler, much like hyperthreading works:

[Ostwind]

NV driver team are wizards but they still release garbage drivers with issues that have been going on for months.

[ozziestig]

This confirms Nvidias driver team is made up of wizards. They can make software do hardware shit.

[Citadelen]

I read something about this earlier in the year, so they can use asynchronous compute, but just a small error in their code would create massive latency, so it means Nvidia has to work harder to get the performance AMD gets with actual hardware capable of it. That's what I've taken away from this, correct me if I'm wrong.

[alamox]

nvidia card were always able to do async compte through drivers, but they had issues with heavy workload, they solved that issue, but you still need to test it and find the trade off, and even if there isn't a software solution wont be as good as a hardware one, especialy for latency, and it being the main reason for async...

[zacRupnow]

54 minutes ago, D14RAP said:

Fixed Formatting:

What's wrong with the source at the top? That's all you changed.

[Prysin]

42 minutes ago, NonaHexa said:

I'm confused. Software can't add hardware, can it? I thought the cards themselves hadn't implemented asynchronous compute?

The cards doesnt have a scheduler in the GPU. So to queue the workloads, you got to write a software based auto-scheduling system rather then having the GPU do it.

Kepler and every AMD card since the HD 7000 series has had this scheduler built into the hardware. Meaning AMD doesnt need to tell the GPU what to do, it knows. Maxwell 2.0 however got NO clue how to prioritize workloads, so unless you write an extensive software solution for it, it would simply try to solve everything in the order it was requested. Ignoring priority and crucially, not making use of the GPU horsepower when other workloads causes idle periods (we are talking nanoseconds to perhaps a microsecond or twoi of idle time. but these minute breaks adds up)

21 minutes ago, ozziestig said:

This confirms Nvidias driver team is made up of wizards. They can make software do hardware shit.

everyone can tell a piece of hardware to do this at this point in time, then next do this. What Nvidia is doing is simply making a work schedule MANUALLY for their GPU rather then have the GPU do it itself. This is costing Nvidia time and money optimizing A LOT more for every game. And worst is, there is still limitations to Maxwell, compute wise, they cannot work around.

What Nvidia is doing is no different then a teacher making a schedual for your homework with a clause that "if you get a test, studying for this test comes before homework"...

 

The reason Maxwell is so efficient is because when Nvidia went from Kepler to maxwell they litterally tore out ALL the scheduling stuff for compute. It may not seem like much, but with those parts gone, power savings were huge.

 

You can see the same with R9 390X vs R9 Fury X... but unlike Nvidia AMD werent stupid enough to rip the schedulers out.

On the contrary, Fiji, aka Fury, is desinged to be a "gaming" card first and foremost. This is why it has non-existent double precision, and also why there are no Fiji based Firepro cards. Whilst its single precision compute is monstrous (8.1TFLOPS) its double precision is 1/32 of that.... Unlike a Hawaii based GPU which is 1/4th for the R9 390X and 1/2 for the Firepro W9100 (the pro version)....

 

Everything can STILL blow up in Nvidias face, because there is ONE crucial element to all of this. Maxwell is fast at compute, faster then any AMD card actually. BUT this only applies if there is a queue depht of 32 workloads OR LESS. Maxwell (and kepler) can only do 31 compute tasks + 1 graphic/compute at a time. Meaning Max 32 compute.

GCN cards, aka HD 7000 series, 200, 300 and Fury series have up to 64 compute taks PLUSS one graphic. Meaning whilst Maxwell can be up to twice as fast at computing workloads, AMD can simply do TWICE MORE at the same time. So for Nvidia, it is all about optimizing so that compute speed is blistering fast, if you can do 2x 32 queues in equal time as 1x 64 queues then there is no issue. Consumers wont notice. However the moment the workload intensifies by ANY amount, and the GPU need to work longer on some aspects, AMDs GCN cards will pull ahead.

 

This all boils down to OXIDE games. Did Nvidia manage to bully them into ONLY using 32 deep queues? in which case Maxwell WILL be faster then AMD cards, because it simply IS faster at computing the workload. However if OXIDE uses deeper then 32 queues, say 64 deep. Then maxwell will struggle, because it is beyond its capacity so it has to do two passes to complete the workload.

 

[Guest]

Just now, zacRupnow said:

What's wrong with the source at the top? That's all you changed.

removed spacing, changed source, removed text formatting so dark theme can see the quoted text.

[Parhelia]

2 hours ago, ozziestig said:

This confirms Nvidias driver team is made up of wizards. They can make software do hardware shit.

I don't think some people are understanding the whole "software scheduler" thing. The CPU the scheduler runs on is hardware and then it's still only the scheduler. The GPUs themselves do the actual simultaneous compute contingent on the driver's scheduler being advanced enough to tell the GPU to do it.

 

They're not trying to add hardware capability with software as if it's not hardware running that software or whatever people think.

 

2 hours ago, alamox said:

nvidia card were always able to do async compte through drivers, but they had issues with heavy workload, they solved that issue, but you still need to test it and find the trade off, and even if there isn't a software solution wont be as good as a hardware one, especialy for latency, and it being the main reason for async...

 

Current Nvidia Maxwell cards, Kepler, and AMD cards before the 7970 too do all scheduling through drivers whether async or not. So far, so good. Any potential latency issues haven't been a problem and seem to be masked by actual execution time. If it wasn't a problem before, why should it be a problem now?

 

And the main reason for async isn't for scheduling -> execution latency, it's for pure maximization of rendering power and minimization of idle shaders. 

[AlTech]

1 hour ago, NonaHexa said:

I'm confused. Software can't add hardware, can it? I thought the cards themselves hadn't implemented asynchronous compute?

They did but they were rather shitty at it.

[AlTech]

1 hour ago, Citadelen said:

I read something about this earlier in the year, so they can use asynchronous compute, but just a small error in their code would create massive latency, so it means Nvidia has to work harder to get the performance AMD gets with actual hardware capable of it. That's what I've taken away from this, correct me if I'm wrong.

loving your rig. Specs are awesome buddy. (Except that 8gb ram)

[Citadelen]

1 minute ago, AluminiumTech said:

loving your rig. Specs are awesome buddy. (Except that 8gb ram)

Heh, thanks, if I weren't upgrading to Zen later in the year then 24GB would be something I would consider.

[AlTech]

3 minutes ago, Arokhantos said:

 

Mostly everything is software anyway remember when nvidia acquired physx and magicaly all their cards that supported cuda also supported physx.

Async is different. The cards support it but not very well.

[AlTech]

1 hour ago, zMeul said:

Huddy and couple of million asshats should eat crow by now

 

there wasn't even in question if Maxwell v2 can do Async Compute

the question was it if it can do it inside the pipeline at the same time as regular graphics - my guess is nVidia added a layer of software to the scheduler, much like hyperthreading works:

Huuuh. Say what? Hyperthreading is software based? WHAT????!!!! 

[patrickjp93]

3 hours ago, NonaHexa said:

I'm confused. Software can't add hardware, can it? I thought the cards themselves hadn't implemented asynchronous compute?

What exactly do you think emulation is? Every algorithm you can implement in software can be implemented in a hardware circuit (with some workload size restrictions for certain algorithms) to increase performance and reduce power consumption.

[samcool55]

Umm, i don't know about you guys, but i read something else than you.

 

I read that the DRIVER from Nvidia supports Dx12. But that doesn't mean the gpu's support it...

 

And there was already support for it because with Async on, Ashes just works. It runs terrible on Nvidia cards, but it works. Which means something is done with it.

Maybe they have optimized it or something but i'm not getting my hopes up.

 

And Nvidia driver devs should make work of Geforce experience, it's terrible! I don't understand how they can make decent drivers, but also make such a big piece of sh*t like geforce experience...

[patrickjp93]

1 hour ago, AluminiumTech said:

Huuuh. Say what? Hyperthreading is software based? WHAT????!!!! 

No, but to get the most out of it, you need to be aware of how it works when you program your software. If two threads on the same core share the same instructions in their pipeline and the same data, you'll lose performance to have both running. If they share data but not instructions, you get the most performance gains.

[patrickjp93]

1 minute ago, samcool55 said:

Umm, i don't know about you guys, but i read something else than you.

 

I read that the DRIVER from Nvidia supports Dx12. But that doesn't mean the gpu's support it...

 

And there was already support for it because with Async on, Ashes just works. It runs terrible on Nvidia cards, but it works. Which means something is done with it.

Maybe they have optimized it or something but i'm not getting my hopes up.

 

And Nvidia driver devs should make work of Geforce experience, it's terrible! I don't understand how they can make decent drivers, but also make such a big piece of sh*t like geforce experience...

Programmers have specialties and weaknesses. Embedded system programmers (driver team) generally are not good with designing user experiences and interfaces. However I imagine Nvidia has a dedicated team for that, and they should start from scratch for sure.