SLI vs one card?

[ameetr]

I'm trying to configure a PC for deep learning and it looks like the for the price I can get one quadro RTX, I can get two 2080 supers/maybe TIs. Which of these would be better? They seem to relatively match in terms of vram, but not sure what the difference is like with specialized deep learning cores, etc. 

[Entropy.]

1 minute ago, ameetr said:

I'm trying to configure a PC for deep learning and it looks like the for the price I can get one quadro RTX, I can get two 2080 supers/maybe TIs. Which of these would be better? They seem to relatively match in terms of vram, but not sure what the difference is like with specialized deep learning cores, etc. 

You mean titan rtx? the titan and quadros are configured better for deeper learning while the 2080S/ti's are less so

[Mick Naughty]

Two is better than one.  For everything. 
 

[ameetr]

1 minute ago, Brok3n But who cares? said:

You mean titan rtx? the titan and quadros are configured better for deeper learning while the 2080S/ti's are less so

I think quadro rtx (5000) - the titan is even more expensive right? I'm able to get an academic discount on it, which is how it comes to be relatively the same as two 2080 supers. I know that a single quadro would do better than a single RTX 2080 super or TI for that matter, but would it do better than two of them running in SLI? SLI might be more of a headache but I'm pretty sure it would work for my usecase.

[ameetr]

Just now, Mick Naughty said:

Two is better than one.  For everything. 
 

This is a true thing, but I guess I'm wondering if two 2080 supers > one quadro, because quadro > 2080 super.

[Entropy.]

13 minutes ago, Mick Naughty said:

Two is better than one.  For everything. 
 

bans from two Minecraft servers you like is better than a ban form one?!?!?!

[Entropy.]

13 minutes ago, Mick Naughty said:

Two is better than one.  For everything. 
 

bans from two Minecraft servers you like is better than a ban form one?!?!?!

[Darkseth]

25 minutes ago, Mick Naughty said:

Two is better than one.  For everything. 
 

Unless the 1 is much better optimized for a specific Task. And that depends on the Drivers and Applications.

1x 100 is still more than 2x 20.

Quality matters too, not just quantity.

 

 

Besides: For ANYTHING that's not Gaming,  SLI does not exist.

SLI is Gaming-ONLY, and that is useless and a waste of Money. Games stopped supporting Sli years ago.

Having 2 Cards for something like CAD, Video rendering etc is NOT SLI. It's just having multiple GPUs that all do their stuff.

 

That's only technical definition-nitpicking tho

[igormp]

38 minutes ago, ameetr said:

I'm trying to configure a PC for deep learning and it looks like the for the price I can get one quadro RTX, I can get two 2080 supers/maybe TIs. Which of these would be better? They seem to relatively match in terms of vram, but not sure what the difference is like with specialized deep learning cores, etc. 

Do you have any idea on what kind of models are you going to work on? If they can fit in only 8gb of vram, or if you can easily split your model across 2 GPUs, then go for the 2 cards, otherwise the quadro is your only option.

 

I guess your best bet would be 2x 2080Tis, since you would have better raw performance from card, more total VRAM if your models can be split, and a way to manage parallel tasks if you want to.

 

17 minutes ago, Darkseth said:

Besides: For ANYTHING that's not Gaming,  SLI does not exist.

SLI is Gaming-ONLY, and that is useless and a waste of Money. Games stopped supporting Sli years ago.

Having 2 Cards for something like CAD, Video rendering etc is NOT SLI. It's just having multiple GPUs that all do their stuff.

 

That's only technical definition-nitpicking tho

The 2080/Super/Ti all have nvlink capabilities, which nets an extra 5~10% performance on some ML tasks.

[ameetr]

1 minute ago, igormp said:

Do you have any idea on what kind of models are you going to work on? If they can fit in only 8gb of vram, or if you can easily split your model across 2 GPUs, then go for the 2 cards, otherwise the quadro is your only option.

 

I guess your best bet would be 2x 2080Tis, since you would have better raw performance from card, more total VRAM if your models can be split, and a way to manage parallel tasks if you want to.

 

The 2080/Super/Ti all have nvlink capabilities, which nets an extra 5~10% performance on some ML tasks.

yeah that's my biggest issue. AFAIK the best way to work on multiple gpus is to literally copy the model on each gpu, then train w/ separate data sets and merge later which is what tensorflow does. That said, my current models are only resnet variants but with video data. That said, if I ever want to go higher I might be out of luck. I'll see if I can get approval for the dual 2080 TIs, which might end up being the best price. 

[ameetr]

1 minute ago, ameetr said:

yeah that's my biggest issue. AFAIK the best way to work on multiple gpus is to literally copy the model on each gpu, then train w/ separate data sets and merge later which is what tensorflow does. That said, my current models are only resnet variants but with video data. That said, if I ever want to go higher I might be out of luck. I'll see if I can get approval for the dual 2080 TIs, which might end up being the best price. 

Well... the TI is much more expensive than the super so maybe it's not the best actually

[ameetr]

6 minutes ago, igormp said:

Do you have any idea on what kind of models are you going to work on? If they can fit in only 8gb of vram, or if you can easily split your model across 2 GPUs, then go for the 2 cards, otherwise the quadro is your only option.

 

I guess your best bet would be 2x 2080Tis, since you would have better raw performance from card, more total VRAM if your models can be split, and a way to manage parallel tasks if you want to.

 

The 2080/Super/Ti all have nvlink capabilities, which nets an extra 5~10% performance on some ML tasks.

but it does look like i can split weight matrices over GPUs/ giving me 16 gb usable space on two 2080 supers https://timdettmers.com/2014/11/09/model-parallelism-deep-learning/

[igormp]

14 minutes ago, ameetr said:

but it does look like i can split weight matrices over GPUs/ giving me 16 gb usable space on two 2080 supers https://timdettmers.com/2014/11/09/model-parallelism-deep-learning/

But doing so you'd have all of the overhead of synchronizing the data between GPUs, and not all models are that easy to split across GPUs.

 

19 minutes ago, ameetr said:

Well... the TI is much more expensive than the super so maybe it's not the best actually

Again, what are your needs in terms of VRAM size? Something like a resnet50 can fit in 8gb of vram, even if barely. If vram is important to you, then you may want to go for the quadro anyway. Otherwise, I'd go for a single 2080Ti since it provides the best performance with a reasonable amount of vram.

 

Another option would be to just use GPUs in the cloud. You can easily get a V100 on GCP.

[ameetr]

1 hour ago, igormp said:

But doing so you'd have all of the overhead of synchronizing the data between GPUs, and not all models are that easy to split across GPUs.

 

Again, what are your needs in terms of VRAM size? Something like a resnet50 can fit in 8gb of vram, even if barely. If vram is important to you, then you may want to go for the quadro anyway. Otherwise, I'd go for a single 2080Ti since it provides the best performance with a reasonable amount of vram.

 

Another option would be to just use GPUs in the cloud. You can easily get a V100 on GCP.

yeah the problem with cloud is that we have a lot of video footage, so the upload time alone would be unacceptable (unless we have a system in place, but we would still need to copy videos back and forth). The first time I did this was on colab and only three videos which took *forever*. They are all resnet variants, but I want the flexibility to move to something bigger if I want... that said, I'm wondering if the extra effort of splitting a model over GPUs (assign layers to GPUs and make data available to all) -- would take longer to train, but might be worth the money? Also, I'm expected a lot of training (think research), so GCP would be a lot of $$ -- probably much more than just buying the two 20180TIs.

[igormp]

13 minutes ago, ameetr said:

yeah the problem with cloud is that we have a lot of video footage, so the upload time alone would be unacceptable (unless we have a system in place, but we would still need to copy videos back and forth). The first time I did this was on colab and only three videos which took *forever*. They are all resnet variants, but I want the flexibility to move to something bigger if I want... that said, I'm wondering if the extra effort of splitting a model over GPUs (assign layers to GPUs and make data available to all) -- would take longer to train, but might be worth the money? Also, I'm expected a lot of training (think research), so GCP would be a lot of $$ -- probably much more than just buying the two 20180TIs.

You'll spend more time tuning your model rather than training, that's for sure. If you're really sure that you need the vram for larger models with larger batch sizes, I'd just jump the gun with a single quadro card, which would save both time tuning stuff and allow for bigger batch sizes for faster training. If you're sure that your models won't go past 11gb, or if you're ok with smaller batch sizes, then a 2080Ti would be your best bet.

 

If you really want to go multi-gpu, why not multiple 2060Super or even 2070Supers? 3 or 4 of those would net you way more VRAM and raw compute power anyway. Also remember to get blower cards, since dual or triple fan GPUs don't go well together in this scenario.

[igormp]

3 hours ago, Brok3n But who cares? said:

You mean titan rtx? the titan and quadros are configured better for deeper learning while the 2080S/ti's are less so

That's not true, the underlying chip in all of those is the same.

[Mick Naughty]

3 hours ago, Darkseth said:

Unless the 1 is much better optimized for a specific Task. And that depends on the Drivers and Applications.

1x 100 is still more than 2x 20.

Quality matters too, not just quantity.

 

 

Besides: For ANYTHING that's not Gaming,  SLI does not exist.

SLI is Gaming-ONLY, and that is useless and a waste of Money. Games stopped supporting Sli years ago.

Having 2 Cards for something like CAD, Video rendering etc is NOT SLI. It's just having multiple GPUs that all do their stuff.

 

That's only technical definition-nitpicking tho

 Sli Does exist. Just because there are more game that dont support it doesn’t mean it isn’t there. 
 

Two is better. If there’s a lack of effort it isn’t the cards fault. 
 

3 hours ago, Brok3n But who cares? said:

bans from two Minecraft servers you like is better than a ban form one?!?!?!

Yes. Then you can move on to a real game. 

[igormp]

4 minutes ago, Mick Naughty said:

 Sli Does exist. Just because there are more game that dont support it doesn’t mean it isn’t there. 

He said "For ANYTHING that's not Gaming,  SLI does not exist.", which is true. SLI is a gaming thing, not for compute. Keep in mind that we're not talking gaming in this thread.

[Mick Naughty]

11 minutes ago, igormp said:

He said "For ANYTHING that's not Gaming,  SLI does not exist.", which is true. SLI is a gaming thing, not for compute. Keep in mind that we're not talking gaming in this thread.

Yea I’m well aware. Which is why I didn’t bring it up. Other people did. So again, 2 is better. 

[Darkseth]

Only if we're talking about the exact same Product.

But we're not.

 

One Quadro can destroy 2 RTXs, depending on which software and drivers are beeing used, and how the optimization is because of that.

[igormp]

35 minutes ago, Mick Naughty said:

So again, 2 is better. 

It really depends, and that's the whole point of the topic.

 

3 minutes ago, Darkseth said:

One Quadro can destroy 2 RTXs, depending on which software and drivers are beeing used, and how the optimization is because of that.

True, but not in this scenario. Quadros, teslas and their Geforce counterparts perform the same on ML tasks, the only difference being the available vram

 

@OP, have you seen this benchmark?