Hardware for Neural Network Training

[AndrewZ]

Hello all, me and my friend are currently in college and are getting into computer science and more specifically deep learning. We have been programming several neural networks but the problem is that they take forever to train on my crappy old laptop. I'm looking in to building a new computer specifically for training these networks. Since the graphics card is what does all the heavy work, I'll probably go with a 1080. My question is, what CPU and RAM setup would I need to go along with this build? Thanks

[reniat]

This seems relevant and useful: http://timdettmers.com/2018/12/16/deep-learning-hardware-guide/

(it was also the 2nd google result )

 

tl;dr: with a gtx 1080 I would probably go with 16GB of standard ram (don't worry about RAM speed), and something cheaper but with a good thread count like an AMD 2600.

[kendoka]

2 hours ago, reniat said:

This seems relevant and useful: http://timdettmers.com/2018/12/16/deep-learning-hardware-guide/

(it was also the 2nd google result )

 

tl;dr: with a gtx 1080 I would probably go with 16GB of standard ram (don't worry about RAM speed), and something cheaper but with a good thread count like an AMD 2600.

Also might be Intel Xeon E5-2670, which has 2 more cores (4 more threads), but clock speed is lower (2.6 GHz)

[straight_stewie]

9 hours ago, AndrewZ said:

Since the graphics card is what does all the heavy work, I'll probably go with a 1080. My question is, what CPU and RAM setup would I need to go along with this build? Thanks

16GB of ram will set you off for a good start, but be sure to choose a motherboard that will allow you heft upgrade options. If you stay with it, you may very well end up working with extremely large data sets during your tenure at university, and having the ability to increase the amount of RAM you can use can keep your machine up to speed.

As far as the CPU goes, something with a few extra cores/threads would be preferable, but you still want a relatively high clock speed. There is a balancing point there if you are going to be working on both small and large data sets.

[reniat]

4 hours ago, straight_stewie said:

If you stay with it, you may very well end up working with extremely large data sets during your tenure at university, and having the ability to increase the amount of RAM you can use can keep your machine up to speed.

I would think that any data science course that involved legitimately huge data sets would allocate some compute time to the students on the unversity's compute cluster. It's really important for courses not to discriminate between students levels of computers imo, and it would be a pretty big problem to me if a class forced students to buy a machine beefier than what is provided in a campus computer lab.

 

To be fair, I only took one grad course in machine learning, so I don't have the experience of an entire data science/stats degree course to back that up.

[straight_stewie]

6 hours ago, reniat said:

I would think that any data science course that involved legitimately huge data sets would allocate some compute time to the students on the unversity's compute cluster. It's really important for courses not to discriminate between students levels of computers imo, and it would be a pretty big problem to me if a class forced students to buy a machine beefier than what is provided in a campus computer lab. 

Well with paging one could technically handle a data set of any size with only a minimal amount of RAM, assuming infinite hard drive size.

Classes usually won't have you working with abnormally large datasets. However, I get the sneaking suspicion that @AndrewZ and his friend are doing this for fun, and not just for a class. Therefore, it's worthwhile to to allow ones self upgrade options, since one can never tell where a hobby might take you.

Additionally, since you did go to university, I'm sure you're aware that different programs have different requirements for what you will need to complete the coursework. This is normal and acceptable: For example, the CS department at my uni only required a Windows 7 machine of any type, but suggested a machine with an i5 or greater and 16 gb of RAM. While the aerospace engineering department (which was my major), recommended a minimum of an i7 with 16 gb of ram, a 500 gb hard drive or greater, and highly suggested a discrete graphics card (CAD systems can benefit greatly from discrete graphics cards). While the business department (excluding the IT track) simply required the students to have internet access.

[reniat]

6 minutes ago, straight_stewie said:

While the aerospace engineering department (which was my major), recommended a minimum of an i7 with 16 gb of ram, a 500 gb hard drive or greater, and highly suggested a discrete graphics card (CAD systems can benefit greatly from discrete graphics cards)

did your school provide a pc lab? I'm not saying schools can't recommend decent hardware, what i'm saying would be a problem would be "Assignment X is gonna be really compute intense, and you're gonna need a powerful discrete GPU to complete it, but the PC lab's machines are not strong enough so if you don't have that kind of PC already and don't have disposable income, I guess you don't get an A"

There's a difference between "recommended" and "required". I could be wrong, but i'm willing to bet you could have used campus resources to graduate without having an i7 machine. It would have been annoying to not be able to work from your dorm/apartment sure, but squarely possible. I just want to make sure no one without much money feels that if they can't go out and buy a behemoth machine, they can't get certain STEM degrees like data science.

 

Back to the original topic (my bad for getting us off course in the first place)

 

I re-read what you said and realized I mis-read it. I definitely encourage getting a good motherboard that is scalable. I was under the incorrect impression you were telling OP to get all that RAM initially. I just can't reading.

[straight_stewie]

24 minutes ago, reniat said:

Assignment X is gonna be really compute intense, and you're gonna need a powerful discrete GPU to complete it, but the PC lab's machines are not strong enough so if you don't have that kind of PC already and don't have disposable income, I guess you don't get an A"

It is standard operating policy in the United States that students of any age, from K to P.h.D, and their families, are responsible for seeing that they can complete the coursework. If one needs help in that regard, it is their responsibility to find a program that gives them access to such help (many such programs exist for students and their families at virtually every level of education).

From another perspective, university can cost anywhere from $5,000 USD to $50,000 USD, just in tuition, per semester. Considering that the minimum time to a bachelors degree is 8 semesters, it's not unreasonable for a university to expect that if you can afford to go there, you can also afford the equipment necessary to complete the coursework.

[AndrewZ]

9 hours ago, reniat said:

I would think that any data science course that involved legitimately huge data sets would allocate some compute time to the students on the unversity's compute cluster. It's really important for courses not to discriminate between students levels of computers imo, and it would be a pretty big problem to me if a class forced students to buy a machine beefier than what is provided in a campus computer lab.

 

To be fair, I only took one grad course in machine learning, so I don't have the experience of an entire data science/stats degree course to back that up.

Yeah this machine is more for personal projects, but I definitely agree on universities not discriminating based on how good your PC is. Unfortunately, we can't really use the universities hardware for our own personal projects so we need to build our own.

[AndrewZ]

2 hours ago, straight_stewie said:

Well with paging one could technically handle a data set of any size with only a minimal amount of RAM, assuming infinite hard drive size.

Classes usually won't have you working with abnormally large datasets. However, I get the sneaking suspicion that @AndrewZ and his friend are doing this for fun, and not just for a class. Therefore, it's worthwhile to to allow ones self upgrade options, since one can never tell where a hobby might take you.

Additionally, since you did go to university, I'm sure you're aware that different programs have different requirements for what you will need to complete the coursework. This is normal and acceptable: For example, the CS department at my uni only required a Windows 7 machine of any type, but suggested a machine with an i5 or greater and 16 gb of RAM. While the aerospace engineering department (which was my major), recommended a minimum of an i7 with 16 gb of ram, a 500 gb hard drive or greater, and highly suggested a discrete graphics card (CAD systems can benefit greatly from discrete graphics cards). While the business department (excluding the IT track) simply required the students to have internet access.

Yep we'd be using this machine more for our own personal use (not to say that it wouldn't help with our current assignments). 

[AndrewZ]

Thank you everyone for your advice! Hopefully we'll be up and running pretty soon.

[reniat]

2 hours ago, straight_stewie said:

From another perspective, university can cost anywhere from $5,000 USD to $50,000 USD, just in tuition, per semester. Considering that the minimum time to a bachelors degree is 8 semesters, it's not unreasonable for a university to expect that if you can afford to go there, you can also afford the equipment necessary to complete the coursework.

I know in theory everyone should be able to afford an i7 with 16GB of ram, and that university expectations don't NEED to go beyond "hey, this is the reqs take em or leave em", but given that financial assistance programs often only provide a base level of discretionary assistance, i do firmly believe that a university should really provide computer labs with the minimum specs to complete the coursework of a given degree program. 

 

For example, when I went to college I did get financial aid, where you got X dollars per semester. And you received a check for X - tuition as discretionary funds, but that had to cover living expenses, food, books, etc. It's really not always as easy as just "use financial aid to buy kickass computer hardware".

 

I think it's not unreasonable for a university to supply these kinds of utilities, especially given how much tuition costs. It's pretty backasswards to go into heaps of debt and live on scraps for 4+ years to go to a place that said "You don't have an i7? tsk tsk tsk we don't want to pay for a computer lab, so you'd better pick a different course this semester". also, for the record before it's asked, I worked as much as I was allowed to as a student, and then did freelance work on the side, and still struggled. Shits expensive.

 

Though this is DRAMATICALLY off topic. Also I do realize I might be a tad biased, but I do feel pretty strongly about this. At least for large academic institutions with a healthy amount of resources.