Going to build a Machine Learning Rig

[Moon Man]

Hey guys,

I am going to build a machine learning focused build, for my studies in university.

Maybe going to have a budget of 100,000 PHP (1300 - 1800 USD).

Right now this is the build  I have planned: https://pcpartpicker.com/list/h8BccY

CPU: Ryzen 7 1800x

CPU Cooler: Noctua NH-D15

Mobo: Asus Crosshairs Hero (Wifi-AC)

RAM: G-Skill Ripjaws (2 x 16) 32 GB

Storage (boot): 500 GB Samsung 960 Evo

Storage (Mass): 2, 2TB WD Black in RAID1

GPU: Titan XP

PSU: Corsair AX760

Case: Phanteks Enthoo Pro M

 

Please note that:

I will primarily use Tensorflow and Scikit-Learn Libraries.

I am relatively new to PC building, and machine learning.

The Titan XP will be lent to me by the university not bought.

 

So, I would gladly accept any advice to make my rig better.

Also, please say if there are any incompatibilities with my hardware choices, so I can make the appropriate modifications.

Any recommendations will be greatly appreciated.

Again, I am relatively new to pc building and hardware, so please be nice.

 

Thank you for your response.

 

 

[owencrispy]

could machine learning benefit from a higher core count cpu, rather than the 1800x?

[Moon Man]

As far, as I know, core count does not really matter much, as shown in Linus' video about the Xeon Phi Machine.

Although, I picked Ryzen in general for their extra PCI-E Lanes,

so for me, Threadripper would be excellent, but right now, don't have the budget for it.

For the reason in picking the 1800x specifically, it was just recommended to me, but I think the R7 1700, or even R5 would do just fine.

[owencrispy]

9 minutes ago, Moon Man said:

As far, as I know, core count does not really matter much, as shown in Linus' video about the Xeon Phi Machine.

Although, I picked Ryzen in general for their extra PCI-E Lanes,

so for me, Threadripper would be excellent, but right now, don't have the budget for it.

For the reason in picking the 1800x specifically, it was just recommended to me, but I think the R7 1700, or even R5 would do just fine.

would it be worth you waiting and saving for a threadripper cpu, rather than buying a ryzen and finding out how much better the TR wouldve been so you spend extra money to upgrade?

[brob]

8 hours ago, Moon Man said:

As far, as I know, core count does not really matter much, as shown in Linus' video about the Xeon Phi Machine.

Although, I picked Ryzen in general for their extra PCI-E Lanes,

so for me, Threadripper would be excellent, but right now, don't have the budget for it.

For the reason in picking the 1800x specifically, it was just recommended to me, but I think the R7 1700, or even R5 would do just fine.

Ryzen 7 offers 16 PCIe lanes for gpu. The remaining 8 are reserved for other uses, 4 to communicate with the chipset and 4 for an NVMe M.2 connection. Intel Coffee Lake cpu have 16 PCIe lanes on chip that can be used for gpu. In addition the chipset has an additional 24 PCIe 3.0 lanes.

[Moon Man]

@owencrispy

Well, my budget is supplied to me by my university, so the budget is not really in my control.

But, in my opinion, you are correct, buying a threadripper, would be excellent for the number of PCI-E lanes alone.

But this time, unless someone could give me one, it really is not an option.

[Moon Man]

@brob

Well, correct me if I am wrong, but If I remember correctly, you are correct Z370 does support a maximum of 24 PCI-E lanes, but it would only be implemented if the CPU that is put on it supports 24 PCI-E lanes. In the case of the 8700k, which only supports 16 PCI-E Lanes, the whole system would only have 16 PCI-E lanes. Compared to a ryzen system, which supports 24 PCI-E lanes, 16 for the GPU, 4 for NVME,  and 4 for other stuff it would still be better since I do plan to have a 960 EVO.

 

That is how I remember it though, so please correct me if I am wrong.

[brob]

11 minutes ago, Moon Man said:

@brob

Well, correct me if I am wrong, but If I remember correctly, you are correct Z370 does support a maximum of 24 PCI-E lanes, but it would only be implemented if the CPU that is put on it supports 24 PCI-E lanes. In the case of the 8700k, which only supports 16 PCI-E Lanes, the whole system would only have 16 PCI-E lanes. Compared to a ryzen system, which supports 24 PCI-E lanes, 16 for the GPU, 4 for NVME,  and 4 for other stuff it would still be better since I do plan to have a 960 EVO.

 

That is how I remember it though, so please correct me if I am wrong.

Not what I said. The i7-8700K cpu has 16 PCIe 3.0 lanes. It is the chipset that has 24 PCIe 3.0 lanes for things like NVME drives. On Z370 motherboards there are usually 2 M.2 connectors that run through the chipset. 

 

Ryzen 7 cpu have 24 PCIe lanes. However, only 16 are available for general use. The other eight are dedicated, 4 for communications with the chipset, (Intel uses a different bus), and 4 for an NVMe drive.

 

While the architectures differ, there is little practical difference in gpu support between Ryzen 7 and Intel Coffee Lake.

[Moon Man]

Ohh, then

if that is the case, even though the Chipset has 24 PCI-E lanes, it is usually used for other components. such a USB, SATA, etc.

so, in the case of the 8700k, all 16 of those lanes would be allocated to the GPU. and the 4 PCI-E lanes for my NVME SSD, would go through the PCH which mean less performance overall.

So, if Ryzen has 24 PCI-E lanes, you have 16 allocated for the GPU, which means that the CPU would still have extra lanes such that my SSD would have a full 4 PCI-E lanes, going through the CPU having better performance, and an additional 4 for other expansion cards.

Again, please correct me if my logic is wrong. Thanks

[brob]

Coffee Lake uses DMI 3.0 for communications between chipset and cpu. Speed of this bus is roughly equivalent to 4 PCIe 3.0 lanes.

[Moon Man]

But, having the PCI-E lanes directly connected to the chip, would still be better compared to the lanes connectes to the chipset, right?

[brob]

3 hours ago, Moon Man said:

But, having the PCI-E lanes directly connected to the chip, would still be better compared to the lanes connectes to the chipset, right?

I'm not aware of any benchmark data that proves or disproves the statement. I would not presume that a particular connection architecture guaranteed better performance.

[llama95]

Hi, I am also a beginner in machine learning and I did some research on the associated hardware.

First up the CPU, I recommend getting a ThreadRipper CPU, a 1900X will be enough as TensorFlow (as you mentioned) mainly use CUDA cores for computation. You could use CPU for this but the difference will be night and day compare to GPU. The features we want from a TR system is the PCIE lanes, and more importantly the quad channel memory. Often I read other researcher mentioned they have their learning speed bottle-necked at the RAM speed, if your RAM is not fast enough to keep up with your GPU, more PCIE bandwidth will not help either.

For the extra PCIE lanes, it will be useful if you planning to have a multi acceleration card configuration in the future. But from what I read configuring multi card system could potentially be very tedious on the software side. So keep in mind on the future upgradability and scalability.

For the RAM size, the rule of thumb is at least double the size of you VRAM, so the 32 GB will be more than enough in your case of Titan Xp.

For the motherboard, I recommend buying one with 10gbps nic port. However, this depends on your future plan. Many lab with super large data set, they like to setup an external JBOD (Just a Bunch of Disk) for storing their data, with a certain degree of striping and redundancy. In this case the bottleneck will happened on your nic port. Or if you would like to store all your data within your computer case, I would recommend a SAS controller for expanding storage.

All and all, everything I mentioned is highly depending on the data set you are working on. If you are not planning on future upgrade, a simple system with enough PCIE lanes will do. If you experience bottle-necking in the future, check your code/algorithm/data set, as there are always more ways to optimize on software level before upgrading your hardware. Bottle-necking may not always happened on your hardware.

[brob]

@llama95

 

If there is no plan for multiple gpu, is there any reason for all the extra PCIe lanes of TR? Would not Ryzen 7 or Intel i7 do just as well?

 

Do datasets get into the TB range?

[Moon Man]

@brob

Well, the lanes for the chipset usually handles, other things, such as SATA, etc.

So from my understanding, if the NVMe drive is directly connected to the chip, well the chipset can use its lanes for other for its intended purpose.

So, yeah, there may not be any speed benefit, for the drive but there might be other things at play.

 

The NVMe drive aside, wouldn't it still be better if to have more PCI-E lanes in general for other expansion cards?

[Moon Man]

@llama95

Is the memory speed really that important?

From what I know, the only thing necessary is the GPU.

But, you are correct getting a system with plenty of PCI-E lanes would be excellent, but the university budget, I think would not allow that.

As for JBOD, I think our current infrastructure, would prevent us from doing that, so I will just stick with my Raid 1 WD black.

Thanks for the input though!

And more information will be appreciated.

 

PS:

Also, would the benefits of going HEDT, outway the cost, just for the PCI-E lanes? or would more cores help?

[brob]

@Moon Man

 

PCIe lane allocation is done by motherboard designers, i.e. are hard. If one has definite thoughts about upgrades then making sure the motherboard can handle them makes sense. But spending budget because more is better is generally unwise.

 

Benchmarks for Ryzen have established that faster memory improves cpu performance.

 

More generally, memory intensive workloads tend to benefit from higher speed memory. On Intel the benefit of higher speed memory is muted, but still present.