Boot Disk Versus Storage

[jtk700cln]

Hey folks,

I'm trying to understand the relation between your boot disks speed and the drive you use for storage, and setup a workstation as optimized as possible to work with 8-14tbs worth of stoage

I'm a vfx artist, building a new computer, and the build will be similar to the one Linus built recently for his vfx editors--

https://www.youtube.com/watch?v=EXBGRxmLqOU

So 

Asus Zenith Motherboard,

Threadripper 3970 or 3990,

optane for the boot disk,

 

and then.....STORAGE?

 

This is where Im a bit lost.  

 

I would like a storage solution with up to 14 tbs of storage (8 tb will be okay for now).  I know this is a lot, far more than a high speed onboard ssd can handle.  

However, I do NOT want to sacrifice speed for the prize of massive storage, so if that means if i need to create a two part storage solution, so something fast and reasonably sized (nvme mirrored raid 2 tb), and then a slower 8 tb storage solution i back stuff up to on a nightly basis or something, I am all ears.  

 

Again, any suggestions will be helpful.  

 

I considered for a bit trying to do a nas, and forego local storage altogether, but I am worried that there is too much dependency there (internet connection speed, personal ignorance lol).  I know that they are selling nas devices that claim they support 10 GB/s connection speed, but how realistic is that really?  

 

Lets say I go the other route, something a bit cray lol, like this:

https://www.youtube.com/watch?v=nkrHGnSGrk4

https://www.youtube.com/watch?v=5MhOA2IHC9E

 

i could use this for ludicrously fast live read and writes, and then back it up to a slower 8 tb external at night if needed.  Only downside is management isnt as easy, and I dont really have access to my work from other workstations.  This isnt priority, because its just me at the moment, but having access to my files at multiple locations is something that would be nice to have in the future.  If I can get reasonably high speed from a nas (im talking something really impressive), and forego onboard storage, im game to buckle down and learn a thing or two to get it working.   

 

 

So theres a lot of speculation here but I had two main questions, namely

1) the relation between storage and your boot disk, what is it, and how do you not bottleneck yourself/

2) how can i build a 8-14 tb redundant storage solution without losing a lot of speed.

     As I said, I am open to any and all suggestions, including NAS, maybe with DAS hookup (so I can just directly plug in with a usb 3.2 gen 2 if needed), or even two part storage solutions.  

 

Any and all information would be hugely helpful.  

I am willing to spend quite a bit for a forward looking solution.  

 

 

Thanks!!!!!

 

 

 

 

 

[Electronics Wizardy]

The speed of the what your doing disks io(I think that make sense) depends on the drive its running off of, the other drives in the system won't matter.

 

So if you editing video on your system with a hdd and a ssd, and the project is on the ssd, it will load much faster than if it was on the hdd. The other drive won't matter in thise case.

 

Probably something like a 500gb-1tb boot  ssd drive, a 2tb ssd for current projects, and a big hdd or two for less used projects.

 

 

[jtk700cln]

gotcha.  when people work off a network then are they always taking a performance hit?

[Kisai]

12 minutes ago, jtk700cln said:

Hey folks,

I'm trying to understand the relation between your boot disks speed and the drive you use for storage, and setup a workstation as optimized as possible to work with 8-14tbs worth of stoage

I'm a vfx artist, building a new computer, and the build will be similar to the one Linus built recently for his vfx editors--

https://www.youtube.com/watch?v=EXBGRxmLqOU

So 

Asus Zenith Motherboard,

Threadripper 3970 or 3990,

optane for the boot disk,

 

and then.....STORAGE?

 

 

 

You can sometimes have up to 4 M2 SSD's in a system. You are not required to have all drives be SSD's, you can use mechanical SATA HDD's for storage which some systems can accept up to 8 of. You can also just put mechanical drives on USB 3.0 ports.

 

Your boot disk doesn't need to be the fastest drive, only the drive that any pagefile/scratch files are on. If you are capturing video in real time, you need a fast drive. If you are scrubbing video a lot, you need a fast drive. For all other considerations (eg transcoding video) the GPU and CPU make more of a difference. 

 

Like for video work, I'd suggest, if you can afford it, to get fewer larger drives than several smaller drives as the larger SSD's are also the fastest models. So if you need 8TB of storage, you should consider 2 x 4TB rather than 4 x 2TB. If you however are working with SATA drives, they cap at 530MB/sec, where as PCIe NVMe SSD's cap at 4GB/sec (4 lane PCIe3). Mechanical drives more or less cap at around 120MB/sec. So 1 PCIe 4 lane SSD = 8 SATA SSD's = 32 Mechanical drives, in terms of speed. Roughly. In practice most PCIe 3.0 4 lane drives are somewhere between 2GB and 3GB/sec.

 

Optane is intended to be faster than NVMe drives, but most of what has been made with the technology has been NVMe drives, not RAM options. 32GB Optane modules are basically intended to be read/write cache. Not boot drives. We're actually at a point where Optane's use might only be to bring under-performing systems based on mechanical drives up to the same performance as machines based on SATA SSD's. It won't really help with systems that already use SSD's.

 

Intel claims Optane has 1000x the durability of flash memory, but we don't know if that's really the case, as the drives are too darn small and puts them into the same capacity and use case as SLC flash memory.

 

So if anything, if you are working, entirely with video, the far more expensive flash drives may not even be justified if you can fit an entire project onto one drive, and then use cheap mechanical drive storage for archiving it.