Speeding up data transfer/downloads to hard drives faster?

[natsuu]

Hi Guys, I am an aspiring content creator just wondering if there was any way to speed up downloading/transferring video from an SD/CF card y 2x12TB Western digital Gold HDD in raid 1. I was looking at Intel optane but to my understanding and not that well of it but those only work for files that are open frequently? I have an I7 6850k GTX 1080TI and 64GB of ram and an Intel PCIE SSD 400gb drive for startup. specifically the intel optane ssd 900p for video editing

Some of my work if you want to check it out:

 

Thank you,

Natsuu

[Electronics Wizardy]

How fast are the files copying? Your probably limited by the card or the card reader not the hdd. That array should get over 200mB/s writes.

 

Optane and most other caches won't help as the cache will just fill up and then back to the hdd speeds.

 

 

900p isn't really a great editing drive. Its made for low queue depth random reads, and you don't do that much in editing.

 

What type of footage are you working with. Id probably get a drive like this. https://www.amazon.com/Micron-MTFDDAK2T0TBN-1AR1ZABYY-Solid-State-Drive/dp/B01LB05TOO/ref=sr_1_1?ie=UTF8&qid=1526352051&sr=8-1&keywords=micron+1100

 

If you want more speed, just get a few of the above in raid 0(backups of course, but ssds don't fail that often anyways)

[natsuu]

20 hours ago, Electronics Wizardy said:

How fast are the files copying? Your probably limited by the card or the card reader not the hdd. That array should get over 200mB/s writes.

 

Optane and most other caches won't help as the cache will just fill up and then back to the hdd speeds.

 

 

900p isn't really a great editing drive. Its made for low queue depth random reads, and you don't do that much in editing.

 

What type of footage are you working with. Id probably get a drive like this. https://www.amazon.com/Micron-MTFDDAK2T0TBN-1AR1ZABYY-Solid-State-Drive/dp/B01LB05TOO/ref=sr_1_1?ie=UTF8&qid=1526352051&sr=8-1&keywords=micron+1100

 

If you want more speed, just get a few of the above in raid 0(backups of course, but ssds don't fail that often anyways)

It writing at about 60mB/s, any particular reason for the Micron SSD or is Samsung and Western digital ssds good too?

[Electronics Wizardy]

40 minutes ago, natsuu said:

It writing at about 60mB/s, any particular reason for the Micron SSD or is Samsung and Western digital ssds good too?

Then thats not your hdds thats your problem, its the card or the reader. Don't expect these speeds to be any faster with an ssd.

 

The micro ssd is a good deal. Also micron is huge, they make the flash for lots of other companys like intel. Samgsung and wd drives are also fine.

[natsuu]

1 minute ago, Electronics Wizardy said:

Then thats not your hdds thats your problem, its the card or the reader. Don't expect these speeds to be any faster with an ssd.

 

The micro ssd is a good deal. Also micron is huge, they make the flash for lots of other companys like intel. Samgsung and wd drives are also fine.

my sd is rated for 95MB/s and my card reader is rated up to 500 MB/s second transfer speed so I'm not sure what wrong.

SD card: https://www.amazon.com/dp/B01J5RH5Y2/ref=twister_B01M3P8TUQ?_encoding=UTF8&psc=1

 

card reader: https://www.amazon.com/SanDisk-ImageMate-Reader-Writer-SDDR-289-X20/dp/B005GP7WKU/ref=sr_1_4?ie=UTF8&qid=1526429921&sr=8-4&keywords=sandisk+card+reader&dpID=31zpdXXHVJL&preST=_SX300_QL70_&dpSrc=srch

 

[Electronics Wizardy]

Just now, natsuu said:

my sd is rated for 95MB/s and my card reader is rated up to 500 MB/s second transfer speed so I'm not sure what wrong.

SD card: https://www.amazon.com/dp/B01J5RH5Y2/ref=twister_B01M3P8TUQ?_encoding=UTF8&psc=1

 

card reader: https://www.amazon.com/SanDisk-ImageMate-Reader-Writer-SDDR-289-X20/dp/B005GP7WKU/ref=sr_1_4?ie=UTF8&qid=1526429921&sr=8-4&keywords=sandisk+card+reader&dpID=31zpdXXHVJL&preST=_SX300_QL70_&dpSrc=srch

 

The sd card might not be up to spec. Do you have other cards or readers to use?

 

Also the sd cards often can only hit the speeds on some specfic cases, like a big queue depth, or in bursts.

 

Why does this matter? The copy from the sd card is normally a short part of the whole process?

[natsuu]

I mean I shoot a lot of photos and video in raw and especially with a 50-megapixel camera- Canon 5DSR the files sizes are huge. So I am trying to reduce off time as much as I can. Also, unfortunately, I do not have any other card readers.

[Electronics Wizardy]

1 minute ago, natsuu said:

I mean I shoot a lot of photos and video in raw and especially with a 50-megapixel camera- Canon 5DSR the files sizes are huge. So I am trying to reduce off time as much as I can. Also, unfortunately, I do not have any other card readers.

If you want a faster card get a CF card, they can be a good amount faster(up to about 250mB/s read.)

[natsuu]

1 minute ago, Electronics Wizardy said:

If you want a faster card get a CF card, they can be a good amount faster(up to about 250mB/s read.)

I do have a 256MB/S 32GB CF card but it still does the same thing any idea why it is doing that?

[Electronics Wizardy]

Just now, natsuu said:

I do have a 256MB/S 32GB CF card but it still does the same thing any idea why it is doing that?

Then it feels like the card reader issue. Try a different one if you can.

[natsuu]

Just now, Electronics Wizardy said:

Then it feels like the card reader issue. Try a different one if you can.

Alright, thanks