Interesting Camera/Computer transfer question

[TFields]

Hey, So I have an interesting question about transferring photos to a computer as fast a possible. I have a rig of about 100 Dslr cameras and I need to transfer photos as fast as possible to a set of computers. I'm currently transferring the photos between 4 computers. The cameras go into a Usb 2.0 hub then those go to specific computers which then go across a network to a main computer. If I'm shooting Raw it takes about a min to transfer all the photos. Is there a faster method to transfer all these images? 

[scottyseng]

3 minutes ago, TFields said:

-snip-

Is USB 3.0 a option? It would boost your speed a lot if you can use a USB 3.0 Hub and a USB 3.0 card reader.

 

This is assuming your cameras have reasonably fast cards.

 

Or did you mean the speed between transferring PC to PC? That's kind of limited by 1Gb/s Ethernet. You can upgrade to 10Gb/s Ethernet, but that may cost a bit depending on your Ethernet cable system currently.

[TFields]

8 minutes ago, scottyseng said:

Is USB 3.0 a option? It would boost your speed a lot if you can use a USB 3.0 Hub and a USB 3.0 card reader.

 

This is assuming your cameras have reasonably fast cards.

 

Or did you mean the speed between transferring PC to PC? That's kind of limited by 1Gb/s Ethernet. You can upgrade to 10Gb/s Ethernet, but that may cost a bit depending on your Ethernet cable system currently.

So I "think" part of the issue is going straight out of the camera's. The camera's only have a USB 2.0 output, I'm not sure if there's another method of transfer other then that from the camera? the cameras are Rebel SL1 if that helps   

[Jake0210]

TFields, if the cameras use an SD card you can try to find a SD to USB 3.0 adapter.

[TFields]

Interesting Jake0210, Thanks I look into that!

[scottyseng]

3 minutes ago, TFields said:

Interesting Jake0210, Thanks I look into that!

Yeah, I was going to recommend the same thing with the USB 3.0 card readers.

 

Actually surprised you weren't using card readers already. Make sure to get a USB 3.0 hub as well though.

[TFields]

I think the main reason for not was I didnt realize they made such a thing and money, lots of cameras cost a bit, now that money is freed up I can focus on optimizing transfer rates, thanks for the help!

[xQubeZx]

28 minutes ago, TFields said:

I think the main reason for not was I didnt realize they made such a thing and money, lots of cameras cost a bit, now that money is freed up I can focus on optimizing transfer rates, thanks for the help!

May I out of curiosity ask what you need 100 DSLR's for? But as others said USB 3 card readers and fast SD cards is the way to speed up transfers.

[TFields]

Just now, xQubeZx said:

May I out of curiosity ask what you need 100 DSLR's for? But as others said USB 3 card readers and fast SD cards is the way to speed up transfers.

Working with photogrammetry, taking images and converting them to 3D images. A fast workflow is needed because we shoot several images in the day, I'm just interested in figuring out how to transfer directly to computers as fast as possible. Optimizing the workflow is super important. 

[TFields]

Another Question I have is whether software would make a difference when it comes to transferring, currently I use smart shooter, would using something like lightroom for tethered cameras make a difference? I feel like this wouldn't but I'm interested if this has any effect?

[.spider.]

1 hour ago, Jake0210 said:

TFields, if the cameras use an SD card you can try to find a SD to USB 3.0 adapter.

But it would take ages to remove the memory cards from 100 cameras.

[Jake0210]

1 hour ago, .spider. said:

But it would take ages to remove the memory cards from 100 cameras.

Yeah I thought about that. But the transfer speeds would be quicker. Pick your poison, right?

[cc143]

Could you theoretically just connect 25 cameras to each computer and shoot tethered? or is the computer usually not in the same location as the shoot?

 

Have you looked at the individual speeds of transfer from each camera? it is possible you are bottlenecked by your usb2.0 hubs or ports as well, in which case using a faster adapter on faster ports would theoretically allow for the overall transfer speed to be higher.

 

Usb 2.0 handles up to 60mbps (according to google), if the overall bandwidth of a single usb adapter or port is 60mbps that is divided between all cameras you import pictures from. If therefore you used a usb 3.0 port or hub, the overall speed may increase even if the camera's usb port is of lower bandwidth. 

 

That only applies if all files are transferred simultaneously by your OS or whatever software you use to transfer though.

[AkiraDaarkst]

I know what the OP is trying to do, and with the camera models he has, there are only two options:

• Extract memory cards and off load them using a card reader, takes time to extra memory cards off 100 cameras.
• USB tethering and offload, speeds depends on whether he is shooting JPEGs or RAW.  Basically speeds depends on file size and number of files.

The question to the OP, do you need to shoot in RAW for the type of stuff you do?  Wouldn't JPEGs (perhaps the highest quality JPEGs the camera offers) be enough?

 

Unless the OP can create an SD card adapter that can go into the memory card slot of the camera and directly stores data onto an external storage system, he will be stuck with the above mentioned limitations.  I'm talking about something similar in concept to the CFast Card to SSD adapter that Black Magic Ursa camera users sometimes use.

[mama_fluxus]

This seems like an ideal situation for those wifi SD cards in my eyes. One computer can connect to multiple cameras directly with the files instead of using canon camera connect or something.

[AkiraDaarkst]

24 minutes ago, mama_fluxus said:

This seems like an ideal situation for those wifi SD cards in my eyes. One computer can connect to multiple cameras directly with the files instead of using canon camera connect or something.

Like the EyeFi cards?  They can be just as slow if not slower.

 

What the OP needs is a custom engineered solution and not really off the shelf stuff.

[TFields]

I think I'll have to look into a custom solution, we need it to be fast with transferring RAW photos, its a very VFX heavy pipe and requires the most information from the camera's for delighting textures and getting the correct color correction. I'll look into something that goes straight from the ssd to usb 3.0 the only issue is the camera's have the ssd right beside the battery which is a headache :(. 

[AkiraDaarkst]

5 minutes ago, TFields said:

I think I'll have to look into a custom solution, we need it to be fast with transferring RAW photos, its a very VFX heavy pipe and requires the most information from the camera's for delighting textures and getting the correct color correction. I'll look into something that goes straight from the ssd to usb 3.0 the only issue is the camera's have the ssd right beside the battery which is a headache :(. 

Pro bodies like the Nikon D4/D5 and Canon 1D series have ethernet ports that should offer faster offloading of files.  However they cost more and buying 100 bodies would cost you about 10x the price of 100 Canon SL1 type of cameras.

 

Ask some engineers if they can design a dummy SD card that can go into the memory card slot of the camera, but with a cable connected that can offload data to an external storage system or computer.  Basically the dummy SD card will have electronic contact points that will make contact with the contact points inside the memory card slot to be able to read the data, but instead of having data storage cells inside the card it just transfers the data to something else.

 

Basically something like this, but for SD cards instead of CFast.  The one below is CFast to SATA I think.

 

Even faster offloading would be possible if you have engineers who can "mod" the camera so that the camera can offload data from the internal buffer to an external storage system bypassing the memory card route altogether.  But this would require serious modding that may not be possible or the design of custom circuit boards to replace some internal components of the camera.

 

But why do you need RAW, can't you work with high quality JPEG?

[TFields]

That's good to know, I figured I'd have to do a custom solution, and RAW is the way, Jpeg is too compressed, we made textures with both and Raw gives us the flexibility and quality we need.