Jump to content

USB expantion

Flyguygamer

I have a client that has asked me to look in to the possibility of a server or two that are able to ingest large amounts of data from USB C external SSD's to local storage and then to offsite storage. The user requests 100 USB 3.0 ports for the drives. I've looked a bit, but haven't yet found a good way to do this. Any ideas for 100 USB ports on a single server?

Link to comment
Share on other sites

Link to post
Share on other sites

They expect to have up to 100 people with 100 external SSDs all dumping footage simultaneously? I think you should probably talk to them and explain that that's kind of unrealistic.

 

Most large operations for stuff like this would use a series of desktops users can sit at with a 10Gbit or 20/40Gbit aggregated link to the server.

Link to comment
Share on other sites

Link to post
Share on other sites

2 minutes ago, Windows7ge said:

They expect to have up to 100 people with 100 external SSDs all dumping footage simultaneously? I think you should probably talk to them and explain that that's kind of unrealistic.

 

Most large operations for stuff like this would use a series of desktops users can sit at with a 10Gbit or 20/40Gbit aggregated link to the server.

Not video, it's all data. Just dumping from SSD to local storage so they can use it. One or two people using 1-3 devices and all they do is connect SSD and start the transfer. 

Link to comment
Share on other sites

Link to post
Share on other sites

100 ports and all capable of maximum speed is gonna be near impossible.

 

You're probably gonna be limited to maybe maximum 7  usb controller cards, where each controller card can create 4 or maybe 7 usb 3.0 ports

You may be able to get a "mining" motherboard with 10+ pci-e x1 slots where you could plug usb controller cards, but you'd hit various resource issues and limitations and the drivers may be unstable crash, hence why probably limiting to 7 cards  or less would be a good idea.

 

You could connect an USB hub to each of those USB ports so let's say you connect a 4 port or 7 port usb hub to each of the 4 usb ports on a controller - that means you convert 4 ports into 16 or 28 ports ... but those groups of 4 or 7 ports will share the bandwidth of 5 gbps.

 

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

4 minutes ago, Flyguygamer said:

Not video, it's all data. Just dumping from SSD to local storage so they can use it. One or two people using 1-3 devices and all they do is connect SSD and start the transfer. 

Doesn't really justify what it is that they want. It doesn't make sense for a central system to have 100+ USB C ports. The whole purpose of high speed networking is so that people DON'T have to do crazy things like this.

 

If they need this much bandwidth setup a 40Gbit aggregated link to a switch and create a few ingest stations people can walk up to and dump data onto the server form there.

Link to comment
Share on other sites

Link to post
Share on other sites

5 minutes ago, Flyguygamer said:

Not video, it's all data. Just dumping from SSD to local storage so they can use it. One or two people using 1-3 devices and all they do is connect SSD and start the transfer. 

 

You should consider suggesting the use of  those 5.25" bays where you can plug a 2.5" SSD ... you can enable hot plug and play in bios and you just plug the ssd into the slot and flip a switch and they're connected to the SATA controller, and you can transfer at up to 550 MB/s

Transfers would be faster than going through usb 3.0 and taking out the SSD from the external case should not be very hard.

 

Link to comment
Share on other sites

Link to post
Share on other sites

15 minutes ago, mariushm said:

 

You should consider suggesting the use of  those 5.25" bays where you can plug a 2.5" SSD ... you can enable hot plug and play in bios and you just plug the ssd into the slot and flip a switch and they're connected to the SATA controller, and you can transfer at up to 550 MB/s

Transfers would be faster than going through usb 3.0 and taking out the SSD from the external case should not be very hard.

 

They do large scale data collection, recorded to USB C external SSD's, so unfortunately that isn't practical and we are limited to USB for connectivity. 

Link to comment
Share on other sites

Link to post
Share on other sites

13 minutes ago, Windows7ge said:

Doesn't really justify what it is that they want. It doesn't make sense for a central system to have 100+ USB C ports. The whole purpose of high speed networking is so that people DON'T have to do crazy things like this.

 

If they need this much bandwidth setup a 40Gbit aggregated link to a switch and create a few ingest stations people can walk up to and dump data onto the server form there.

That is the backup plan if I can't get it all on one system is 2 or 3 systems with 7 port PCIE cards in 6-7 slots. 

Link to comment
Share on other sites

Link to post
Share on other sites

11 minutes ago, Flyguygamer said:

That is the backup plan if I can't get it all on one system is 2 or 3 systems with 7 port PCIE cards in 6-7 slots. 

I mean, technically you can connect (I believe) up to 127 devices to a single USB port. The bandwidth will get choked in no time though and it'd look very unprofessional.

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, Windows7ge said:

I mean, technically you can connect (I believe) up to 127 devices to a single USB port. The bandwidth will get choked in no time though and it'd look very unprofessional.

Yeah, looking to keep transfer bandwidth high. They'll have a poor intern who's only job it is to take the drives and connect them and transfer data, but I guess they'll just need to use 3 workstations to do that. Thankfully the office is already setup for 10gb

Link to comment
Share on other sites

Link to post
Share on other sites

17 minutes ago, Flyguygamer said:

Yeah, looking to keep transfer bandwidth high. They'll have a poor intern who's only job it is to take the drives and connect them and transfer data, but I guess they'll just need to use 3 workstations to do that. Thankfully the office is already setup for 10gb

Is this data transfer a one-off situation or would this be standard operation?

Link to comment
Share on other sites

Link to post
Share on other sites

57 minutes ago, Windows7ge said:

Is this data transfer a one-off situation or would this be standard operation?

Standard operation. They have 250 of these USB SSD's in circulation for a data collection operation. 

Link to comment
Share on other sites

Link to post
Share on other sites

3 hours ago, Flyguygamer said:

Standard operation. They have 250 of these USB SSD's in circulation for a data collection operation. 

IMO if this were to be remotely feasible while being at least a little professional it'd be to find docks that accept like 10 of these SSDs at a time.

 

Another issue we haven't even addressed is if they use Windows, Windows only allows up to 26 drives to have drive letters. Beyond that the other 74 would have to manually be assigned mountpoints. That would be a lot of work and I don't see a way to maintain it seeing how they'll be plugged and unplugged at random.

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×