Share HDD over internet?

[GrandFatMan]

Hello everyone. 

Me and my friend both work on the same projects, mainly Audio and Video work. And we share access to the same files. Currently, It's a bit, old school. I go round there with a USB drive, get what I need, then go back, and put it on the HDD so he can view it. 

Is there any way I can add that harddrive to my system as if it was a local "network" drive?

I should add, his setup is a little different. He is using an Nvidia Shield as his NAS. Which runs android, not windows. 

[Crunken]

Google drive or something similar would probably be the best option

[GrandFatMan]

The data is constantly changing. And it's terrabytes. Plus, we'd rather not entrust our data to someone else. 

 

[GrandFatMan]

Oh, and another reason the "cloud" can't be an option, Is he edits locally off the drives. Can't do that if it's hosted elsewhere

[Oshino Shinobu]

5 minutes ago, GrandFatMan said:

The data is constantly changing. And it's terrabytes. We both have fairly decent down speeds. Plus, we'd rather not entrust our data to someone else. 

 

Google's upload rate is going to be a hell of a lot faster than either of yours and more reliable.

 

Also, you'd rather not trust a company that hosts sensitive information for loads of companies but you're willing to set up a connection that's open for attack from anywhere on the internet over a home network with consumer grade equipment?

 

If you insist, the best way of doing this would be to set up a VPN connection on a host machine to allow access to the local network. I'd suggest doing this over Windows server and use SSTP if you're both using Windows to access the share. There's a load of info on how to setup VPNs using Windows Server's RAS roles. You can manage access using AD if you wish with Windows Server.

 

EDIT: In terms of real-time editing, he's not going to be able to edit off the share unless your upload bandwidth is at least on par with his download, So I don't really see why cloud options aren't viable, seeing as he'll need to download the files, edit them locally and then re-upload anyway.

[Snipergod87]

Not really no, if you could easily like that your data rate would be limited by the upload speed which tends to be extremely slow.

Only real option is a VPN

[GrandFatMan]

2 minutes ago, Snipergod87 said:

Not really no, if you could easily like that your data rate would be limited by the upload speed which tends to be extremely slow.

Only real option is a VPN

His upload speed is sufficient. My download speed is sufficient. 

[Mr Technician]

As others have said, a site to site VPN could work but only if you have fast upload, think 100 mbps minimum and more like 250-500 mbps since this is for video. You definitely don't have those speeds unless you have fiber internet.

[GrandFatMan]

4 minutes ago, Oshino Shinobu said:

Google's upload rate is going to be a hell of a lot faster than either of yours and more reliable.

 

Also, you'd rather not trust a company that hosts sensitive information for loads of companies but you're willing to set up a connection that's open for attack from anywhere on the internet over a home network with consumer grade equipment?

 

If you insist, the best way of doing this would be to set up a VPN connection on a host machine to allow access to the local network. I'd suggest doing this over Windows server and use SSTP if you're both using Windows to access the share. There's a load of info on how to setup VPNs using Windows Server's RAS roles. You can manage access using AD if you wish with Windows Server.

There are reasons we don't want to host the data elsewhere. 

Also, The upload speed is going to be dependant on the local upload speed anyway, which is fine. If i could see the HDD "locally" then it's just more convinient

[Oshino Shinobu]

Just now, GrandFatMan said:

There are reasons we don't want to host the data elsewhere. 

Also, The upload speed is going to be dependant on the local upload speed anyway, which is fine. If i could see the HDD "locally" then it's just more convinient

Not if you're accessing the share remotely... Or are you just wanting to be able to access the drive when you're at his house, on his network?

[GrandFatMan]

No, I want to be sat at home, and see the drive as if it was a locally attatched network storage drive. 

[Alex Atkin UK]

As others have said, a VPN is the most flexible option as then you can use almost any method to access it you could on the LAN.

 

I've accessed my NAS over a VPN using normal Windows File Sharing absolutely fine, well apart from being on crappy hotel WiFi at the time so it was a tad slow.

 

With the latency of the Internet though, its definitely not going to work as well as on the LAN itself, especially for video work.  Its likely going to be quite painful if he is accessing it directly as if its a local drive.  You have to remember that LAN latency is about .1ms and Internet is likely 100 times that or more.

[mariushm]

Does he have the upload speed for you to copy the files through the internet?

Do you have the download speed to do it every day?

 

He can set up a FTP server, do some port forwarding on his modem/router and then you can simply connect to his computer using a ftp client and retrieve the files to your computer. Or upload from your computer to his. 

Or instead of ftp, you could use something else, doesn't matter what you decide to go with. Can be a simple http server with auto index set and authentification, and then you can simply use the web browser to go through the files.  

 

If you don't have such a great speed, an option would be for one of you to be the "master" and the other one to be the "slave" editor. The master creates a "proxy" files which can be let's say a 720p 1-2mbps (low bitrate) files, the slave can download these and do everything with these proxy files and at the end of the day he sends you the project and you render it using the high quality files instead of the proxy.

 

ps.  Having the remote drive as if a local one may be attractive, but it would work really slow when you're in various software products.

The problem is that such products constantly seek through files, or jump at various locations, they seek, extract a number of frames, then may seek into the file somewhere else to extract a chunk of audio, and repeat ... that latency of your requests going all over the internet to the other computer, other computer aknowledging command and replying and starting a small transfer... it takes a few ms for transfers to ramp up due to dynamic tcp windows ... basically it would be really laggy no matter how fast the connection you each have.

Makes more sense to copy the files locally and work on them.

 

 

 

[Oshino Shinobu]

Just now, GrandFatMan said:

No, I want to be sat at home, and see the drive as if it was a locally attatched network storage drive. 

Then it's not dependent on the local upload speed, it's dependent on your internet upload/download speeds, same as if you were uploading to Google.

 

If it's hosted on his network, anything you download will depend on his upload speed and vice-versa. Whoever hosts it will only depend on their local upload/download.

 

Anyway, best way to set this up is to use a VPN. You can set this up on your router, depending on the model. Most consumer models don't support it. If it doesn't, then I'd suggest investing in a dedicated PC to host a VPN and possibly the share as well. It gives you some more control over access.

[GrandFatMan]

1 minute ago, Oshino Shinobu said:

Then it's not dependent on the local upload speed, it's dependent on your internet upload/download speeds, same as if you were uploading to Google.

 

If it's hosted on his network, anything you download will depend on his upload speed and vice-versa. Whoever hosts it will only depend on their local upload/download.

 

Anyway, best way to set this up is to use a VPN. You can set this up on your router, depending on the model. Most consumer models don't support it. If it doesn't, then I'd suggest investing in a dedicated PC to host a VPN and possibly the share as well. It gives you some more control over access.

The speed is irrelevant. Mine is fairly decent too. Take the speed out of the equation, it's not a concern. 

[GrandFatMan]

5 minutes ago, mariushm said:

Does he have the upload speed for you to copy the files through the internet?

Do you have the download speed to do it every day?

 

He can set up a FTP server, do some port forwarding on his modem/router and then you can simply connect to his computer using a ftp client and retrieve the files to your computer. Or upload from your computer to his. 

Or instead of ftp, you could use something else, doesn't matter what you decide to go with. Can be a simple http server with auto index set and authentification, and then you can simply use the web browser to go through the files.  

 

If you don't have such a great speed, an option would be for one of you to be the "master" and the other one to be the "slave" editor. The master creates a "proxy" files which can be let's say a 720p 1-2mbps (low bitrate) files, the slave can download these and do everything with these proxy files and at the end of the day he sends you the project and you render it using the high quality files instead of the proxy.

 

ps.  Having the remote drive as if a local one may be attractive, but it would work really slow when you're in various software products.

The problem is that such products constantly seek through files, or jump at various locations, they seek, extract a number of frames, then may seek into the file somewhere else to extract a chunk of audio, and repeat ... that latency of your requests going all over the internet to the other computer, other computer aknowledging command and replying and starting a small transfer... it takes a few ms for transfers to ramp up due to dynamic tcp windows ... basically it would be really laggy no matter how fast the connection you each have.

Makes more sense to copy the files locally and work on them.

 

 

 

It's not to edit using that drive. It's an export drive. So finished, or files needing review. 

So i'd just "send them" to that harddrive just before i went to bed. So the speed isn't really an issue. 

[mariushm]

Then have him set up a ftp server on his computer, and forward the ports in his modem/router and tell him to leave the pc running over night.

When you're done, you simply connect to the ftp server and upload the videos to his computer.

 

[thisforumsucksassdeletemya]

FTP server?

 

god damn ninja mariushm

[Oshino Shinobu]

Just now, GrandFatMan said:

The speed is irrelevant. Mine is fairly decent too. Take the speed out of the equation, it's not a concern. 

I've already given you an answer then. Ignore the speed. Use a VPN.

 

I run a Windows Server with a RAS role on my own network for remote file access and CCTV. I also have a few friends who I've given access to some shared files. That's the way I'd recommend doing it, using AD to manage NPS and file sharing permissions gives you a lot of control.

 

I limit my friends to only an hour connection to the VPN before disconnecting them and a 10 minute idle timeout. There's a lot of videos and guides for how to set this up. It's not really an intensive usage, so you don't need a super powerful PC to do it.

[mariushm]

Quote

FTP server?

 

god damn ninja mariushm

It's "ancient" tech, but supports encryption, supports passwords, supports bandwidth limits if needed, supports concurrent connections, it's just a pain in the ass to configure it to go through firewalls but moderns routers and modems support it quite well..

[GrandFatMan]

6 minutes ago, Oshino Shinobu said:

I've already given you an answer then. Ignore the speed. Use a VPN.

 

I run a Windows Server with a RAS role on my own network for remote file access and CCTV. I also have a few friends who I've given access to some shared files. That's the way I'd recommend doing it, using AD to manage NPS and file sharing permissions gives you a lot of control.

 

I limit my friends to only an hour connection to the VPN before disconnecting them and a 10 minute idle timeout. There's a lot of videos and guides for how to set this up. It's not really an intensive usage, so you don't need a super powerful PC to do it.

The Shield is always on anyway. 

[Oshino Shinobu]

Just now, GrandFatMan said:

The Shield is always on anyway. 

Through the shield, I'm not sure it's possible. At least, not in a particularly secure way.

 

Android has good support for client side VPN connections, but I don't have much experience with hosting from Android. There are solutions like OpenVPN and SoftEther, though I'm not sure what the hosting is like on Android for those.

[Alex Atkin UK]

41 minutes ago, mariushm said:

It's "ancient" tech, but supports encryption, supports passwords, supports bandwidth limits if needed, supports concurrent connections, it's just a pain in the ass to configure it to go through firewalls but moderns routers and modems support it quite well..

FTP doesn't support encryption though, unless you mean SFTP which in my experience kills the speed advantage.

[mariushm]

4 minutes ago, Alex Atkin UK said:

FTP doesn't support encryption though, unless you mean SFTP which in my experience kills the speed advantage.

 

https://en.wikipedia.org/wiki/FTPS

 

 

I'm downloading stuff from my server in Netherlands at 100 mbps, any time of day, with encryption enabled... no worries about speeds.

 

[Cra1gst1]

next cloud is what i use for what your after a local hosted google drive