AWS S3 Glacier - Best Way to Upload Data

gutz00 · April 25, 2021

Hi folx,

I use AWS S3 glacier to store and archive any long term data, I mainly do video production and I don't get rid of files (over the past year alone I've created 3TB worth of uncompressed data).

I try and do an archive upload once a quarter - but since moving to Phoenix I haven't been able to do that on my DSL connection (10 mbps up). Other than the expensive Snowball service, is there a way to reach out to a local data center as buy time to pop a couple high capacity drives into a machine and start a upload to AWS.

In the past I was in the Chicago suburbs with a gigabit connection so it wasn't as much of an issue, but now it just keeps snowballing.

Bombastinator · April 25, 2021

39 minutes ago, gutz00 said:

Hi folx,

I use AWS S3 glacier to store and archive any long term data, I mainly do video production and I don't get rid of files (over the past year alone I've created 3TB worth of uncompressed data).

I try and do an archive upload once a quarter - but since moving to Phoenix I haven't been able to do that on my DSL connection (10 mbps up). Other than the expensive Snowball service, is there a way to reach out to a local data center as buy time to pop a couple high capacity drives into a machine and start a upload to AWS.

In the past I was in the Chicago suburbs with a gigabit connection so it wasn't as much of an issue, but now it just keeps snowballing.

Does this data spark joy?

gutz00 · April 25, 2021

50 minutes ago, Bombastinator said:

Does this data spark joy?

The data sparks income. Income buys joy.

Bombastinator · April 25, 2021

5 minutes ago, gutz00 said:

The data sparks income. Income buys joy.

There is that I guess. There’s always the various forms of sneakernet. There’s a really old saying: “never underestimate the bandwidth of a van full of tape barreling down the road at 60mph”. You can even get decent bandwidth from messenger pidgins carrying SD cards. The latency is terrible of course.

Edited April 25, 2021 by Bombastinator

gutz00 · April 25, 2021

4 minutes ago, Bombastinator said:

There is that I guess. There’s always the various forms of sneakernet. There’s a really old saying: “never underestimate the bandwidth of a van full of tape barreling down the road at 60mph”. You can even get decent bandwidth from messenger pidgins carrying SD cards. The latency is terrible of course.

Does AWS have the option to ship the data to them? The only option I know about is snowball and for the amount of data I have it's not cost effective. (Too little data, too high of a cost)

Bombastinator · April 25, 2021

3 minutes ago, gutz00 said:

Does AWS have the option to ship the data to them? The only option I know about is snowball and for the amount of data I have it's not cost effective. (Too little data, too high of a cost)

Sounds like a question for AWS. I have heard of very high capacity stuff. I don’t have details though. For a mere 3tb you could just buy a mechanical HD and FedEx it.