Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

Lifetime storage + Stuff?!?

Hooowdy!

 

I'll star by giving you a quick intro to who i am, and what i do!

 

I'm a professional photographer, i take a crapton of photos and i record very many videos. I run a team of 3 people ( me + two others). 

 

To all our customers, we sell lifetime storage of ALL the photos and videos we make for them for a set rate, around 85$ one time payment. (usually wedding stuff + some product photography and family shoots). Nothing complicated.

 

Up until now i have been solving this by getting premium deals on usb flashdrives from a local tech shop i have ties to, but this is just excruciatingly painful, due to the fact that a single accident could wipe out everything. There's obvious problems with this, as you all understand.

 

NOW! I would like to solve this issue for myself, and i need your help to do it!

 

Below is some criterias for what i want.

 

A machine that can:

 

1. Store huge amounts of data (20-30 TB or more)

2. Double as a render PC which i or a member of my team can edit off when we're on the road.

3. Not break my bank, this is a part-time mission with limited amount of resources. (3000-4000$, defo not more than 5k)

4. Live forever.

 

Any help and suggestions is greatly appreciated! :)

 

Thanks for your time

Link to post
Share on other sites

Just clarify your budget so we know what we're working with.

Link to post
Share on other sites
3 minutes ago, just tilt said:

Just clarify your budget so we know what we're working with.

3000-4000$, defo not more than 5k)

Link to post
Share on other sites

My sound system costs more than my PC.        Check out my S340 build log "White Heaven"        The "LIGHTCANON" flashlight build log        Project AntiRoll (prototype)        Custom speaker project

Spoiler

Intel i7 4790k | AMD Vega 64 | ASUS Sabertooth Z97 Mark S | Corsair Vengeance Pro 32GB | NZXT S340 | Seasonic Platinum 760 | modded H100i | Coolermaster SK630 White | Logitech MX Master 2S | 2x Samsung 850 Pro 512GB | WD Red 4TB Samsung 58" 4k TV | 2x Behringer NEKKST K8 | BIC Acoustech H-100II | Scarlett 2i4 | 2x AT2020

 

Link to post
Share on other sites

Could look at a QNAP TVS-671 with 6TB Red Pro drives for around $2.5-$3k US. Not 100% sure on the offloading render jobs though.

 

EDIT - you might also want to look at how you're backing that data up. I know Azure offers remote storage support for QNAP so you could use their cheap cold storage option.

Link to post
Share on other sites
3 hours ago, Verymanythings said:

To all our customers, we sell lifetime storage of ALL the photos and videos we make for them for a set rate, around 85$ one time payment. (usually wedding stuff + some product photography and family shoots). Nothing complicated.

 

Up until now i have been solving this by getting premium deals on usb flashdrives from a local tech shop i have ties to, but this is just excruciatingly painful, due to the fact that a single accident could wipe out everything. There's obvious problems with this, as you all understand.

Wait, let me get this straight... You offer your clients the indefinate preservation of any media you produce for them but then you only store a SINGLE instance of that data on a USB key?

 

...How fucking stupid can you be?  Like, I literally think that this might be FRAUD.

Link to post
Share on other sites

if you're a photography company that is offering "lifetime" storage, this is something you should already have....do your clients know you store their potentially irreplaceable data on a single flash drive?

 

EDIT: Miswording

Judge the product by its own merits, not by the Company that created it.

 

Don't dilute <good thing> by always trying to focus on, and drag conversation back to, <bad thing>.

Link to post
Share on other sites
9 minutes ago, Sierra Fox said:

if you're a photography company that is offering "lifetime" storage, this is something you should already have....do your clients know you store their potentially irreplaceable data on a single flash drive?

 

EDIT: Miswording

I don't even think OP is going to properly correct this.  This thread is asking about build a better local storage system and as well built as it can be it is still a SINGLE location for the data.  The OP thinks the system should 'Live Forever' so they seem to be pretty ignorant about data presevration.  No matter how well you build the basket, you still have all your eggs in ONE basket.  One fire or the right kind of power surge or PSU failure or USER ERROR could ruin all that data.

 

The smart plan is to outsource that to something offsite, Glacier or who knows what.  Then use some kind of 'colder' for local access like MDISC Blurays in a well organized and sorted system.  So in MOST cases when the client calls up because they want additional copies off their wedding photos, you PROBABLY have an optical disc on hand with it.  From then onward you have additional versions elsewhere incase those have failed.

 

ONE instance of data, no matter HOW well stored, is vunlerable.  The only real security for data is MULTIPLE instances, so that if you discover that an instance has been compromised, you can copy one of the other instances to replace the instances.  This is especially true if you're offering this as a COMMERCIAL SERVICE.

Link to post
Share on other sites
7 minutes ago, AshleyAshes said:

-a big snip-

soooo, 3 flash drives per customer? one in the right pocket, one in the left, and one in a shipping container next door.

Judge the product by its own merits, not by the Company that created it.

 

Don't dilute <good thing> by always trying to focus on, and drag conversation back to, <bad thing>.

Link to post
Share on other sites

Thanks for the personal insults and whatnot!

 

Anyways, the pictures i take already has 3 backups(a couple of big eHDD's and a cold storage pc), this is not an issue, there's no need for you to debate this. The issue is i need to get rid of the ~100 flash drives i have on my hands aswell as get a better way of accessing these images and videos without having to (FIGURATIVELY SPEAKING) dig through a mountain of flash drives or take a 1 hr drive to my summerplace where my cold pc is, or plug in 6 different external HDD's trying to find what i'm looking for.

 

Enough of that.

The 45drives option seems appealing, but very very costly, i was able to push the price just below 4k by dropping phone support and pre-installation.

Also, what is Unraid? I have heard it mentioned but never really had an explanation of it.

 

The NAS thing is pretty much a no-go. I need to be able to double it as a render offloading machine. 

 

Looking out for more suggestions!

Link to post
Share on other sites

I don't know anything about "render offloading", but I don't get why you would want your storage computer to do this?

I would try to find a HP Microserver Gen8. It is confirmed to be able to use 6TB drives, but I don't think there is a reason it would not be able to use bigger drives.
Put a P222 or P420 Raid controller in it, or run Windows Storage spaces and put 4 10TB drives in it. Add a boot drive to the 5th SATA port, find a E3-1260L or 1265L on Ebay for a bit more power and you are good to go. With the P222 you could even add an external enclosure. Maybe upgrade the RAM, if you want to (it requires ECC UDIMM).
And you could use Windows Server or Windows 10 Pro as OS.
Or a more professional focussed towerserver from HP or Dell.

Link to post
Share on other sites
1 hour ago, Pyramiden said:

I don't know anything about "render offloading", but I don't get why you would want your storage computer to do this?

I would try to find a HP Microserver Gen8. It is confirmed to be able to use 6TB drives, but I don't think there is a reason it would not be able to use bigger drives.
Put a P222 or P420 Raid controller in it, or run Windows Storage spaces and put 4 10TB drives in it. Add a boot drive to the 5th SATA port, find a E3-1260L or 1265L on Ebay for a bit more power and you are good to go. With the P222 you could even add an external enclosure. Maybe upgrade the RAM, if you want to (it requires ECC UDIMM).
And you could use Windows Server or Windows 10 Pro as OS.
Or a more professional focussed towerserver from HP or Dell.

What i mean with render offloading is that if i'm on the train and working with a project in Premiere or likewise, i need to be able to throw the project over to the server and render it there, instead of rendering it on my laptop and effectively locking it down for 30-40 minutes.

Link to post
Share on other sites
1 minute ago, Verymanythings said:

What i mean with render offloading is that if i'm on the train and working with a project in Premiere or likewise, i need to be able to throw the project over to the server and render it there, instead of rendering it on my laptop and effectively locking it down for 30-40 minutes.

What I mean to say is that I have no experience doing that. Is it CPU or GPU rendering? And what kind of application or OS do you need for it? I guess with a tower server you can just throw in a beefy CPU or GPU for rendering. Microserver Gen8 only has one PCIE slot, and Gen10 has no iLO and a slow CPU which can't be exchanged, so then I would look for something bigger.

Link to post
Share on other sites

If it was me i would keep a local copy of the files in a redundant storage (hdd based software raid + secondary backup offsite) then keep a cold copy in AWS Glacier storage as a worse case solution.

 

4k should be enough for a redundant 40tb solution with the remaining budget will get you 30tb of glacier storage for at least a year.

Link to post
Share on other sites

Have you looked into the Storinator Workstation? https://www.45drives.com/products/workstation/
It's essentially a PC that has a ton of storage, it writes to HDD or SSD’s, providing control of where your data lives. The SSDs provide fast access to your data until you are finished with it, and then it can be migrated to the HDDs for longer-term storage. 

Link to post
Share on other sites
On 3/19/2018 at 6:07 AM, Verymanythings said:

What i mean with render offloading is that if i'm on the train and working with a project in Premiere or likewise, i need to be able to throw the project over to the server and render it there, instead of rendering it on my laptop and effectively locking it down for 30-40 minutes.

You need to get one of the storinators and use windows server. You can have windows handle the raid and for render offloading windows will probably be the most capable os for what you want to do. I don’t have much experience with render offloading but if any os would support it, it would be windows server especially for adobe products like premiere. 45 drives man this what you need. They also have amazing support they will help you if you need it. 

This is my opinion, it doesn't mean I'm right and is liable to change at any time. I may offend of which I apologize in advance.


(Our lord and savior: GabeN)

Link to post
Share on other sites

Sounds to me like you basically just want an actual PC with a ton of HDD space...Grab you a 8700k, 1060 (don't know if you're gpu or cpu rendering, but if GPU you can upgrade), 32gb ram, and a nice ventilated case, and a bunch of HDD lol...

 

Then keep up with your current storage methods if you truly use 3 copies of backup.

Osmium: NFC Skyreach 4 // i7-8700k (delidd) // GTX 1080 // 32GB DDR4-3200Mhz // 1TB 960 Evo M.2 // 1.1TB MX300 m.2
Peripherals: Razer Blackwidow // Razer Orbweaver // Razer Kraken // Logitech G502 // Logitech K830 // LG 34UC88-B

Usage: Adobe Lightroom // Adobe Photoshop // Web Dev // Recording Gameplay // Video Editing // Portable Gaming

Link to post
Share on other sites
On 3/19/2018 at 12:22 AM, Verymanythings said:

--SNIP--

get a better way of accessing these images and videos without having to (FIGURATIVELY SPEAKING) dig through a mountain of flash drives

--SNIP--

 

The 45drives option seems appealing, but very very costly, i was able to push the price just below 4k by dropping phone support and pre-installation.

Also, what is Unraid? I have heard it mentioned but never really had an explanation of it.

The NAS thing is pretty much a no-go. I need to be able to double it as a render offloading machine. 

In this case, I'd 100% recommend investing in a 45drives machine, or another dedicated server machine with lots of hot swap drive bays. You're asking essentially for a $2000-$3000 rending server with hot storage attached to it for about half what it's going to cost in hardware, drives, and time setting it up. (This is not an insult to you; just stating a fact.)

 

You've got 2 choices:

  1. Spend around $5000-$8,000 and do it right the first time (hardware in a storage chassis with enough HDDs and the time to set this up)
  2. Spend between $2500-$5000 and plus a few thousand when (not if) you have a failure and need to shutdown business while restoring backups

I'm just throwing rough figures around here, but my point is I'd do it right the first time since this is storing media files that essentially make up the bread and butter of your business operations, even if it costs a bit more up front. I'd recommend looking into either building out a PC and attaching some fast NAS storage appliance to the network, or spending a bit more on a dedicated server chassis (such as one from Norco or SuperMicro) and putting a dual Xeon workstation into production for rendering.

 

On a side note, it's great that you have both on-site and off-site cold backups, but do you also have cloud backups should disaster strike at both locations at the same time?

Desktop: KRySTaLoGi-PC Build Log (i7-4790K, RTX2060) Mobile: OnePlus 5T | Bell - Unlimited Calling & Texting + 10GB Data
Laptop: Dell XPS 15 9560 (the real 15" MacBook Pro that Apple didn't make) Tablet: iPad Mini 5 | Lenovo IdeaPad Duet 10.1
Camera: Canon SX280 + Rebel T1i (500D) | Sony HDR-AS50R | Panasonic DMC-TS20D Music: Spotify Premium (CIRCA '08)

Link to post
Share on other sites

I can appreciate the OP wanting to take on the task of storage in house. The fact that the OP also recognized the terrible solution they've been using as...inadequate is proof of acknowledging the problem. But from what I gather, it would probably be better to have a service handle that storage task. Even if you had a great machine that can store more than any data you can throw at it you still should think about off site backups to protect against things like fire, flood, natural disasters, etc.

 

Sorry to not be much of help in the original questions but my opinion would be to take the money you would have spent on a giant server and shop around for a business account with a data storage firm. You'll have a lot more flexibility and would not have to worry about redundancy, backups, software licenses, hardware failures, or obsolescence of said hardware. I have no idea what sort of guarantee you give your customers for the lifetime storage thing but at least if you leave that data to a company that focuses in keeping the data safe and sound, you can focus on your pictures.

There's no place like ~

Spoiler

Problems and solutions:

 

FreeNAS

Spoiler

Dell Server 11th gen

Spoiler

 

 

 

 

ESXI

Spoiler

 

 

 

 

 

 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×