surveillance drive in a storage server

[wylie the coyote]

hello I'm more likely to light a fire with this one i don't mean to..

so I'm thinking making my self a storage server as my hoard of drives I'm hitting over 700gb now of lose drive laying around 

as i like reliable storage i was thinking using surveillance drives speed im not worried about as the data im collecting is not always needed i can go as long two years needing a file ect,

to the point i was thinking that surveillance best choice of drive for reliability. the data i keep is terrain telemetry data as i map parts of world turn them in to game maps with all satellite imaging with it as well,

the thing worries me buying drives and have bunch of drives fail losing about 7years of work i am going to run raid for back up if few drive fails i can just plop in a new drive back it up to that all that shizz.

 

so what you think people any ideas whats the best drive type and cost effective.

 

[BiotechBen]

The surveillance drives are designed for 24/7 writes of sequential data, so if you are using them for raw long term storage, with predominantly sequential read/write, if there is a good deal on it: sure.

[Aaron_T]

For very important data, individual drive reliability is less important that multiple redundant backups.

Personally, for very important data storage (for personal or small business) I'd grab 4-6 NAS drives, and configure them in a RAID 6 to read and write to/from locally. Then I'd either use a cloud storage provider for an off site backup, or I'd get another NAS that was the same, or at least RAiD 1, and copy my data to it every week or two and store it somewhere other than where I work. Family or friends house.

[Senzelian]

Surveillance drives will be great for such an application. 

If you want to go a step further for reliability, buy them from different manufacturers. The likelyhood of all of them failing at the same time massively decreases if you do that. 

[wylie the coyote]

yes i will be writing a lot but rarely read so ill collect like 400sq km of sat data pull it off make the map move it back after 

[wylie the coyote]

i want start really building my DB more as finding sat data coming harder for what i do.. i was think having a rack getting say like 100 drives brake them down in to groups of 20 those groups running them raid  all so have another like 50 sitting on standby if something to happen ect, basically ill see 5 network drives. each drive say be 1tb if i have 20 ill only have 10tb in that group. if u see what im getting at. sorry if im not making sense I'm all self-taught. or could have NAS drives as the main storage the have Surveillance drives as my back up they only turn on to back up the data maybe i want like 99.9% safe data storage lol sorry for all the questions

18 minutes ago, AaronThomas said:

For very important data, individual drive reliability is less important that multiple redundant backups.

Personally, for very important data storage (for personal or small business) I'd grab 4-6 NAS drives, and configure them in a RAID 6 to read and write to/from locally. Then I'd either use a cloud storage provider for an off site backup, or I'd get another NAS that was the same, or at least RAiD 1, and copy my data to it every week or two and store it somewhere other than where I work. Family or friends house.

18 minutes ago, Senzelian said:

Surveillance drives will be great for such an application. 

If you want to go a step further for reliability, buy them from different manufacturers. The likelyhood of all of them failing at the same time massively decreases if you do that. 

 

[Aaron_T]

4 minutes ago, wylie the coyote said:

i want start really building my DB more as finding sat data coming harder for what i do.. i was think having a rack getting say like 100 drives brake them down in to groups of 20 those groups running them raid  all so have another like 50 sitting on standby if something to happen ect, basically ill see 5 network drives. each drive say be 1tb if i have 20 ill only have 10tb in that group. if u see what im getting at. sorry if im not making sense I'm all self-taught. or could have NAS drives as the main storage the have Surveillance drives as my back up they only turn on to back up the data maybe i want like 99.9% safe data storage lol sorry for all the questions

 

If you're going to have that many drives per rack (20), then you should be getting enterprise grade drives. Like the EXOS series. NAS and Surveillance drives are good, but only the enterprise drives are designed to be in a chassis with that much vibration from other drives.
 

Also, running RAID 10 or RAID 6 is your own personal preference, IE either 10 or 18 TB per rack (based on 1 tb individual disk size). But I don't think enterprise drives come that small. So maybe just get two racks, with 8-10 TB enterprise drives and one of the racks can be your cold storage that you store "off-site".

[Senzelian]

4 minutes ago, wylie the coyote said:

like 100 drives

That's not the right way to approach such a project.
You configure the amount of drives and the size of each drives depending on your needs. And you'll have to decide which hardware and software should run such a storage server and how it will be accessed.

 

[wylie the coyote]

2 minutes ago, AaronThomas said:

If you're going to have that many drives per rack (20), then you should be getting enterprise grade drives. Like the EXOS series. NAS and Surveillance drives are good, but only the enterprise drives are designed to be in a chassis with that much vibration from other drives.
 

Also, running RAID 10 or RAID 6 is your own personal preference, IE either 10 or 18 TB per rack (based on 1 tb individual disk size). But I don't think enterprise drives come that small. So maybe just get two racks, with 8-10 TB enterprise drives and one of the racks can be your cold storage that you store "off-site".

 

1 minute ago, Senzelian said:

That's not the right way to approach such a project.
You configure the amount of drives and the size of each drives depending on your needs. And you'll have to decide which hardware and software should run such a storage server and how it will be accessed.

 

im not getting that small drives just using it as an example i was thinking like 20tb drives,  it's looking like im going to need enterprise drives lol oh this getting expensive  

thread ripper cpu? mite need it with all these pci lanes taken up lol  

[Aaron_T]

I see. So if you are talking about 2 petabytes across multiple racks, then you definitely have to get enterprise grade drives. Also, yes, you will need beefy CPUs to handle running a NAS with that much storage. You should look at the LTT petabyte project videos as a starting place.

 

Also @Senzelianhas a good point. You need to start worrying about what OS and software you're going to run. What file system is right for you. How you want to access the data stores, physical layer and protocol, remotely vs in network only. You have a lot to think about when building this large of storage solution on your own.

If you're going from loose drives to multiple racks of drives there is going to be a huge learning curve, and if your data is only at 700gb now, it sounds like you are planning for an astronomical amount of overkill that you likely don't need.
 

[wylie the coyote]

1 hour ago, AaronThomas said:

I see. So if you are talking about 2 petabytes across multiple racks, then you definitely have to get enterprise grade drives. Also, yes, you will need beefy CPUs to handle running a NAS with that much storage. You should look at the LTT petabyte project videos as a starting place.

 

Also @Senzelianhas a good point. You need to start worrying about what OS and software you're going to run. What file system is right for you. How you want to access the data stores, physical layer and protocol, remotely vs in network only. You have a lot to think about when building this large of storage solution on your own.

If you're going from loose drives to multiple racks of drives there is going to be a huge learning curve, and if your data is only at 700gb now, it sounds like you are planning for an astronomical amount of overkill that you likely don't need.
 

i going to run Linux as there is lots of free tools are really good software out there for it also why i only got 700gb is to do with my download speed lol i got 1gb now so like i can fill these drive like in no time

[tkitch]

go watch the new LTT Video.