Database Server Storage Configuration

[kno3]

I need to build a server for a set of databases totaling ~10TB. The data will be used by online tools so querying/data access performance is critical (let's assume I have the right database software infrastructure). There are multiple points which I'm looking at:

 

- Storage 'structure': I've been looking at data storage options: NAS vs. DAS vs SAN. NAS is out of the question for this task and so it seems to be DAS vs. SAN. Since for now there will be one host which will be accessed locally and remotely, I'm leaning more towards DAS and avoid switches and all that.

- Storage configuration: I'm almost certain I will be going for RAID 10 for redundancy and performance

- Drive type: SSD vs HDD vs SSHD vs SAS. Due to the relatively 'low' lifetime of an SSD (and *cough* we've also seen Linus' disaster *cough*), I'd say its not a reliable option for the main storage. Performance-wise SAS are surely better but with 10TB size, this is not worth it money-wise. Even worse if I use RAID 10. Does SSHD make sense as a compromise? I've also read that SATA drives should be avoided at all costs; not sure what other options are left....I've also seen that Linus is using SATA HDDs for his 100TB server though?

----

 

EDIT: Some background about the usage

 

The database will be more of a data repository, with occasional scheduled updating (e.g. once/twice a week) and not continuous updating. Once the data subset is pulled out, it will be used for processing and analytics. Regarding OS, quite flexible although Unix is probably favored at this stage. The repository will be required to be consistently available however the load and querying rates are unknown at this stage since this will depend on user usage. Having said that, this is more of a 'preliminary' server to establish a system, gather the statistics of usage over time and if needed, update the hardware accordingly in the future. 

 

Any input is appreciated.

 

 

[tt2468]

7 minutes ago, kno3 said:

I need to build a server for a set of databases totaling ~10TB. The data will be used by online tools so querying/data access performance is critical (let's assume I have the right database software infrastructure). There are multiple points which I'm looking at:

 

- Storage 'structure': I've been looking at data storage options: NAS vs. DAS vs SAN. NAS is out of the question for this task and so it seems to be DAS vs. SAN. Since for now there will be one host which will be accessed locally and remotely, I'm leaning more towards DAS and avoid switches and all that.

- Storage configuration: I'm almost certain I will be going for RAID 10 for redundancy and performance

- Drive type: SSD vs HDD vs SSHD vs SAS. Due to the relatively 'low' lifetime of an SSD (and *cough* we've also seen Linus' disaster *cough*), I'd say its not a reliable option for the main storage. Performance-wise SAS are surely better but with 10TB size, this is not worth it money-wise. Even worse if I use RAID 10. Does SSHD make sense as a compromise? I've also read that SATA drives should be avoided at all costs; not sure what other options are left....I've also seen that Linus is using SATA HDDs for his 100TB server though?

 

Any input is appreciated.

 

 

10TB of databases is not something we should be helping you with. You might want to look into some sort of cloud solution, as database servers stop getting more powerful at around 6 to 8 cores. 10TB of databases sounds like 100s of queries per second, which is where a professional should be handling it. Just saying.

[leadeater]

Whats the budget?

Is high availability required?

What is the projected database load?

Whats is the type of load, OLTP or OLAP?

 

Basically there is no where near enough information to give a proper response.

[Mark77]

Are you just querying this data, or are you constantly updating/changing it?  What's the query rate?  What's your uptime requirement?  Linux/Unix or Windows? 

[leadeater]

13 minutes ago, kno3 said:

 

- Drive type: SSD vs HDD vs SSHD vs SAS. Due to the relatively 'low' lifetime of an SSD (and *cough* we've also seen Linus' disaster *cough*),

 

SSD's have extremely good life spans now days, much more than HDDs. Also the server failure at LMG had nothing to do with the disks, it was the RAID controller.

 

For performance workloads and anything not archival the industry consensus and movement is to all SSD storage, this is still rather costly however.

[zMeul]

I haven't seen enterprise grade SSHDs, if they exists I'd like to see them - thus, avoid using SSHDs in this type of situation

 

if access time is a issue, depending on the OS you can setup a cache with SSDs and have main storage on HDDs

[Mark77]

Just now, zMeul said:

I haven't seen enterprise grade SSHDs, if they exists I'd like to see them - thus, avoid using SSHDs in this type of situation

 

if access time is a issue, depending on the OS you can setup a cache with SSDs and have main storage on HDDs

The DBMS may even facilitate the use of hierarchal storage, depending on what's used. 

 

For example, in the Oracle DB's, you can place certain parts of the database on HDDs, and other parts on SSDs.  Thus even modest use of SSD storage can compliment HDD-based storage quite nicely. 

[leadeater]

51 minutes ago, Mark77 said:

The DBMS may even facilitate the use of hierarchal storage, depending on what's used. 

 

For example, in the Oracle DB's, you can place certain parts of the database on HDDs, and other parts on SSDs.  Thus even modest use of SSD storage can compliment HDD-based storage quite nicely. 

Yea even MSSQL can do this with Partitioned Tables, Multiple Data File and File Groups etc.

[leadeater]

1 hour ago, kno3 said:

EDIT: Some background about the usage

 

The database will be more of a data repository, with occasional scheduled updating (e.g. once/twice a week) and not continuous updating. Once the data subset is pulled out, it will be used for processing and analytics. Regarding OS, quite flexible although Unix is probably favored at this stage. The repository will be required to be consistently available however the load and querying rates are unknown at this stage since this will depend on user usage. Having said that, this is more of a 'preliminary' server to establish a system, gather the statistics of usage over time and if needed, update the hardware accordingly in the future. 

 

Any input is appreciated.

 

 

Well if this is more of a pilot then I guess the real difference is what you do and how you want to expand after it.

 

The simplest thing to do would be a single server with local disk and use this to gather the required metrics. Downside is expanding from this really isn't possible, not easily anyway.

 

The next cheapest option is to use a SAS connected storage array i.e. Lenovo V3700 or Netapp E2600. This will allow you to connect a few hosts without having to buy any extra SAS/FC/Network switches and will give very good performance.

 

However the disk I/O requirements may not be as big as you might expect, database engines are very good at keeping data resident in memory so as long as you have a good amount of this you won't be hitting disk at all, unless you are doing huge queries/table scans.

[kno3]

On 10/4/2016 at 1:13 AM, leadeater said:

Well if this is more of a pilot then I guess the real difference is what you do and how you want to expand after it.

 

The simplest thing to do would be a single server with local disk and use this to gather the required metrics. Downside is expanding from this really isn't possible, not easily anyway.

 

The next cheapest option is to use a SAS connected storage array i.e. Lenovo V3700 or Netapp E2600. This will allow you to connect a few hosts without having to buy any extra SAS/FC/Network switches and will give very good performance.

 

However the disk I/O requirements may not be as big as you might expect, database engines are very good at keeping data resident in memory so as long as you have a good amount of this you won't be hitting disk at all, unless you are doing huge queries/table scans.

Cheers for your input @leadeater. Much appreciated.