Ubuntu server disk use

[Joveice]

Hi, I got two 120GB ssd's and two 1TB HDD's + three 3TB HDD's.

I'm wondering if this would be a good way to do it until I can buy more SSD's. OS is installed on the RAID 1 (2 x 120GB SSD). Database storage is on the RAID 1 (2 x 1TB HDD) and website + other applications is on the RAID 5 (3 x 3TB HDD).

[Electronics Wizardy]

What are you using for raid?

 

What specs?

 

How big is the data base?

 

how much do you need iops wise?

[Joveice]

3 minutes ago, Electronics Wizardy said:

What are you using for raid?

 

What specs?

 

How big is the data base?

 

how much do you need iops wise?

I'm using the motherboard for raid, the specs are 16GB ram (upgraded later) 1x E5-2630v4 and I don't have a iops as I'm not currently have any load at all as I'm still developing my website. The database as of now isent bigger than 5MB as I only have 13 users registered without any data for them so all those numbers are still unknown, but I'm not expecting any big load at all.

[Electronics Wizardy]

1 minute ago, Joveice said:

I'm using the motherboard for raid, the specs are 16GB ram (upgraded later) 1x E5-2630v4 and I don't have a iops as I'm not currently have any load at all as I'm still developing my website. The database as of now isent bigger than 5MB as I only have 13 users registered without any data for them so all those numbers are still unknown, but I'm not expecting any big load at all.

Don't use mobo raid, use the software raid in linux. Id use btrfs for raid 0/1/10 and md/zfs for raid 5/6

 

For the data base, might as well store it on the ssd if its small. You can always move it later.

[Jonny]

I would use ZFS on this, might be a little tricky to setup, because will require some command line first, same with MD but md is old and ZFS has much better data integrity.

 

Set the mount point on the SSD's to /

 

With ZFS you can turn "RAID 1 (2 x 1TB HDD)  RAID 5 (3 x 3TB HDD)" in to one big storage pool. Which you then have 7TBish of usable space.

 

Then you can create a dataset in ZFS for /var

 

/var is where most data goes on Linux,including /var/www/html the default place for websites.

 

If you then want you can create another dataset for /home and limit the amount of space it can use if needed. Lots of features you can do in ZFS. Snapshots will probably be really handy for you. I highly recommenced you do do some googling of ZFS and maybe some YouTube videos explaining it.

 

[Guest]

As other have said ZFS is a good option for storage that needs reliability. [Your ram is ECC right? The usual thinking is 1gb of ram per terabyte of Raw storage] While ECC ram is not a requirement it is STRONGLY recommended for the best data reliability. You can create storage pools. As for your ssds do NOT use your motherboard raid. Instead create a mirrored drive configuration for your root drive in the Ubuntu server install config. 

[Jonny]

3 hours ago, BurningSmile said:

As other have said ZFS is a good option for storage that needs reliability. [Your ram is ECC right? The usual thinking is 1gb of ram per terabyte of Raw storage] You can create storage pools. As for your ssds do NOT use your motherboard raid. Instead create a mirrored drive configuration for your root drive in the Ubuntu server install config. 

I'm guessing its not. ZFS is fine with none ECC memory, of course ECC is better but will run fine with normal RAM. (Just don't overclock it)

Yeah using the mirror feature in the installer would be good. But I prefer to use ZFS for everything, rather then having MD and ZFS running together.

But using the build in feature of the install and using MD would make setting up the system easier.

 

Another way to set the system would be this.

 

Install ubuntu OS to one SSD. 

Create your ZFS pool with all the HDD's then use the 2nd SSD as a cache. That way it will store all the commonly used files in the ZFS Pool on the SSD. Which would improve performance depending how the pool is used.

 

I have a 60GB Boot SSD in my server.

ZFS pool with 6x 3TB WD Reds. in two sets of raidz1. Which then has a Samsung SSD 250gb, for cache. Then I store all my files on the pool.

I have a lot of video files on my server, various data, torrents. So when I Seed Linux ISO's, as the file is consistently being used. The file will be loaded in to the cache SSD, instead of reading it off the HDD's everytime.

[Joveice]

6 hours ago, Jonny said:

I'm guessing its not. ZFS is fine with none ECC memory, of course ECC is better but will run fine with normal RAM. (Just don't overclock it)

Yeah using the mirror feature in the installer would be good. But I prefer to use ZFS for everything, rather then having MD and ZFS running together.

But using the build in feature of the install and using MD would make setting up the system easier.

 

Another way to set the system would be this.

 

Install ubuntu OS to one SSD. 

Create your ZFS pool with all the HDD's then use the 2nd SSD as a cache. That way it will store all the commonly used files in the ZFS Pool on the SSD. Which would improve performance depending how the pool is used.

 

I have a 60GB Boot SSD in my server.

ZFS pool with 6x 3TB WD Reds. in two sets of raidz1. Which then has a Samsung SSD 250gb, for cache. Then I store all my files on the pool.

I have a lot of video files on my server, various data, torrents. So when I Seed Linux ISO's, as the file is consistently being used. The file will be loaded in to the cache SSD, instead of reading it off the HDD's everytime.

 

9 hours ago, BurningSmile said:

As other have said ZFS is a good option for storage that needs reliability. [Your ram is ECC right? The usual thinking is 1gb of ram per terabyte of Raw storage] You can create storage pools. As for your ssds do NOT use your motherboard raid. Instead create a mirrored drive configuration for your root drive in the Ubuntu server install config. 

My ram is ECC, so I should install it on one of the disks, mirror it to the other and setup ZFS for raid 1 of the 2 1TB disks (Database disks when that is needed). and a raid 5 for my 3x3TB which will be used for programs and such?

[Jonny]

On 1/27/2017 at 3:27 PM, Joveice said:

 

My ram is ECC, so I should install it on one of the disks, mirror it to the other and setup ZFS for raid 1 of the 2 1TB disks (Database disks when that is needed). and a raid 5 for my 3x3TB which will be used for programs and such?

Seems like a plan :), have you started yet?

[Joveice]

On 1.2.2017 at 9:48 AM, Jonny said:

Seems like a plan :), have you started yet?

No, my motherboard havent arrived yet, hopefully it will at the start of the next week.

[Jonny]

8 minutes ago, Joveice said:

No, my motherboard havent arrived yet, hopefully it will at the start of the next week.

Oh fair enough,

 

Keep us Posted with the build

[Joveice]

On 3.2.2017 at 10:20 AM, Jonny said:

Oh fair enough,

 

Keep us Posted with the build

Well done with the build and everything is working as planed, tho now I want to install that ZFS and make the 2x1TB HDD to a mirror and the 3x2TB to a "Raid 5" and I'm wondering where I can find information about this? and also where do I snapshoting? I want to do that instead of mirroring if that is better.

[Joveice]

On 3.2.2017 at 10:20 AM, Jonny said:

Oh fair enough,

 

Keep us Posted with the build

So I created a mirror of 2x 1TB amd zpool list says size 928G so thats correct, then I did a raidz of 3x 2TB and it says total 5.44, thats not correct, thats the total size of all the disks? And when that is fixed I want to mount it as /var but I have no clue on how that is done.

[Jonny]

54 minutes ago, Joveice said:

So I created a mirror of 2x 1TB amd zpool list says size 928G so thats correct, then I did a raidz of 3x 2TB and it says total 5.44, thats not correct, thats the total size of all the disks? And when that is fixed I want to mount it as /var but I have no clue on how that is done.

What you using to show the total space?

 

little tricky. Why do you want /var as it? why not just create new datasets for /var/websites and that. keep the logs and that on the SSD, keep the logging fast.

[Joveice]

2 minutes ago, Jonny said:

What you using to show the total space?

 

little tricky. Why do you want /var as it? why not just create new datasets for /var/websites and that. keep the logs and that on the SSD, keep the logging fast.

I used zfs list, but now after I destroyed it and created it again it shows 3.51T so I guess it's working now ?

And yea that might also work :P, should I just keep it mounted as "storage" it's found at /storage.

[Jonny]

1 minute ago, Joveice said:

I used zfs list, but now after I destroyed it and created it again it shows 3.51T so I guess it's working now ?

And yea that might also work :P, should I just keep it mounted as "storage" it's found at /storage.

Yeah or that. I just created a mount point for mine in /mnt/red/

 

Easy to set apache to use /storage/websites/....

same for other things

[Joveice]

2 minutes ago, Jonny said:

Yeah or that. I just created a mount point for mine in /mnt/red/

 

Easy to set apache to use /storage/websites/....

same for other things

Well I will be using Nginx, but I know how to do it there too

[Jonny]

You can do datasets in datasets, if you want to limit storage quota per website or something to that effect.

[Joveice]

1 minute ago, Jonny said:

You can do datasets in datasets, if you want to limit storage quota per website or something to that effect.

Nah I'm fine with a big storage. But now let's say one of my 3 drives dies, how do I know and what do I do when I'm going to replace it? (that rebuild thing)

[Jonny]

I have SMART Daemon running which is set to email me, if it see's disk errors.

 

Then I have a email script which mails me the status of the pools and their snapshots list. See below for example.

Does this email everyday at 8:30am

 

[Joveice]

3 minutes ago, Jonny said:

I have SMART Daemon running which is set to email me, if it see's disk errors.

 

Then I have a email script which mails me the status of the pools and their snapshots list. See below for example.

Does this email everyday at 8:30am

 

 

Okey thats really cool

[Jonny]

7 minutes ago, Joveice said:

Okey thats really cool

Here's the script https://paste.lemonjuice.tk/view.php?i=q6QncvPpBg

Uses Postfix to send the email so your need to install that.

[Joveice]

4 minutes ago, Jonny said:

Here's the script https://paste.lemonjuice.tk/view.php?i=q6QncvPpBg

Uses Postfix to send the email so your need to install that.

Thanks alot! (maby I will need to ask you about that )

 

My thread about webmail and my needs

I liked this IRedMail and I would like to use that becouse it looked nice and had all of those security things preinstalled.

Basicly, I need to be able to get emails and send them over my domain, and I need to be able to use send in PHP (I found a library that can use a SMTP server for that)

[Jonny]

2 hours ago, Joveice said:

Thanks alot! (maby I will need to ask you about that )

 

My thread about webmail and my needs

I liked this IRedMail and I would like to use that becouse it looked nice and had all of those security things preinstalled.

Basicly, I need to be able to get emails and send them over my domain, and I need to be able to use send in PHP (I found a library that can use a SMTP server for that)

Mail

OK, I would take a look at Open-Xchange. Might be nicer interface than roundcube, I've used roundcube no way as near as good as other free mail providers but its self hosted.

https://oxpedia.org/wiki/index.php?title=AppSuite:Open-Xchange_Installation_Guide_for_Debian_7.0

That's a guide in how to install it in Debian 7 (Wheezy).

 

LXC

I would look in to LXC containers, that way you can containerize Nginx, Mail, torrents, etc.

You can then have a Debian Wheezy container for Mail, they share the host kernel, you can put more or less any Linux distro in a container. They use a lot less resources then virtual machines.

 

You will probably want to create a bridge on your eth0 Adapter so they use the same subnet as the server it self. Might take you a little while to configure it to your needs. But its worth it. I have everything in containers on my home server. Only thing I haven't got in a container is ZFS and SAMBA.

 

Using Containers will improve your Server Security, as if they hack on container they haven't got access to the host or other containers.

 

Any questions on setting up LXC just ask me. Been using it for years.

 

Basic Guide here

https://help.ubuntu.com/lts/serverguide/lxc.html

 

Follow this before creating your first container so they share the same network as the server (In your case home LAN), otherwise you have iptables to mess with and having each container have its own LAN IP much better.

https://insights.ubuntu.com/2015/11/10/converting-eth0-to-br0-and-getting-all-your-lxc-or-lxd-onto-your-lan/

 

I have 14 containers on my server. Couple of them externally facing. Also have mine on a DMZ network, but you need a proper firewall/router for that. Stop my web server being able to see my laptops, Desktops and TVs if they got hacked.

 

How much time have you spent on Linux Systems? Trying to get an idea, what you know.

[Joveice]

22 minutes ago, Jonny said:

Mail

OK, I would take a look at Open-Xchange. Might be nicer interface than roundcube, I've used roundcube no way as near as good as other free mail providers but its self hosted.

https://oxpedia.org/wiki/index.php?title=AppSuite:Open-Xchange_Installation_Guide_for_Debian_7.0

That's a guide in how to install it in Debian 7 (Wheezy).

 

LXC

I would look in to LXC containers, that way you can containerize Nginx, Mail, torrents, etc.

You can then have a Debian Wheezy container for Mail, they share the host kernel, you can put more or less any Linux distro in a container. They use a lot less resources then virtual machines.

 

You will probably want to create a bridge on your eth0 Adapter so they use the same subnet as the server it self. Might take you a little while to configure it to your needs. But its worth it. I have everything in containers on my home server. Only thing I haven't got in a container is ZFS and SAMBA.

 

Using Containers will improve your Server Security, as if they hack on container they haven't got access to the host or other containers.

 

Any questions on setting up LXC just ask me. Been using it for years.

 

Basic Guide here

https://help.ubuntu.com/lts/serverguide/lxc.html

 

Follow this before creating your first container so they share the same network as the server (In your case home LAN), otherwise you have iptables to mess with and having each container have its own LAN IP much better.

https://insights.ubuntu.com/2015/11/10/converting-eth0-to-br0-and-getting-all-your-lxc-or-lxd-onto-your-lan/

 

I have 14 containers on my server. Couple of them externally facing. Also have mine on a DMZ network, but you need a proper firewall/router for that. Stop my web server being able to see my laptops, Desktops and TVs if they got hacked.

 

How much time have you spent on Linux Systems? Trying to get an idea, what you know.

Well I'll look into this, first gotta fix a issue with nginx displaying 404 even tho it should be right configured 

How much time? Well not alot really, have used ubuntu server for almost 3 years now, but I havent gotten so much into it yet, I normaly just look for some documentation or tutorial that explains what stuff does and not just how to do it. I have setup a few servers (game servers) that havent worked out of the box becouse it wasent designed for linux and made some of them work by doing alot of work arounds, also I have started to setup a website project for a while now and have configured a few server things by doing it.

Most of my time goes into programming (PHP, some HTML, and little CSS). I do want to learn alot more of PHP and maby get into OOP PHP with some frameworks, or get into something like angularjs.

 

As of now I might be getting 2 years in a company that does programming from my school, if this happens I want to go more into that way, or I might go more into Networking and Infrastructure, have not made up my mind yet. So for now thats basicly it.