Unraid guide

[ChalkChalkson]

EDIT: This is heavily outdated and half the stuff in here doesn't work anymore due to some repos no longer being maintained. I probably won't be updating it though since A - time, B - my current setup is pretty different, C - I think other people can do this better.

 

Steps:

1. What is unraid and how does it work
2. Components
3. installing unraid
4. setting up the shares
5. configuring your router
6. plugins and docker
7. OpenVPN
8. Plex
9. Apache
10. MineOS
11. backups
12. crontab
13. further reading

1-Preamble

 

When it comes down to it unraid is a linux distribution, under the hood it uses a standard linux kernel and kvm, so in theory you can do everything unraid can with open-source freeware. Note however, that the UI and a few of the plugins definitely are worth the 50 bucks you pay for it.

The NAS side works the following way:

You have shares, which can be exported via SMB or NFS, so you can access them over the network like any other drive. The files are then distributed on the actual drives, but not striped, so even if you lose 2 drives at the same time, you wouldn't lose all the data. Each share has its own privacy rules. but non are encrypted by default.

The parity is something rather interesting, each part of the parity drive holds the value of all other drives x-ored, so you can rebuild any one drive. Note that this only works, when you replace that drive on its own, not, when you replace 1 failed drive with 2. Adding a second parity allows any 2 drives to fail but that math is orders of magnitude harder.

 

2-Components

 

Here is what you need for a server:

• CPU (should go with at least a modern dualcore)
• Motherboard
• Power Supply
• Case
• A USB drive (should go with at least 8GB)
• 2 or more Harddrives (see below)

With the harddrives there are a few things to consider, first, a failing harddrive is a real danger to your data, no matter what you do, so you shouldn't go with the sketchiest drives you can find, but you also don't need anything fancy. Also: you should get a drive that is at least as large, as the largest drive you want to add in the future.

If you want to have a proper backup, you need all the components twice of course.

 

 

3-installing unraid

 

First start downloading the unraid server zip from this page, then insert your USB drive, rightclick in the explorer, hit format, select FAT32 and name it 'UNRAID'.

After that, put all files from the zip onto the flashdrive and execute "makebootable.bat" as administrator. Now insert it into the newly built computer.

Go into the BIOS and set USB as the top boot priority.

Now you can unplug keyboard and screen from the server, you won't need them anymore.

Go to your normal PC, open a browser and type in "http://tower" and hit enter. On the page you will land on you can either get a trial key or copy-paste your key.

 

First things first: you want to set a password for your "root" (admin) account:

Go to the "users" tab, click on the "root" account and add a password. Hit "done".

After you got that sorted out, go to the "Main" tab. Now select your largest drive as parity and put the others in as disks.

Now just hit the format button and, when finished, start the array.

 

 

4-shares

 

Now this depends heavily on usecase, I decided that the best option for me is giving each server 2 shares, one being the "main" share of that server, the other one holding backups of the other one's main share.

To create shares, go to the "Shares" tab, click add share, type in the name and optionally select "use cache drive", I would advise to keep the rest unchanged. Hit "add share".

Now, you can select the export settings, if your household is only using windows (apart from the unraid server(s)) you can just hit "done". If you have linux machines that are supposed to be able to access the data, you need to enable NFS. 

If you want to make it only accessible to certain people, hit "done" for the moment, go to the "users" tab, create a new user, go back to the "shares" tab, click the name of the share, and select "private" or "secure" for all exports. "secure" means only the selected users have write access, but everyone has read access, "private" gives no permission to other users.

Usually you should make it "secure" with read/write for root.

 

 

5-The router

 

First, you should think about what parts of the server you want to make public. I would recommend only making a potential website, a VPN and potentially your Plex public.

Here are some common ports:

HTTP (Websites) Port 80

Plex Port 32400

OpenVPN Port 1194 (changeable) 

Minecraft Port 25565

CoD 4 Ports 28960 20800 20810

 

Now go to your router page (most commonly 192.168.0.1) and add a port forward from port 80 to port 80 on the IP of your server.  Use a page like this one to check whether it is open or not and immediately delete this port forward!

If the port did not show up as open, give your ISP a call and ask whether you are on IPv4 and whether your IP is capsuled. Try to sound annoyed and ask them to give you your own IPv4, while you are at it, you could also ask, whether you have a DNS running, and what the domain for it is and, if not, whether your IP is static. After they worked their magic, try again.

Now just forward all the ports to your server. If you want to host a website (public port needs to be 80), do not forward to the internal port 80, use 8080 internally instead, otherwise anyone can access the settings of you unraid server and thus your private network.

 

 

6-plugins and docker

 

Time to add stuff to listen to these ports.

First, lets add the unofficial, "app store". Go to the plugins tab, install plugins and copy paste this url:

Quote

https://raw.githubusercontent.com/Squidly271/community.applications/master/plugins/community.applications.plg

After this is done, go to the new "Apps" tab and search for "openvpn". Install the "OpenVPN Server" and, if you want to back up the server to another one in a different network, install the "OpenVPN Client" on the other one. While you are at it, go to the "docker" tab, click add container, select the "plex" template and hit "done". 

 

 

7-OpenVPN

 

Now go to "settings", openVPN Server, "OpenVPN Server Cert/Misc" and configure to your desire, 2048bit RSA is still considered safe, but the only trade off for using 4096bit is a bit of time during signing and connecting. I'd also change the path to store server and client certificates to "/boot/config/plugins/openvpnserver/" for convince sake. Hit "save configuration" and go to the next tab. Here you could leave anything at default as well, but if you already have a DNS it doesn't hurt to specify your domain.

Hit "save" and go to the first tab.

Install EasyRSA with the button and generate the server certificate. Go to the clients, keep the default selected and type in the certificate name (doesn't really matter). Go to the first tab again and start the openvpn server.

 

Now navigate to the path you specified for the client certificates to be stored in ("boot" is exported as "flash") and go into the folder with the name of your client certificate. Copy the ".ovpn" file.

On the client side (could be another unraid server, or your PC, install the client software and navigate to the openVPN folder.

Windows: C:\Program Files\OpenVPN\config , unraid with plugin: \\[ServerIP]\flash\config\plugins\openvpnclient and store the certificate there.

 

At this point you really want a either a static IP or a DNS. I decided to buy a domain from No-IP, but this service also offers free (ugly) domains.After you registered your domain and account, go to the "apps" tab and search for "noip" and install. Now navigate to \\[ServerIP]\appdata\No-Ip\ and edit the .config file (I'd recommend Notepad++) and fill in your information. Save, go to the "Docker" tab and start No-IP (might also want it to auto-start), if your information in the .config file is correct it will start, otherwise it won't. 

 

Now you can navigate back to the .ovpn file on your client, edit it with Notepad++ and change the first line to "remote [yourdomain].[tld]".

 

Congratulations, after saving you should now be able to remote into the network where your server is at any time like

 

 

8-Plex

 

Plex is way easier to set up, after you added a container using the plex preset, you are almost good to go. Click the plex-icon under "Docker", click "edit", "Add another path/port/variable" name it however you please, under container path just type /plex or /movies etc. under host path type /mnt/user/[your share]/[folder] give it read/write access and hit "apply". Now start the plex server, click the icon again to open the "web-ui". Login or register, press "add library" and select the folder you specified as /plex or /movies in the last step. Press "next" and "done".

 

Now you can access your movies from everywhere by either getting a plex app or just typing "[yourdomain].[tld]:32400" into your browser.

 

[If you want something similar for mine-os (minecraft server) and or apache (a website), please comment. I think these are pretty niche things]

 

9-Apache

 

So, you want a website and already have your website as a .html file(s), but are wondering how to get it out there? My answer: get a professional web-hosting service. 

But if you really want to host it yourself Apache is a good solution. 

First step: go to your router again and add a forward from port 80 to port 8080 on your server, I can only say it again, do not forward anything to port 80 on your server!

Now go to the "Apps" tab, search for "Apache" and "Apache" by "linuxserver". 

You will be send to a Docker configuration page. Here you need to change "Host Port 1" from 80 to 8080.

Hit "apply", then "done".

Go to the "Docker" tab, click on the Apache icon and press restart. Now wait for the page to reload.

Open explorer and navigate to apache on the "appdata" share. 

Place your .html file(s) directly in the folder "www" and make sure the page people are supposed to land on is named "index.html"

Hit the apache logo in the "Docker" tab again, restart and wait for the page to refresh.

Now you should have a nicely working website!

 

 

10-MineOS

 

To get a (or in fact multiple) Minecraft servers running, you should install MineOS. To to "apps" search for MineOS and install. 

On the docker configuration page change the variable in "key1" to change your password, then hit apply and done.

Go to the "Docker" tab, click the MineOS icon and open the WebUI. If you have a decent browser installed you will now get a warning, pass that warning and login with account name "minecraft" and your password.

Click on "manage profiles" now it depends a bit on what you want to do:

 

If you just want a vanilla server click on "create stock profile" and select your version. Now wait for the page to refresh and press the green "update" button next to the new profile. Wait for the page to refresh again and check that there is a check mark in the "ok" column.

Now click "create server" name it and continue to the next page. Configure to your liking (I'd leave the port at 25565). Hit next again and select "vanilla[game version]" as the profile and hit submit.

You'll be redirected to the "Dashboard", click the name of your server and hit start.

The server will shut down after a few moments, refresh the page and navigate to the server again. Where there used to be the "start server" button, there should now be a "accept EULA" one. Accept the EULA and start your server.

 

If you something more sophisticated, like a forge server click "create custom profile" and use a variation on the following settings

profile name: "forge-installer"
type: standalone, runnable jar
Download URL: http://files.minecraftforge.net/maven/net/minecraftforge/forge/1.11.2-13.20.0.2228/forge-1.11.2-13.20.0.2228-installer.jar
save download as: forge-installer-1-11-2
jarfile to run: forge-installer-1-11-2
jarfile arguments: --installServer

Hit the update button and wait for the refresh and check that the check-mark is there under "ok".

Now click "create server" name it and continue to the next page. Configure to your liking (I'd leave the port at 25565). Hit next again and select "forge-installer" as the profile and hit submit.

You'll be redirected to the "Dashboard", click the name of your server and hit start.

After a few moments the server will shut down again.

Create another profile with a variation of the following settings:

profile name: forge-1-11-2
type: unmanaged
jarfile to run: forge-1.11.2-13.20.0.2228-universal.jar
jarfile arguments: 

Now click the name of your server on the Dashboard again, go to "server.config" and change the first variable to "forge-1-11-2".

Now start your server.

You might need to refresh the page and accept the EULA before the server starts.

 

 

11-backups

 

Now we come to the nice parts: For this you need 2 servers that can see each other (eg over the vpn you set up). The IP of the main server will be 1.2.3.4 the IP of the backup server will be 1.2.3.5 from now on. The folder you want to backup is in the share "share" and is called "important" and the backup shall be stored in the share "backup". 

I will now assume you are working on windows. Open the old control panel, programs, programs and feature. On the left hand side, select "active windows features" and select the "Telnet client".

Hit windows and r, to open the "execute" window, type in "telnet 1.2.3.4" and hit enter. Use "root" as your login. 

Type in:

rsync -avz -e ssh /mnt/user/share/important root@1.2.3.5:/mnt/user/backup

you might need to confirm and enter your credentials now, so this isn't good for regular backups, but a great way to test, whether it works at all.

Assuming that it works, you now want a certificate, so you can back up without prompts whenever you want. So execute the following command:

"ssh-keygen -t rsa -b 2048 -f /root/.ssh/rsync-key"

when promted for a password just hit enter.

scp /root/.ssh/rsync-key.pub root@1.2.3.5:/root/

now switch to the other server with

telnet 1.2.3.5

here you need to make the newly generated key an authorised key (hit enter after each like):

then mkdir .ssh
chmod 700 .ssh
fi 
mv rsync-key.pub .ssh/ 
cd .ssh/ 
if [ ! -f authorized_keys ]; then touch authorized_keys ; chmod 600 authorized_keys ; fi 
cat rsync-key.pub >> authorized_keys 
chmod 700 .ssh/
chmod 600 .ssh/rsync-key.pub
chmod 644 .ssh/authorized_keys

Now go back top the other server:

telnet 1.2.3.4

And check out if it works or not:

rsync -avz -e "ssh -i /root/.ssh/rsync-key" /mnt/user/share/important root@1.2.3.5:/mnt/user/backup

Now you just need to make it sick for the next reboot:

cp /root/.ssh/* /boot/config/sshroot/
cd /root/.ssh
cp /boot/config/sshroot/* /root/.ssh/
chmod 600 *

[Big thanks to tr0910 who wrote this post on the unraid forum, this ssh based solution is way more elegant than my previous one]

Also note, that rsync works both ways: by switching the order of the folders, you copy data the other way

 

 

12-cron

 

So, now you have a simple 1 line solution to back up folders, but you don't want to do it manually, you want the machine to remember. A solution for this is cron.

First step is making sure cron is doing its thing, just execute:

crontab -l

If you see a whole bunch of commented lines roll by, you are good.

 

Open up explorer and navigate to \\1.2.3.4\flash\config\plugins and make a folder called "cron", make a new text file and rename it to "backup.cron" (make sure you override the .txt ending)

I will now assume you want to back up at 3:30 every tuesday, here you can read more on the time format of cron.

30 3 * * 2 rsync -avu --numeric-ids --progress  -e "ssh -i /root/.ssh/rsync-key  -T  -o Compression=no -x "  /mnt/user/share/important  root@1.2.3.5:/mnt/user/backup/

save the file.

 

Now go back into the telnet window and execute:

cd /boot/config/plugins/cron
crontab backup.cron
crontab -l

if it displays the line inside your cron file, you are good.

 

 

But note this will only move new files to the backup share. 

To get the IMO best possible result, jump over to the backup server

telnet 1.2.3.5

and repeat the process for a new cron file on that server called "movebackup.cron" containing the following code:

20 3 * * 2 shopt -s extglob
21 3 * * 2 cd /mnt/user/SyncedShare/
22 3 * * 2 mkdir "Backup-$(date +%Y-%m-%d)"
23 3 * * 2 mv ls !("Backup-"*) "Backup-$(date +%Y-%m-%d)"

This will create a folder with the date of the backup in the name and move the synced files there every week

If there are folders that only contain files that don't change and are too big to reasonable sustain this process (like movies or pictures) create folders for them in the "important" folder and replace the last line in movebackup.cron with:

23 3 * * 2 mv ls !("Backup-"*|movies|pictures) "Backup-$(date +%Y-%m-%d)"

 

Note that from my understanding the "ls" shoudn't be needed, but without it it did not work in my case.

 

 

13-Further reading:

 

Getting started with unraid

ssh rsync setup

cron

a random linux shell tutorial i just googled

 

[Screenshots will be added on request]

[budgidiere]

oh hello...

[ChalkChalkson]

Just now, budgidiere said:

oh hello...

Hi, I fcked up and hit submit  writing in an other tab atm

[budgidiere]

[budgidiere]

It's night in 'Merica... i'll check on this in the morning!

[ChalkChalkson]

Yeah, night in Merica and 5AM in Naziland

[Nikolithebear]

Followed.  Might switch to unraid for my server for the "lols" so we'll see.  

[ChalkChalkson]

I'd be very happy, if you could give some feedback, so that I can make it easier to follow (btw, that took waaaay longer than expected  )

@Nikolithebear @budgidiere

[budgidiere]

i would add more page breaks makes  it easier to read.

 

[budgidiere]

@Slick @LinusTech

[Jarsky]

21 hours ago, ChalkChalkson said:

 

here you need to make the newly generated key an authorised key (hit enter after each like):

then mkdir .ssh
chmod 700 .ssh
fi 
mv rsync-key.pub .ssh/ 
cd .ssh/ 
if [ ! -f authorized_keys ]; then touch authorized_keys ; chmod 600 authorized_keys ; fi 
cat Tower-rsync-key.pub >> authorized_keys 
chmod 700 .ssh/
chmod 600 .ssh/rsync-key.pub
chmod 644 .ssh/authorized_keys

 

there you concatenate Tower-rsync-key.pub but in previous steps you've simply named the key rsync-key.pub 

 

I would also cover User security - so configuring guest accounts as read-only, admin accounts (for read/write) and service accounts for VM's/dockers. 

 

Not really a fan of people port forwarding their NAS storage page in their router either - making it publicly available, leaving your storage open to attack. 

I think ideally what you would want to do is create a seperate linux VM on unRAID for 1) SSH access from external, which you can then jump into CLI of unRAID - and 2) for running your rsync backups from the same physical host (so you only need 1 server).  

[ChalkChalkson]

15 hours ago, Jarsky said:

there you concatenate Tower-rsync-key.pub but in previous steps you've simply named the key rsync-key.pub 

Thanks! 

[ChalkChalkson]

16 hours ago, budgidiere said:

i would add more page breaks makes  it easier to read.

 

how do I add page breaks?

[ChalkChalkson]

15 hours ago, Jarsky said:

I would also cover User security - so configuring guest accounts as read-only, admin accounts (for read/write) and service accounts for VM's/dockers. 

I'll add that

[ChalkChalkson]

15 hours ago, Jarsky said:

Not really a fan of people port forwarding their NAS storage page in their router either - making it publicly available, leaving your storage open to attack. 

I think ideally what you would want to do is create a seperate linux VM on unRAID for 1) SSH access from external, which you can then jump into CLI of unRAID - and 2) for running your rsync backups from the same physical host (so you only need 1 server).  

What are you referring to with "NAS storage page" I literally put "do not forward port 80" in bold and did not list the ports for SMB, FTP and NFS. Unless I missed something the only access to the actual NAS happens via the local network or via the VPN. And even if someone forwards port 20 to work around the vpn, as long as they set up their users correctly no one should be able to edit their data, since the rsync uses ssh.

But if you can tell me what I should add, go ahead, I'll do some more research and add it.

[domandric034]

On 21. 05. 2017. at 3:59 AM, ChalkChalkson said:

 

Steps:

1. What is unraid and how does it work
2. Components
3. installing unraid
4. setting up the shares
5. configuring your router
6. plugins and docker
7. OpenVPN
8. Plex
9. Apache
10. backups
11. crontab
12. further reading

 

@ChalkChalkson Very very informative guide for begginers in unRAID and noobies which got first time home server. Keep it up great work

[Jarsky]

Can we please not quote entire how to posts...

[ChalkChalkson]

33 minutes ago, Jarsky said:

Can we please not quote entire how to posts...

yeah, I think this post isn't suited well for that...

@domandric034 with the @ you can tag people, and you can delete parts of a quoted post, that makes it way easier to read your comment

I'd very much appreciate if you could edit your comment, but thanks for your positive feedback!

[ChalkChalkson]

20 hours ago, budgidiere said:

@Slick @LinusTech

IF (and I wouldn't recommend it) we were to tag anyone here it should be the "Storage and NAS" mod(s), Linus and Luke are pretty busy people, and I assume they know this stuff. And I wouldn't recommend to tag the mods because A: I have no idea who they are, and B: this post is pretty unrefined atm and I hope to get a lot more feedback

[ChalkChalkson]

Ok, now that I added MineOS and Apache, any other ideas?

[ChalkChalkson]

21 hours ago, domandric034 said:

@ChalkChalkson Very very informative guide for begginers in unRAID and noobies which got first time home server. Keep it up great work

thanks for the edit BTW  

[Koitosake]

Yeah this is an amazing guide. Finally got a Job in IT and I'm looking to build an unRaid server for my self as soon as I can afford to do so!

[ChalkChalkson]

19 hours ago, Koitosake said:

Yeah this is an amazing guide. Finally got a Job in IT and I'm looking to build an unRaid server for my self as soon as I can afford to do so!

I am very happy that this guide still finds people to read it  

[ChalkChalkson]

@Alfred14 Wow, people still reading this?

I mean it was meant to hold up for a while...

You got everything from it that you needed?

[Alfred14]

@ChalkChalkson Yeah stumbled upon this thread while researching rsync, i currently have about 10TB worth of data on an unraid server and thinking about setting up a cold backup on another server that i only turn on every now and again because unfortunately i have suffered a severe failure in the past where i lost about 70% of my data so dont want that to happen again!