Howto: create your own linux home server using Debian

[Shaqalac]

The line starting with ARRAY should be appended to the mdadm.conf file in order for mdadm to discover and start the array at boot time. It's weird that the system doesn't allow you to append the line to mdadm.conf by using sudo. You could try logging in as  root by executing `sudo su` and then try to execute

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

again. If it still gives you 'permission denied', there's something really funky going on with your system.

This is what I get. Is it as it should be or were I supposed to get some kind of confirm. If not, is there a way I can check whether the configuration is updated or not?

[alpenwasser]

I'm following this guide and as you can see it says that after you've created your RAID array you should the configuration to the mdadm conf with this command:

sudo mdadm --detail --scan >> /etc/mdadm/mdadm.conf

If I execute that it give me this

-bash: /etc/mdadm/mdadm.conf: Permission denied

 

 

- snip -

Side note: In my experience, running

sudo anycommand >> outputfile

will fail if your regular user does not have permission to write

to the output file. As far as I can tell, the reason for this is that

only the first command is executed with elevated privileges. After

it has been completed, the output redirect (the >> ) is only executed

with your normal user privileges, and if your normal user does not

have write permission to the file to which you are trying to append,

you'll get a premission denied error.

So, you either need to change the file permissions (usually not

adviseable IMHO) or log in as root user and execute all commands

with elevated privileges (as I see you've done).

I'm not absolutely 100% sure on this, so if anyone can find some proper

documentation about the subject free to correct me if I'm wrong or just

post it for the FYI value.

If not, is there a way I can check whether the configuration is updated or not?

Open the file and check if the line has been appended I'd say?

[MG2R]

This is what I get. Is it as it should be or were I supposed to get some kind of confirm. If not, is there a way I can check whether the configuration is updated or not?

 

`tail /etc/mdadm/mdadm.conf` should do the trick

 

EDIT: if such a command doesn't give any feedback, it usually succeeded

[Shaqalac]

Open the file and check if the line has been appended I'd say?

Yeah, I don't know why I didn't think of this... It is Sunday, my brain is relaxing.

 

It has been appended to the configuration file. Thanks.

 

EDIT1:

MG2R

I've managed to format the /dev/md0 with XFS and now I have to mount it and add it to fstab so it will mount each time I boot the system.

To do this I need to create a new directory where I can mount /dev/md0 to. I've chosen to create a new directory - /dev/drivepool.

After that I've added /dev/md0 to /etc/fstab:

<file system> <mount point> <type> <options> <dump> <pass>/dev/md0      /dev/drivepool xfs   default   0      0

The problem is when I reboot the system /dev/drivepool no longer exist. Can't I make new directories to /dev? I'm sure I've seen someone do it in the 10TB show off topic.

 

Just to be completely sure, I have to mount /dev/md0 to a directory, or can I just share the entire drive (/dev/md0). I'm really not interested in making folders within a folder, I rather have my Video-, Music- and Picturefolder right on the drive rather making a ton of folders within drivepool.

RandomNoob's post in the 10TB show off topic shows how the RAID array is shown a drive in Explorer rather than a folder. That's how I want it, to show like a drive.

Edited June 9, 2014 by Shaqalac

[MG2R]

<stuff>

 

You should NOT make files in /dev. Ever. /dev is a directory where all DEVices available on your system are represented. The directory is populated by Linux itself and should not be altered by users. The File Hierarchy Standard states /mnt to be the prefered directory for mounting file systems in. This is where I mount my file systems there as well (/mnt/raid is the mount point for my RAID device).

 

RandomNOOB's setup is very similar to mine. You that by making a directory in /mnt (ex: /mnt/raid). You add

/dev/md0       /mnt/raid        xfs       defaults      0       0

to fstab and then create a share in samba by appending

#======================= Personal Shares =======================[sharename]        comment=The RAID array        path=/mnt/raid        browsable=yes        guest ok=yes        read only=no        create mask=0755

or sth similar to /etc/samba/smb.conf and reloading samba with `sudo service samba reload`.

 

You should now be able to access your XFS filesystem on a windows system at \\servername\sharename

[IdeaStormer]

You should NOT make files in /dev. Ever. /dev is a directory where all DEVices available on your system are represented. The directory is populated by Linux itself and should not be altered by users. The File Hierarchy Standard states /mnt to be the prefered directory for mounting file systems in. This is where I mount my file systems there as well (/mnt/raid is the mount point for my RAID device).

 

RandomNOOB's setup is very similar to mine. You that by making a directory in /mnt (ex: /mnt/raid). You add

/dev/md0       /mnt/raid        xfs       defaults      0       0

to fstab and then create a share in samba by appending

#======================= Personal Shares =======================[sharename]        comment=The RAID array        path=/mnt/raid        browsable=yes        guest ok=yes        read only=no        create mask=0755

or sth similar to /etc/samba/smb.conf and reloading samba with `sudo service samba reload`.

 

You should now be able to access your XFS filesystem on a windows system at \\servername\sharename

 

Yea, /dev is no place for userland items, OS could just wipe you stuff because it thinks its junk.

 

Still you can (should) create your own directory under / for your mounts to keep the path as short as possible and avoid the dreaded /mnt, so you could have your Samba shares under "/data/bla bla" or "/backups/bla bla" making it obvious from either end what/where things are, there is no limit to what you can put under / as long as its not one of the system names under / .

[MG2R]

Yea, /dev is no place for userland items, OS could just wipe you stuff because it thinks its junk.

 

Still you can (should) create your own directory under / for your mounts to keep the path as short as possible and avoid the dreaded /mnt, so you could have your Samba shares under "/data/bla bla" or "/backups/bla bla" making it obvious from either end what/where things are, there is no limit to what you can put under / as long as its not one of the system names under / .

Why would /mnt be dreaded? I hate making extra directories under /.... Makes the whole thing look messy.

[IdeaStormer]

Why would /mnt be dreaded? I hate making extra directories under /.... Makes the whole thing look messy.

 

You can always remove /mnt

[MG2R]

You can always remove /mnt

You can always remove any directory

[IdeaStormer]

You can always remove any directory

 

Yea, you can!

 

The take home message is at least where I've worked, /mnt has always been the temp mount locations as it won't survive a reboot. Anything in /etc/fstab has its own dedicated mount point directory off of /  . This keeps any accidental unmappings to test/use  a temp drive, the more sysadmins you have the more you have to protect yourself.

[MG2R]

The take home message is at least where I've worked, /mnt has always been the temp mount locations as it won't survive a reboot. Anything in /etc/fstab has its own dedicated mount point directory off of /  . This keeps any accidental unmappings to test/use  a temp drive, the more sysadmins you have the more you have to protect yourself.

Well, yeah, it's a matter of protocol either way, but I get where you're coming from. I'd still make a folder /persistent or similar where all persistent filesystems are located. I can't stand a clogged up root, I like to keep it as FHS-compliant as possible.

[alpenwasser]

Yea, you can!

 

The take home message is at least where I've worked, /mnt has always been the temp mount locations as it won't survive a reboot. Anything in /etc/fstab has its own dedicated mount point directory off of /  . This keeps any accidental unmappings to test/use  a temp drive, the more sysadmins you have the more you have to protect yourself.

 

 

Well, yeah, it's a matter of protocol either way, but I get where you're coming from. I'd still make a folder /persistent or similar where all persistent filesystems are located. I can't stand a clogged up root, I like to keep it as FHS-compliant as possible.

Yeah, I tend to use /mnt for temporary mounts only as well. I

don't really have any additional directories in / though, usually

I put them in a specific user's home dir. For public stuff that

needs to be shared and doesn't belong to a specific user, I then

have special public users (say, a user "media" for sharing films,

TV shows and so on on the network, and if need be more users for

other purposes).

I pondered for quite a while until I reached this approach, read

up on best practices and so on. I also considered additional dirs

in /, but I don't really like my root system to be cluttered,

and so far my files have been organized in such a fashion that

my current approach leads to sensible results. If (when?) I

encounter a scenario where that's no longer the case, I'll probably

go with additional directories under /.

[Shaqalac]

I've finally set up the share and are connected from my Win7 machine.

 

I went with making directories under /, so this is how it looks

/dev/md0     7.3T     34M     7.3T     1%     /Storage

For future expansion I'll be making /Backup and /TimeMachine.

I do however how a problem with write, I can't create any folders from my Win7, it says "You need permission to perform this action".

My appended this section to /etc/samba/smb.conf

[FileServer]comment = RAID array /dev/md0path = /Storagebrowsable = yesguest ok = yesread only = nocreate mask = 0755

I did some search and found this site.

The first thing I noticed was that Linux permissions overrule Samba permissions, which makes sense. Before I started to work with Linux permissions I tried a few things with Samba:

• writeable = yes
• write list = marc
• create mask = 0775

None of this worked, so I figured I need to play with the Linux permissions.

 

If I execute this command

ls -ld /Storagedrwxr-xr-x  2  root  root  6  Jun  8  16:47  /Storage

it shows that I should be able to read, write and execute as owner, which is what logged onto the server as from my Win7.

 

What am I doing wrong?

[MG2R]

-- snip --

AFAIK, Samba performs actions to the share as your user, which wouldn't allow it to write that directory, as it does not have write permission set for others.

The way I have my share mounted at home is just with rwx for everyone, but that might not be the best idea if you have a multi-user setup (which I don't).

Another way you could alleviate the problem is to create a group for people that have write access to the drive.

[Shaqalac]

AFAIK, Samba performs actions to the share as your user, which wouldn't allow it to write that directory, as it does not have write permission set for others.

The way I have my share mounted at home is just with rwx for everyone, but that might not be the best idea if you have a multi-user setup (which I don't).

Another way you could alleviate the problem is to create a group for people that have write access to the drive.

The plan is that this will act as media server for me (mainly), my dad and my 2 younger brothers.

I'm about to move to my own apartment soon, so I'll have to figure out some non-local sharing, thinking of Plex, anyway that's another problem for another time.

The setup I wan't to run is me as owner with rwx permissions and I'm thinking of adding my dad and brothers to a group called family and give them r-x permissions.

Primarily I only want me to have rwx, so if anything goes wrong I know for sure that my dad and brothers didn't have permissions to mess with anything.

Edited June 16, 2014 by Shaqalac

[Shaqalac]

I think I found the solution

sudo chmod -R 755 /Storage

At least, I think that was the command that did the trick. Last night I tried a couple of things and after I had run a command I reconnected to the system thinking that relogging from my Win7 to the NAS it would refresh/update my settings. That wasn't the case, so I ran a couple of commands last night and today after a proper restart the permissions have been changed. I'm able to rwx and my family members are allowed to r-x.

 

Before I start populating the RAID array I think it would be smart to do some kind of test/benchmark, especially when I've had so many problems with a bricked controller etc.

What have you done to stress test your system and check transfer speeds or what would you recommend?

 

EDIT:

I don't know if that could be anything worth to add the guide, a post installation guide. I'm thinking of something corresponding to my questions above and similar to what Linus have added to their recent build guides.

Edited June 17, 2014 by Shaqalac

[IdeaStormer]

Your problem was the group and owner are root. At least have the group be family.

[MG2R]

Before I start populating the RAID array I think it would be smart to do some kind of test/benchmark, especially when I've had so many problems with a bricked controller etc.

What have you done to stress test your system and check transfer speeds or what would you recommend?

 

Testing sequential R/W speed:

#Test write speed:dd if=/dev/zero of=/path/to/raid/somefilename bs=64M count=100#Test read speed: ('if' is the file you wrote with the prev command)dd if=/path/to/raid/somefilename of=/dev/null

You could test some drive failure scenarios by unplugging drives at random, afterwards note the steps you need to take to rebuild the array, so you know how it is done in case of an actual emergency.

If you want to be safe about it, you can shut down the server, unplug a drive, and boot again. Or, if you like to live on the edge, you can do like I did and just yank a SATA data cable during operation

[Shaqalac]

Testing sequential R/W speed:

These are my results. I don't really know where I should go look if they are good or bad, does it look reasonable to you guys?

 

You could test some drive failure scenarios by unplugging drives at random, afterwards note the steps you need to take to rebuild the array, so you know how it is done in case of an actual emergency.

If you want to be safe about it, you can shut down the server, unplug a drive, and boot again. Or, if you like to live on the edge, you can do like I did and just yank a SATA data cable during operation

The test of how to rebuild the array is a pretty good idea. I think I'll skip the other exercises

[MG2R]

These are my results. I don't really know where I should go look if they are good or bad, does it look reasonable to you guys?

Very good speed! Could you check your CPU utilization while writing the file? I'm interested!

 

For reference: my array gets about 130 - 140 MB/s writes.

[Shaqalac]

Very good speed! Could you check your CPU utilization while writing the file? I'm interested!

Now you mention it I am too. I'm not quite sure how I check the CPU utilization while writing the file though.

 

For reference: my array gets about 130 - 140 MB/s writes.

I can only say I'm extremely happy with my results then. With all the trouble I've had with the controllers it's nice to get some kind of approval that it's all working just fine.

[Shaqalac]

I want to install Transmission to my Debian machine and I want to be able to monitor and manage it from my Windows machine through a browser window. So obviously I need to setup a webserver on my Debian machine.

 

I've only installed apache2 as it looks like it's the only package I need. It seems to work as if I type in 192.168.1.49 I will get the "It works!" message across the screen.

Next step is to make it more secure. I've followed all the instructions to the part where I'm going to configure the server to redirect from http to https. I'm able to enter https://lianlinas and https://192.168.1.49 and I'll get to this page.

You might notice that there's a cross at the lock, it's because the certificate hasn't been verified.

 

I have a problem or two.

 

First one is when I restart or reload apache2 I'll get this error:

Restarting web server apache2apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName... waiting apache2:Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName

I found this discussion from askubuntu with a solution. I'm pretty sure I'm supposed to use the suggested solution for 13.10 or newer but when I try to safe my ServerName localhost (LianLiNAS 127.0.0.1) to

sudo vim /etc/apache2/conf-available/servername.conf

I'm getting an error: "E212: Cannot open file for writing". I'm guessing this is because the conf file doesn't exist, can I create conf file in /etc/apache2/conf-available/ called servername.conf just like that?

 

Next problem.

As mentioned earlier I'm at the point where I need to configure so people will be redirected to https.

When I run

sudo cp /etc/apache2/sites-available/000-default /etc/apache2/sites-available/000-default.bak

as mentioned in the part Making your webserver more secure I'll get this message:

cp: cannot stat `/etc/apache2/sites-available/000-default´: No such file or directory

Obviously the file doesn't exist but from the guide it seems like that the file should be there or exist?

 

 

One last question.

Anyone got a good guide to set up Transmission? I've found this but whether it's good or not I don't know.

 

 

Cheers Shaqalac.

[MG2R]

I want to install Transmission to my Debian machine and I want to be able to monitor and manage it from my Windows machine through a browser window. So obviously I need to setup a webserver on my Debian machine.

Actually, Transmission has its own built-in web server, no Apache needed

 

 

First one is when I restart or reload apache2 I'll get this error:

Restarting web server apache2apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName... waiting apache2:Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName

I found this discussion from askubuntu with a solution.

 

That warning basically means that Apache didn't find any ServerName directive defined and it can't find any reverse DNS records for your IP (which is logical, as your IP is local). Neither Ubuntu nor Debian define a ServerName in the default Apache2 package, because they don't know what your server's name is. You can ignore this warning if you want, but if you want it resolved, simply create a file under /etc/apache2/conf.d and type "ServerName yourservername" without quotation marks in that file. For example:

sudo vim /etc/apache2/conf.d/servername# type your server's name in the file, save and closesudo service apache2 restart

I'm getting an error: "E212: Cannot open file for writing". I'm guessing this is because the conf file doesn't exist, can I create conf file in /etc/apache2/conf-available/ called servername.conf just like that?

The folder /etc/apache2/conf-available doesn't exist, that's why you get that error

 

 

cp: cannot stat `/etc/apache2/sites-available/000-default´: No such file or directory

Obviously the file doesn't exist but from the guide it seems like that the file should be there or exist?

Apparently, in newer versions of Apache2, they've renamed the site to 'default'. Thanks for the heads-up. Consider it fixed.

 

 

One last question.

Anyone got a good guide to set up Transmission?.

 

I've been saying forever that I'd be doing a Transmission guide. Let me write that up for you real quick. It's quite easy after all...

[MG2R]

Installing a torrent client (Transmission)

Installing a torrent client with web UI is quite simple. Basically all you need to do is install the package 'transmission-daemon', that will install the Transmission client and a web UI on your machine.

sudo apt-get install transmission-daemon

After doing that, you should be able to visit http://192.168.1.150:9091 and be greeted with this message:

 

 

You're seeing that because, by default, transmission-daemon only allows requests from localhost. We can change this by editing the settings. We do this by editing the settings.json file.

Be sure to shut down transmission before altering settings.json, transmission-daemon overwrites that file on reboot!

simon@brol3:~$ sudo service transmission-daemon stop[ ok ] Stopping bittorrent daemon: transmission-daemon.simon@brol3:~$ sudo cat /etc/transmission-daemon/settings.json{    "alt-speed-down": 50,    "alt-speed-enabled": false,    "alt-speed-time-begin": 540,    "alt-speed-time-day": 127,    "alt-speed-time-enabled": false,    "alt-speed-time-end": 1020,    "alt-speed-up": 50,    "bind-address-ipv4": "0.0.0.0",    "bind-address-ipv6": "::",    "blocklist-enabled": false,    "blocklist-url": "http://www.example.com/blocklist",    "cache-size-mb": 4,    "dht-enabled": true,    "download-dir": "/var/lib/transmission-daemon/downloads",    "download-limit": 100,    "download-limit-enabled": 0,    "download-queue-enabled": true,    "download-queue-size": 5,    "encryption": 1,    "idle-seeding-limit": 30,    "idle-seeding-limit-enabled": false,    "incomplete-dir": "/root/Downloads",    "incomplete-dir-enabled": false,    "lpd-enabled": false,    "max-peers-global": 200,    "message-level": 2,    "peer-congestion-algorithm": "",    "peer-limit-global": 240,    "peer-limit-per-torrent": 60,    "peer-port": 51413,    "peer-port-random-high": 65535,    "peer-port-random-low": 49152,    "peer-port-random-on-start": false,    "peer-socket-tos": "default",    "pex-enabled": true,    "port-forwarding-enabled": false,    "preallocation": 1,    "prefetch-enabled": 1,    "queue-stalled-enabled": true,    "queue-stalled-minutes": 30,    "ratio-limit": 2,    "ratio-limit-enabled": false,    "rename-partial-files": true,    "rpc-authentication-required": true,    "rpc-bind-address": "0.0.0.0",    "rpc-enabled": true,    "rpc-password": "{3c15cfea495e0f4d6f318475c99094118f136f62zqmr.N0h",    "rpc-port": 9091,    "rpc-url": "/transmission/",    "rpc-username": "transmission",    "rpc-whitelist": "127.0.0.1",    "rpc-whitelist-enabled": true,    "scrape-paused-torrents-enabled": true,    "script-torrent-done-enabled": false,    "script-torrent-done-filename": "",    "seed-queue-enabled": false,    "seed-queue-size": 10,    "speed-limit-down": 100,    "speed-limit-down-enabled": false,    "speed-limit-up": 100,    "speed-limit-up-enabled": false,    "start-added-torrents": true,    "trash-original-torrent-files": false,    "umask": 18,    "upload-limit": 100,    "upload-limit-enabled": 0,    "upload-slots-per-torrent": 14,    "utp-enabled": true}

Above, you can see the settings.json file. To be able to connect to the web UI, you will need to either disable the whitelist by setting "rpc-whitelist-enabled" to false, or by adding your IP to the "rpc-whitelist".

 

You can also set the system so it asks for a password to access the web UI. You do this by making sure "rpc-enabled" is set to true, you can set username and password in the "rpc-username" ans "rpc-password" directives.

Feel free to alter other settings as well (download dirs, for example).

Save and close. Start transmission-daemon.

sudo service transmission-daemon start

For reference, this settings.json file should allow any IP to connect to the web UI and require username 'simon' and password 'passwd' to login:

note that the password will be hashed when transmission-daemon starts. This means that you don't need to lock down read access to the file, as the password will not be stored in plain text.

simon@brol3:~$ sudo cat /etc/transmission-daemon/settings.json{    "alt-speed-down": 50,    "alt-speed-enabled": false,    "alt-speed-time-begin": 540,    "alt-speed-time-day": 127,    "alt-speed-time-enabled": false,    "alt-speed-time-end": 1020,    "alt-speed-up": 50,    "bind-address-ipv4": "0.0.0.0",    "bind-address-ipv6": "::",    "blocklist-enabled": false,    "blocklist-url": "http://www.example.com/blocklist",    "cache-size-mb": 4,    "dht-enabled": true,    "download-dir": "/var/lib/transmission-daemon/downloads",    "download-limit": 100,    "download-limit-enabled": 0,    "download-queue-enabled": true,    "download-queue-size": 5,    "encryption": 1,    "idle-seeding-limit": 30,    "idle-seeding-limit-enabled": false,    "incomplete-dir": "/root/Downloads",    "incomplete-dir-enabled": false,    "lpd-enabled": false,    "max-peers-global": 200,    "message-level": 2,    "peer-congestion-algorithm": "",    "peer-limit-global": 240,    "peer-limit-per-torrent": 60,    "peer-port": 51413,    "peer-port-random-high": 65535,    "peer-port-random-low": 49152,    "peer-port-random-on-start": false,    "peer-socket-tos": "default",    "pex-enabled": true,    "port-forwarding-enabled": false,    "preallocation": 1,    "prefetch-enabled": 1,    "queue-stalled-enabled": true,    "queue-stalled-minutes": 30,    "ratio-limit": 2,    "ratio-limit-enabled": false,    "rename-partial-files": true,    "rpc-authentication-required": true,    "rpc-bind-address": "0.0.0.0",    "rpc-enabled": true,    "rpc-password": "{719994fa4805aaf16f03d9f522d9e076dc84dd7bFxm7vtZg",    "rpc-port": 9091,    "rpc-url": "/transmission/",    "rpc-username": "simon",    "rpc-whitelist": "127.0.0.1",    "rpc-whitelist-enabled": false,    "scrape-paused-torrents-enabled": true,    "script-torrent-done-enabled": false,    "script-torrent-done-filename": "",    "seed-queue-enabled": false,    "seed-queue-size": 10,    "speed-limit-down": 100,    "speed-limit-down-enabled": false,    "speed-limit-up": 100,    "speed-limit-up-enabled": false,    "start-added-torrents": true,    "trash-original-torrent-files": false,    "umask": 18,    "upload-limit": 100,    "upload-limit-enabled": 0,    "upload-slots-per-torrent": 14,    "utp-enabled": true}

 

You should be able to reach the web UI now.

 

 

Back to index

[ionbasa]

Am I the only one who was disappointed at your lack of using LVM :angry: ?

 

Other than that I would have liked to have seen LVM groups and volumes used instead. But that's just me...

 

Also, incase anyone needs extra help: http://www.howtoforge.com/