mhddfs only writing to first drive

[PhilPC]

Ubuntu 13.04 (GNU/Linux 3.8.0-19-generic x86_64)

 

here is my /etc/fstab 

# /etc/fstab: static file system information.## Use 'blkid' to print the universally unique identifier for a# device; this may be used with UUID= as a more robust way to name devices# that works even if disks are added and removed. See fstab(5).## <file system> <mount point>   <type>  <options>       <dump>  <pass># / was on /dev/sda6 during installationUUID=eccfb5b3-f6fd-461b-bdc1-74b71f618b1c /               ext4    errors=remount-ro 0       1# swap was on /dev/sda5 during installationUUID=c76e45aa-6cdb-4152-b179-6150ceb0ff7a none            swap    sw              0       0#/dev/sda1UUID=5580a7c5-b59e-4523-bf9f-169c9193d8a8       /home/phil/harddrives/sda1    ext4    defaults    0    0# /dev/sdb1UUID=cbaa00c3-d1bb-4522-8523-c13b7b92a097       /home/phil/harddrives/sdb1    ext4    defaults    0    0#/dev/sdc1UUID=63baad7c-c12a-42b9-9ae2-f36ac90e7798       /home/phil/harddrives/sdc1    ext4    defaults    0    0#/dev/sdd1UUID=8c63d06b-31b4-4b8c-9ad5-7f1d31b06653       /home/phil/harddrives/sdd1    ext4    defaults    0    0#/dev/sde1UUID=e7ca4a1b-4675-4774-a983-06c4d7c98818       /home/phil/harddrives/sde1    ext4    defaults    0    0#/dev/sdf1UUID=0caec458-2264-4d2b-a9e0-32fc6c9316c7       /home/phil/harddrives/sdf1    ext4    defaults    0    0mhddfs#/home/phil/harddrives/sdb1,/home/phil/harddrives/sdc1,home/phil/harddrives/sdd1,/home/phil/harddrives/sde1  /home/phil/storagepool fuse defaults,allow_other 0 0

All drives appear to be mounted correctly, users phil and root can read & write to all drives listed. Here's the ls -l for the drives mounted to home/phil/hardrives

total 24drwxr-xr-x 2 phil root 4096 Aug  3 20:47 sda1drwxr-xr-x 4 phil root 4096 Aug  7 16:23 sdb1drwxr-xr-x 4 phil root 4096 Aug  7 16:17 sdc1drwxr-xr-x 3 phil root 4096 Aug  7 00:19 sdd1drwxr-xr-x 4 phil root 4096 Aug  7 00:19 sde1drwxr-xr-x 2 phil root 4096 Aug  3 20:45 sdf1

I believe the filesystems provide the possibility to get their parameters correctly, because of my df -h output

Filesystem                                                                                                   Size  Used Avail Use% Mounted on/dev/sda6                                                                                                     52G  2.5G   47G   6% /none                                                                                                         4.0K     0  4.0K   0% /sys/fs/cgroupudev                                                                                                         3.9G  4.0K  3.9G   1% /devtmpfs                                                                                                        797M  1.9M  796M   1% /runnone                                                                                                         5.0M     0  5.0M   0% /run/locknone                                                                                                         3.9G     0  3.9G   0% /run/shmnone                                                                                                         100M     0  100M   0% /run/user/dev/sda1                                                                                                     55G   52M   52G   1% /home/phil/harddrives/sda1/dev/sdb1                                                                                                    1.8T  1.6T  110G  94% /home/phil/harddrives/sdb1/dev/sdc1                                                                                                    1.8T  146M  1.7T   1% /home/phil/harddrives/sdc1/dev/sdd1                                                                                                    1.8T  146M  1.7T   1% /home/phil/harddrives/sdd1/dev/sde1                                                                                                    459G  148M  435G   1% /home/phil/harddrives/sde1/dev/sdf1                                                                                                    2.7T  1.6T  973G  63% /home/phil/harddrives/sdf1/home/phil/harddrives/sdb1;/home/phil/harddrives/sdc1;/home/phil/harddrives/sdd1;/home/phil/harddrives/sde1  5.9T  1.6T  4.0T  29% /home/phil/storagepool

So what is strange is that anything I write in /storagepool, goes directly to /sdb1, rather than any of the other drives with more free space. I have tested this through the command-line with touch, over FTP, and also using samba.

 

I also have snapraid doing snapshots at irregular intervals, though I doubt it's relevant, here's my snapraid.conf

#Parity Format: "parity FILE_PATH"parity /home/phil/harddrives/sdf1/snapraidparity# q-Parity Format: "q-parity FILE_PATH"#q-parity /mnt/diskqar/q-parity# Content Format: "content FILE_PATH"content /home/phil/harddrives/sdb1/contentcontent /home/phil/harddrives/sdc1/contentcontent /home/phil/harddrives/sdd1/contentcontent /home/phil/harddrives/sde1/content# The order is relevant for parity, do not change it# Data Format: "disk DISK_NAME DISK_MOUNT_POINT"disk d1 /home/phil/harddrives/sdb1/disk d2 /home/phil/harddrives/sdc1/disk d3 /home/phil/harddrives/sde1/disk d4 /home/phil/harddrives/sdd1/# Excludes hidden files and directories (uncomment to enable).# nohidden# Defines files and directories to exclude# Remember that all the paths are relative at the mount points# Format: "exclude FILE"# Format: "exclude DIR/"# Format: "exclude /PATH/FILE"# Format: "exclude /PATH/DIR/"exclude *.bakexclude *.unrecoverableexclude /tmp/exclude /lost+found/include /raidredemption/# Defines the block size in kibi bytes (1024 bytes).# Default value is 256 -> 256 kibi bytes -> 262144 bytes# Format: "block_size SIZE_IN_KiB"block_size 256# Automatically save the state when synching after the specied amount# of GiB processed.# This option is useful to avoid to restart from scratch long 'sync'# commands interrupted by a machine crash.# The SIZE argument is specified in gibi bytes -> 1073741824 bytes# Default value is 0, meaning disabled.# Format: "autosave_size SIZE_IN_GiB"# autosize 500# Defines the pooling directory where the virtual view of the disk# array is created using the "pool" command.# The files are not really copied here, but just linked using# symbolic links.# This directory must be outside the array.# Format: "pool DIR"# pool 

I guess it's possible that the content files could be confusing mhddfs as they all have the same name, however that wouldn't explain why mhddfs is happy enough to write in /sdb1

[MG2R]

What happens if you write to the storage pool beyond the capacity of a single drive? It could very well be that mhddfs works a lot like JBOD (haven't used the former), where it just fills up drives sequentially.

[PhilPC]

What happens if you write to the storage pool beyond the capacity of a single drive? It could very well be that mhddfs works a lot like JBOD (haven't used the former), where it just fills up drives sequentially.

 

I actually have no idea, I'm up to 94% of the first drive, I'll have a go at duplicating some legally obtained, G-rated movies until I fill it to see what happens.

 

btw, thankyou for your debian home server walkthrough, your posts on this forum have been a massive help, and you've really eased the learning curve for linux noobs like myself. 

[MG2R]

I actually have no idea, I'm up to 94% of the first drive, I'll have a go at duplicating some legally obtained, G-rated movies until I fill it to see what happens.

 

You can use dd to write some garbage data to the pool as well, if you want:

dd if=/dev/zero of=/mount/point/of/your/pool/somefilename bs=50M count=10000

Make sure you put the right parameters in for if (input file) and of (output file).

As you may have guessed, /mount/point/of/your/pool/ is the place where you right data to your pool, make sure you don't forget somefilename!! You can name it differently, but if you don't put something there, you'll overwrite your whole storage pool.

/dev/zero is a virtual device that just outputs all zero's.

This command should write 10000 times 50MB of zero's to a file called somefilename in you storage pool mounting pount.

You might need to run this as super user, because you are working with a device (/dev/zero).

 

Again, make sure to put the correct parameters in! dd is an awesome tool, but it can be quite dangerous.

 

Alternatively, you can indeed just duplicate some movies.

 

 

btw, thankyou for your debian home server walkthrough, your posts on this forum have been a massive help, and you've really eased the learning curve for linux noobs like myself.

Thank you! It means a lot to me that someone actually uses it

[PhilPC]

Okay, so contrary to the first google result for mhddfs, it turns out that it writes to the first drive, until it gets to the preset max-size, then writes to the next drive. If you edit a file, that becomes larger than what could fit on a drive, it will simply move it to the next drive with free space. I guess this is okay, so the problem never existed, I just got my info from an unreliable source.

[MG2R]

Okay, so contrary to the first google result for mhddfs, it turns out that it writes to the first drive, until it gets to the preset max-size, then writes to the next drive. If you edit a file, that becomes larger than what could fit on a drive, it will simply move it to the next drive with free space. I guess this is okay, so the problem never existed, I just got my info from an unreliable source.

So it's like JBOD. The cool thing is that using that technique ensure that you can still read all of your data even if you tak a drive out of the pool

[PhilPC]

whilst your here, my utorrent 64-bit 3.0 build 27079 for linux (used it to legally share cake recipes with my peers, nothing unscrupulous here), for some reason resets to default settings when I kill it (I've tried editing settings from the web GUI and the utsettings.conf).
Do you have any input on this? I'm kind of at a loss.

[MG2R]

whilst your here, my utorrent 64-bit 3.0 build 27079 for linux (used it to legally share cake recipes with my peers, nothing unscrupulous here), for some reason resets to default settings when I kill it (I've tried editing settings from the web GUI and the utsettings.conf).

Do you have any input on this? I'm kind of at a loss.

Is there a 'proper' way to exit the program? If so, try using that instead of blatantly killing it. I'm not familiar with uTorrent on Linux, but there are a number of ways Linux programs usually get stopped:

• If you start it by issuing /etc/init.d/utorrent start, try issuing /etc/init.d/utorrent stop . Of course, utorrent may be named differently.
• If you start it by issuing /path/to/utorrent [start], with [start] being optional, try issuing /path/to/utorrent stop .
• Try checking the web GUI for an option to stop the service.

Whilst looking it up, I noticed that this software is still in alpha. Keep that in mind, it might just be a "feature" ;)

 

Can you try the following:

1. kill the service
2. edit utsettings.conf with the settings of your choice
3. make utsettings.conf only writeable by root, but readable by others (chown to root and chmod to 644)

This might fix it, as uTorrent shouldn't be able to overwrite the json file (and with it the settings) as long as you don't start it as root (i.e. don't do sudo utorrent)

[PhilPC]

Thanks for your suggestions, but I ended up just getting fed up and moved to transmission. you wouldn't have any suggestions for a decent linux usenet client with a web ui, would you? (I also moved to debian because a friend kept roasting me for using ubuntu)

[MG2R]

Thanks for your suggestions, but I ended up just getting fed up and moved to transmission. you wouldn't have any suggestions for a decent linux usenet client with a web ui, would you? (I also moved to debian because a friend kept roasting me for using ubuntu)

I find Debian quite a bit better than Ubuntu these days.

 

Usenet clients: SABnzdb+ works perfectly, you might also be interested in Sickbeard, which you can use in combination with SABnzdb+ for automated series organisation