My Mini-Offsite Backup Rig

[Bigun]

I've built an offsite backup rig for my 3 TB media center.  I'll share this here in case someone wants something similar, also just to brag to someone.

 

It consists of three main components:

 

1. The Raspberry Pi (Model 1B with LCD shield display and case)
2. The powered USB hub
3. External 4 TB drive

The whole system put together.

My home media center connects offsite to the Pi via a secure shell (SSH), and performs a linux "rsync" command.  Upon success, the media center deposits a day and time into a file on the Pi, a script reads the file, and the Pi displays the contents on the LCD display.

 

The buttons have been programmed to cycle through several statistics (as well as colors).

 

1. Last Sync
2. Free space on drive
3. RAM usage
4. CPU statistics
5. This mode will automatically cycle through the above stats

As an added bonus, a fifth select button can be held down to shutdown the pi.

 

I've let this system run for a few years (as evidenced by the model B pi) and it's been rock solid.

 

If anyone is interested, I'd be happy to post the code and shopping URL's so anyone could build their own.
 

Assumptions:

1. This is a linux environment pushing backup to a linux environment, I'm sure this could be modified to backup a windows environment, but that is outside the scope of this how-to... for now

Shopping list:

1. Some type of Raspberry Pi, I used a Model 1 B, but I don't think it matters
2. Adafruit 16x2 LCD Shield (optional, the rig will work without it, although without it, there is no quick way to verify backup worked - without some kind of display)
3. Powered USB hub, recommended 2 port minimum
4. External hard drive to handle the capacity needed
5. A case (also optional) to handle the Pi and shield combo

Tools:

1. Soldering equipment (the LCD shield will need to be soldered together)
2. An internet connection with ability to forward at least one port and enough bandwidth to handle the throughput.  Preferably with a static IP, unless you want to mess with DynDNS.

Preparation:

1. Assemble the shield
2. Download and install Rasbian, the Lite version will be fine, be sure to set the password before continuing
   1. Also be sure to update the Pi's software before going forward.  There was a very important update that fixes a bug between the Pi and the I2C library that the shield uses.
3. Format the drive into the desired format
4. Setup fstab to use the sync and rw options to disable caching
5. Assign a static IP to Pi
6. Forward an internet connection port to the static IP of the Pi (default is 22, however I recommend using literally anything else for security reasons)

The Setup:

1. Every install of linux will generate it's own unique SSH key.  This step will get your target's key (the Pi), and add it to the source so that the source will not need a password to connect to the Pi.  Follow the instructions here, and substitute computer A as your source, and B as the Pi.  Be sure to test and confirm source machine can login to the Pi without a password.
2. Create a script on your source machine with the following content

   1. if ! ps -A | grep rsync ; then	
          if rsync -vaz --progress --delete -e "ssh -p <your port>" /mnt/<source path> pi@<public Pi IP Address>:/mnt/<Pi path to drive> ; then
          	    date +%a\ %b\ %d\ %R\ %Y >/tmp/last_sync && rsync -vaz --progress --delete -e "ssh -p <your port>" /tmp/last_sync pi@<public Pi IP Address>:~/last_sync
      	fi
      else
      	echo 'Offsite sync still running....'
      fi

      This script will be doing the actual sync, and if it was successful, dump the current date and time into a file, and sync it to the Pi for it to read to the display.  Be sure to set permissions to allow the script to be executed.

3. On the source machine, as a user with access to your media, setup your cron schedule

   1. 0 2 * * * <path to script>

      You can (of course) adjust the schedule to fit your taste

4. Create a directory for the last_sync file to live on the pi

   1. mkdir ~/last_sync

       

5. If you choose not to use a shield, your done.  Just log in to check results, or setup e-mail alerts through crontab if you want to have the output e-mailed to you.

 

LCD Shield Setup:

1. Install the screen utility:  sudo apt-get install screen
2. SSH into pi and install the I2C libraries, instructions on Adafruit's website
3. Setup Pi to autoload I2C module - see here
4. On the target, create a python script named plate_stats.py in the pi's home directory with the following content
   1. A note, I did not make the script, I simply butchered the example script supplied by Adafruit to get what I needed
5. Open the cron schedule on the target as the Pi user (crontab -e)
   1. Add the following line:  @reboot screen -d -m sudo python /home/pi/<path>/plate_stats.py
6. Reboot the Pi

LCD Shield Use:

1. Left and right buttons cycle through colors
2. Up and down buttons cycle through statistics
3. Select button refreshes displayed data
4. Holding down select button will shutdown the pi
5. The shield will auto-refresh data every 30 minutes

Final Notes:

1. This walkthrough obviously assumes some familiarity with Linux, I'm sure me and a lot of users on here will assist if you decide to move forward with the project
2. All of the scripts have a spot or two that will need to be customized to fit your environment
3. I am human, I may have missed a step or two, it's been a few years since I've set all this up.  If you find a mistake, let me know, and I'll fix the original post and credit you for the fix.
4. It may be more sane to do the initial sync locally rather than offsite.  Use your judgement based on the amount of data and throughput present.

Good luck!

 

Change Log:

1. 8/24/2016 - Modified python script to have a mode that cycles through all the stats without touching anything.
2. 8/26/2016 - De-cluttered some of the scripts by assigning some static paths, the last_sync file is now located on the Pi's local SD-Card drive.
3. 9/15/2016 - Added a check to make sure sync isn't running before trying to start it again

 

[tikker]

That's cool! Maybe I'll do something like this as well and finally have my Pi do something useful again after a long time

[MrDrWho13]

5 minutes ago, tikker said:

That's cool! Maybe I'll do something like this as well and finally have my Pi do something useful again after a long time

My pi (Model B version 1) hasn't done anything useful ever 

Looks cool though.

[Bigun]

On 8/15/2016 at 11:06 AM, tikker said:

That's cool! Maybe I'll do something like this as well and finally have my Pi do something useful again after a long time

Let me know if you decide to, I'll post a how-to... or something.

 

This has been done

[Harvey-Specter]

13 minutes ago, Bigun said:

Let me know if you decide to, I'll post a how-to... or something.

I would definitely be interested in seeing a how-to for this

[Bigun]

1 hour ago, Harvey-Specter said:

I would definitely be interested in seeing a how-to for this

10-4, I'll edit the original post.

 

*edit*

 

Done

[Heesleemer]

very nice use of a Pi, very low powered too! 

I may do this as a secondary back up for just pictures and what not. 

[tikker]

4 hours ago, Bigun said:

10-4, I'll edit the original post.

 

*edit*

 

Done

Cool. I'll save it for future reference.

[Nicnac]

saved this one as well. have a Raspi 2 Mod B lying around somewhere....

[Bigun]

Added a feature to have the backup drive not mounted by default.  Should help with flushing the cache upon umounting in case of power failure offsite, also should help with power consumption, allowing the remote drive to sleep if it has the feature.

[Bigun]

Ok, I've ran into a bug.  Keeping the drive unmounted makes it impossible to read the free drive space on the backup drive.  Is showing free space important?  Or having the drive unmounted in an attempt to save power?

 

*edit*

 

Looks impossible to check free space of a filesystem without said filesystem mounted.  Modified scripts to handle drive being mounted (again) and added an option to the fstab config to disable caching to make things a bit safer.

[Bigun]

Modified the push script to check to see if previous sync is still running before trying to start the offsite sync process again.