Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

 A few weeks ago, I had a catastrophic failure with my Linux laptop that was caused by none other than my own stupid self, by issuing a command functionally similar to a sudo rm -rf --no-preserve-root, deleting everything under / seemingly unrecoverably (I tried a couple different backup utilities, and none of them worked). Anyway, I've moved on and reinstalled my OS, and I've come to terms with the (albeit relatively insignificant) data loss I've suffered. This time, however, I want to use a robust backup solution (something I completely lacked before). I've decided I want my script to rsync my home folder (pretty much all I really need) to my NAS and do incremental daily backups, but full weekly backups (so folders would be stored 2018-%w where %w is the week number in the year). That part is relatively straightforward but I have a few more goals that I'd like to accomplish with this script, and I'm not sure how to go about them:

 

1) Transfer files using an SSH destination to my server without having to log in or enter the password on my private key (because I might be away from my computer when the script runs)

2) Run the script while the laptop is sleeping/suspended (because chances are the script will run late at night when I'm sleeping, and I don't want to have to wake up the laptop just to run it). Is there a cronjob that can override suspension and proceed anyway? or an alternative that will accomplish the same goal of always having a backup for that day? Even if I don't use the computer that day?

3) Keep track of files over, say, 10gb that are unchanged and only back them up once (Since I'm only making a new folder every week, I'm referring to the previous week's folder) to save disk space.

Spoiler

My main desktop, "Rufus":

Spoiler

PC Specs:

CPU: AMD Ryzen 5 1600

CPU Cooler: Cooler Master MasterLiquid Lite 120

RAM: 2x8gb Corsair Vengence DDR4 Red LED @ 3066mt/s

Motherboard: MSI B350 Gaming Pro Carbon

GPU: XFX RX 580 GTR XXX White 

Storage: Mushkin ECO3 256GB SATA3 SSD + Some hitachi thing

PSU: Seasonic Focus Plus Gold 650W

Case: Corsair Crystal 460X

OS: Windows 10 x64 Pro Version 1607

Retro machine:

Spoiler

PC Specs:

CPU: Intel Core 2 Quad Q9550

CPU Cooler: Stock heatsink

RAM: GSkill 4gb DDR2 1066mt/s

Motherboard: Asus P5n-e SLI

GPU: 8800 GTS 640mb, I swap between that and my 8800 GTS 512mb

Storage: Seagate 320gb right from 2006

PSU: Ultra 600W 

Case: Deepcool Tesseract SW

OS: Windows XP SP3 32-bit, Linux Mint 18.2 Cinnamon 64-bit, Manjaro Deepin x64 (sorta)

Mac Pro Early 2008: Dual Xeon X5482s w/ 32GB RAM & HD 5770 running macOS High Sierra

More PC's

 

Link to post
Share on other sites

Well.... to answer your question. edit your crontab (crontab -e) and add.

 

@daily rsync -az /home/user/ /mnt/backup

 

Each user has their own individual crontab you can do this on whatever user you want that preformed on. (note the / on the end of /home/user/)

 

but.......

 

I'd recommend maybe duplicity (deja dup in Gnome) as the rsync command is just a straight copy.. it isn't an incremental.

"Only proprietary software vendors want proprietary software." - Michael Dexter

Link to post
Share on other sites
1 minute ago, jde3 said:

Well.... to answer your question. edit your crontab (crontab -e) and add.

@daily rsync -az /home/user /mnt/backup

 

Each user has their own individual crontab you can do this on whatever user you want that preformed on.

 

but.......

 

I'd recommend maybe duplicity (deja dup in Gnome) as the rsync command is just a straight copy.. it isn't an incremental.

Will a cronjob in a crontab run even when the system is sleeping?

Spoiler

My main desktop, "Rufus":

Spoiler

PC Specs:

CPU: AMD Ryzen 5 1600

CPU Cooler: Cooler Master MasterLiquid Lite 120

RAM: 2x8gb Corsair Vengence DDR4 Red LED @ 3066mt/s

Motherboard: MSI B350 Gaming Pro Carbon

GPU: XFX RX 580 GTR XXX White 

Storage: Mushkin ECO3 256GB SATA3 SSD + Some hitachi thing

PSU: Seasonic Focus Plus Gold 650W

Case: Corsair Crystal 460X

OS: Windows 10 x64 Pro Version 1607

Retro machine:

Spoiler

PC Specs:

CPU: Intel Core 2 Quad Q9550

CPU Cooler: Stock heatsink

RAM: GSkill 4gb DDR2 1066mt/s

Motherboard: Asus P5n-e SLI

GPU: 8800 GTS 640mb, I swap between that and my 8800 GTS 512mb

Storage: Seagate 320gb right from 2006

PSU: Ultra 600W 

Case: Deepcool Tesseract SW

OS: Windows XP SP3 32-bit, Linux Mint 18.2 Cinnamon 64-bit, Manjaro Deepin x64 (sorta)

Mac Pro Early 2008: Dual Xeon X5482s w/ 32GB RAM & HD 5770 running macOS High Sierra

More PC's

 

Link to post
Share on other sites
Just now, panther420 said:

Will a cronjob in a crontab run even when the system is sleeping?

no. anacron can take care of that but.. again this is probably not what you want to do for a proper backup.

"Only proprietary software vendors want proprietary software." - Michael Dexter

Link to post
Share on other sites
Just now, jde3 said:

no. anacron can take care of that but.. again this is probably not what you want to do for a proper backup.

Alright, not to be rude but I already said I figured out how to do a basic script such as the one you suggested where it just copies the files. You haven't really answered any of my three questions, which was what I was asking about. I'm also not looking for advice unless you believe dejadup or duplicity will accomplish all three of the things I wanted up there. Besides, I prefer sticking to CLI tools when I can. And rsync is sufficient for my needs, maybe I have the definition of an incremental backup wrong but all I need is a script that will copy my home folder to a directory of the week and update it daily, until the next week runs around. And that doesn't need to be debated.

Spoiler

My main desktop, "Rufus":

Spoiler

PC Specs:

CPU: AMD Ryzen 5 1600

CPU Cooler: Cooler Master MasterLiquid Lite 120

RAM: 2x8gb Corsair Vengence DDR4 Red LED @ 3066mt/s

Motherboard: MSI B350 Gaming Pro Carbon

GPU: XFX RX 580 GTR XXX White 

Storage: Mushkin ECO3 256GB SATA3 SSD + Some hitachi thing

PSU: Seasonic Focus Plus Gold 650W

Case: Corsair Crystal 460X

OS: Windows 10 x64 Pro Version 1607

Retro machine:

Spoiler

PC Specs:

CPU: Intel Core 2 Quad Q9550

CPU Cooler: Stock heatsink

RAM: GSkill 4gb DDR2 1066mt/s

Motherboard: Asus P5n-e SLI

GPU: 8800 GTS 640mb, I swap between that and my 8800 GTS 512mb

Storage: Seagate 320gb right from 2006

PSU: Ultra 600W 

Case: Deepcool Tesseract SW

OS: Windows XP SP3 32-bit, Linux Mint 18.2 Cinnamon 64-bit, Manjaro Deepin x64 (sorta)

Mac Pro Early 2008: Dual Xeon X5482s w/ 32GB RAM & HD 5770 running macOS High Sierra

More PC's

 

Link to post
Share on other sites

Uh.. well umm..

 

1. rsync can use ssh so.. yeah just look into man rsync for the options for it, it works basically like scp.

3. rsync is a delta copy so it will never copy a unmodified file.

 

So you're asking kind of the wrong questions.

"Only proprietary software vendors want proprietary software." - Michael Dexter

Link to post
Share on other sites
6 minutes ago, jde3 said:

Uh.. well umm..

 

1. rsync can use ssh so.. yeah just look into man rsync for the options for it.

3. rsync is a delta copy so it will never copy a unmodified file.

 

So your asking kind of the wrong questions.

Again, you haven't answered my questions. 

1. I'm well aware of the fact that it can use ssh, wouldn't it need me to enter a password or the password on my key? How would I do that if I'm not at the computer?

3. This one is turning out to be kind of a pain in the ass and your answer also doesn't tell me anything I didn't already know so I think I'm just going to exclude all files over a certain size limit upfront and copy big files manually (I won't have a lot of them on this laptop.)

Spoiler

My main desktop, "Rufus":

Spoiler

PC Specs:

CPU: AMD Ryzen 5 1600

CPU Cooler: Cooler Master MasterLiquid Lite 120

RAM: 2x8gb Corsair Vengence DDR4 Red LED @ 3066mt/s

Motherboard: MSI B350 Gaming Pro Carbon

GPU: XFX RX 580 GTR XXX White 

Storage: Mushkin ECO3 256GB SATA3 SSD + Some hitachi thing

PSU: Seasonic Focus Plus Gold 650W

Case: Corsair Crystal 460X

OS: Windows 10 x64 Pro Version 1607

Retro machine:

Spoiler

PC Specs:

CPU: Intel Core 2 Quad Q9550

CPU Cooler: Stock heatsink

RAM: GSkill 4gb DDR2 1066mt/s

Motherboard: Asus P5n-e SLI

GPU: 8800 GTS 640mb, I swap between that and my 8800 GTS 512mb

Storage: Seagate 320gb right from 2006

PSU: Ultra 600W 

Case: Deepcool Tesseract SW

OS: Windows XP SP3 32-bit, Linux Mint 18.2 Cinnamon 64-bit, Manjaro Deepin x64 (sorta)

Mac Pro Early 2008: Dual Xeon X5482s w/ 32GB RAM & HD 5770 running macOS High Sierra

More PC's

 

Link to post
Share on other sites

I'm not going to tell you how to do it wrong. Your implementation you want to create is awful. There is no reason to contently re-create the same data. You are trying to write a script to create a incremental backup solution from scratch when many already exist.. anyone ever tell you not to re-invent the wheel?  ssh does not need a password on a private key.

 

I gave you my advice on the problem.. use duplicity, it's a cli very similar to rsync.

 

@daily duplicity /home/user scp://user@host/backup

"Only proprietary software vendors want proprietary software." - Michael Dexter

Link to post
Share on other sites
10 hours ago, jde3 said:

I'm not going to tell you how to do it wrong. Your implementation you want to create is awful. There is no reason to contently re-create the same data. You are trying to write a script to create a incremental backup solution from scratch when many already exist.. anyone ever tell you not to re-invent the wheel?  ssh does not need a password on a private key.

 

I gave you my advice on the problem.. use duplicity, it's a cli very similar to rsync.

 

@daily duplicity /home/user scp://user@host/backup

Hey, I've been toying around with duplicity a little bit and I think you're right, it would be an easier solution that will manage everything for me. I'll probably just make one directory that does incremental backups forever (Is that a problem? Should you create full backups every so often? Is a week too much?). Sorry for being so harsh earlier.

Spoiler

My main desktop, "Rufus":

Spoiler

PC Specs:

CPU: AMD Ryzen 5 1600

CPU Cooler: Cooler Master MasterLiquid Lite 120

RAM: 2x8gb Corsair Vengence DDR4 Red LED @ 3066mt/s

Motherboard: MSI B350 Gaming Pro Carbon

GPU: XFX RX 580 GTR XXX White 

Storage: Mushkin ECO3 256GB SATA3 SSD + Some hitachi thing

PSU: Seasonic Focus Plus Gold 650W

Case: Corsair Crystal 460X

OS: Windows 10 x64 Pro Version 1607

Retro machine:

Spoiler

PC Specs:

CPU: Intel Core 2 Quad Q9550

CPU Cooler: Stock heatsink

RAM: GSkill 4gb DDR2 1066mt/s

Motherboard: Asus P5n-e SLI

GPU: 8800 GTS 640mb, I swap between that and my 8800 GTS 512mb

Storage: Seagate 320gb right from 2006

PSU: Ultra 600W 

Case: Deepcool Tesseract SW

OS: Windows XP SP3 32-bit, Linux Mint 18.2 Cinnamon 64-bit, Manjaro Deepin x64 (sorta)

Mac Pro Early 2008: Dual Xeon X5482s w/ 32GB RAM & HD 5770 running macOS High Sierra

More PC's

 

Link to post
Share on other sites
5 minutes ago, panther420 said:

Hey, I've been toying around with duplicity a little bit and I think you're right, it would be an easier solution that will manage everything for me. I'll probably just make one directory that does incremental backups forever (Is that a problem? Should you create full backups every so often? Is a week too much?). Sorry for being so harsh earlier.

No, no problem. It does it's own versioning. When you restore or list a restore you can choose a point in time. (if it has it based on it's config)

"Only proprietary software vendors want proprietary software." - Michael Dexter

Link to post
Share on other sites
11 hours ago, jde3 said:

No, no problem. It does it's own versioning. When you restore or list a restore you can choose a point in time. (if it has it based on it's config)

Cool. I'll double check the config and just roll with it. Thanks!

Spoiler

My main desktop, "Rufus":

Spoiler

PC Specs:

CPU: AMD Ryzen 5 1600

CPU Cooler: Cooler Master MasterLiquid Lite 120

RAM: 2x8gb Corsair Vengence DDR4 Red LED @ 3066mt/s

Motherboard: MSI B350 Gaming Pro Carbon

GPU: XFX RX 580 GTR XXX White 

Storage: Mushkin ECO3 256GB SATA3 SSD + Some hitachi thing

PSU: Seasonic Focus Plus Gold 650W

Case: Corsair Crystal 460X

OS: Windows 10 x64 Pro Version 1607

Retro machine:

Spoiler

PC Specs:

CPU: Intel Core 2 Quad Q9550

CPU Cooler: Stock heatsink

RAM: GSkill 4gb DDR2 1066mt/s

Motherboard: Asus P5n-e SLI

GPU: 8800 GTS 640mb, I swap between that and my 8800 GTS 512mb

Storage: Seagate 320gb right from 2006

PSU: Ultra 600W 

Case: Deepcool Tesseract SW

OS: Windows XP SP3 32-bit, Linux Mint 18.2 Cinnamon 64-bit, Manjaro Deepin x64 (sorta)

Mac Pro Early 2008: Dual Xeon X5482s w/ 32GB RAM & HD 5770 running macOS High Sierra

More PC's

 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×