Rsync --delete command

[Vegardbeider]

Now I am confused. I am using rsync to transfer files from my Unraid server to a Synology NAS. Until now I have not been using the --delete tag. But I see that my backup nas is filling up as move files around and reorganise them.

 

I tried running it with the --delete tag today as a DRY-RUN (with the -n tag) and the log files showes that it want to delete files that still are in the source directory. Can someone explain why this might be happening? I thought the --delete tag only deleted files in the destination that were removed from the source.

 

 

A side question to this I have: Is there a recommended command I can run once in a while to get list of files that exist in the destination but not in the source anymore so I can run it manually so I have som time to check that I still have does in the source so I dont delete anything if it has been accidentally removed?

Thanks!

[Biohazard777]

45 minutes ago, Vegardbeider said:

Now I am confused. I am using rsync to transfer files from my Unraid server to a Synology NAS. Until now I have not been using the --delete tag. But I see that my backup nas is filling up as move files around and reorganise them.

 

I tried running it with the --delete tag today as a DRY-RUN (with the -n tag) and the log files showes that it want to delete files that still are in the source directory. Can someone explain why this might be happening? I thought the --delete tag only deleted files in the destination that were removed from the source.

 

 

A side question to this I have: Is there a recommended command I can run once in a while to get list of files that exist in the destination but not in the source anymore so I can run it manually so I have som time to check that I still have does in the source so I dont delete anything if it has been accidentally removed?

Thanks!

https://linux.die.net/man/1/rsync

https://linux.die.net/man/1/cp

https://linux.die.net/man/1/mv

https://linux.die.net/man/1/rm

[tikker]

From the man pages:

  

Quote

--delete

This tells rsync to delete extraneous files from the receiving side (ones that aren't on the sending side), but only for the directories that are being synchronized. You must have asked rsync to send the whole directory (e.g. lqdirrq or lqdir/rq) without using a wildcard for the directory's contents (e.g. lqdir/*rq) since the wildcard is expanded by the shell and rsync thus gets a request to transfer individual files, not the files' parent directory. Files that are excluded from the transfer are also excluded from being deleted unless you use the --delete-excluded option or mark the rules as only matching on the sending side (see the include/exclude modifiers in the FILTER RULES section).

Prior to rsync 2.6.7, this option would have no effect unless --recursive

was enabled. Beginning with 2.6.7, deletions will also occur when --dirs (-d) is enabled, but only for directories whose contents are being copied.

This option can be dangerous if used incorrectly! It is a very good idea to

first try a run using the --dry-run option (-n) to see what files are going to be deleted.

If the sending side detects any I/O errors, then the deletion of any

files at the destination will be automatically disabled. This is to prevent temporary filesystem failures (such as NFS errors) on the sending side causing a massive deletion of files on the destination. You can override this with the --ignore-errors option.

The --delete option may be combined with one of the --delete-WHEN options

without conflict, as well as --delete-excluded. However, if none of the --delete-WHEN options are specified, rsync will choose the --delete-during algorithm when talking to rsync 3.0.0 or newer, and the --delete-before algorithm when talking to an older rsync. See also --delete-delay and --delete-after.

 

1 hour ago, Vegardbeider said:

I thought the --delete tag only deleted files in the destination that were removed from the source.

Yeah it checks what is in your sending directory, then deletes everything in the receiving directory that is not in that sending directory. The important detail is "must have asked rsync to send the whole directory (e.g. lqdirrq or lqdir/rq) without using a wildcard". From what I gather of you specify a file or * while also specifying delete it'll only consider those, but it's a bit vague on this aspect. This also means if you sync an empty directory with delete, it'll delete everything in the destination directory. As you did already, of course always test with a dry run first.

[Vegardbeider]

40 minutes ago, tikker said:

The important detail is "must have asked rsync to send the whole directory (e.g. lqdirrq or lqdir/rq) without using a wildcard". From what I gather of you specify a file or * while also specifying delete it'll only consider those, but it's a bit vague on this aspect.

So than I think I understand the --delete tag correctly. So either I am misunderstanding the log fil or something else wrong is happening.

 

I use this command: 

rsync -av --stats --rsh "ssh -i "sshfile"" Source HOST:destination

 

does it matter where I place the --delete tag?

[tikker]

26 minutes ago, Vegardbeider said:

So than I think I understand the --delete tag correctly. So either I am misunderstanding the log fil or something else wrong is happening.

 

I use this command: 

rsync -av --stats --rsh "ssh -i "sshfile"" Source HOST:destination

 

does it matter where I place the --delete tag?

I don't think the position of --delete matters as it's just a toggle.

 

I just ran an "rsync -avn --stats --delete source/ dest/" on my machine and it correctly only deletes files not present in source. Maybe it is related to the --rsh option. Is it possible to test without it? Just to be sure, you mean it explicitely says "deleting <file in source>" right?

[Vegardbeider]

15 minutes ago, tikker said:

Maybe it is related to the --rsh option. Is it possible to test without it?

Might be but I did not think so. Due to the location of the servers I have not been able to test without that option. 

 

15 minutes ago, tikker said:

Just to be sure, you mean it explicitely says "deleting <file in source>" right?

Yes it says "deleting" then it points to a file that are still in the source. I have been testing some smaller shares now (with not so important content) and the --delete tag works as expected there. 

 

But if I understand it correctly the --delete tag can only delete files in the destination right? Cuz then I can just test and in worst case I loose the files in my backup and I would just have to run the command again to get it to sync up again.

[tikker]

8 minutes ago, Vegardbeider said:

But if I understand it correctly the --delete tag can only delete files in the destination right? Cuz then I can just test and in worst case I loose the files in my backup and I would just have to run the command again to get it to sync up again.

Yeah --delete should only delete on the receiving side. There is --remove-source-files to delete on the sending side after successful transfer.

8 minutes ago, Vegardbeider said:

Yes it says "deleting" then it points to a file that are still in the source. I have been testing some smaller shares now (with not so important content) and the --delete tag works as expected there. 

Strange. Maybe the --rsh option or something about the paths gets it confused about the actual sending side? Would be odd, but I haven't used rsync's --rsh option.

[Vegardbeider]

1 minute ago, tikker said:

Yeah --delete should only delete on the receiving side. There is --remove-source-files to delete on the sending side after successful transfer.

Ok thank you! 

 

Than I think I will just try to use --delete and see what happens.

[tikker]

Just now, Vegardbeider said:

Ok thank you! 

 

Than I think I will just try to use --delete and see what happens.

Do make a backup if possible. Even if they are less important it sucks to lose stuff.

[Vegardbeider]

Just now, tikker said:

Do make a backup if possible. Even if they are less important it sucks to lose stuff.

Yes I will. 

Think I will test with small dirs first and try it from there.