Which files are not duplicated

[drRodneyMcKay]

Hi,

There is several hundred images. I have categorized some of them into directories.

However, due to recent back ups, i have a directory wich containes every photo that our familiy has ever made.

I need to find out which images has NOT been categorised yet. How can i achieve this?

[Slottr]

You can write a script that will take the file name and search a directory for  matching data

 

if you're on UNIX as your title suggests, theres about a million ways to do it

[mariushm]

Use a tool or something to generate a md5 or sha256 hash for each file that's categorized.

Repeat for your other folder.

 

make a list with the hashes and paths to file, one for each folder.

 

Compare lists and remove duplicates.

 

 

[D_Lan]

You could use a CLI program like fdupes to scan the directory they are in. Like mariushm suggested, this program does do a hash comparison. Quick warning, the -d flag will delete the duplicates. So don't use it unless you really really want to. I have omitted it in the examples. Check out the links for more on fdupes.

 

Install:

sudo apt-get install fdupes

Example Command:

fdupes --recursive --sameline /path/to/photos

You can also direct the output to a file with:

fdupes --recursive --sameline /path/to/photos > /home/$USER/fdupes_out.txt

 

fdupes tutorial: https://www.tecmint.com/fdupes-find-and-delete-duplicate-files-in-linux/

fdupes documentation: https://linux.die.net/man/1/fdupes