Data corruption

[ggkfox]

i always hear linux talk about backing up, raid, and just overall protection in case of hard drive failure. however I never hear anything about file corruption. I have tons of music files on an HDD, and over the years a good 10% of those files have become corrupted from moving from copying from drive to drive. overall i think that HDDs are ******, but i want to store a ton more data than just music (web server, music, movies, roms) and this is just wayyyyy too much to be spending on SSDs. is there a way to prevent backing up of already corrupted data? 

 

TLDR: is there a way to verify data integrity (no corruption) before i back it up to another drive? im scared of backing up corrupted data. (and don't say raid, it just duplicates the corrupted data im sure)... thanks btw!

[ggkfox]

not quite what i asked for. and regardless, it still happened.

i want to ensure nothing like this ever happens to my movies, so if anyone other than arika knows of a way to help prevent corruption of any way, i would appreciate it. 

[homeap5]

5 minutes ago, ggkfox said:

not quite what i asked for (...)

It's true, but what you're asked for is question like "what to prevent my computer to move from one place to another by themselve".

Data integrity is part of hard drive hardware / firmware and filesystem. If your data is corruted, then probably your harddrive is bad. Not like bad sector bad, just damaged electronically. Linus in some of episodes talk about professional data recovery and explain about unique electronic chips for every drive and how sensitive it is so you cannot just, for example, move that electronic print circut board even from the same model of hardware to another.

 

Anyway - method is to have second hard drive and makes backups. Just like that. From one working HDD to another.

[manikyath]

this sounds like some exceptional bad luck, or a form of hardware issues that wouldnt be solved by SSDs.

 

there are storage formats "like RAID but not RAID" that supposedly can pick up on data getting corrupted while stored, and correct it, but i never bothered to look into it because the only files i've ever had corrupted were fished from dead or dying storage devices.

[Electronics Wizardy]

12 minutes ago, ggkfox said:

not quite what i asked for. and regardless, it still happened.

i want to ensure nothing like this ever happens to my movies, so if anyone other than arika knows of a way to help prevent corruption of any way, i would appreciate it. 

Look up par files, they can check for and fix this http://www.quickpar.org.uk/

 

But normally this is a bad hdd or system(bad ram). This won't just randomly happen. Also some file systems like zfs, btrfs, and refs can check for and fix these errors. Also hdds and ssds have their own error protection.

[ggkfox]

ok thanks