help about linux disks

[apoyusiken]

so i have a few disks with a few bad sectors (thats what crystal disk info said on windows) but qdiskinfo says no bad sectors?

 

i have disk with ntfs but i will convert it to ext4.

 

a few times while moving some files it bugged out and i lost the files.

 

also ive noticed linux is kinda wonky with files. it is hard to play partially downloaded videos and if an app is working on a directory the other app seems not to be able to?

 

so the question is, is there anything critical i should know or do regarding this?

[Alex Atkin UK]

2 hours ago, apoyusiken said:

so i have a few disks with a few bad sectors (thats what crystal disk info said on windows) but qdiskinfo says no bad sectors?

 

i have disk with ntfs but i will convert it to ext4.

 

a few times while moving some files it bugged out and i lost the files.

 

also ive noticed linux is kinda wonky with files. it is hard to play partially downloaded videos and if an app is working on a directory the other app seems not to be able to?

 

so the question is, is there anything critical i should know or do regarding this?

There's nothing I'm aware of that should cause bugging out to lose the files.  Like any other modern OS if they haven't been moved to the destination they should remain on the source, unless the source and/or destination drive is failing in some way.  Bad sectors is not necessarily a sign of bad drive, its how many and how quickly they are occurring that signals a failing drive.

 

Ability to play partially downloaded files depends on how much and which parts of the file you have, plus which application you are using to play them.  It heavily relies on the start of the file being complete but depending on the container (mkv, ts, avi, etc) this can impact your chances greatly too.  This isn't really different to Windows though as far as I recall.

 

What exactly do you mean by "working on a directory"?  There are certainly some cases where the file manager you are using might get locked up while the drive is busy, and other apps may use the same API as that file manager so also end up waiting on the task to finish.  This should not necessarily lock out anything not using that API, such as a console window.

[OddOod]

32 minutes ago, Alex Atkin UK said:

Bad sectors is not necessarily a sign of bad drive, its how many and how quickly they are occurring that signals a failing drive.

Fully agree. Though, I will say, almost every time I've had bad sectors, replacing the SATA cable has solve the issue. YMMV

[apoyusiken]

1 hour ago, Alex Atkin UK said:

What exactly do you mean by "working on a directory"?  There are certainly some cases where the file manager you are using might get locked up while the drive is busy, and other apps may use the same API as that file manager so also end up waiting on the task to finish.  This should not necessarily lock out anything not using that API, such as a console window.

i was moving some files and jellyfin couldnt see the new files until the operation was complete (i was moving unrelated files) (its another hdd than my main ssd so maybe this is whats up cuz when i was moving files to my phone on windows i couldnt access the folders either)

[apoyusiken]

1 hour ago, OddOod said:

Fully agree. Though, I will say, almost every time I've had bad sectors, replacing the SATA cable has solve the issue. YMMV

yea but the issue is i had bad sectors on windows and linux doesnt see them, wtf is up? @Alex Atkin UK

[Alex Atkin UK]

2 hours ago, apoyusiken said:

yea but the issue is i had bad sectors on windows and linux doesnt see them, wtf is up? @Alex Atkin UK

Jellyfin likely has a delay as you don't want it doing constant rescans while you are still copying files over as it would just slow everything down.  It will also need the files to be complete before it can index, as it needs to generate the meta data, scan for intro, credits, etc.  I have my Plex server set to never automatically refresh the library as I found it more reliable to do so manually when I know I have finished adding new content.

 

Bad sectors are handled by the drive and reported in SMART.  May be the Linux util is not reporting the right field.  A lot of SMART data is only relevant to the manufacturers health utility and can be misinterpreted by third party software.

[OddOod]

1 hour ago, Alex Atkin UK said:

I have my Plex server set to never automatically refresh the library as I found it more reliable to do so manually when I know I have finished adding new content.

One cool thing that you can do with Sonarr/Radarr is have it handle the file moving, then, when it's done moving the files it will actually hand Plex/JF a mostly completed metadata file and tell it to scan for intro/credits etc. 

 

[apoyusiken]

3 hours ago, Alex Atkin UK said:

Bad sectors are handled by the drive and reported in SMART.  May be the Linux util is not reporting the right field.  A lot of SMART data is only relevant to the manufacturers health utility and can be misinterpreted by third party software.

doesnt the drive avoid bad sectors?

[NoLeafClover]

1 hour ago, apoyusiken said:

doesnt the drive avoid bad sectors?

Yes and no. Drives will have a limited amount of spare sectors. When a bad sector is identified, the drive's controller will attempt to "fix" it the next time data is written onto it. If it succeeds, the sector wasn't bad, rather corrupted. If forcing a write does not "repair" the sector, then the controller will remap the bad one to one of the spare ones. Subsequent read/write commands to the original sector's address will then point to the remapped one. There's probably a bit more complexity in the actual logic, but the gist of it remains the same.

 

The amount of spare sectors varies, but is usually in the order of a few hundred. When that gets exhausted, future bad sectors cannot be remapped. A drive that has a stable number of remapped sectors is likely in okay health. An increase in the number of remapped sectors is generally an early sign of failure.

 

You can use a tool such as smartctl (command line) or, for example, KDE's SMART Status tool to get information about the current number of remapped sectors. You can also use smartctl to look at the controller's log and observe any listed errors, such as failed read / write commands at a given LBA. Drive self-test can also be initiated, in fact I would recommend that you run both a short (~2-5 min) and extended (1+ hours depending on drive capacity) to see if the controller itself will be able to detect and correct any issues. Take note of the remapped sector count before and after tests.

 

Edit: I assume this concerns a traditional spinning platter hard drive, not an SSD. SSDs can also have "bad sectors" and there's a similar process in place, but it's a bit more nuanced as the controller will be doing a lot more remapping for wear-levelling. Permanently failing "sectors" in SSDs are eventually caused by general wear as drives have a limited number of rewrite cycles that the memory cells can handle and can be used as a metric to how far through its usable lifespan a drive is.