Defragging a mechanical HDD - How much free space do you need?

[Vectraat]

1.) So, some people, such as the person in this article:

https://www.howtogeek.com/324956/how-much-free-space-should-you-leave-on-your-windows-pc/

Are suggesting 15% free space for a mechanical hard drive defrag.

However, I believe I've heard others say this practice has been irrelevant since Windows Visa, and depending on who I talk to, a little over 200 MB to a few GB is enough to defrag a mechanical hard drive. My OS isn't installed on this HDD, just self-created media files ranging from 4 GB to 80 GB. It's a high-use 10 TB HDD. 

So 15% of that would be 1.5 terabytes. That's an insane amount of storage space I'd be giving up if this were true.

But some are saying if It's not 15-20% free, the HDD will work a lot harder, be slower and the risk of failure will go up.

And I guess Microsoft's own documentation says 15%
https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/defrag

But still, giving up 1.5 terabytes seems kinda insane. So what should I do?

2.) If I recall correctly, MS automatically defrags internal HDDs via the "Optimize Drives" application? My "Scheduled Optimization" is "On" so if I have less than 15% free space, that could cause a problem?

[emosun]

just defrag it as is , if it's just a storage drive then every time it defrags its going to have less and less to do becuase not much has changed between defrags anyways

[Vectraat]

1 minute ago, emosun said:

just defrag it as is , if it's just a storage drive then every time it defrags its going to have less and less to do becuase not much has changed between defrags anyways

I add / delete files a lot throughout the day depending on my work flow. 

[Taf the Ghost]

3 minutes ago, Vectraat said:

I add / delete files a lot throughout the day depending on my work flow. 

It should be able to defrag with very minimal free space, but the main thing is having at least more free space than the largest files you have on the computer. A Defrag is a Copy/Rewrite function because spindle drives have high Continuous Read/Write but really bad Random. Really, really limited space would require a lot more small copy/rewrite actions. You'd probably be fine with 150-200 Gb of free space, given a 10 TB drive. I'd recommend leaving more for performance reasons, but being able to defrag should be doable.

 

I'd also recommend a 1 TB SSD scratch drive. You can get good enough ones for 100-120USD right now.

[Lady Fitzgerald]

Gee, experts recommend 15% (I've always read 10-15%) free space and you are questioning it?  Yes, that is a lot of space to leave free but you still need it. It's something HDDs need to work properly (it's even more for SSDs) so build a bridge and get over it.

[mariushm]

Ideally, you'd want around 10% of disk space free.

I found that with a 4 TB hard drive, anything less than around 10 GB would slow down the defragmenting software a lot. Above that threshold, the speed increase is minimal. So I'd say anything more than around 10 GB would make the defragmenting software very happy.

 

I really recommend trying out O&O Defrag, I found it works great, much better than free defragmenting programs like Defraggler. It's not free, but you can download a 30day trial from their website and it works without limitations: https://www.oo-software.com/en/products/oodefrag

 

Has some smart things in it ... for example, if you're patient and spend around 5-10 minutes, you can configure "zones", basically you can say "this folder contains movies, big static files that won't change or fragment, and they're read sequentially by movie players so access times don't matter much, so move these files to the end of the hard drives"  , "this folder called Steam will change often as games are updated, or I launch games, so keep the files in the middle of the partition, where there would often be free space"

 

 

[Vectraat]

1 hour ago, Lady Fitzgerald said:

Gee, experts recommend 15% (I've always read 10-15%) free space and you are questioning it?  Yes, that is a lot of space to leave free but you still need it. It's something HDDs need to work properly (it's even more for SSDs) so build a bridge and get over it.

Ok, well, let's put this into perspective.
When you spend $400 for a 10 TB HDD that's only 9 TB, then you gotta remove another 1.5 TB. You're left with 7.5 of usable storage. I'm sure you can understand why some people may be irritated by that. 

[Vectraat]

2 hours ago, Taf the Ghost said:

It should be able to defrag with very minimal free space, but the main thing is having at least more free space than the largest files you have on the computer. A Defrag is a Copy/Rewrite function because spindle drives have high Continuous Read/Write but really bad Random. Really, really limited space would require a lot more small copy/rewrite actions. You'd probably be fine with 150-200 Gb of free space, given a 10 TB drive. I'd recommend leaving more for performance reasons, but being able to defrag should be doable.

 

I'd also recommend a 1 TB SSD scratch drive. You can get good enough ones for 100-120USD right now.

You mean, the largest file on that drive--I think that's what you meant to say. So if the largest file on that drive is 80 GB, then you'd want a minimum of 80 GB free space on that HDD. That's MINIMUM. Right now I have about 741 GB free on that drive. 

[RobbinM]

Ahh, the good old TB vs TiB debate. As much as I agree with you here, I'd first want to give you some advice before defragging your drive.

Back it up. Seriously.

[mariushm]

Well, it's not really true.

The size of files doesn't really matter, and you certainly don't need as much free space as the biggest file.

 

Like I said, just have enough free space so that the defragmenting application can quickly move the data in a bunch of clusters to a different area freeing the space to move in another defragmented file. Can be done with as little as 50-100 MB free disk space, but it would be horribly inefficient.  Try to go for at least 5-10 GB of free disk space. 

 

As an example, here's O&O Defrag I recommended above, working on a 1 TB drive with 1 GB free space - it works, but would take 3-4 days to complete (17 days is bad estimation as I just started it)

the yellow squares are all or a part of the fragments belonging to the file currently in process of being defragmented. The software detects the fragments and possibly reads those fragments in memory, then moves data that's located where this file should be repositioned to other parts of the drive, then makes a copy of these yellow segments to the new place, edits the file allocation table to point to new location of those fragments and then "deletes" the old fragments that now don't belong to any file. So at any point, there can be no data loss. 

With little disk space, the defrag tool is forced to constantly look for enough consecutive sectors to move in one shot segments of files. With more disk space, finding a big enough area of free space is much faster, and the program can copy bigger chunks of files (instead of reading in memory only 20-50 of those yellow squares, it can work with 500 at a time)

 

 

[Vectraat]

7 minutes ago, mariushm said:

Well, it's not really true.

The size of files doesn't really matter, and you certainly don't need as much free space as the biggest file.

 

Like I said, just have enough free space so that the defragmenting application can quickly move the data in a bunch of clusters to a different area freeing the space to move in another defragmented file. Can be done with as little as 50-100 MB free disk space, but it would be horribly inefficient.  Try to go for at least 5-10 GB of free disk space. 

 

As an example, here's O&O Defrag I recommended above, working on a 1 TB drive with 1 GB free space - it works, but would take 3-4 days to complete (17 days is bad estimation as I just started it)

the yellow squares are all or a part of the fragments belonging to the file currently in process of being defragmented. The software detects the fragments and possibly reads those fragments in memory, then moves data that's located where this file should be repositioned to other parts of the drive, then makes a copy of these yellow segments to the new place, edits the file allocation table to point to new location of those fragments and then "deletes" the old fragments that now don't belong to any file. So at any point, there can be no data loss. 

With little disk space, the defrag tool is forced to constantly look for enough consecutive sectors to move in one shot segments of files. With more disk space, finding a big enough area of free space is much faster, and the program can copy bigger chunks of files (instead of reading in memory only 20-50 of those yellow squares, it can work with 500 at a time)

 

 

Eh. Is using the Windows "Defragment and Optimize Drives Desktop app" really that bad? I mean, Windows does automatically schedule weekly defrags if I'm not mistaken, so...... if It's already automatically defragging my HDD, why do I want do it manually with another piece of software?

[mariushm]

10 minutes ago, Vectraat said:

Eh. Is using the Windows "Defragment and Optimize Drives Desktop app" really that bad? I mean, Windows does automatically schedule weekly defrags if I'm not mistaken, so...... if It's already automatically defragging my HDD, why do I want do it manually with another piece of software?

It's super basic and slow and kinda dumb.

 

Why do you use Photoshop when you have MS Paint built in? Why use custom video players when you have Media Player built into Windows? Why use Chrome or Firefox when you have IE built in?

 

From experience, O&O Defrag is much faster and also offers some functions I consider useful, like zones (I put movies at the end of the drive, music and game files in the middle, executables and dll files at the start for reduced access time ... and the software can defragment data within these zones and arrange data as I want)

Or for example, if you want, the software can also stay in background (runs as service) and monitors how how files are accessed and how game files are read and after a while, you can run a "defrag/access" and the software can use that info it gathered to put at the start of the drive files that were most often accessed by programs. I don't do that, as I mostly defragment once a month or defragment specific folders as needed.

 

For a while I used Defraggler which works faster than native defrag tool, but it's basic, mostly defragments by sorting files by name and that's about it..and it gives up to quickly... if there's not enough continuous free space it will leave a file fragmented, considering it good enough (had lots of files in 5-10k fragments and Defraggler gave up after reducing to 50 fragments or so)

 

If you're curious why so many fragments, 7zip has a dumb way of writing data to files, especially if you use multiple threads to compress data, so you can end up with 4-10 GB archives in 30k fragments on disk.

[Lady Fitzgerald]

1 hour ago, Vectraat said:

Ok, well, let's put this into perspective.
When you spend $400 for a 10 TB HDD that's only 9 TB, then you gotta remove another 1.5 TB. You're left with 7.5 of usable storage. I'm sure you can understand why some people may be irritated by that. 

 

41 minutes ago, Vectraat said:

Eh. Is using the Windows "Defragment and Optimize Drives Desktop app" really that bad? I mean, Windows does automatically schedule weekly defrags if I'm not mistaken, so...... if It's already automatically defragging my HDD, why do I want do it manually with another piece of software?

You don't. Windows defrag on a schedule wrks good enough. True, there are better defraggers but why bother when good enough is good enough. You don't need a $200 tool when a $10 tool can get the job done adequately.

22 minutes ago, mariushm said:

It's super basic and slow and kinda dumb....

So what. If you set Windows to automatically defrag on a schedule, such as while you are sleeping (if you leave the computer on overnight; mine desktop runs 24/7), it will get done without inconveniencing you any. And, again, good enough is good enough; you don't need the best defragger as long as your frafgmentation is kept below 10%.

[RobbinM]

Zoning looks quite useful actually. Movies can be kept at the end of a disk and system files at the start for best performance.

[LunaP0n3]

There's a good reason why IT experts recommend 15-20% of free space. Defrag utilities use this space as a sorting area.