What does defragmentation do?

[Darell3]

I defraged my HDD and it shaved 300mb of running processes?

 

ANYWAYS, What does it actually do? and why is it bad for SSD's?

[Guest]

It moves stored files around on mechanical drives to increase speed. I can't remember if it's closer to the centre or further away but mechanical drives perform best when the data is stored on one of those extremes. Defragmenting will move data to those areas to increase speed and performance.

 

This will have no effect on an SSD because SSD's are not mechanical. 

[unidentical]

It moves stored files around on mechanical drives to increase speed. I can't remember if it's closer to the centre or further away but mechanical drives perform best when the data is stored on one of those extremes. Defragmenting will move data to those areas to increase speed and performance.

 

This will have no effect on an SSD because SSD's are not mechanical. 

I'm pretty sure it's closer to the center.

 

Defragmenting does have an effect on SSDs but it's negative. Flash memory has a finite number of writes so when you defrag you're basically just wearing down that life expectancy. Also, flash doesn't immediately erase, it just removes the reference to the data, so as it's moving around it's leaving chunks of data all over the place that then need to be cleaned up by trim, or overwritten when you add more data, which also has a negative impact on performance.

[pcman2000]

Say [D] is pieces of a specific, large file, and [O] is all other data.

 

The data is stored on your HDD like this;

 

x [D][O][O][O][D][O][D][O][O][O][D][D] y

 

Once you defrag it, it looks like this;

 

x [D][D][D][D][D][D][O][O][O][O][O][O] y

 

This way, your HDD does not need to spin from place x to place y to somewhere inbetween to read the entire file. Instead, it can just read in one go, sequentially.

 

On a SSD, this is not needed as it does not take any time to go from x to y than to anywhere inbetween, as there is no spinning disk and moving head.

[CouchSloth]

When you start out with a fresh formatted hard drive, as you write files to the drive they are all kept together in close location to each other, By doing this the hard drive can simply move its reading heads small amounts to read sequential files. When you delete a file gaps are left scattered across the hard drive. As the hard drive fills up Windows will place new files in these blank spaces, but by doing so files that belong to the same program are now scattered across the hard drive. When it comes time to load these files, as they are now scattered across the drive the read heads need to travel further between reading each of the files. This takes longer and also gives you the mechanical 'crunching' sounds that you can sometimes hear on older drives. This scattering of files and data is called fragmentation.

 

By de-fragging your hard drive, the program attempts to sort out your files into a more logical order as weel as removing the small gaps. The idea is to leave all vacant space in one location on the disk so then next time you install a new program there is a large enough room to sequentially store the files. If you are frequently removing (un-installing programs) then your HDD is more likely to fragment more.

[CouchSloth]

Re-reading your question, with SSD's they have a limited number read and write cycles. Far less than a traditional HDD. By de-fragging an SSD you are chewing into those limited read write cycles. Plus an SSD is much faster than an HHD so any speed improvements as a result of de-fragging will be less than that of a HDD.

[SteepGnomeKing]

Well everyone has answered it so far. Puts the files closer together on the disk towards the centre (so less path of travel) of the HDD so it can be read faster without it having to move around reading "fragmented" files at different parts of the disk. SSD's don't need this because it erodes the drive because you can only have so many reads/writes and defragging the drive is heavy on moving files reading from point A and writing to part B. Plus an SSD can read from more than one part of a drive at a time so it doesn't matter if the files are fragmented because it can still read everything it needs at the same time.

[Lukiose]

Just to give a simple way of looking at it:

Defragmentation is like sorting out a file, where you put similar articles / topics in the same section for easy and quick referencing without needing to sort through the whole pile to find the document you need. 

[bradscoolio]

Say [D] is pieces of a specific, large file, and [O] is all other data.

 

The data is stored on your HDD like this;

 

x [D][O][O][O][D][O][D][O][O][O][D][D] y

 

Once you defrag it, it looks like this;

 

x [D][D][D][D][D][D][O][O][O][O][O][O] y

 

This way, your HDD does not need to spin from place x to place y to somewhere inbetween to read the entire file. Instead, it can just read in one go, sequentially.

This is completely true^

 

But also imagine your HDD Is a spinning disk and then think how if the read head moves above the disk.

 

If the read head is located on the outer edge of the disk the data will be read a faster speed because a larger surface area is moving underneath the read head.

 

A = center of the platter = less performance

B = outer edge of the platter = best performance

 

Defragging will not only re organize the files so they're close together and the read head doesn't have to move very far but it will also move large & commonly used files on the outer edge of the platter and smaller and rarely used files to the inner edge of the platter.

 

Like this:

[Lukiose]

This is completely true^

 

But also imagine your HDD Is a spinning disk and then think how if the read head moves above the disk.

 

If the read head is located on the outer edge of the disk the data will be read a faster speed because a larger surface area is moving underneath the read head.

 

A = center of the platter = less performance

B = outer edge of the platter = best performance

 

Defragging will not only re organize the files so they're close together and the read head doesn't have to move very far but it will also move large & commonly used files on the outer edge of the platter and smaller and rarely used files to the inner edge of the platter.

 

Like this:

Great stuff!

[GoodBytes]

Ok there is some wrong stuff being said here, but also right.

 

Let me explain:

 

To understand what is defragmentation, you must first know how an HDD works when writing data.

For speed reasons, HDD writes any information you create at the first blank space it finds from where the head current position is at. When it writes information, it will eventually come to an area where there is already data stored. So it continues what it needs to write at the next available open spot. It has a system to know where each peace of data is located on the disk.

This system is great, as it makes writhing data very efficient and fast. However, as you saw, it splits data into parts. This slows down the reading process, as the head needs to get all the little peaces to have the file in full to give to the program. This makes interacting with your programs and files go slow, especially if it's cut into many little peaces.

Defragmentation will move files to empty space, and regroup files that where once split, into 1 block.

 

To better understand pcman2000 diagram:

Imagine that your entire hard drive is made of this:

 

[V][V][V] {[V]} [V][ ][ ][ ][F][F][F][ ][ ][ ][ ][ ][ ][ ]

 

Where V, is a video file, F is a normal file.. lets say text file, and S is some operating system file.

The empty space is free available location to put data (frees space).

The { } represent the location of the head of the hard drive.

 

You want to download this kick ass Music you purchase.. let's call it M.

M is made of [M][M][M][M][M][M][M] (7x [M]).

 

So this is what happens once

 

1- The head goes to the first available spot, closest to it:

[V][V][V][V][V] {[ ]} [ ][ ][F][F][F][ ][ ][ ][ ][ ][ ][ ]

 

2- It writes the Music files.

[V][V][V][V][V][M][M][M] {[F]} [F][F][ ][ ][ ][ ][ ][ ][ ]

Oh wait, we only put 3 M's, out of 7.

 

3- We the HDD now skips over the F's, to the next available free spot and continues to write the Music file. The split of the file, is recorded, but for simplicity sakes, let's ignore this, and assume it's super smart.

[V][V][V][V][V][M][M][M][F][F][F][M][M][M][M] {[ ]} [ ][ ]

 

4- Done. But now the Music file is cut into 2 parts, in this example. This means, as the hard drive is mechanical, it will take more time for the hard drive to read the music file, instead of being all in 1 block.

 

This is fragmentation. Defragmentation, regroups the parts of files into 1 block. So:

[V][V][V][V][V] [M][M][M][M][M][M][M] [F][F][F] [ ][ ][ ]

 

 

As for location of defragmentation of file, this varies based on the defragmentation program you use. Each defragmentation program have their own algorithm.

 

Some defrag program will put anything related to the Windows startups, one after another, from the beginning from the disk, so that the head doesn't need to move, and can just read all the files it need continuously in a row. And, as it's the beginning of the disk inside the HDD (so closest to the center), the heads read data faster, as you have more rotation from the disk that occurs. And then it position other data and program in blocks right after. O&O Defrag Complete process does this. Hence when you do this "Complete" defrag mode, it can take well over 8h. But then Windows boots in a flash compared to before. When I had Windows 7 with an HDD. I had Windows boot up, to the desktop with all background process loaded in 19sec.

 

Other defrag programs, simply puts data in the middle as it defrags. It makes the defrag process much quicker, as it doesn't need to shift data around, constantly, and give good performance results.

 

Others defrag porgrams, does what bradscoolio diagram shows.

 

It all depends. Each program has their own algorithm.

[bradscoolio]

Other defrag programs, simply puts data in the middle as it defrags. It makes the defrag process much quicker, as it doesn't need to shift data around, constantly, and give good performance results.

 

Others defrag porgrams, does what bradscoolio diagram shows.

 

It all depends. Each program has their own algorithm.

 

You're extremely knowledgeable in technology, I aspire to have equal or greater knowledge than you do after I get my Computer Science degree. You're a great addition to the forum :)

 

I have a little question for you. Can you get a defrag program that allows you to choose what algorithm you want? I think if someone were to make a program that could do that they'd make a lot of money.

[GoodBytes]

Well, I have deifnitly not tried all the defrag software that exists. But I do know that O&O Defrag, gives you 3 defrag options which they call

"Optimize", "Optimize/quick", and "Optimize/Complete". Where Optimize/Quick does a standard style defrag, where it puts data at the center, and does nothing special. It's like the Windows defrag (which is actually not as bad as people make it to be.. it just doesn't have the graph status, and a bit slow, but the job is done). "Optimize/Complete" does what I explained above, with trying to sort data, and "Optimize" is something in between the two.

 

You can also modify the algorithm. It uses "Zones" as it calls it to sort and place type of data on the disk. You can include and exclude folder and data in each type of zones.

 

It's a fairly impressive program, sadly its a paying software. There is a fully features and working trial version available.

[Huntsman]

Uh.. You guys got something wrong here... The fastest region of the platter is the outter side, not the center. Since given a fixed rpm, the magnetic materials would go past the header fastest at the outer perimeter compared to the inner.

 

http://www.techflare.com.au/blogs/2012/8/14/75/Improving_your_hard_disk_performance

 

Also, a software that will let the user select the algorithm for defrag is Perfect Disk. It will also let the user customize their own algorithm.

[zoltan]

Defragmentation is a necessary solution for an unnecessary problem.

The real problem is that Windoze+NTFS is so old that it fragments data. Modern operating systems with modern filesystems do not fragment data, therefore do not need defragmentation.

[SkilledRebuilds]

I think of defrag as a huge block chain, delete files, it removes blocks, putting new files on,..without defragging fills in those deleted block spaces, putting data in a randomized placement pattern.

Defrag mitigates this by moving data blocks around whether in the last accessed order/your order of choosing/the best suited defaults for the file system.

 

PerfectDisk or O&O work great

Zone Filling is good for this, telling the drive which folders to put to the start of the drive enhancing performance slightly for reads and writes.

(zone 1 - windows/games/sys32)

(zone 2 - program files/program files x86/users)

(zone 3 - the rest of the data not chosen for #1 or #2)

 

Before SSD's, I had my games set to be located on the drive before anything else.c:\games at the start of the drive then c:\windows would be moved afterwards.

[GoodBytes]

Defragmentation is a necessary solution for an unnecessary problem.

The real problem is that Windoze+NTFS is so old that it fragments data. Modern operating systems with modern filesystems do not fragment data, therefore do not need defragmentation.

No. Who ever told you this, knows nothing.

All File System works IDENTICALLY. What changes is the header information and footer information of files, and how things are being indexed.

 

If files are can't be fragmented, that means you'll fall into situation where you can never come close to fill your HDD size, as files can't be split. Moving data is VERY time consuming process. As you have have 1 arm in an HDD, performance of the system will be total abysmal. In other words, it creates fragmented free space. So you still need to defrag the drive.

 

If you think that MacOS or Linux don't need defrag, it's because it DOES defrag, but on the back. Windows give you a utility in front of you, Linux and MacOS, does not.

 

One thing in technology that has not been changed since the 70's, is how HDD works in storing data. And yes, many have tried many different methods... they all proudly failed to even come to the stage of being released anywhere, as it simply didn't work.

[GoodBytes]

Uh.. You guys got something wrong here... The fastest region of the platter is the outter side, not the center. Since given a fixed rpm, the magnetic materials would go past the header fastest at the outer perimeter compared to the inner.

 

http://www.techflare.com.au/blogs/2012/8/14/75/Improving_your_hard_disk_performance

 

Also, a software that will let the user select the algorithm for defrag is Perfect Disk. It will also let the user customize their own algorithm.

Correct, my fault. I meant to say outer layer.. for some reason, I got confused.

[zoltan]

No. Who ever told you this, knows nothing.

All File System works IDENTICALLY. What changes is the header information and footer information of files, and how things are being indexed.

 

If files are can't be fragmented, that means you'll fall into situation where you can never come close to fill your HDD size, as files can't be split. Moving data is VERY time consuming process. As you have have 1 arm in an HDD, performance of the system will be total abysmal. In other words: fragmented free space. So you need to defrag the drive. This results in still needing to defrag your drive.

 

If you think that MacOS or Linux don't need defrag, it's because it DOES defrag, but on the back. Windows give you a utility in front of you, Linux and MacOS, does not.

 

One thing in technology that has not been changed since the 70's, is how HDD works in storing data. And yes, many have tried many different methods... they all proudly failed to even come to the stage of being released anywhere, as it simply didn't work.

 

Interesting.

 

I do agree that the filesystem as such doesn't determine the degree of fragmentation (at least I think that's part of what you're saying, correct me if I understand it wrong).

 

The operating system itself determins where the data comes on the drive, and it's not NTFS or EXT4 that determins where data is written, but it's Windoze or GNU/Linux or BSD/MacOS. Which is also why GNU/Linux can perfectly write on an NTFS formatted disk and not fragment data just like it does with any other filesystem, so that it doesn't need later defragmenting, but Windoze can't write to it's own NTFS filesystem and keep the data unfragmented and needs defragmentation at a later moment, which is a very time consuming abysmal experience like you say, and Windoze can't use other filesystems either. Since only Windoze uses NTFS as native filesystem, is it in your opinion safe to say that operating systems that natively use NTFS and cannot work with EXT4 are older and less evolved than operating systems that are compatibel with any filesystem known to mankind and can use all of these without fragmenting the hard drive in such a way that it's necessary to go through the abysmal timeconsuming process of defragmenting it with some application or tool later? And is it safe to say in that respect that the time consuming separate defragmentation that is only necessary in Windoze, is therefor a necessary solution (otherwise the system slows down even more) for an unnecessary problem (only Windoze has this problem, other operating systems don't)?

[SteepGnomeKing]

Uh.. You guys got something wrong here... The fastest region of the platter is the outter side, not the center. Since given a fixed rpm, the magnetic materials would go past the header fastest at the outer perimeter compared to the inner.

http://www.techflare.com.au/blogs/2012/8/14/75/Improving_your_hard_disk_performance

Also, a software that will let the user select the algorithm for defrag is Perfect Disk. It will also let the user customize their own algorithm.

If it is a fixed RPM there is not a faster region. The middle just wold hold less information than the outer region. The whole disk is moving at the same rpm no faster or slower region. I didn't read the article I just know from experience and common sense. The inner region would hold less information while the outer would hold more. But in the centre the head of the head would have to move less distance if the information was all centred. That might not be how it is but there is no faster region because (inner vs outer) only the head moving would make a difference.

[GoodBytes]

First of all, I don't know what "Windoze" is.. But I think you mean Windows.

If not, then perhaps you are talking about this:

And that's a different topic.

 

The operating system itself determins where the data comes on the drive, and it's not NTFS or EXT4 that determins where data is written, but it's Windoze or GNU/Linux or BSD/MacOS. Which is also why GNU/Linux can perfectly write on an NTFS formatted disk and not fragment data just like it does with any other filesystem, so that it doesn't need later defragmenting, but Windoze can't write to it's own NTFS filesystem and keep the data unfragmented and needs defragmentation at a later moment, which is a very time consuming abysmal experience like you say, and Windoze can't use other filesystems either.

Basically, it's

 -> Take more time to write data each time, but doesn't need time consuming defrag process, BUT it still needs defragging, it's just won't be as time consuming

vs

 -> Be as fast as possible, and require a more time consuming defrag process.

 

2 different school of thought. But are good, and both sucks.

 

However, Microsoft approach is best for SSD's as SSD is always fragmented, and that 's how it works. Unless Linux updated something when it detects a SSD, it's process is even more time consuming, as data is all fragmented and evenly spaced out in the SSD, so it will have no choice to fragment it. (the reason why SSDs are always fragmented, is to lengthily use the chips, and it doesn't mather, as it can get every peace of forming a requested file simultaneously. Hence why there is 0 performance gain when you force a defrag on an SSD)

 

Since only Windoze uses NTFS as native filesystem, is it in your opinion safe to say that operating systems that natively use NTFS and cannot work with EXT4 are older and less evolved than operating systems that are compatibel with any filesystem known to mankind and can use all of these without fragmenting the hard drive in such a way that it's necessary to go through the abysmal timeconsuming process of defragmenting it with some application or tool later?

No, that's not why Windows doesn't support Ext4. Again, is has nothing to do with drive format. The reason why Windows doesn't support Ext4, is that there is no benefits for Windows to support it. 99% of people won't care, and it's not a feature to be exited about that will push sales. Linux NEEDS to support NTFS, as 90%+ of computers uses NTFS. And even then, Linux is reluctant in support NTFS as it's doesn't fully support it.

 

And is it safe to say in that respect that the time consuming separate defragmentation that is only necessary in Windoze, is therefor a necessary solution (otherwise the system slows down even more) for an unnecessary problem (only Windoze has this problem, other operating systems don't)?

Both OS have defragmenters. Linux (and MacOS) is invisible to the user, but still executes it on the back. Windows goes: "Here is a utility so that you can manually do it, and get status on the operation, oh and tell you and give you the option to when you want to do it. Enjoy!"

 

In other words.. I am not sure if this representation is correct, as I have not thought it through, but here goes.

What's better?

 An office building, where people works on their stuff, and at night you have a team of hard working, and awesome, people come and clean the building. (Windows)

 Or, is it best that every employees empty the trash, and clean the floor, as they work.

[zoltan]

First of all, I don't know what "Windoze" is.. But I think you mean Windows.

If not, then perhaps you are talking about this:

And that's a different topic.

 

 

 

Basically, it's

 -> Take more time to write data each time, but doesn't need time consuming defrag process, BUT it still needs defragging, it's just won't be as time consuming

vs

 -> Be as fast as possible, and require a more time consuming defrag process.

 

2 different school of thought. But are good, and both sucks.

 

However, Microsoft approach is best for SSD's as SSD is always fragmented, and that 's how it works. Unless Linux updated something when it detects a SSD, it's process is even more time consuming, as data is all fragmented and evenly spaced out in the SSD, so it will have no choice to fragment it. (the reason why SSDs are always fragmented, is to lengthily use the chips, and it doesn't mather, as it can get every peace of forming a requested file simultaneously. Hence why there is 0 performance gain when you force a defrag on an SSD)

 

 

 

No, that's not why Windows doesn't support Ext4. Again, is has nothing to do with drive format. The reason why Windows doesn't support Ext4, is that there is no benefits for Windows to support it. 99% of people won't care, and it's not a feature to be exited about that will push sales. Linux NEEDS to support NTFS, as 90%+ of computers uses NTFS. And even then, Linux is reluctant in support NTFS as it's doesn't fully support it.

 

 

 

Both OS have defragmenters. Linux (and MacOS) is invisible to the user, but still executes it on the back. Windows goes: "Here is a utility so that you can manually do it, and get status on the operation, oh and tell you and give you the option to when you want to do it. Enjoy!"

 

In other words.. I am not sure if this representation is correct, as I have not thought it through, but here goes.

What's better?

 An office building, where people works on their stuff, and at night you have a team of hard working, and awesome, people come and clean the building. (Windows)

 Or, is it best that every employees empty the trash, and clean the floor, as they work.

 

Can't say I agree.

 

First of all, not nearly 90+% of computers use NTFS, I say not even half, there is only one single full featured RISC device that uses NTFS, and that's the Windoze NT tablet. I would estimate that not even half of all x86 CISC computers uses NTFS as filesystem, but I would like to know your source for stating that 90+% of all computers uses NTFS. In terms of storage, I think GFS and other large storage UNIX/Linux filesystems govern the most storage on the planet, in terms of amount of hard drive storage, UNIX/Linux filesystems are obviously governing much more storage than any other filesystem, no other filesystem comes even close.

 

Second of all, NTFS is not faster, it's slower, a lot. Source: http://www.phoronix.com/scan.php?page=news_item&px=ODIxNw

That's not the only benchmark, there are others that come to the same conclusion that ext4, even though it's not a high performance filesystem, but rather a mainstream allround filesystem in Linux, that has regressed a lot to incorporate security features, so isn't performing to it's maximal potential, performs a lot better than NTFS in an unbiased benchmark.

That NTFS is faster or that it works better with SSD's is also not true. SSD's rely on trim to stay healthy, which is supported by the Linux kernel, so it doesn't need extra userspace software (a separate "tool" like the defragmenting software) to work, and it works with all SSD's and flash memory drives supporting trim and with most filesystems. On Windoze, trim only works with ordinary AHCI drives, and is not part of the kernel functionality, and is unusable when the disk is also encrypted, because Windoze doesn't support native encryption, this is also done with an extra "tool" that runs as an application, which means that trim reveals the encryption, which is a fundamental security leak in Windows, so if you encrypt your SSD in WIndows, you can't keep it healthy because you can't use TRIM or your encryption will not serve any security purpose, and on top of that, all operations on that disk in Windoze, whether read/write or encrypt or trim, will perform a lot slower, and the data will need lengthy time consuming maintenance later because it was not written efficiently, but fragmented in such a way that the performance even further decreases.

[GoodBytes]

Can't say I agree.

 

First of all, not nearly 90+% of computers use NTFS, I say not even half, there is only one single full featured RISC device that uses NTFS, and that's the Windoze NT tablet. I would estimate that not even half of all x86 CISC computers uses NTFS as filesystem, but I would like to know your source for stating that 90+% of all computers uses NTFS. In terms of storage, I think GFS and other large storage UNIX/Linux filesystems govern the most storage on the planet, in terms of amount of hard drive storage, UNIX/Linux filesystems are obviously governing much more storage than any other filesystem, no other filesystem comes even close.

 

Second of all, NTFS is not faster, it's slower, a lot. Source: http://www.phoronix.com/scan.php?page=news_item&px=ODIxNw

That's not the only benchmark, there are others that come to the same conclusion that ext4, even though it's not a high performance filesystem, but rather a mainstream allround filesystem in Linux, that has regressed a lot to incorporate security features, so isn't performing to it's maximal potential, performs a lot better than NTFS in an unbiased benchmark.

That NTFS is faster or that it works better with SSD's is also not true. SSD's rely on trim to stay healthy, which is supported by the Linux kernel, so it doesn't need extra userspace software (a separate "tool" like the defragmenting software) to work, and it works with all SSD's and flash memory drives supporting trim and with most filesystems. On Windoze, trim only works with ordinary AHCI drives, and is not part of the kernel functionality, and is unusable when the disk is also encrypted, because Windoze doesn't support native encryption, this is also done with an extra "tool" that runs as an application, which means that trim reveals the encryption, which is a fundamental security leak in Windows, so if you encrypt your SSD in WIndows, you can't keep it healthy because you can't use TRIM or your encryption will not serve any security purpose, and on top of that, all operations on that disk in Windoze, whether read/write or encrypt or trim, will perform a lot slower, and the data will need lengthy time consuming maintenance later because it was not written efficiently, but fragmented in such a way that the performance even further decreases.

 

Most PC runs Windows, and therefor the HDD must be NTFS in order for Windows to operate properly/

If you are saying that most people are running on Linux, then you live in a dream land.

 

Some benchmark on a pro Linux... sorry I have to do like you... Lamelux.... web site, doesn't make your argument valid.

Also, benchmark performance is meaningless. Real world performance is what maters. Not theories.

 

1- TRIM keeps SSD healthy, not make things faster.

2- You do not need AHCI for TRIM in Windows. You are miss informed. However, people enable AHCI for Native Command Queue (NCQ) support for SSDs and mostly HDDs. NCQ is a AHCI feature.

3- TRIM not working on an encrypted drive is a TRIM limitation not Windows.

4- Its the SSD controller that fragment data on purpose, not Windows.

[SirReallySam]

Puts your files in the most efficient place on your HDD :)

[orangecat]

This post has allot of replies but I'm going to say it very simple. It puts stuff together to make it quick.