file size Wrong 'used space' on external SSD

[yokitosh]

I've got brand new SanDisk External SSD 500GB drive (SDSSDE80-500G-A25) (I'm aware that not ALL of 500GB is available for direct use), speed are fine, but what I gather almost immediately, is strange calculation of free size on that drive. Basic calculation gives me that sizes are 5.5 times larger that it should be!
Here is calculations:

Spoiler

*For simplicity I've used 500 instead of whatever actual in bytes size is

500 - 65.57 = 434,43  - [Drive capacity - actual total used space by files = expected available scape]

500 - 140.76 = 359.24  - [Drive capacity - current `available space` = current `used space`]

 

359.24 / 65.57 = 5.47  - [Current `used space` / actual total used space by files = ratio]

 

 

I've know for sure that all of those files can't be more than 70 GB, since I've copied them from another external drive (HDD), and there they use not more than 70 GB.

I've attached screenshots from Mac (slightly edited tho, since my main language is not English, so I've renamed key labels to English), but it's not tight to Mac issue, I've tried Windows as well, Total Commander gave the same results, then I've run WinDirStat, and it calculated about right..

 

I've tried reformat drive, both from mac and windows. Doesn't help.

Maybe I'm missing something, but will appreciate any help&information, thanks in advance!

[Origami Cactus]

Sounds like you have too big of an "allocation unit size".

Try reformatting it?

 

In windows, if you go to file explorer, and go to that drive, and right click on a folder/file, it shows you an "actual size", and "size on disk", that should indicate if you have the drive setup wrong.

[yokitosh]

22 minutes ago, Origami Cactus said:

Sounds like you have too big of an "allocation unit size".

Try reformatting it?

Well, I've formatted it twice..
But, your mention of `allocation unit size` is really helpful, I've forgot to change it when formatted the drive... Thank you!

Is there any advice, about what should I use if I have couple large single files and a lot small files?

[mariushm]

Looks like you're using exFAT file system.

 

Determine the allocation unit size, it may be a value higher than the standard 512 bytes or 4096 bytes.

That's the minimum amount of disk space that's used by a file on your disk - if your allocation unit size is 1 MB and a file is 1.1 MB, then 2 x 1 MB would be reserved for that file on your disk so 0.9 MB are lost and unusable.

 

In your picture above, you have a folder with 24 410 files (I think) which use 27,327,987,712 bytes in total, so in the worst case scenario where all those files are very small, each file would use minimum 27,327,987,712 / 24410 = 1,119,540 bytes each on average which is very close to 1 MB, so the picture makes me think you have a 1 MB allocation unit size, or a value a bit lower than this, like 512 KB.

 

For example, a nice and easy way to figure out this allocation unit size is to open notepad and type a few sentences .... aim for around 2-300 characters. You could just copy 12345678901234567890123456789012345678901234567890 (50 characters) in Clipboard and paste it around 10 times in the text document. 

If you type just a few characters, the operating system may resort to some tricks to save disk space, like saving those few characters in the area where the operating system stores the last access date, last modified date, file attributes etc instead of actually reserving actual disk space for the file.

 

So once you do this, simply save and close Notepad and then right click on File and select Properties. You will see actual bytes used, and actual bytes used on disk, which should be higher and will tell you the actual allocation unit size.

 

With exFat, the partition may be configured to use minimums of 128 KB or 512 KB or even 1 MB, because they assumed you'd use the drive for pictures, music or videos, which are all bigger file sizes in general.

 

So if you use lots of very small files, a lot of disk space will be wasted and unusable. You can re-format the drive and before you start to format, you can usually change the allocation unit size.

But be careful, if you format you lose the data on the drive.

 

 

 

[yokitosh]

14 minutes ago, mariushm said:

You will see actual bytes used, and actual bytes used on disk, which should be higher and will tell you the actual allocation unit size.

 

With exFat, the partition may be configured to use minimums of 128 KB or 512 KB or even 1 MB, because they assumed you'd use the drive for pictures, music or videos, which are all bigger file sizes in general.

 

So if you use lots of very small files, a lot of disk space will be wasted and unusable. You can re-format the drive and before you start to format, you can usually change the allocation unit size.

But be careful, if you format you lose the data on the drive.

Thanks for the reply!
I've just made another reply, confirming that it's in fact a `allocation unit size`, and I just recheck I do have 1mb allocation unit size. So, what is my option as for chosing this in format? Is there any drawback if I chose really small size like 32kb? 128kb? 1mb? 4mb?

[mariushm]

Very small values can result in fragmentation if you copy and delete files often, making seeking into big files a bit harder... so that could cause problems for example if you play a 4K video from the drive directly on your TV (drive as an external usb drive). Not a problem with SSD drives, but can be with mechanical drives.

 

It may also take a bit longer to copy big files from the drive but not by a significant amount.

 

Otherwise it's not really a problem... like file safety, stressing the hard drive, whatever.

 

I'd probably use 64 KB as a good compromise ... it's a nice multiple of 512 bytes and 4 KB and if you have lots of small files (like 10-20 KB) you'll only lose up to a block of 64 KB so it won't be a big loss.

 

[Origami Cactus]

36 minutes ago, yokitosh said:

Well, I've formatted it twice..
But, your mention of `allocation unit size` is really helpful, I've forgot to change it when formatted the drive... Thank you!

Is there any advice, about what should I use if I have couple large single files and a lot small files?

I use about 64-128kb allocation sizes. 64 is the most common.

But with your drive there maybe something else, allocation units shouldn't cause 10x the file size?

So if reformating doesn't help i would try to find out if the drive you have is actually a 500gb one, or a fake one.

[yokitosh]

21 minutes ago, Origami Cactus said:

I use about 64-128kb allocation sizes. 64 is the most common.

But with your drive there maybe something else, allocation units shouldn't cause 10x the file size?

So if reformating doesn't help i would try to find out if the drive you have is actually a 500gb one, or a fake one.

Reformat doesn't helped previously, cuz I didn't change the AUS, and left it on default from reformat to reformat...

I've have multiple node_modules folders for projects (this a folder that contain large amount of small files, libraries, usually less than 1mb). So I do see how quick that could stack up..

 

Currently I've reformatted it with 64kb block and half-way-thought copying files back to SSD, and things looks very promising!

So I'd like to thank for helping you guys @mariushm @Origami Cactus!