Python, File to PNG Compressor.

[Kilrah]

17 hours ago, Poet129 said:

I've configured it to only clean up the "junk" repeated data in the file not data that is potentially important.

You're supposedly making a compressor that works for any file, so there will be no "junk" data to remove. It makes no sense testing the performance of your thing on one specific file type which has some - it's irrelevant. 

Any valid test must be able to take any file (/dev/urandom ?) and after compression/decompression yield exactly the same thing.

[OrionFOTL]

OP, this thread started with you trying to design an algorithm that converts any binary data into its representation as an image of its individual bytes, trying to compress it by leveraging PNG compression on the resulting image.

 

Applying additional existing compression methods onto input files before putting it through your algorithm, like you just did with the Linus photo, doesn't make your algorithm any better and in no way does it make any point about the validity or effectiveness of your method.

 

Imagine I take the Linus photo, convert it to a 50kB JPG by saving it with 1% quality, then I put it through your program, which spits out a 50kB image of black and white dots.

I obviously can't claim my program had any significant contribution in reducing the file's size this way.

[Poet129]

7 hours ago, Kilrah said:

You're supposedly making a compressor that works for any file, so there will be no "junk" data to remove. It makes no sense testing the performance of your thing on one specific file type which has some - it's irrelevant. 

Any valid test must be able to take any file (/dev/urandom ?) and after compression/decompression yield exactly the same thing.

Here you go, also the pngcrush can be easily deactivated...

1003.11KB

File File.7z

[Poet129]

Also here's a decompressor take note you will need python and php with the gd extension. Also it only works on linux for me. Feel free to modify it to get to work on windows. Also they both need stored in /Programs/TXT_to_PNG/ this just how it is written again feel free to modify it.

Decompess.1.py Decompress.2.php

[OrionFOTL]

17 minutes ago, Poet129 said:

Here you go, also the pngcrush can be easily deactivated...

1003.11KB

File 1000 kB · 0 downloads File.7z 1013.74 kB · 0 downloads

Your "compressor" made the file larger... The original File has 1 024 000 bytes, while your picture has 1 027 180 bytes.

 

I also compressed your original File myself with WinRAR and with 7zip. Both results are larger than the original file by 148 and 168 bytes, but your image is larger by 3180 bytes.

 

 

Perhaps LinusTechTips processed your image attachment somehow? Try uploading your image to a filehost like Zippyshare, to make sure this forum's attachment system doesn't process it.

[Poet129]

1 minute ago, OrionFOTL said:

Your "compressor" made the file larger... The original File has 1 024 000 bytes, while your picture has 1 027 180 bytes.

 

I also compressed your original File myself with WinRAR and with 7zip. Both results are larger than the original file by 148 and 168 bytes, but your image is larger by 3180 bytes.

 

I don't know why your 7zip is smaller but I posted mine as well perhaps settings if so please share.

[OrionFOTL]

4 minutes ago, Poet129 said:

I don't know why your 7zip is smaller but I posted mine as well perhaps settings if so please share.

I used WinRAR with simply choosing "Best" in a dropdown menu, and 7zip with "Ultra" and "LZMA2".

[Poet129]

1 minute ago, OrionFOTL said:

I used WinRAR with simply choosing "Best" in a dropdown menu, and 7zip with "Ultra" and "LZMA2".

Okay I don't know where that is set for linux though.

[Poet129]

47 minutes ago, OrionFOTL said:

Perhaps LinusTechTips processed your image attachment somehow? Try uploading your image to a filehost like Zippyshare, to make sure this forum's attachment system doesn't process it.

The png is the correct size however the 7zip is smaller than I have on my system.

[mariushm]

Download 7zip and configure the compression manually.

 

Also, you achieve better compression if you use a single thread for compression, though on such small file sizes, the compression software probably defaults on single thread anyway.

Also, LZMA is better compression than LZMA2 in a lot of scenarios, LZMA2 is LZMA "optimized" to work well with lots of threads, which is useful when you're compressing big files.

And... in lots of cases, PPMd algorithm gives higher compression ratio.

 

Anyway... your File  (whatever it is, some already compressed stuff)  1024000 bytes ... you 7z file is 1038068 bytes ... the png is 1027180  bytes ... and no wonder, it's a png file with Deflate compression but set at level 0 - copy uncompressed, basically your file, wrapped in PNG container.

As proof, I attached a xdelta patch which can generate the PNG file from the original File and this 30 KB patch... the 30 KB patch file contains the binary differences between your PNG file and the original file diff.7zThere's 30 KB worth of differences between Deflate works with 32 KB or 64 KB chunks of data, so your 1024000 bytes are split in a few hundred chunks which are copied 1:1 but between these chunks there's a few bytes of Deflate stuff.

 

As a side note... file is very compressed, but paq8jc still managed to inflate it to only 1025472 bytes ... so 1472 bytes above the original file size.

 

[Poet129]

40 minutes ago, mariushm said:

Anyway... your File  (whatever it is, some already compressed stuff)

 

9 hours ago, Kilrah said:

You're supposedly making a compressor that works for any file, so there will be no "junk" data to remove. It makes no sense testing the performance of your thing on one specific file type which has some - it's irrelevant. 

Any valid test must be able to take any file (/dev/urandom ?) and after compression/decompression yield exactly the same thing.

It is a /dev/urandom file.