More Cat photos Online: Google releases open source JPEG Algorithm

[WMGroomAK]

Now we can easily send more cat photos online and take up less space.   Google has developed and released a new open sourced JPEG algorithm that reduces file sizes by about 35% called Guetzli.  From Ars Technica (https://arstechnica.com/information-technology/2017/03/google-jpeg-guetzli-encoder-file-size/), it accomplishes this by:

 

Quote

There are numerous ways of tweaking JPEG image quality and file size, but Guetzli focuses on the quantization stage of compression. Put simply, quantization is a process that tries to reduce a large amount of disordered data, which is hard to compress, into ordered data, which is very easy to compress. In JPEG encoding, this process usually reduces gentle colour gradients to single blocks of colour and often obliterates small details entirely.

The difficult bit is finding a balance between removing detail, and keeping file size down. Every lossy encoder (libjpeg, x264, lame) does it differently.

Guetzli, according to Google Research, uses a new psychovisual model—called Butteraugli, if you must know—to work out which colours and details to keep, and which to throw away. "Psychovisual" in this case means it's based on the human visual processing system. The exact details of Butteraugli are buried within hundreds of high-precision constants, which produce a model that "approximates colour perception and visual masking in a more thorough and detailed way" than other encoders

Here are some samples of this compression with the original on the left, the libjpeg in the middle and the Guetzli on the right

 

 

 

Best thing I can think of this is that it can help to conserve data on upload and download of files and optimize load times on websites.

[vanished]

I guess until people are willing and able to switch to other superior formats entirely this will have to do.

 

come on webp... where are you?

http://jjcm.org/blog/real_world_analysis_of_webp/

[Ya_Mi]

More compression..urgg

[Bouzoo]

As long as they don't go as far as Facebook went with their compression I'm happy. 

[themctipers]

1 hour ago, Ryan_Vickers said:

I guess until people are willing and able to switch to other superior formats entirely this will have to do.

 

come on webp... where are you?

http://jjcm.org/blog/real_world_analysis_of_webp/

webp shit format

doesn't even work in discord, barely works on chrome, wont work on ios, wont work on android..

 

if people started to use it, sure.

-but ill take my png over webp any day, because i can see it on my phone

 

why are we developing a better jpeg algorithm over using png?

[vanished]

1 hour ago, themctipers said:

webp shit format

doesn't even work in discord, barely works on chrome, wont work on ios, wont work on android..

It works perfectly in Discord and Chrome, and support for it could easily be added to more things.  That's just an issue with adoption, not the format.

1 hour ago, themctipers said:

why are we developing a better jpeg algorithm over using png?

They serve different purposes.  png is lossless, jpg is lossy.  you will always get smaller files with jpg, but the quality is lower too, and it doesn't support transparency.  Depending on your needs, you'll select one or the other.  They don't replace each other, just as mp3 and flac can coexist.

[vanished]

1 hour ago, Bouzoo said:

As long as they don't go as far as Facebook went with their compression I'm happy. 

 

2 hours ago, Ya_Mi said:

More compression..urgg

 

I think you're both missing the point here...

[themctipers]

8 minutes ago, Ryan_Vickers said:

It works perfectly in Discord and Chrome, and support for it could easily be added to more things.  That's just an issue with adoption, not the format.

They serve different purposes.  png is lossless, jpg is lossy.  you will always get smaller files with jpg, but the quality is lower too, and it doesn't support transparency.  Depending on your needs, you'll select one or the other.  They don't replace each other, just as mp3 and flac can coexist.

Discord doesn't embed it for me, LTT doesn't like it either. Chrome on desktop takes a few refreshes for it to load and Chrome on mobile doesn't work. Ever. 

Rather see webp die, it's an annoying format for me since nothing can read it. 

 

They do serve different purposes but can you really not spare 600kb for a nice high quality photo? 

[vanished]

9 minutes ago, themctipers said:

Discord doesn't embed it for me, LTT doesn't like it either. Chrome on desktop takes a few refreshes for it to load and Chrome on mobile doesn't work. Ever. 

Rather see webp die, it's an annoying format for me since nothing can read it. 

Well you seem to have bugs or something   Ultimately it's just an issue of adoption.  You can imagine that when jpg was new few things could use it at first either.

9 minutes ago, themctipers said:

They do serve different purposes but can you really not spare 600kb for a nice high quality photo? 

In some images the difference is more like several megabytes, and if you have thousands of images, that adds up.

[themctipers]

1 minute ago, Ryan_Vickers said:

Well you seem to have bugs or something   Ultimately it's just an issue of adoption.  You can imagine that when jpg was new few things could use it at first either.

In some images the difference is more like several megabytes, and if you have thousands of images, that adds up.

Then compress PNG if you can't spare a few gigabytes

 

for me, webp is like using rar files, except 7zip and winrar aren't supported on any of your devices. Fucking useless. 

So stop sending them to me ?

[Doobeedoo]

You know you can never have too many cat photos though.

[Zodiark1593]

1 hour ago, themctipers said:

 

 

They do serve different purposes but can you really not spare 600kb for a nice high quality photo? 

Using a 5 GB data plan as my sole Internet source, one or two 600 KB files are no problem. When there are a multitude of them on a Web page however, which is common on certain threads here, the impact on my data cap can be quite substantial. 

[vanished]

1 minute ago, Zodiark1593 said:

Using a 5 GB data plan as my sole Internet source, one or two 600 KB files are no problem. When there are a multitude of them on a Web page however, which is common on certain threads here, the impact on my data cap can be quite substantial. 

For that matter, gif is insanely antiquated and better formats could be used if they would just be supported

[WMGroomAK]

13 minutes ago, Zodiark1593 said:

Using a 5 GB data plan as my sole Internet source, one or two 600 KB files are no problem. When there are a multitude of them on a Web page however, which is common on certain threads here, the impact on my data cap can be quite substantial. 

With the number of people switching over to mobile devices from hardwired connections and being always on the go, we definitely need the better compression.  I know this isn't the main reason Google pursued this, but any side benefit to avoid throttled data or overage charges from a cellular provider has to be good.  Especially if their algorithm is actually producing better results on the visual impacts of compression over what's existing.

[cesrai]

2 hours ago, themctipers said:

600kb for a nice high quality photo?

When I'm paying data, I'm gonna do everything to save some.

[LAwLz]

Nice, but I'd rather they created a new format instead of trying to polish a turd.

 

 

5 hours ago, Ryan_Vickers said:

I guess until people are willing and able to switch to other superior formats entirely this will have to do.

come on webp... where are you?

http://jjcm.org/blog/real_world_analysis_of_webp/

I blame Microsoft. They were not interested in supporting WebP so as soon as some websites such as Facebook changed over, people with IE and Windows photo viewer ran into a ton of issues.

We need all companies to be on board if we want a new format to succeed. It can't just be Google.

 

 

4 hours ago, themctipers said:

why are we developing a better jpeg algorithm over using png?

Because PNG is not a be-all-end-all format. PNG is absolutely atrocious for certain kinds of images.

Here is a photo of my dog saved as jpeg and PNG. Both have the exact same quality (because the master is the jpeg image).

 

The file size almost quadrupled for 0 increase in quality.

 

You should only use PNG in a few instances.

1) When your image contains large spaces of solid colors. I am not talking like a blue sky either. I am talking the exact same hex code for the color. Like what you get with the bucket tool in paint.

2) If you got lots of fine lines with perfect curves, like in a lot of vector art.

3) When you got medium to small size text in the image which you want to be readable.

4) Image that contains gradient which you want to keep as high quality as possible.

5) When you need transparency.

6) If you want your image to be lossless.

 

 

2 hours ago, themctipers said:

Then compress PNG if you can't spare a few gigabytes

Optimizing the PNG compression won't help much, and it eats up quite a lot of computational resources.

You're trying to screw in a screw using a hammer... PNG was meant for specific types of images, and using it for other types of images is just a really bad idea.

 

 

3 hours ago, Ryan_Vickers said:

you will always get smaller files with jpg

If we're going to be pedantic, PNG can quite often produce files which are smaller than jpeg.

Just create a 1000x500 picture in paint and fill half of it in one color, and half of it in another color. Save it as PNG and JPEG. The jpeg PNG version will not only be significantly smaller (like 1/2 the size) but it will also be higher quality.

Or you could just use a single color, in which case the PNG will have an even bigger lead.

[vanished]

9 minutes ago, LAwLz said:

Nice, but I'd rather they created a new format instead of trying to polish a turd.

Same

Quote

I blame Microsoft. They were not interested in supporting WebP so as soon as some websites such as Facebook changed over, people with IE and Windows photo viewer ran into a ton of issues.

Oh, I didn't know that.  What was in it for them to not support it?  Does Microsoft have a competing format or something?

 

On the flip side, the default viewer can preview RAW files which is pretty cool  

Quote

We need all companies to be on board if we want a new format to succeed. It can't just be Google.

Yes

Quote

If we're going to be pedantic, PNG can quite often produce files which are smaller than jpeg.

Just create a 1000x500 picture in paint and fill half of it in one color, and half of it in another color. Save it as PNG and JPEG. The jpeg version will not only be significantly smaller (like 1/2 the size) but it will also be higher quality.

Or you could just use a single color, in which case the PNG will have an even bigger lead.

Fair enough.  I was thinking more about photos (the kind of thing you'd typically use jpg for), but you're right, I guess I shouldn't say jpg will always be smaller.

Edit: I just tried this (4K image, two colours only) and confirmed the jpg even at maximum compression was still larger  

Edit 2: I think you made a typo in your quote that I mentally fixed when I read it the first time:

Quote

The jpeg version will not only be significantly smaller (like 1/2 the size) but it will also be higher quality.

I think you meant PNG  

Edited March 17, 2017 by Ryan_Vickers

[LAwLz]

13 minutes ago, Ryan_Vickers said:

Oh, I didn't know that.  What was in it for them to not support it?  Does Microsoft have a competing format or something?

I don't think they have. Just seems like they are lazy and generally against supporting more than the bare minimum of formats.

I mean, Windows just recently started supporting FLAC and MKV. Still no support for Vorbis, Opus, VP8 or VP9 though (Edge might now support some of those formats, but not Windows).

 

Hopefully that will change now that they are working with Mozilla, Google, Cisco and a few others on NetVC (going to be a next gen video codec that is completely free and open source, projected to be done near the end of this year).

It would be pretty awesome if they could develop a format like BPG but based on NetVC.

 

 

58 minutes ago, Ryan_Vickers said:

I think you meant PNG  

Damn you're right.

[Shakaza]

Hmm, I've been testing this out a bit and the results seem to be pretty good, though I'm not sure how to best go about doing an objective comparison of image quality. And, I tried out using a random 7 MB .PNG I had lying around, and it's been running for the last 15 minutes or so, and it decided to use 4 GB of RAM at some points. 

 

I find this kind of thing fascinating, and I really want to learn more. I'm a noob when it comes to compression, image and video encoding, etc, etc.

[vanished]

30 minutes ago, LAwLz said:

I don't think they have. Just seems like they are lazy and generally against supporting more than the bare minimum of formats.

I mean, Windows just recently started supporting FLAC and MKV. Still no support for Vorbis, Opus, VP8 or VP9 though (Edge might now support some of those formats, but not Windows).

 

Hopefully that will change now that they are working with Mozilla, Google, Cisco and a few others on NetVC (going to be a next gen video codec that is completely free and open source, projected to be done near the end of this year).

It would be pretty awesome if they could develop a format like BPG but based on NetVC.

Yeah supporting more things would be great.  Just imagine if the built-in apps could actually play everything you needed  

[WMGroomAK]

2 minutes ago, Ryan_Vickers said:

Yeah supporting more things would be great.  Just imagine if the built-in apps could actually play everything you needed  

I think the closest thing I've found for audio/video is VLC player...  One of the first things I install on any new device now.  At work, most of the image viewing is through InfranView although there are still some formats I've had issues with that recognizing.

[vanished]

Just now, WMGroomAK said:

I think the closest thing I've found for audio/video is VLC player...  One of the first things I install on any new device now.  At work, most of the image viewing is through InfranView although there are still some formats I've had issues with that recognizing.

I used to use that for everything but I've since moved on to MPC-HC

[Zodiark1593]

1 hour ago, Ryan_Vickers said:

I used to use that for everything but I've since moved on to MPC-HC

MPC-HC does seem to be the more optimized of the two, and better supports hardware decode (VLC does not yet support hardware decode of HEVC). In addition, tinkering produces excellent results. Though, I find it difficult to find a combination of settings that works well across most videos, at least, without MadVR. MadVR can greatly enhance the quality of even sub-par source files, at the expense of requiring heavy GPU compute. 

 

VLC I find leans heavily on the CPU for much of it's enhancements beyond decode. I find it much easier to come up with a good, general purpose setting that works acceptably across all videos. Also had not seen an update for awhile. 

 

Really, no matter which you choose, it's hard to go wrong with either. I keep both around tbh. 

[Bleedingyamato]

5 hours ago, Zodiark1593 said:

Using a 5 GB data plan as my sole Internet source, one or two 600 KB files are no problem. When there are a multitude of them on a Web page however, which is common on certain threads here, the impact on my data cap can be quite substantial. 

Why not get some sort of internet service?    Even a slow speed internet service, assuming no data cap, would be an improvement over a 5GB limit.

 

How can you get by with only 5GB a month?  Downloading games or music or other large things must be tricky.

[Zodiark1593]

8 minutes ago, Bleedingyamato said:

Why not get some sort of internet service?    Even a slow speed internet service, assuming no data cap, would be an improvement over a 5GB limit.

 

How can you get by with only 5GB a month?  Downloading games or music or other large things must be tricky.

I chose not to eat a 66% price hike from Comcast is why. As for how I get ahold of large files, my Intel Atom tablet is key. Though not fast, it's cheap, and lasts an entire work day on battery alone (assuming the display shuts off like it should). I connect it to the work wifi, download what I need, and tuck it away somewhere to work on it. 

 

A fan cooled device would actually be quite poor at this, as I found out before with my older laptop. I pulled it out of the bag with the fan screaming one day.   

 

Home Internet browsing is my biggest consumer of bandwidth on my data plan, and chews through the data in a hurry if I leave images enabled.