Samsung is improving qualities of moon-picture with pre-stored detail, you can try it out yourself!

[betalars]

I've heavily edited the post, as the reddit post did one more test, that actually convinced me it is real.

This is a story, that kind of goes in and out of the cycle once in a while. Accusations, that Samsung actually stores pictures of the moon on their device, that they overlay to artificially improve their camera quality. And it makes sense: when you deliberately blur an image of the moon, and take a screenshot from across the room there suddenly will be a lot more more detail than before. So where is that coming from? Reports on similar things in the past have come to the conclusion that more likely than not it comes down to the aggressive post-processing all phone cameras do, says inverse.com.

 

I've actually tried my own Methodology to reproduce that, and first came to the conclusion, that the Reddit Story likely does not check out, however:

 

When putting a cut-off moon and a full moon next to each other, samsung will only recover detail on the full moon.

You can use this image to see if your own phone will put more detail in the full moon than in the cut-off moon when taking a screenshot from really far away.

 

Me testing the Theory (I have not seen the last result before)
Okay, so I've made this fake moon in blender. It is entirely fake and the detail on it looks nothing like the real moon. If a camera would introduce fake moon detail over any moon-shaped object, we would expect a Screnshot made from across the room to look a lot more like the real moon than this image does:

 

Here's what my Pixel did (the stars are amplified dust on my monitor):

It would be interesting to see someone getting their Phones to make it more moon-ish than it actually is. But it may not work, because the broad details on my moon and the patterns on the real moon do not really add up.

 

What is Detail, that can be explained away with image sharpening?

 

 

This is the second test I did with a space Image I had laying around already and very little effort mind you.

• I've blurred and down-scaled it, just like the reddit poster did
• I've added artificial camera noise
• I put that into Rawtherapee and did noise-reduction, detail-recovery and local-contrast amplification. So pretty much the manual version of what your phone is doing auto-magically.

And as you can see: Rawtherapee managed to "recover" a lot more detail, than what was in the original photo. But it was also not really the detail, that has been there before.

 

HOWEVER: it does not manage to properly reconstruct the flow of fine details, it just recovers random detail that wasn't there before. And Samsungs algorythm recovers detail, that is more correct, than it should be with just sharpening.

 

I've attached the fake moon and the image-degradation-setup, and the raw-therapee-setup so you can replicate everything at home.

 

My Opinion

This is on the one hand a really great example to understand, how you can find patterns in noisy/blurred data and can give a great understanding, of what kinds of post processing phone cameras can do nowadays.

 

However Samsung clearly also did train it's detail recovery algorithms on good pictures of the moon. So while their raw data has to be good enough to recognize some amount of detail, such as the orientation of the moon, some of it is also added just from training data.

 

Sources

 In the Text itself and in the linked files.

fake moon.blend raw-therapee-setup.pp3

[Bananasplit_00]

Anyone with the latest Samsung phone around to test this? Did anyone actually buy it?

[betalars]

Oh btw: I'm not saying there's no sharp images of the moon are being used as training data for image enhancement AI, and those may help boost quality beyond what that sensor can actually capture.

 

But it's not so plain cheating as people make it out to be.

[LAwLz]

The sooner people realize that modern cameras "fill in" detail that the sensor isn't actually capturing, the better.

Here is a 5 year old quote from Isaac Reynolds, who (or was) the product manager for the Google Pixel's camera:

Quote

>One of the other things we did. When you're in very very low light, it's very hard to figure out what color the photo should be. So we've actually used machine learning to analyze the picture itself and try to determine what the right true-to-life colors are. That's another really interesting innovation that we have in nightsight.

 

In other words, when the sensor can't gather enough information, Google makes their camera "fill in" information it thinks should be in the image.

That's what Samsung is doing for their moon shots as well, as explained in this article from Samsung.

https://r1.community.samsung.com/t5/camcyclopedia/달-촬영/ba-p/19202094

 

 

You can turn this feature off by disabling scene optimization if you want. But I am not sure why you'd want to. It just makes your pictures look worse. 

(General) photography has never been about recreating what the camera sensor "sees".

[mr moose]

25 minutes ago, LAwLz said:

Quote

So we've actually used machine learning to analyze the picture itself and try to determine what the right true-to-life colors are.

 

But that means I can't copyright my photo's. 

 

This world sucks.  I'm going to create my own world with AI copyright and hookers and blackjack. 

[Vishera]

45 minutes ago, betalars said:

But where does the Detail Come from?

I am a Photoshop artist,

There are ways the expose and highlight detail that is not seen by the naked eye or difficult to see but is there in the image.

I took the image from your example and did just that (I used some cheap and quick tricks because i am not going to spend a hour on it):

[Arika]

s22. looks exactly how i would expect it would.

 

[mr moose]

25 minutes ago, Arika S said:

s22. looks exactly how i would expect it would.

 

Nope, I definitely see the old David Jones pattern in your photo,  This means samsung have been training their AI with retro Australian shopping images.

 

 

[betalars]

26 minutes ago, Arika S said:

s22. looks exactly how i would expect it would.

Did you use maximum zoom for this? the pixel looks like crazy sharp.

[betalars]

Update: i did a take on this using the Moon Image form the original post:

 

And I couldn't really get it to match Samsung's result, the reddit user posted on imgur

 

And while some of the patterns the algorythm managed to recover seem a bit too-good-to-be-true to me, right now the more plausible solution still are high-Quality Pictures of the moon being used as training data in training detail reconstruction and not samsung recognizing a moon and putting a fake overlay on top of that.

 

BTW: have a blurred picture of the fake moon to toy around with, maybe this will yield interesting results.

 

[betalars]

Oh, they edited their post and I have not realised that and they actually managed to "break" the algorythm. I take everything back, it is a lot better than it should be able to do on merely detail recovery without knowledge about how the moon actually looks.

[Arika]

51 minutes ago, betalars said:

Did you use maximum zoom for this? the pixel looks like crazy sharp.

nope, i opened the camera app, and clicked the button.

 

here's one of the blurred photo (wit night mode on this time)

 

and what ever that other image is

 

 

seems like a pretty fucking stupid claim.

[betalars]

12 minutes ago, Arika S said:

seems like a pretty fucking stupid claim.

that's also what I thought until I saw them making a picture that has one moon half cut off and Samsungs camera would only "recover" detail on the moon, that was not cut off.

[Doobeedoo]

Well AI is taking over, can't wait for planet images next.

[Spotty]

Moved to Photography & Videography. Tech News section requires news articles.

[HenrySalayne]

On 3/12/2023 at 2:50 PM, betalars said:

that's also what I thought until I saw them making a picture that has one moon half cut off and Samsungs camera would only "recover" detail on the moon, that was not cut off.

This user "edited" the moon and all the edits where recovered by the phone. This is probably just a case of machine learning.

[betalars]

15 hours ago, Spotty said:

Tech News section requires news articles.

Yeah I tried and didn't really find only articles on related subjects. Well at least I'm now on tech news, as in the video \o/

[Gimmick21]

48 minutes ago, HenrySalayne said:

This user "edited" the moon and all the edits where recovered by the phone. This is probably just a case of machine learning.

Of course it's machine learning.

The main reason of the diskussion is, that there are different opinions on what "fake" means.

 

If I take a shitty photo of a street sign and I recreate the text on that sign in photoshop (I can, because I read it), is that fake?

What if I change the text to something else?

What if the AI "enhances" the text? What if the AI text recognition failed and the text is wrong?

What about changed colors, contrasts, saturations? Does this make a photo "fake"?

Are old photos, taken with B&W film fake? Because the world is not B&W....

 

One single word is simply not enough to fully describe all those situations. From my point of view, something becomes "fake" if there is an intentional fraud. If you auto-optimize the hell out of your face and you post that image online to show "how you look" -> that's a fake. If you tell someone that "the zoom of the phone camera is better than the optical zoom of expensive cameras" and you "prove it" with AI reconstruction -> that's a fake.

 

If you describe the "optimized" image with all details, like all used filters -> not a fake. That's more like the typical information about a photo like lense, aperture, film simulations, grain added, color removed...

 

The might be situations where this information is not neccessary, like art galleries.

 

Because of all that it is maybe better to create a new category like "smartphoto", so everyone knows: Nobody knows what happend to this image.

[HenrySalayne]

9 minutes ago, Gimmick21 said:

Because of all that it is maybe better to create a new category like "smartphoto", so everyone knows: Nobody knows what happend to this image.

Every picture taken with a smartphone of the last five years is a "smartphoto".

10 minutes ago, Gimmick21 said:

If you tell someone that "the zoom of the phone camera is better than the optical zoom of expensive cameras" and you "prove it" with AI reconstruction -> that's a fake.

They were pretty open about it and machine learning is only one thing of many they are doing to improve the image.

https://r1-community-samsung-com.translate.goog/t5/camcyclopedia/달-촬영/ba-p/19202094?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-GB

 

The reddit user starting this outrage-train was claiming that they 'added details back into the picture that weren't there'. Which bears some truth but also not. The increased detail doesn't come from a database of Moon pictures, it's extracted from the blurry image ("Blurring" is spreading the information of a single pixel across several pixels. The problem with "unblurring" is that a pixel contains the combined information of all the surrounding pixels - that's why the process is not reversible. With machine learning the algorithm can make a "good guess" what the original "unblurred" image looked like and get details back that no human would ever imagine in the picture).

Quote

And it's not sharpening, it's not adding detail from multiple frames because in this experiment, all the frames contain the same amount of detail. None of the frames have the craters etc. because they're intentionally blurred, yet the camera somehow miraculously knows that they are there. And don't even get me started on the motion interpolation on their "super slow-mo", maybe that's another post in the future..

 

[betalars]

27 minutes ago, Gimmick21 said:

The main reason of the diskussion is, that there are different opinions on what "fake" means.

I agree with you and it comes to managing expectations and being honest.

Nobody would look at this Video I've made and call it fake, even tough all of it is clearly generated.

 

But what about this Photo of a rural street I've made?

It didn't look as vibrant IRL, it didn't look as spectacular in the sensor data, but I've did a bunch of manual contrast adjustments, I've shifted hues and saturation around so it looks more authentic to what the scenery felt like.

 

I think especially professionals should be more transparent about their process. Indicating when a models proportions have been changed or freckles have been removed comes to mind.

 

And if Samsung would just outright tell people they're doing Content-Aware Detail reconstruction or maybe even Moon Super-resolution this wouldn't be a story with a "faking" connotation.

 

But I get that doing maximum transparency comes with tons of bloat too, and you have to think, if this is worth it. I mean they posted about what their doing on the samsung forum, so it's not like they are trying to cover it up.

[Gimmick21]

49 minutes ago, HenrySalayne said:

Every picture taken with a smartphone of the last five years is a "smartphoto".

Not every smartphone/smartphone-mode adds (or removes) plausible details. But maybe "photo" is wrong in the first place and it's more like a composition.

 

40 minutes ago, betalars said:

I agree with you and it comes to managing expectations and being honest.

Nobody would look at this Video I've made and call it fake, even tough all of it is clearly generated.

 

But what about this Photo of a rural street I've made?

It didn't look as vibrant IRL, it didn't look as spectacular in the sensor data, but I've did a bunch of manual contrast adjustments, I've shifted hues and saturation around so it looks more authentic to what the scenery felt like.

 

I think especially professionals should be more transparent about their process. Indicating when a models proportions have been changed or freckles have been removed comes to mind.

 

And if Samsung would just outright tell people they're doing Content-Aware Detail reconstruction or maybe even Moon Super-resolution this wouldn't be a story with a "faking" connotation.

 

But I get that doing maximum transparency comes with tons of bloat too, and you have to think, if this is worth it. I mean they posted about what their doing on the samsung forum, so it's not like they are trying to cover it up.

Exactly. There is nothing wrong with editing, creating compositions, blending operations and so on.... just don't hide it.

A photo is art "paint with light" and not a measurement. Therefore I would suggest: Brightness, saturation,.. is included in "photografy", adding/removing details/objects is maybe a composition (editing with masks, too, I guess?) and  if something happens and the user does not know why: add "Auto" or "ML". 

 

Nice ML-Composition of the moon xD.

 

[Needfuldoer]

So that's what those tens of gigs of "system files" are. It's all moon pictures!

[HenrySalayne]

1 hour ago, betalars said:

And if Samsung would just outright tell people they're doing Content-Aware Detail reconstruction or maybe even Moon Super-resolution this wouldn't be a story with a "faking" connotation.

 

But I get that doing maximum transparency comes with tons of bloat too, and you have to think, if this is worth it. I mean they posted about what their doing on the samsung forum, so it's not like they are trying to cover it up.

In this particular case I just see a bunch of people being satisfied with the picture they've taken of the moon and not caring the slightest about the how and why.

But did Samsung advertise it correctly? Was this just an "example" of how amazing their cameras are (implying "in general" and that they will always perform like this) or how was it communicated?

[Tams]

So in summary:

 

Some reddit 'expert' did some easy tests and found the Samsung camera app to be doing what it is supposed to do.  They then misunderstood what was happening and created an outrage-train.

 

Now, could Samsung have been a bit clearer about it (not in their explanations, which are fine, but in their promotional material)?  Sure.  But they don't appear to have lied at all.

 

Good job reddit.  You strike again.

[betalars]

1 hour ago, Tams said:

Now, could Samsung have been a bit clearer about it (not in their explanations, which are fine, but in their promotional material)?  Sure.  But they don't appear to have lied at all.

They did a poor job in managing expectations, so some of the outrage is justified. I mean it even MKBXD had to do a correction about misleading statements he made because of this saga ...