Halide Developer Takes a Deep Dive Into iPhone XS Camera and Skin Smoothing Rumors

[x.marX]

Exactly right. No idea why people keep posting year after year that they miss SJ. If Apple is so bad then move on, stop giving Apple money.

There is no other option !
(Please don't mention android, it's too fragmented and buggy)

 

[ignatius345]

The details are still there — they are just invisible.

 

[lordofthereef]

So I'd hakide worth the $6? The phones we would use it on are iPhone X, 8, and maybe an SE, but the first two for sure.

Certainly my seens priced nicely if it does everything it says it does. I just hate tossing money st an app before being able to use it. Also imagine there's no way to set it as default camera, so launch g would require a dedicated app loading?

 

[Ahmad Sami]

Sebastiaan de With, the developer behind photography app Halide, has taken an in-depth look at the front and rear-facing cameras in the iPhone XS and XS Max, providing some insight into complaints about a possible skin smoothing "beauty mode" that results in less realistic selfies than prior iPhone models.

A soft filter on selfies "doesn't exist," says de With, with the smoother look attributed to more aggressive noise reduction techniques and the merging of exposures that eliminates sharp light and dark contrasts where light hits the skin.

Camera differences between iPhone X and XS Max demoed by a Reddit user​

Both the front and rear-facing cameras in the iPhone XS and XS Max are using computational geography to improve photo quality, a feature that takes multiple shots at once and then merges them into a single perfect photo.

Apple's iPhone XS website explains all of the different things that are going on behind the scenes when an image is captured, and part of the process involves taking shots that are underexposed, overexposed, and captured at different times, with the camera taking the best elements of each shot and combining them.

Apple says the iPhone XS performs 1 trillion operations per photo​

According to de With, this results in a "whole new look" for photos that represents a "drastic departure" from images captured with previous-generation iPhones. As the Halide blog post explains, the small areas of contrast seen in photos from older iPhones can make images look sharper, but these areas of deep contrast have been eliminated in the iPhone XS models.

Using the exposure merging techniques, the iPhone XS reduces the brightness of the bright areas and the darkness of the shadows. While the detail is still there, we see it as less sharp because of this loss of contrast. Skin ends up looking smoother because the "light isn't as harsh."

The iPhone XS and XS Max are also using much more aggressive noise reduction techniques than in previous iPhones. de With says this is because the iPhone XS models prefer a faster shutter speed and a higher ISO level, capturing photos quicker but resulting in more noise. Taking care of that noise sacrifices some detail and contrast.

Noise in a RAW iPhone X image (left) compared to RAW iPhone XS image (right)​

In the front-facing camera specifically, which uses a smaller sensor than the rear-facing camera, the heavy noise reduction paired with the image merging techniques leads to the noticeably smoother selfies that people have seen with the iPhone XS camera.According to de With, Apple can tweak these settings through software updates if the majority of people are unhappy with the selfie camera, so we may see a better compromise between noise reduction and the filtering of harsh lighting in the future.

All in all, de With believes the iPhone XS camera is better than the iPhone X camera thanks to its superior dynamic range, with some post processing able to re-add the contrast where necessary.

There are issues with RAW image capture because of the noise level. When taking RAW photos with an app like Halide, de With says it is a must to do so manually and to reduce the exposure. Otherwise, RAW images will end up looking worse than Smart HDR JPEGs.

To account for the changes to RAW image capture, Halide is gaining a new Smart RAW feature that uses the new sensor tech in the iPhone XS to get better images. It avoids Smart HDR all together to cut down on noise reduction and pull out more detail.

Halide's full blog post is well worth a read for anyone interested in an in-depth look at the camera changes Apple has introduced in the iPhone XS and XS Max. The Halide app can be downloaded from the App Store for $5.99. [Direct Link]

Article Link: Halide Developer Takes a Deep Dive Into iPhone XS Camera and Skin Smoothing Rumors

" the iPhone XS reduces the brightness of the bright areas and the darkness of the shadows. While the detail is still there," isn't that an improved (Dynamic Range)?

 

[Dorje Sylas]

So the solution again goes to "give users control". If Apple exposed various settings of the algorithm in a user accessable way, that could be tweeked on a device by device, shot by shot bases, this would be a non-issue and be hailed as a powerful feature.

Too little user control remains one of Modern Apple's problems.

 

[zerolight]

Seems to me that any photo I take. Even at direct sunlight. I get the same result with smart HDR on and off. I think there's a bug that keeps it enabled always. Even in 3rd party apps like Camera+.

 

[pika2000]

Sebastiaan de With, the developer behind photography app Halide, has taken an in-depth look at the front and rear-facing cameras in the iPhone XS and XS Max, providing some insight into complaints about a possible skin smoothing "beauty mode" that results in less realistic selfies than prior iPhone models.

A soft filter on selfies "doesn't exist," says de With, with the smoother look attributed to more aggressive noise reduction techniques and the merging of exposures that eliminates sharp light and dark contrasts where light hits the skin.

Camera differences between iPhone X and XS Max demoed by a Reddit user​

Both the front and rear-facing cameras in the iPhone XS and XS Max are using computational geography to improve photo quality, a feature that takes multiple shots at once and then merges them into a single perfect photo.

Apple's iPhone XS website explains all of the different things that are going on behind the scenes when an image is captured, and part of the process involves taking shots that are underexposed, overexposed, and captured at different times, with the camera taking the best elements of each shot and combining them.

Apple says the iPhone XS performs 1 trillion operations per photo​

According to de With, this results in a "whole new look" for photos that represents a "drastic departure" from images captured with previous-generation iPhones. As the Halide blog post explains, the small areas of contrast seen in photos from older iPhones can make images look sharper, but these areas of deep contrast have been eliminated in the iPhone XS models.

Using the exposure merging techniques, the iPhone XS reduces the brightness of the bright areas and the darkness of the shadows. While the detail is still there, we see it as less sharp because of this loss of contrast. Skin ends up looking smoother because the "light isn't as harsh."

The iPhone XS and XS Max are also using much more aggressive noise reduction techniques than in previous iPhones. de With says this is because the iPhone XS models prefer a faster shutter speed and a higher ISO level, capturing photos quicker but resulting in more noise. Taking care of that noise sacrifices some detail and contrast.

Noise in a RAW iPhone X image (left) compared to RAW iPhone XS image (right)​

In the front-facing camera specifically, which uses a smaller sensor than the rear-facing camera, the heavy noise reduction paired with the image merging techniques leads to the noticeably smoother selfies that people have seen with the iPhone XS camera.According to de With, Apple can tweak these settings through software updates if the majority of people are unhappy with the selfie camera, so we may see a better compromise between noise reduction and the filtering of harsh lighting in the future.

All in all, de With believes the iPhone XS camera is better than the iPhone X camera thanks to its superior dynamic range, with some post processing able to re-add the contrast where necessary.

There are issues with RAW image capture because of the noise level. When taking RAW photos with an app like Halide, de With says it is a must to do so manually and to reduce the exposure. Otherwise, RAW images will end up looking worse than Smart HDR JPEGs.

To account for the changes to RAW image capture, Halide is gaining a new Smart RAW feature that uses the new sensor tech in the iPhone XS to get better images. It avoids Smart HDR all together to cut down on noise reduction and pull out more detail.

Halide's full blog post is well worth a read for anyone interested in an in-depth look at the camera changes Apple has introduced in the iPhone XS and XS Max. The Halide app can be downloaded from the App Store for $5.99. [Direct Link]

Article Link: Halide Developer Takes a Deep Dive Into iPhone XS Camera and Skin Smoothing Rumors

Anybody that have dealt with RAW photo editing with Lightroom can tell this is more due to the more aggressive noise reduction.

Of course, once Apple tone this down, the same youtubers/tech bloggers will all scream and yell how the photos of iPhones are so noisy, and how they prefer the "beautified" photos from the Chinese phones. Seriously, Apple won't make these noise-makers happy, so I hope Apple has its own target quality, not changing the algorithm here and there just because some tech youtubers need more views.
[doublepost=1538470765][/doublepost]

This is silly. Anyone obsessing this much over their selfies is the exact market segment that Apple is trying to pacify with the "beauty filter"

Imo we should stop saying this as a "beauty filter." It is not. It's just an unintentional side effect of the new ISP algorithm and noise reductions. Noise reduction tools in photography software like Lightroom creates the same effect.

Actual beauty filter won't just smooth out the skin, it will whiten your skin, tighten your cheeks, and enlarge your eyes, just like in those Chinese phones.

 

[Diatribe]

This image shows how much over-processing is done in the native camera app. The RAW image from Halide (left) is NIGHT AND DAY amazing, almost looks like its from a film camera. Who cares if it has more noise, the color, mood, clarity beats the living hell out of the overly processed camera.app hands down.

I was going to post this same picture. The difference is astounding and I do not know who would ever choose the right one over the left one. Now if only I could access Halide from my lock screen...

 

[dilbert99]

This isn't a big deal, just some very minor skin smoothing on a camera that is primarily used as a selfie camera. It's a good thing, leave it on.

And what about the people that need/want it turned off?
[doublepost=1538480161][/doublepost]

Anybody that have dealt with RAW photo editing with Lightroom can tell this is more due to the more aggressive noise reduction.

So are you saying it is not a beauty filter and we have a choice between lots of noise or "beauty/smoothing" effect noise reduction?
[doublepost=1538480232][/doublepost]

It's a smart filter - and that is nice. but has it come to the point where we are obsessing over a smart filter? Is this a rumor, is this a game changer? Incredible strides in computer technology reduced to pandering to narcissism? There are amazing things computers can do, and this is just a pimple.

I thought the complaints were over not being able to turn the feature off.
[doublepost=1538480555][/doublepost]

This debate is hilarious. Professional photographer takes a smoothes things in Photoshop. Perfectly acceptable. Phone software does the same thing so that everyone can enjoy the same professional-style results? OMG THE WORLD IS ENDING!!!!

The same "professionals" that complained about the iPhone for photography for years now think their use case (the 1%) is more important than the use case of the 99% of iPhone users who aren't professionals and will benefit greatly from this software application to improved photography.

So nothing to do with not being able to turn the feature off?

 

[H3LL5P4WN]

So I'd hakide worth the $6? The phones we would use it on are iPhone X, 8, and maybe an SE, but the first two for sure.

Certainly my seens priced nicely if it does everything it says it does. I just hate tossing money st an app before being able to use it. Also imagine there's no way to set it as default camera, so launch g would require a dedicated app loading?

Halide is worth the money. It's the only 3rd party camera app that has graduated to homescreen status (the rest are generally one-trick ponies and get tossed in the "Camera Bag" folder).

 

[Spazturtle]

All digital cameras process the image, even a DSLR shooting in RAW mode will process the image. If you can't zoom in and see the sub pixels then the image has been processed. Modern DSLRs will even automatically fill in holes in the image caused by dead pixels on the sensor using the neighboring pixels as a reference.

There is enormous amounts of processing done to digital images, people who are asking for an option to disable processing simply do not understand what they are asking for.

 

[beermode]

All digital cameras process the image, even a DSLR shooting in RAW mode will process the image. If you can't zoom in and see the sub pixels then the image has been processed. Modern DSLRs will even automatically fill in holes in the image caused by dead pixels on the sensor using the neighboring pixels as a reference.

There is enormous amounts of processing done to digital images, people who are asking for an option to disable processing simply do not understand what they are asking for.

What is the point of your reply? It's been a long time since I've seen NR this aggressively bad.

 

[JWiLL1988]

I mean you can see the photos right?

I don't know the technical details, but this "beauty mode" is totally unacceptable to me.

 

[steveando]

As a photographer myself, I completely agree with his explanation. Essentially Apple is lowering the contrast of the skin by bringing the highlight levels down and the shadow levels up with their new image processing. This creates a flatter looking image. Lower contrast means less detail and makes the skin look smoother. If you like to edit your photos after they are taken, working with a flatter image can have its advantages. At the same time, I do feel like they may have gone a little too heavy handed on the image processing.

 

[ignatius345]

All digital cameras process the image, even a DSLR shooting in RAW mode will process the image. If you can't zoom in and see the sub pixels then the image has been processed. Modern DSLRs will even automatically fill in holes in the image caused by dead pixels on the sensor using the neighboring pixels as a reference.

There is enormous amounts of processing done to digital images, people who are asking for an option to disable processing simply do not understand what they are asking for.

But it's not an all-or-nothing thing either, is it? Nobody (but you) is saying anything about applying zero processing to images right off the sensor. Obviously sensor output needs processing to produce an image that most people would accept as a photograph, but if it's is so strong that detail is lost, it starts looking like something else.

Since you brought up DSLRs, take a look at an in-depth camera review over at dpreview or something and notice how carefully they evaluate each camera to see how much noise reduction is applied and what settings one can choose in that regard.

A phone-sized sensor and lens are much smaller and require more processing to output a pleasing image, but the principle is the same: noise reduction is a necessary evil, and if it's too heavy, the image starts to diverge from the user's expectation of what a photographed image should capture.

Contrary to what you say, people do know what they're asking for: a less aggressive noise-reduction algorithm -- or at least one that's smarter about separating out signal noise from details in the photographed subject.

 

[PickUrPoison]

But it's not an all-or-nothing thing either, is it? Nobody (but you) is saying anything about applying zero processing to images right off the sensor. Obviously sensor output needs processing to produce an image that most people would accept as a photograph, but if it's is so strong that detail is lost, it starts looking like something else.

Since you brought up DSLRs, take a look at an in-depth camera review over at dpreview or something and notice how carefully they evaluate each camera to see how much noise reduction is applied and what settings one can choose in that regard.

A phone-sized sensor and lens are much smaller and require more processing to output a pleasing image, but the principle is the same: noise reduction is a necessary evil, and if it's too heavy, the image starts to diverge from the user's expectation of what a photographed image should capture.

Contrary to what you say, people do know what they're asking for: a less aggressive noise-reduction algorithm -- or at least one that's smarter about separating out signal noise from details in the photographed subject.

I have no doubt Apple will improve their image processing algorithms. The hysteria and great offense some have expressed is rather amusing though. Nice payday for the YouTubers

 

[ignatius345]

I have no doubt Apple will improve their image processing algorithms. The hysteria and great offense some have expressed is rather amusing though. Nice payday for the YouTubers

I dunno, man. You spend $1K on a phone, you have a right to complain if the camera is weird.

And since the iPhone is a lot of people's primary/only camera it does matter if it's spitting out unnecessarily artificial looking images.

 

[PickUrPoison]

I dunno, man. You spend $1K on a phone, you have a right to complain if the camera is weird.

And since the iPhone is a lot of people's primary/only camera it does matter if it's spitting out unnecessarily artificial looking images.

Sure, and sending comments to Apple via the feedback page at https://www.apple.com/feedback/ would actually be constructive. But too many won’t bother, they’d rather rant and complain.

Of course many of them don’t even have an iPhone XS, so they really don’t care whether it gets fixed or not. They’re in it for the righteous indignation.

 

[Unimpossible]

Sure, and sending comments to Apple via the feedback page at https://www.apple.com/feedback/ would actually be constructive. But too many won’t bother, they’d rather rant and complain.

Of course many of them don’t even have an iPhone XS, so they really don’t care whether it gets fixed or not. They’re in it for the righteous indignation.

I’ll bet you it’s more effective to raise awareness on forums and social media, than using that inane feedback link.

And why do you have to own one to care? People are potential buyers.

 

[PickUrPoison]

I’ll bet you it’s more effective to raise awareness on forums and social media, than using that inane feedback link.

And why do you have to own one to care? People are potential buyers.

You’re assuming Apple doesn’t know about the issue or somehow needs to be pressured into improving the pictures. Rather, I think Apple is well aware of the issues and limitations of the XS’s v1.0 computational photography algorithms. No doubt they’ve been working on improvements since 12.0 was frozen.

And I’m sure potential customers do care, and they’re not who I’m referring to when I talk about people who really don’t care whether or not the problem gets fixed.

 

[kbk75]

Bought Halide a couple of days ago when I realised the photos on my Xs looked significantly flatter (and just flat-out worse...less contrast, less detail, unnatural skin tone) than the photos my partner’s X took with the default cam app. Halide makes the Xs images better than the X native cam app because you get the benefit of the Xs’ improved HDR performance without that crappy flat, no-contrast, smoothed-out look that the native cam app causes.

$6 well spent for me. I don’t doubt Apple will get around to fixing this but I’m glad a relatively inexpensive app allows me to enjoy the cam on my new phone without having to wait for Apple’s fix to happen!

 

[lordofthereef]

Halide is worth the money. It's the only 3rd party camera app that has graduated to homescreen status (the rest are generally one-trick ponies and get tossed in the "Camera Bag" folder).

Cool. I'll give it a go.

The. Most intriguing thing about the app is some of the features it seems to have that are exclusive (natively) on Xs models. Have you seen or done any comparisons between halide on an X and stock xs shots for photo correction?
[doublepost=1538514195][/doublepost]

Sure, and sending comments to Apple via the feedback page at https://www.apple.com/feedback/ would actually be constructive. But too many won’t bother, they’d rather rant and complain.

Respectfully, how do you know they don't do that too?

My most recent experiences with social media AND using a feedback link (email in this case) were with T-Mobile just last month. I got my issues sorted and they got back to me on social media and through a phone call.

I don't feel these two avenues of making one's voice heard are mutually exclusive. Social media has become a very really place to air grievances and get yourbissues solved. I used to be the guy that felt it was a waste of time to "rant" on places like that, but sometimes, increasingly more often, you do see results. And this isnt even acknowledging the potential solutions the community might offer up. I don't have an Xs but I discovered Halide as a direct result of these discussions. And I am all the better for it.
[doublepost=1538514408][/doublepost]

You’re assuming Apple doesn’t know about the issue or somehow needs to be pressured into improving the pictures.

The converse of thisnwould be to assume that everything will just get better (Siri, I'm still waiting) without mentioning anthing. That's the thing about discussing your issues. At worst you've wasted some time typing them out, as we have here lol.

 

[PickUrPoison]

Respectfully, how do you know they don't do that too?

My most recent experiences with social media AND using a feedback link (email in this case) were with T-Mobile just last month. I got my issues sorted and they got back to me on social media and through a phone call.

I don't feel these two avenues of making one's voice heard are mutually exclusive. Social media has become a very really place to air grievances and get yourbissues solved. I used to be the guy that felt it was a waste of time to "rant" on places like that, but sometimes, increasingly more often, you do see results. And this isnt even acknowledging the potential solutions the community might offer up. I don't have an Xs but I discovered Halide as a direct result of these discussions. And I am all the better for it.
[doublepost=1538514408][/doublepost]
The converse of thisnwould be to assume that everything will just get better (Siri, I'm still waiting) without mentioning anthing. That's the thing about discussing your issues. At worst you've wasted some time typing them out, as we have here lol.

I’m sure Apple is well aware of Siri’s shortcomings; no amount of hysterical outrage and ranting will get it fixed any faster.

I’m all for discussing the issues; blood-pressure spiking, vein-popping anger and faux-outrage trolling at a first world problem gets a little old. #GateFatigue

 

[tehStickMan]

If you watch the damn linked video you'll see the image processing happens to FACE ONLY after face is detected in the photo.

All y'all points about "Apple knows the best" are completely wrong.

 

[lordofthereef]

I’m sure Apple is well aware of Siri’s shortcomings; no amount of hysterical outrage and ranting will get it fixed any faster.

I’m all for discussing the issues; blood-pressure spiking, vein-popping anger and faux-outrage trolling at a first world problem gets a little old. #GateFatigue

I personally am not one to assume that a company is aware of all shortcomings of their product. That mentality defeats the purpose of your own recommendation of using the feedback page.

As far as trolling, I agree. But there's a difference between trolling and airing a grievance or explaining a customer issue. I'm defending the latter, never the former.