How Samsung uses AI to improve photos taken on the Galaxy S21

It's always exciting to see what technology smartphone manufacturers use to improve the images on their latest smartphones. Samsung, in its blog post, details the magic it uses to improve portraits captured on its phones in fair detail.

First of all, it's all about making faces look natural and a little bit out of the subject at the same time. To do this, the software detects the two faces of the subjects and separates them from the rest of the image for post-processing. Samsung then uses "natural enhancements" to tweak things like the eyes, hair, face shape, and white balance to create natural faces. Here, Samsung basically takes the same approach as it does with manual digital photo editing, which also uses masks to tweak only certain areas of the image.

Precise portrait mode with sharpness gradient

Masks are also about blurring backgrounds. It's been a trend in camera smartphones for a few years now to mimic the depth of field of portrait lenses with a small open aperture, and in fact it worked very well in my review of the Galaxy S21+. For portraits, Samsung takes advantage of the depth information gained from the Time-of-Flight sensor on the S21 Ultra and the additional and slightly differently positioned cameras on the S21+.

As with the faces in the first example, Samsung's camera software creates a mask and then optimizes the transition between subject and background. By using a tri-mask consisting of background, subject and the area of transition, Samsung can also identify hair with greater accuracy and omit it when blurring.

At the same time, the software segments subjects if they are at different distances from the camera. This explains the quite natural-looking sharpness gradient you can see in my test photo of the Galaxy S21.

Source: Samsung