Portrait Retouching Using Masks

If you want to make friends, learn how to retouch a portrait. Nobody likes themselves as-is, despite what they may say. It’s often been said that your goal should be to make a person look 10 years younger, but not more. If you go too far with retouching, you’ll make a portrait that looks completely fake. You won’t get thanked for that.

On the other hand, it’s commonly expected that portrait photographers are also dermatologists, plastic surgeons, dentists, and opthalmologists.

You will be doing yourself a favor if you shoot your photographs using camera picture controls such as “portrait” or “neutral”. Avoid “vivid” like the plague. My favorite is “neutral”. Use a slightly-long lens, like the classic 85 mm, to get a pleasing perspective. Back in the day, the 105 mm was king; it’s still a great choice for portraiture.

Face parts have many completely opposite requirements; some need sharpening and others need softening. Some parts need more saturation, some need less saturation.

To meet these contradictory retouching needs, the best tool is the mask. Many image editors support masking. I’m still a die-hard Nikon Capture NX2 fan, so I’m going to concentrate on how that program uses masks.

Lightroom (ala the ‘Adjustment Brush’) has masks. Lightroom’s “Auto Mask” can select non-circular shapes by looking for similar coloration, but I find their masking a bit too limiting. This is probably my least-favorite program for masks.

Photoshop, of course, has masks. It has always struck me as being just a bit too complicated and time-consuming for my taste, but that may be because I haven’t invested sufficient time in it. If you’re comfortable with it, then by all means use it.

Zoner Photo Studio Pro supports masks via the “Selection Brush”, lassos, circles, rectangles, etc. along with “Mask: Do Not Show”, “Mask: Normal/Inverted” etc. Similar to Capture NX2, you can soften the edges of the selection and erase your selection mistakes. Once you make your mask, you can apply softening, sharpening, or other effects that only affect what’s inside the mask.

Zoner Photo Studio Pro mask in the “Editor” tab

Zoner Photo Studio Pro masking example using the Brush Selection

Portrait Retouching Using Nikon Capture NX2

I want to show you how I use Capture NX2 to accomplish retouching, but I’ll let you pick your own favorite editor to get the same job done. I realize that Capture NX2 is now un-supported by Nikon. If you want to keep using it with your RAW images, then you can check out this article to convert your newer camera files into a RAW format that Capture NX2 can understand.

In this program, you perform mask selection/ adjustment pairs. After the adjustment, you click on “New Step”. Next, you select another mask and the adjustment associated with that mask.

Mask Tools

You need to know how to add a selection mask and, just as important, how to erase a selection mask. Some picture details, such as the corner of an eye, would be extremely difficult to accurately select in a single step. The mask selection brush is typically too wide to easily get into little nooks and crannies. Constantly adjusting the mask brush diameter is horribly inefficient and a losing proposition.

It’s much easier to paint outside of the lines, and then switch to the mask “eraser” to clean up your mask.

Capture NX2 mask controls: Add and Subtract

Don’t be afraid to use a mask that goes beyond the area you want

Switch to the mask “eraser” and clean up the corners

Finished mask after erasing around the nooks and crannies

Retouching Teeth

Few people have really white teeth. Nobody has perfectly-white teeth. When retouching, you need to “de-saturate” them and also brighten them. Don’t go too far with this. You want to reduce yellowing by lowering their color saturation, which will leave teeth looking gray. Next, you need to brighten the teeth (without making them pure white).

You want to select only the teeth to whiten and brighten them

Paint the mask (green) over the teeth in Capture NX2

After you mask an area, you will need to make it invisible while you apply an adjustment. In Capture NX2, you hide the mask by changing the selection from “Show Overlay” to “Hide Selection” as indicated by the arrow in the picture above on the right-hand side.

Fine-tune the saturation

After you use the “Selection Brush +” to mask only the teeth, select the Saturation/Warmth adjustment. Avoid 100% opacity, or your edits will look a bit “fake”. Also adjust the mask feathering, to avoid hard edges. Change the mask selection from “Show Overlay” to “Hide Selection” to see the progress of the saturation effect.

If you make mistakes while painting the (green) selection mask, simply click on the “Selection Brush –“ to erase the parts of the mask that you don’t want. Masks don’t have to be continuous, so that you can do things like selecting both eyes.

Teeth after de-saturation. Gray is better than yellow, but not by much.

Change the mask selection to “Hide Selection” while adjusting the saturation (or to see any effect you’re working on). The teeth may still look a little disappointing, since they changed from yellow to gray. Not to worry.

Increase brightness, but with a mask selecting only the teeth.

Choose the Brightness adjustment, while using the mask over the teeth. Change the mask to “Hide Selection” again, while increasing the brightness. Avoid the temptation to over-brighten the teeth; real teeth are slightly yellow and slightly gray.

When you’re happy with the way the teeth look, click the “New Step” to finish (assuming you’re using Capture NX2).

Fixing Eyes

Most eyes need three different adjustments. The iris typically looks better when its color is more saturated. Similar to teeth, the whites of the eyes may need some de-saturation and they always need brightening. You also want the eyes, brows, and lashes to be very sharp (via the ironically-named un-sharp mask).

Eyes with typical issues that need improvement

Mask used for brightening and de-saturation of any red color

Eyes and brows need extra sharpening

A portrait just won’t look good if the eyes, lashes, and brows aren’t sharp. Make a mask for them and apply the Unsharp Mask.

Repair Eye Bags

Bags under the eyes are typically a two-step process. First, they usually need more Gaussian Blur than the rest of the face, and possibly even some Healing Brush. They usually need extra brightness adjustment, too.

Make a separate mask for enhancing under the eyes

If makeup isn’t used under the eyes, then they usually need to be brightened and have a little healing brush applied. The brightening needs a mask, but the healing brush doesn’t.

Skin

Here’s some advice: don’t go crazy with the “Healing Brush”. You can waste a lot of hours trying to heal every blemish on an entire face. Try this instead: Gaussian Blur. You’ll find that you can usually hide skin blemishes in a single step by simply blurring the skin.

Moderation in all things. You really, really don’t want “Barbie Skin”. When you apply Gaussian Blur, remember to adjust the opacity away from 100%. Skin shouldn’t look blemish-free. And feather the mask edges, too. For males, you’ll generally use much less blur.

Make a face mask that avoids the eyes, brows, nostrils, and mouth

Perhaps the biggest improvement in most portraits is getting the skin blurred. This does not include the eyes and mouth, however.

Gaussian Blur for the skin

The Gaussian Blur can be pure magic. Again, don’t forget to allow a little of the original skin to show through. Keep the opacity around 80 percent, and use a generous “feather” for the face mask. Use a large enough blur radius to hide blemishes, but avoid making the skin look fake.

Conclusion

Portraits typically take more editing work than any other type of picture. Most pictures work just fine with ‘global’ adjustments, without any masking at all, but pictures of people rarely look good with this treatment.

A good job of portrait editing leaves the viewer with a sense that something’s different, but they can’t really put their finger on it.

Cosmetics and good lighting can certainly help portraits and reduce the retouching labor, but there’s really no substitute for skilled retouching.

#howto