Ember Razement ________________________________ The Fine Art Photography of Malcolm Smith

 

NOISE CONTROL AND SHARPENING

PART B – SHARPENING TOOLS AVAILABLE

LINK TO LINK TO
PART A - Background Theory
PART C - Some Other Useful Sharpening Tools and Pointers
PART D - Edge Masks
PART E - Real World Sharpening Examples

This section looks at the sharpening tools available and how they work.  Some results are noted and some are contrary to popular belief.

GAUSSIAN BLUR

An integral part of Unsharp Mask (see next section) and image processing is Gaussian Blur and it is instructive to examine how it works and how the blurred pixel values are calculated from the surrounding pixels.  Gaussian Blur when applied to an image is a two dimensional filter but can be applied as two one dimensional filters first in the X direction then in the Y direction with the same result as a two dimensional filter.  So to make the discussion easier I will describe it in one dimension.  A very useful description of Gaussian Blur can be found at http://en.wikipedia.org/wiki/Gaussian_blur and even has some useful information for the mathematically challenged.

How does Gaussian Blur work?  Well in a one dimensional view it looks at the intensities of the pixel (or more than one pixel) on either side of the one we are calculating the blur for.  In fact it multiplies all the pixel intensities (the ones to either side and the centre one) by predetermined numbers (which depend on the type and spread of the blur) and sums the results to give the new blurred intensity.  These numbers (weights in mathematical terms) used to multiply the intensities are predetermined from a formula and depend on the type and spread of the blur.  For example lets look at the following weights  [ 0, 0.25, 0.5, 0.25, 0] – Note that they add up to one and also note the numbers have been chosen out of thin air to illustrate the process rather than giving any particular type of blur.

Now let us do some examples.  In the first calculation below the new blurred value is:
          New value := 0.25*70 + 0.5*70 + 0.25*70  := 17.5 + 35 + 17.5 := 70
This new value of 70 is the same as the original but this is not surprising as the original three pixels used in the calculation were 70.  But look at the following example that takes in more than one value of original pixel

Original unblurred Intensities

70

70

70

30

30

30

             
Weights 0.25 0.5 0.25      
Calculation 0.25X70 0.5x70 0.25x70      
Results 17.5 35 17.5      
New Value - 70 -      
             
Weights   0.25 0.5 0.25    
Calculation   0.25x70 0.5x70 0.25x30    
Results   17.5 35 7.5    
New Value   - 60 -    
             
Weights     0.25 0.5 0.25  
Calculation     0.25x70 0.5x30 0.25x30  
Results     17.5 15 7.5  
New Value     - 40 -  
             
Weights       0.25 0.5 0.25
Calculation       0.25x30 0.5x30 0.25x30
Results       7.5 15 7.5
New Value       - 30 -
             
Resultant new blurred pixel values 70 70 60 40 30 30

Now to quickly consider a real example.  The list of three numbers used in the previous example were out of my own immagintion and had no relation to any blurring function (as far as I know).  For Gaussian Blur the values are calculated with (you don’t actually have to know this):
      gaussian Blur equation
Where σ is the standard deviation of the distribution or, in our terms, the Radius of the blur and x is the number of pixels out from the center (the centre pixel has x:= 0)  To simplify things lets assume the Radius := 1 pixel.  We get the numbers:

0.0001 0.0044 0.0540 0.2420 0.3989 0.2420 0.0540 0.0044 0.0001

The numbers actually continue to the left and right but they are very small and can be considered zero.  The usual rule applied to this calculation is to assume numbers more  than 3*Radius out are zero.  For example a large Radius of, say, 4 pixels requires 12 pixel values on each side of the centre  An example of a 1 pixel blur using these numbers in a continuous (non discrete) world is:

jjjjj

Graph of Demonstration "blurring"

The important point to remember from this is that in two dimensions a Gaussian Blur of n pixels takes account all the pixels in the box 6*n pixels a side centered on the pixel currently being blurred.

The Wikipedia says the following about Gaussian Blurring “ The visual effect of this blurring technique is a smooth blur resembling that of viewing  the image through a translucent screen, distinctly different from the bokeh effect produced by an out-of-focus lens”.  So to properly blur a background out of focus use the Photoshop Lens Blur filter.

UNSHARP MASK – How it Actually Works

The strangely named Unsharp Mask is a traditional tool used in the wet darkroom to enhance the edges in images being optically enlarged and printed.  The technique in terms of a digital image is to:

  • Let the original be
    f(x,y)
  • Create a blurred version of the original
    blurred(x,y)
  • Subtract the blurred from the original image from the original.  This will give light and dark halos where there were rapid changes in intensity ie (edges) in the original image (ie where the original and blurred differ).  This original minus blurred could be called the “unsharp mask”.
    unsharp mask(x,y)  := f(x,y) – blurred(x,y)
  • The next step is to add the unsharp mask to the original and the result is Unsharp Mask (USM) sharpened.
    sharpened(x,y) := f(x,y) + unsharp mask(x,y)
  • In the above add process consider the amplitude of the “unsharp mask” as unity (100%) but this could be changed so that the lightness and darkess of the sharpen halos can be made greater or less.  The amplitude is represented by K in the equation below
    sharpened(x,y) := f(x,y) + K * unsharp mask(x,y)                (K=1 for 100%)

To illustrate this suppose we have an image with two shades of gray (lighter to the left and darker to the right) with a narrow transition between them.  If we take a profile of the intensities at right angle to the join we get the graph below (marked original).  Then you can follow through the blurring, subtracting to form the Unsharp Mask and finally adding the unsharp mask to the Original to now have sharpening halos.

How Unsharp Mask Works

How Unsharp Mask Works

You can actually try this using blend modes in multiple steps but you will need to use Image>Apply Image to get access to Add and Subtract blend modes.

In the Photoshop Unsharp Mask filter dialog box the Radius is the spread of (Gaussian) blur used to form the mask and Amount is the multiplier applied to the amplitude when the unsharp mask is added to the original image (K in the equation above but as a %).  A radius of 1 roughly equates to a halo width of 1 pixel spacing.

The third parameter is Threshold and all I can say is what the Photoshop books say Threshold sets a minimum difference between pixels before sharpening is applied.  Ie at lowish settings (eg 1 to 4) it can protect skin texture and slightly noisy skies from being sharpened - but using threshold zero and layer edge masks (discussed later) to protect these areas gives better control.

Overdo Unsharp Mask sharpening and you get artifacts giving the image a crunchy appearance.  The cause of this is twofold; the halos can become wide enough to become obvious (seen with the naked eye – see previous discussion on visual acuity) and secondly in the lighter and darker areas the amplitude of the halos can clip (or near clip) to white or black. Clipping to white is particularly bad as it really stands out.  There are ways of controlling this discussed later and this is particularly important as many Photoshop users control this by under sharpening (usually by massively reducing the Amount) which is really not very productive.  I commonly see comments that an image is over sharpened – this is usually wrong as it is the artifacts introduced (and which are generally controllable) that they are referring to and not the sharpness of the image lines.  The other aspect is that noise should not be made more noticeable and which is also controllable.

Because the problem is with the lighter and darker areas where the halos clip to white and black the primary method of control is to keep the sharpening away from the highlights and darks.  Do this by applying the sharpening to a new layer and then using the layer Blend If sliders for this layer (set in the Layer Style dialog box) to prevent the clipped halos from being seen.  The result will be the ability to set Amount to a much higher value and therefore give enhanced sharpening while eliminating the clipped artifacts.  The setting of Blend If sliders, Radius and Amount can be difficult to determine from a real image but can be modeled – at the appropriate points in the application of sharpening to address the different areas I will suggest some values which work for me.

Sharpening and Threshold Test Demonstration

Sharpening Demonstartion with Threshold Test

In the illustration above (a) is the original unsharpened image (b) has been sharpened with Radius 2, Amount 300 and Threshold 0 (c) using a Threshold adjustment layer the white lines show the part of (b) where the sharpened image exceeds 240 and (c) the sharpened image with Blend If sliders set to 190/235 for the white sliders.  The Threshold adjustment layer test, which is further explained later (see Part C – Some Other Tools and Pointers) would show for (a) and (d) above no pixels over 240.

So for Unsharp Mask AMOUNT Controls the intensity of the sharpening halos.  The RADIUS setting controls the width (and as demonstrated below to some extent the intensity) of the sharpening.  Larger Radius gives wider sharpening halos

Most books, web references and this tutorial describe the USM process using a continuous graphical representation but, of course, our images are not continuous but are sampled at discrete intervals (the sensor cell spacing).  If you model in Photoshop an edge transition correctly and vary Radius you will see that for radius 1 or less (assume radius 1 is the sensor pixel spacing) the width of the halo is pretty well one pixel spacing in width however as the Radius setting decreases below one the amplitude of the halo decreases (See table below)

The table below was created by setting up a 30% gray area adjoining a 70% grey area in Photoshop.  The image was sharpened and the light halo measured for a given Radius (corresponding results could have been obtained from the dark halos).  Each time the image was sharpened the Amount in USM was set to 100%.  Of course the sharpening was done on the original 30% & and 70% grey step each time.

LIGHTER HALOS (The figure listed below is the difference
between the halo height in % gray and the original 30% gray)

HALO # 1st 2nd 3rd etc    
RADIUS            
0.2 1 0        
0.3 6 0        
0.4 8 0 Blank = data not entered
0.5 9 0        
0.7 10 1        
1 12 3        
1.5 14 6 2      
2 15 9 4      
3 17 12 8 5 3 1
5 18 15 12 10 7 6
10 19 18 18      

Conclusions from this table

  • A Radius of 0.2 is not narrower than a Radius of 0.7.  The second halo for Radius 1.0 I would consider essentially not there as it is down to 25% of the first.  However if I wanted a one pixel halo I usually use Radius = 0.7)
  • The main difference between the halos for Radius between 0.2 and 1 is the amplitude of the halo (with the width essentially 1 pixel spacing)
  • As the Radius increases (over the useful range) the amplitude of the halos increases as does the likelihood of clipping (and crunchiness).  This increase is sometimes mentioned in discussions but is not understood and usually greatly underestimated.
  • The maximum halo height in this example is a step of 20 % in the gray levels as this is half the difference between the two modeled gray areas ( 0.5 * (70% - 30% ) = 20% ).  This is the limit of the Gaussian blur in the construction of the Halos in USM,
  • The Amplitude of the Halos is quite close to the theoretical maximum at radius 5 and virtually at maximum at Radius 10+ for this example.

So a fractional Radius setting does not change the width of the halos in our discrete Photoshop universe as you can see from the table but only the Amplitude changes and those that think otherwise are only fooling themselves.  If you truly want a halo of 1 pixel then I would use radius of 0.7.

To model this accurately the influence of the image blur should probably be included but it is difficult to interpret the results and in any case the images we are usually working with are considerably more complex than the model above.  But when you do your tests to determine your sharpening plan, you should consider the above in your choice of USM Radius.  Bruce suggests some settings for USM (se later in this tutorial) but he had not thought of the effect of our images being in discrete steps.

One comment I hear all the time is that different Radius settings in USM sharpens smaller or larger objects in the image – is this true?  For example I have found these statements associated with Unsharp Mask:

“Fine detail needs a smaller Radius, or else you may obliterate tiny detail of the same size as the Radius width.”

“The wide sharpening haloes obscure some of the fine detail”

“...created wide sharpening haloes that were much wider than the details in the image, so it ended up looking soft and losing detail.”

“No matter how much we increased the intensity of the sharpening …we’d still end up obscuring detail, because the haloes are too wide”

First most sharpening the values of Radius actually used are three or smaller which are mostly below the level you can usually see; and secondly when we use a very wide Radius (eg 50) for USM for local contrast (see below under Contrast Adjustment Using Sharpening Tools) we don’t obliterate large portions of the image.

I decided to test this so I set up an image with background of 60% gray and constructed three vertical bands of 40% gray four, two and one pixel wide.  Then using the marquee tool to select strips across the image I first applied Gaussian blur in three bands with Radius of one, two and four pixels.  Then lower down again with the marquee tool I sharpened four bands with radius of one, two, four and ten pixels.  The resulting image was:


Blur and Sharpen Examples

Blur and Sharpen Example

I don’t know if this will reproduce well so I have extracted some data.  Looking at the image it is obvious that with Gaussian Blur as the Radius gets larger the objects get more blurred, the smaller objects getting blurred before the larger.  Eventually the objects disappear with increasing radius.  The following date was extracted from the one pixel wide band with the eyedropper – remember the steady state level is 60% gray.

Pixels Out From the Centre 0 1 2 3
Blur radius        
1 52 55 58 60
2 55 56 58 59
4 58 58 59 59

Gaussian Blur

So Gaussian Blur blurs out objects.  But if you look at the way USM works you will realize that as the object is being blurred out the unsharp mask is getting bigger and consequently so are the sharpening halos.  You can see this in the image above but let’s look at the data I extracted:

Pixels Out From the Centre 0 1 2
USM radius      
1 27 65 62
2 25 64 62
4 22 62 62
10 22 62 62

Unsharp Mask

In this table the 0 pixel is the originally 40% gray and the 1 and 2 pixels out were originally 60% gray.  So in this model the lighter halos don’t dull out but get brighter as Radius goes from 1 to 4 then reaches a steady state.  The darker pixels don’t change from the 60% gray by much but actually are darker at USM Radius = 1. 

I wondered if my pattern was influencing the results so I tried other patterns including a checkerboard pattern of one pixel cells 40% and 60% gray with similar results with the brighter pixels getting brighter.  I also did some tests with a range of real images amd came to the same conclusion that the quotes above are myth.

Sometimes light texture (and similarly dark) can be burnt out to white and, of course, the texture disappears when this happens, but this is another problem and one that can be addressed by the Blend If slider in Layer Style.  So it looks like the belief that increasing the radius causes larger objects to be sharpened but not the smaller is not true.  There also seems to be some who believe that wider halos have a life of their own and will overwrite smaller ones (but see the earlier discussion of Gaussian Blur) – to test this out draw some image intensity changes and blur.  In any case if we are considering Radius of the order of 3 pixels and objects of this size will be at the limits of visability.

What may be correct is that a low contrast edge (ie one where the difference in intensity between say a face and background is quite small) then a larger Radius may be used successfully.  But in the normal scheme of things this difference in intensity between the face and background will not be trivial and with a wide radius you may see the halos.  I am not saying that face edges against backgrounds usually require wide Radius sharpening.  I suspect this whole area grew like the passing of messages in the childrens game of Chinese Whispers http://en.wikipedia.org/wiki/Chinese_whispers from some observation made years ago.

One effect that could have started this myth however, is that for wider halos (at least up to Radius 5 pixels) as the halo width is increased so also does the intensity (amplitude) of the halos.  So the effect could be our old friend clipping to white rather than the effect being described in the quotes.

SMART SHARPEN

The Photoshop (Filter>Sharpen>Smart Sharpen) has more controls and variations than Unsharp Mask and supports different algorithms for addressing:

  • GAUSSIAN BLUR (may be the same as Unsharp Mask)
  • LENS BLUR (Uses a different algorithm than Gaussian and Unsharp Mask); and
  • MOTION BLUR Designed to remove blur caused by subject motion. 

 

There are advanced controls to target some of the sharpening to the Shadows and Highlights areas.  I don’t at the moment have enough information about how the different algorithms work (particularly the Lens Blur tab) and have not done enough modeling so I currently don’t use Smart Sharpen but this may change as I develop further.  See also my comments below under Deconvolution sharpening.

HIGH PASS FILTER

Sharpening with the High Pass filter works on a different principle to Unsharp Mask.  Here the High Pass filter (Filter>Other>High Pass) when used on a new layer of the image creates a mostly 50% gray layer with lighter and darker halos where there are rapid changes (edges) in intensity in the original.  When a contrast enhancing blend mode (such as Hard Light or Soft Light) is used with this layer it results in no change on 50% gray and when lighter or darker than 50% grey (the edge halos) the blend mode Screens or Multiplies the image sharpening the original image edge transitions.

This “High Pass sharpening” is done on a separate layer and the sharpening process is controlled by adjusting the layer Opacity (this can only adjust the amount of sharpening down.  As said above use Hard Light blend mode (or Soft Light if you particularly want a minimal sharpening).  A common mistake is to use the Overlay blend mode with High Pass filter sharpening but if you look at the mathematics of the Overlay and Hard Light blend modes you will see that Overlay (although it appears to sharpen the image) will not do quite what you desire – it has to do with which layer (the current high pass layer or the one below) is used to control the switching between Screen and Multiply.

It is interesting to compare USM and High Pass sharpening.  Consider the previous 30% to 70% step we used to look at the discrete USM and compare the corresponding halos for a Radius of 3 pixels and Amount 100% in USM.  We get:

HALO HEIGHTS FOR HIGH PASS AND USM

HALO # 1 2 3 4 5 6
High Pass 10 7 5 3 2 1
USM 17 12 8 5 3 1

The amplitude of the halos is less for High Pass but the ratios of the heights between the two approaches is close.

DECONVOLUTION SHARPENING

Deconvolution sharpening works on a different principle.  USM and High Pass sharpening don’t really sharpen – they create edge halos or edge Acutance (remember this from film discussions in the dim past?) which gives the appearance of making the image sharper.  Deconvolution sharpening actually sharpens by reversing mathematically the blurring process.  For example lens blur can be modeled mathematically but there are many variables and the success of a deconvolution sharpener depends on how close the blur modeling is to reality.

Photoshop Smart Sharpen (at least on the Remove Lens Blur tab) is claimed by some to be based on deconvolution – if so presumably Radius refers to the amount of blur (from the lens and the camera anti-aliasing filter?) and Amount of correction applied (100% being full correction but if so what does 500% correspond to).  Unfortunately Adobe don’t seem to be very forthcoming in telling us how their software works which makes it very difficult to gain proper insight to their use – with sharpening it is particularly important that the tools be applied with some knowledge.

I have used for some time a deconvolution sharpening product called Focus Magic.  I quite like this product because there is a description of how the parameters work and, most importantly, there is a built in test probe you can use to examine the amount of out-of-focus blur in various parts of the image.  They also document how the blur radius can be estimated by examining the pixels at high zoom around a point light source or very bright object (such as a specular source in the image).  Although the probe doesn’t always work (usually it is obvious that it hasn’t been able to estimate the blur) by pointing it at various hard edges such as eyelashes, fingernails, jewelry, defined textures etc, it can come up with an estimate.  I usually use this product for source sharpening whereas content and output sharpening I use USM or High Pass to enhance the acutance of edges,

The downside is that Focus Magic costs something like A$100 to buy.

Go to TOPTOP

LINK TO LINK TO
PART A - Background Theory
PART C - Some Other Useful Sharpening Tools and Pointers
PART D - Edge Masks
PART E - Real World Sharpening Examples