Combining Methods

Except for summing the frames together, combining frames may require correcting for variations between the frames due to different exposure times, sky backgrounds, extinctions, and positions. Currently, scaling and shifting corrections are included. The scaling corrections may be done by exposure times or by using statistics in each frame over a selected part of the image. The statistics can reveal (depending on the keyword `exp'_STA) setting, (where `exp' is the exposure type) for each image the mean, median, or the mode. In the following we refer to the value by MMM. Additive shifting is also done by computing the statistics in the frames.

The region of the frames in which the statistics is computed can be specified by the keyword `exp'_SEC. By default the whole frame is used. A scaling correction is used when the flux level or sensitivity is varying. The offset correction is used when the sky brightness is varying independently of the object brightness. If the frames are not scaled then special routines combine the frames more efficiently.

Below follows a simple overview how the weighting, scaling and offset parameters are determined. All obviously depend on the settings of the keywords `exp'_SCA `exp'_OFF, `exp'_WEI, and `exp'_EXP. The overview makes clear that offset corrections will only be applied if the scaling correction is switched off. The same is true for applying an exposure time correction.

==========================================================================
  o_i = 0.0 
  w_i = 1.0
  s_i = 1.0

  exp_SCA=yes
     s_i = M_i                        
     exp_WEI=yes
        w_i = sqrt(N*s_i)
    
  exp_SCA=no  
     exp_EXP=yes
        s_i = e_i 
        exp_WEI=yes
           w_i = sqrt(N*s_i) 
       
     exp_OFF=yes
        o_i = M_i/s_i
        exp_WEI=yes
           w_i = sqrt(N*s_i/o_i)

  s_i = s_i/s_mean
  o_i = (o_i - o_mean) * s_mean
  w_i = w_i/w_sum
--------------------------------------------------------------------------
  key: o_i:    offset for frame i
       o_mean: mean offset over all input frames
       s_i:    scale factor for frame i
       s_mean: mean scale factor over all input frames
       w_i:    weight factor for frame i
       w_sum:  sum over all weight factors of all input frames
       e_i:    exposure time of frame i
       M_i:    MMM of frame i
       N:      number of of frames previously combined
==========================================================================

In the combining no checks are done on the reduction status of the input frames and no attempts are made for any calibration correction like for bias or dark. Hence, in more complicated reduction sequences the user should be sure not to combine e.g. flat fields that have been corrected for bias and dark with flats fields that are not corrected.

Except for medianing and summing, the frames are combined by averaging. The average may be weighted by

weight = (N * scale) ** 1/2 (3.10)

where N is the number of frames previously combined (the command records the number of frames combined in the frame descriptor), scale is the scale factor depending on the keyword settings listed above (s_i or s_i/o_i). In most of the applications N = 1, i.e. the input calibration frames are the original ones and not the result of previous combinings.

There are a number of algorithms which may be used as well as applying statistical weights. The algorithms are used to detect and reject deviant pixels, such as cosmic rays. The choice of algorithm depends on the data, the number of frames, and the importance of rejecting cosmic rays. The more complex the algorithm the more time consuming the operation. For every method pixels above and below specified thresholds can be rejected. These thresholds are stored in the keyword `exp'_MET. If used the input frames are combined with pixels above and below the specified threshold values (before scaling) excluded. The sigma frame, if requested, will also have the rejected pixels excluded.

The following list summarizes the algorithms. Further algorithms are available elsewhere in MIDAS (see COMPUTE/..., AVERAGE/...), or may be added in time.

• Sum - sum the input frames.
  The input frames are combined by summing. Summing is the only algorithm in which scaling and weighting are not used. Also no sigma frame is produced.

• Average - average the input frames.
  The input frames are combined by averaging. The frames may be scaled and weighted. There is no pixel rejection. A sigma frame is produced if more than one frame is combined.

• Median, MMedian - (mean) median the input frames.
  The input frames are combined by medianing each pixel. Unless the frames are at the same exposure level they should be scaled. The sigma frame is based on all input frames and is only a first approximation of the standard deviations in the median estimates. The second method does an averaging around the found median in a certain interval in order to take into account the distribution of the values near the median. This is in effect the same what AVERAGE/IMAGE also does using the parameter setting 'options = median,low,high'. The required data interval has to be defined by the exp_CLP keyword and is assumed to specify relative limits to the determined median - same as in AVERAGE/IMAGE (both limits positive).

• Minreject, maxreject, minmaxreject - reject extreme pixels.
  At each pixel after scaling the minimum, maximum, or both are excluded from the average. The frames should be scaled and the average may be weighted. The sigma frame requires at least two pixels after rejection of the extreme values. These are relatively fast algorithms and are a good choice if there are many frames (>15).

• Sigclip - apply a sigma clipping algorithm to each pixel.
  The input frames are combined by applying a sigma clipping algorithm at each pixel. The frames should be scaled. This only rejects highly deviant points and so includes more of the data than the median or minimum and maximum algorithms. It requires many frames (>10-15) to work effectively. Otherwise the bad pixels bias the sigma significantly. The mean used to determine the sigmas is based on the "minmaxrej" algorithm to eliminate the effects of bad pixels on the mean. Only one iteration is performed and at most one pixel is rejected at each point in the output image. After the deviant pixels are rejected the final mean is computed from all the data. The sigma frame excludes the rejected pixels.

• Avsigclip - apply a sigma clipping algorithm to each pixel.
  The input frames are combined with a variant of the sigma clipping algorithm which works well with only a few frames. The images should be scaled. For each line the mean is first estimated using the "minmaxrej" algorithm. The sigmas at each point in the line are scaled by the square root of the mean, that is a Poisson scaling of the noise is assumed. These sigmas are averaged to get a line estimate of the sigma. Then the sigma at each point in the line is estimated by multiplying the line sigma by the square root of the mean at that point. As with the sigma clipping algorithm only one iteration is performed and at most one pixel is rejected at each point. After the deviant pixels are rejected the file mean is computed from all the data. The sigma frame excludes the rejected pixels.

The "avsigclip" algorithm is the best algorithm for rejecting cosmic rays, especially with a small number of frames, but it is also the most time consuming. With many frames (>10-15) it might be advisable to use one of the other algorithms ("maxreject", "median", "minmaxrej") because of their greater speed.

The choice of the most optimal combining algorithm will clearly depend on the nature of the data and on the exposure type. Therefore, for every supported exposure type the CCD context contains a default combining setup. Currently, there are five combining setups stored in the CCD keywords, all starting with a specific two letter prefix: for bias BS_, dark DK_, dome flats FF_, sky flats SK_, and for all other exposure types OT_. At initialization these keywords are filled with sensible defaults. Below we will shortly comment on combining the various calibration frames and list the default keywords settings.