QC documentation system: QC procedure QCocam_flat.py for OMEGACAM

This documentation is intended both for QC scientists and SciOps astronomers (who may want to ignore the technical information displayed in grey).
 
HOME
 
CRIRES
FORS1
FORS2
KMOS
NACO
 
GIRAFFE
UVES
XSHOOTER
 
ISAAC
SPHERE
VIMOS
VISIR
 
HAWKI
MUSE
SINFONI
 
AMBER
MIDI
PIONIER
 
OMEGACAM
VIRCAM
[best viewing with style sheets and javascript enabled]
QCocam_qcheck.py QCocam_shutter.py QCocam_readnoise.py QCocam_lingain.py
QCocam_bias.py QCocam_dark.py QCocam_dome.py QCocam_flat.py
QCocam_std.py QCocam_sci.py
 
bottom  plots   |   contact
NAME QCocam_flat.py
VERSION 1.0 -- 2012-06-10 First draft of this QC report
SYNTAX Python
CALL measureQuality of data contained in $DFS_PRODUCT/TWILIGHT/$DATE
within processQC: processQC -a $AB
explicit call: QCocam_flat.py -a [$AB_name] -e [$ext_number: 1-32] -i [$level_of_interaction: 0-2]
INSTRUMENT OMEGACAM
RAWTYPE FLAT (DPR.TYPE = FLAT,SKY)
PURPOSE a) generates 3 QC reports for each OmegaCAM CCD
b) writes QC1 parameters into local QC1 database omegacam_twilight
PROCINPUT No input is required:
a) $DATE is read from AB
b) the primary file is set in QCocam_flat.py and is the PRO.CATG=MASTER_FLAT product with the _0000.fits extension.
c) raw frames are implicitly read from the AB.
d) All reference products: MASTER_FLAT, MASTER_FLAT_TWILIGHT, and BAD_PIXELS_MAP are found and read in from $DFO_ROOT_DIR_FC/calib/references/TWILIGHT/.
QC1TABLE trending | table(s) in QC1 database:
omegacam_twilight
TRENDPLOT trending | HealthCheck plot(s) associated to this procedure:
trend_report_TWILIGHT_raw_rSDSS_HC.html
QC1PAGE trending | associated documentation:
twilight_QC1.html
QC1PLOTS
top

twilight1.png
display each of 32 MASTER_FLAT frames on ds9 display.

twilight2.png
display each of 32 MASTER_FLAT_TWILIGHT frames on ds9 display.

twilight3.png
display each of 32 BAD_PIXELS_MAP frames on ds9 display. Note that cold pixels have a value of 1. Normal pixels = 0.

twilight4.png
left: full-field image of the master flat (twilight + dome flat) of the extension listed at the top of the image
center left: 4x zoom of the center area of the master flat
center right: the difference of the reference master flat and the current master flat frames
The median level of the difference image is listed to the right of the frame
right: full-field image of the bad pixel map (hot + cold pixels). The positions of the reference frame bad pixels are shown
in red, while the bad pixels of the current dome flat frame are shown in blue. The total numbers of bad pixels are listed
in the frame legend.

twilight5.png
top left: a) three single column traces of the current and reference master flats through:
x = 256 (dark grey)
x = 1024 (middle grey)
x = 1792 (light grey)
b) average of all columns (normalized) of current master flat (thick blue line)
c) average of all columns (normalized) of reference master flat (thick red line)
top center: a) three single row traces of the current and reference master flats through:
y = 256 (dark grey)
y = 2048 (middle grey)
y = 3840 (light grey)
b) average of all rows (normalized) of current master flat (thick blue line)
c) average of all rows (normalized) of reference master flat (thick red line)
top right: a) logarithmic histogram of current master flat (blue line)
b) logarithmic histogram of reference master flat (red line)
The median levels (as computed from a Gaussian fit) are shown as vertical dashed lines and listed in the legend
lower left: difference between the average column traces of the reference and current master flat frames
lower center: difference between the average row traces of the reference and current master flat frames
In both plots, the zero line is shown in orange as a reference.
lower right: logarithmic histogram of the difference between the reference and current master flat frames.
Ideally, the histogram should straddle the zero line (shown in orange).
The median difference (as computed from a Gaussian fit) is shown as vertical dashed green line.

twilight6.png
left: full-field image of the second raw twilight flat executed in the template
center: full-field image of the last raw twilight flat executed in the template
top right: a) six single column traces of the current raw twilight flats through:
x = 256 (dark grey) for the second and last raw twilight flats
x = 1024 (middle grey) for the second and last raw twilight flats
x = 1792 (light grey) for the second and last raw twilight flats
b) average of all columns of the second raw twilight flat (thick red line)
c) average of all columns of the last raw twilight flat (thick blue line)
bottom right: a) six single row traces of the current raw twilight flats through:
x = 256 (dark grey) for the second and last raw twilight flats
x = 1024 (middle grey) for the second and last raw twilight flats
x = 1792 (light grey) for the second and last raw twilight flats
b) average of all rows of the second raw twilight flat (thick red line)
c) average of all rows of the last raw twilight flat (thick blue line)
QC1PARAM QC1 parameters written into QC1 table(omegacam_twilight) 
SOURCE(header or script)      DBNAME                  value       description

 PRO.DATANCOM                datancom                 5           the number of raw files combined in pipeline processing
 QC.MASTER.FLAT.MEDIAN       qc_flat_median           1.003423    median counts in normalized master flat (ADU)
 QC.MASTER.FLAT.MEAN         qc_flat_mean             0.945673    mean counts in normalized master flat (ADU)
 QC.MASTER.FLAT.STDEV        qc_flat_std              0.047343    standard deviation of normalized master flat (ADU)
 QC.MASTER.TWILIGHT.MEDIAN   qc_twilight_median       1.003423    median counts in normalized master twilight flat (ADU)
 QC.MASTER.TWILIGHT.MEAN     qc_twilight_mean         0.987343    mean counts in normalized master twilight flat (ADU)
 QC.MASTER.TWILIGHT.STDEV    qc_twilight_std          0.034223    standard deviation of normalized master twilight flat (ADU)
 QC.NUMBER.BAD.PIXELS        qc_bad_pixels            3564        number of bad pixels (cold + hot pixels)
 QC.RAW.TWILIGHT.MIN         qc_raw_twilight_min      8567.54     median flux level of input raw twilight flat having the minimum average flux level (ADU)
 QC.RAW.TWILIGHT.MAX         qc_raw_twilight_max      28567.54    median flux level of input raw twilight flat having the maximum average flux level (ADU)
 QC.RAW.TWILIGHT.MEAN        qc_raw_twilight_mean     18567.54    mean flux level of all input raw twilight flats (ADU)
 QC.RAW.TWILIGHT.MEDIAN      qc_raw_twilight_median   17567.54    median flux level of all input raw twilight flats (ADU)
 QC.RAW.TWILIGHT.STDEV       qc_raw_twilight_std      450.23      standard deviation of the flux level of all input raw twilight flats (ADU)
 QCocam_flat.py              qc_temp_mosaic_plate     146.5       average temperature probe of mosaic plates from DET.TLM5/8 (QC script)
ALGORITHM Description of algorithms:
qc_flat_median: the master flat frame is created from 4 input, twilight flat exposures (each with >20,000 ADU flux levels), and the master dome flat. Although the twilight template generally consists of 5 sky flats, the first frame is made to determine a good count-to-exposure time and therefore it is excluded from the master twilght recipe. The input frames are bias-corrected, trimmed, corrected for bad pixels (hot and cold pixels), and averaged together with a 5 sigma rejection of outliers (intended to reduce photon shot noise and to remove cosmic rays. This results in two primary products: a master_flat_twilight and a master_flat, with the latter a combination of the master dome flat (for the pixel-to-pixel variations) and the master twilight flat (to account for the global shape of the flat field). This QC parameter is the median level of the full master flat frame.
qc_flat_mean: the mean of the full master flat frame.
qc_flat_std: the standard deviation of the full master flat frame.
qc_twilight_median: the median level of the full master twilight flat frame.
qc_twilight_mean: the mean of the full master twilight flat frame.
qc_twilight_std: the standard deviation of the full master twilight flat frame.
qc_bad_pixels the total number of bad (cold + hot) pixels flagged in the master dome flat frame (cold pixels) and in the master bias frame (hot pixels).
qc_raw_twilight_min: the median flux level of the input raw twilight flat having the minimum average flux level (ADU). This QC parameter is used to ensure that the weakest sky flat has enough flux to create a statistically accurate master twilight flat.
qc_raw_twilight_max: the median flux level of the input raw twilight flat having the largest average flux level (ADU). This QC parameter is used to ensure that the sky flat with the most flux does not reach the saturation limits of the detector.
qc_raw_twilight_mean/median/std: the statistics (mean, median, and standard deviation) of all input raw twilight flats (ADU).
CERTIF - Since the master flat produced by the OmegaCAM pipeline is normalized, it is not easy to detect whether or not one of the input sky flat frames was either saturated or had insufficient flux. For this reason, the raw twilight flat statistics are monitored. QC report #3 will clearly show a saturated raw twilight, or a raw twilight with too little flux. Both parameters are scored.
- A saturated twilight flat should be rejected and reported to Paranal.
- It may also happen that the u_SDSS band twilight flat flux levels are very low (i.e. <5000 ADU). This is OK.
COMMENTS OmegaCAM twilight flat frames are normally taken daily to match the science filters used and consist of 5 raw, filter-specific sky flats. It should be noted that the first sky flat obtained is used to set the counts-to-exposure time and hence is excluded from the pipeline processing.
top [2015-11-05T15:16:47]         created by qcDocu v1.1.2, a tqs tool