Gentlemen and Ladies
Below a description of the EWS (Expert Work Station:
pronounced "Ooze") software for coherent estimation
of MIDI visibilities. The package consists of C-routines
that do most of the work and IDL front-ends for selecting
data and viewing results.
I. Summary of C-Language programs II. Summary of tcsh scripts calling C-Language Programs III. Summary of IDL routines calling tch scripts(probably what you want to know!) IV. Summary of other IDL routines V. More details about (I) VI. More details about (II) VII. More details about (III) VIII. More details about (IV) IX. Description of file selection GUI (Gorgonzola) X. Standard file names and contents used by scripts
1. oir1dCompressData "inFiles" maskFile outFile 2. oirFormFringes infile outfile [-smooth nSmooth] [-removeAverage] [-no20] 3. oirRotateInsOpd infile outfile [waveRefFile] 4. oirGroupDelay infile outfile [-smooth gSmooth] [-weight weightFile] [-search] 5. oirRotateGroupDelay infile delayFile outfile 6. oirAutoFlag infile delayFile flagFile [-maxopd maxOpd] [-minopd minOpd] [-jump jumpOpd] [-side nDrop] 7. oirAverageVis infile flagFile outFile 8. oirChopPhotometry "Afiles" "BFiles" maskFile outFile [-nDrop nDrop] [-dSky dSky] (OBSOLETE) 9. oirChopPhotoImages -in "inFiles" -out outFile [-nDrop nDrop] [-dSky dSky] [-ref curveFile] 10. oirMakePhotoSpectra -A Afile -B Bfile -mask maskFile -out outFile 11. oirSci2HiPhot -AB ABfile -outbase outFileBase -ref refCoordFile -cross CrossCoeffFile 12 oirRescaleSpectra templateFile scaleFile outFile 13. oirCrossCoeff Afile Bfile coordFile outFile 14. oirRedCal sourceTag 15. oirCalibrateVis targTag calTags [-calflux calFlux10] [-cald calDiameter] 16. oirCurveCal -A Afile -B Bfile -out outFile [-fudge fudge]
1. dispVis "infiles" maskFile outTag fringeSmooth delaySmooth remAve minopd 2. dispPhot "AFiles" "Bfiles" maskFile outTag dSky 3. dispSciPhot "ABFiles" maskFile CurveFile CrossFile outTag dSky 4. dispSciPhotPhot"ABFiles" maskFile CurveFile CrossFile photoFile outTag dSky 5. dispCrossCoeff AFile Bfile coordFile outFile
1. midiPipe, tag, files=filelist[,files] [,mask=maskfile] [,smooth=smooth] [,gsmooth=gsmooth] [,dSky=dSky] [,/dAve],[,minopd=minopd],[/fringeimage]
2. midiSPipe, tag, files, cross=cross [,photoFile=photoFile]
[,curve=curve] [,mask=maskfile] [,smooth=smooth] [,gsmooth=gsmooth] [,dSky=dSky] [,/dAve]
3. midiVisPipe, tag, filelist [,mask=maskfile] ,[smooth=smooth] [,gsmooth=gsmooth] [,/dAve],[minopd=minopd],[/fringeimage]
4. midiPhotoPipe, tag, filelist [,mask=maskfile] [,dSky=dSky] [,curve=curve]
5. midiCalibrate, sourceTag, calTag, [photoTag=photoTag] [,calFlux10=calFlux10], [,diam=diam] [,/noPhot] [,/print]
6. midiSearch, tag, fileList [,mask=mask] [,smooth=smooth] [,gsmooth=gsmooth]
7. midiCrossCoeff, tag, fileList [,curveFile = curveFile]
1. midiDelayPlot, tag, title 2. corr = midiGetCorr (tag[,wave=wave]) 3. phase = midiGetPhase (tag[,wave=wave]) 4. phot = midiGetPhot(tag) 5. opd = midiGetOpd(tag) 6. delay = midiGetDelay(tag) 7. choppedData = midiChopImage(inputFiles) 8. keyValue = midiGetKeyword(keyword, inputFiles) 9. p = midiPhase (cArray) 10. c = pseudoComplex (pcArray)
Purpose:
Compress MIDI raw (PRISM/GRISM) files by multiplying each MIDI
window by a floating point mask (which has zeros in rows
not containing useful data) and summing in the y-direction
(perpendicular to the PRISM/GRISM dispersion). The result
is a 1-dimensional spectrum for each window region.
Parameters:
"inFiles" a list of raw MIDI (PRISM/GRISM) input files representing
one exposure. There may be more than one file because they
were broken up into 100MB files by the on-line system.
If there is more than one file they should be separated
by spaces and included between double quotes: "file1 file2 file3"
maskFile: a MIDI style FITS binary table whose imaging_data section
contains the masks described above. If this is not
specified it defaults to an environmental variable specified
externally (typically in the vltisetup script):
for PRISM/HIGH_SENS $prismhmask
for PRISM/SCI_PHOT $prismsmask
for GRISM/HIGH_SENS $grismhmask
for GRISM/SCI_PHOT $grismsmask
outFile: the name of the output file. This will be a MIDI style
FITS binary table in exactly the same format as "inFiles" and
maskFile except that the y-dimension of each DATA array is 1.
Purpose:
Take the output of oir1dCompressData and form spectrally dispersed
fringes by subraction of the two interferometic channels and
supression of background. The latter is accomplished by
the subtraction itself, by a high-pass filter (in
the time domain) applied to the individual spectral channels,
and finally, optionally, by removal of the mean flux (over wavelength)
from the individual spectral channels.
This removes instrumental and sky backgrounds that vary
more slowly than the fringe. The fringe should vary quickly
because of the OPD modulation imposed by the MIDI piezos.
Parameters:
inFile: name of the output from oir1dCompressData
outFile: output file name. The format is a MIDI style FITS table
imaging_data file with only one DATA region (
the combined I1-I2 channels).
nSmooth: (optional) width of boxcar used in highpass filtering.
Default = 50
Data is smoothed in time with boxcar, and this smoothed version
is subtracted from input data, yielding a highpass filter.
-removeAverage: If specified, the all pixels in the spectrum in
each frame are averaged together, and this average is subtracted
from this individual pixels. The sky background often generates
a "DC" offset of the whole spectrum, which is not entirely removed
by the preceeding filtering actions. This offset generates
an artefact at zero OPD that can confuse oirGroupDelay if the
source flux is very weak. Specifying -removeAverage removes
this artefact, but it also removes your source if it crosses
zero during a OPD scan.
Default: no removal of average
Suggestions:
specify -removeAverage if you observed with offset OPD tracking
(then there are no OPD zero crossings)
specify -removeAveage if your source is very weak and the
delay solutions returned by oirGroupDelay show a zigzag
pattern typical of zero OPD artefacts.
if you use -removeAverage be sure to use it on your calibrators too.
Purpose:
Remove the known Instrumental OPD components from the fringe spectra
that oirFormFringes produced. Each data point at frequency
k (in spatial units: 2 pi/lambda) is multiplied by exp(-i*k* OPD)
where OPD is the sum of the OPD from the MIDI piezos and the
VLTI delay lines.
Special Notes:
1. The result is a FITS imaging_data file in
"pseudo-complex" format. I.e. it is in REAL format
but each row is twice as long as the original data:
each value is a (real, imaginary) pair.
2. The VLTI Delay values may contain large fictitious
offsets of up to 1cm if the system was not correctly
zeroed at setup. To avoid applying these large
fictitious offsets, the program determines the
MODE of the tracking positions during the run and
subtracts this before applying the OPD shift. This
modal value is printed by the program and also
stored in the output header as "OPD0" in meters.
Parameters:
inFile: name of the output from oirFormFringes
outFile: output file name. The format is a MIDI style FITS table
imaging_data file in pseudo-complex format.
Purpose:
Estimate the group delay for each measured spectrum. The
group delay is the peak of the Fourier Transform of the
spectrum (thus in the delay domain, not the frequency
domain). Each spectrum is fourier transformed. Note
that because the MIDI beam-combiner has only two output
phases it is essentially a cosine correlator and not
a complex correlator. This means that the group-delay
has TWO peaks: at positive and negative delays.
Because the actual OPD was modulated by the MIDI piezos
and this modulation was deRotated in oirRotateInsOpd,
one of the two peaks is nearly stationary from frame
to frame (only atmospheric OPD movement remains) while
the other peak moves around with TWICE the instrumental
modulation. Thus if we average a few frames together
the wrong side band is strongly suppressed. In this
program a gaussian smoothing of sigma = gSmooth frames
(default 4) used in this averaging.
Parameters:
infile: output of oirRotateInsOpd
outFile: output file containing TWO interesting tables:
1. A pseudo-complex imaging_data table containing
the raw FFTs of the input data
2. A DELAY table containing the value of the peak
delay (after smoothing) of the above table at
each frame. This DELAY table has three columns:
TIME (same as raw data), TELESCOPE (always 1,2)
and DELAY (peak of group delay, with OPD0 added),
in seconds (IAU standard). Thus you have to
multiply by the speed of light to get meters.
gSmooth (optional; Default: 4)
If specified can change the smoothing in the frame domain
before looking for a delay peak. Default is gaussian sigma = 4
frames. Choose larger value, like 10-15 for weak sources in good weather,
shorter for strong sources in bad weather.
weightFile (optional; Default: uniform spectral weighting)
If specified this is the name of a MIDI type file containing an
IMAGING_DATA table with at least one record containing
a 1 x nwavelength array. This will be used as relative weights
for the input spectra before looking for the group delay. This
can improve the S/N for sources with a non-flat spectrum.
-search (optional; Default: tracking mode)
If specified oirGroupDelay assumes that this is SEARCH mode
data covering a large OPD range, rather than TRACK mode data.
In SEARCH mode, the program slides its relative OPD zero point
along with each scan to avoid running out of range during the FFT.
Data of this type can be analyzed with the IDL program midiSearch.
Purpose:
Remove the group delay measured by oirGroupDelay as
well as the instrumental OPD from fringe data
(the output of oirFormFringes). Same algorithm
as specified in oirRotateInsOpd.
Additionally estimate an offset PHASE for each
frame and remove it. This phase is primarily
the result of water vapor dispersion (i.e.
non-constant index of refraction). Several
spectra are averaged together to remove the
beats from the off-side-band, then the
phase offset is determined and removed. The
smoothing is a rolling gaussian filter with
sigma = 6 pixels.
Parameters:
infile: output of oirFormFringes
delayFile: output of oirGroupDelay
outfile: output, in format identical with
output of oirRotateInsOpd
Purpose:
Try to choose (automatically) which frames to include in the output
visibility estimation: This is currently NOT done on the basis of
estimated amplitude since this biases the result. The current
criteria are:
1. OPD tracking distance from 0-point. Experience shows significant
attenuation of fringe (a few procent) for PRISM if this is larger
than about 150 microns. So if ABS(instrumental OPD-group OPD) >
maxOpd the frame is flagged.
When the OPD tracking point crosses zero MIDI can get confused with
sky background, especially if oirFormFringes is run
with the -removeAverage option. So if ABS(instrumental OPD-group OPD) <
minOpd the frame is flagged.
2. Jumps in OPD. There are some instrumental OPD jumps (due to
telescope refocusing) where the fringe is presumably attenuated.
If the 2nd difference in the group OPD is > jumpOpd the frame is
flagged.
3. In addition, nDop points on either side of jumps are flagged.
Output is written into a FLAG Fits table that specifies the flagged
time intervals.
Parameters:
inFile: output of oirFormFringes (only used to read instrumental
OPD)
delayFile: output of oirGroupDelay
flagFile: name of file to contain FLAG tables
maxOpd: max acceptable difference between tracking and true OPD (microns)
minOpd: min acceptable difference between tracking and true OPD (microns)
jumpOpd: OPD jumps larger than this (microns) are flagged
Purpose:
Average all unflagged but phase rotated frames together to arrive
at a single (complex) visibility.
Output is in IAU/ESO OI_VIS format FITS tables which is different
from imaging_data. These can be accessed with my IDL oirGetVis
routine: data_array = oirGetVis(file, wave=wave) where
wave contains the channel wavelengths in microns. data_array
is an array of structures containing (among other things)
visamp and visphi (amplitude and phase).
Parameters:
infile: output of oirRotateGroupDelay
flagFile: output of oirAutoFlag (or manual flagging)
outFile: name of OI_VIS file
Purpose:
This program is now obselete. Please use oirChopPhotoImages)
and oirMakePhotoSpectra.
Reduce separate chopping photometry observations for use
in normalizing visibilities. An estimate of sky is made
and removed from arrays. They these are summed in the y-direction
both with and without Masking (as described in oir1dCompressData).
The mask should be IDENTICAL with the one specified there.
Input are two separate (sets of) files containing raw chopping
data for AOPEN and BOPEN shutter positions.
Output is a imaging_data FITS table file with 12 rows, each row
containing a single DATA1 array:
1. Total Flux (ADU/s/channel) for AOPEN shutter position
2. Total Flux (ADU/s/channel) for BOPEN shutter position
3. Total Flux GeometricMean of A&B (i.e. of rows 1 and 2).
4. Masked Flux (ADU/s/channel) for AOPEN shutter position
5. Masked Flux (ADU/s/channel) for BOPEN shutter position
6. Masked Flux GeometricMean of A&B (i.e. of rows 4 and 5).
7-12 are the (poorly) estimated rms of the above quantities.
Parameters:
"AFiles" blank separated list of AOPEN chopped photometry files
"BFiles" blank separated list of BOPEN chopped photometry files
maskFile mask as described in oirCompressSPec
outFile FITS imaging_data file containing output
nDrop exclude this many frames after a transition from Sky to Target
if chop mirror has not settled (if not specified default = 2)
dSky Normally after Target-Sky subtraction, an additional sky correction is
estimated at the top and bottom of the slit. This is necessary for
faint sources (under 1Jy). The program normally estimates the sky from
rows 7-11 and 23-27 (PRISM) or 4-8 and 27-31 (GRISM). If specified,
dSky indicates that the user wishes to move these rows dSky pixels apart
(dSky positive) or together (dSky negative). If dSky is large
enough so that the sky region moves out of the measured window, no
on-slit sky subtraction is performed (this is indicated in the
text output of the program).
Purpose:
The first step of the operations for reducing chopping photometry.
Sky and Target frames are averaged target-sky computed, and an on-slit estimate of residual
sky is made and removed. The result is output as a set of detector images for one, possibly
multiple, input file. For HIGH_SENS photometry this must be run sequentially for the AOPEN
and BOPEN photometry input. For SCI_PHOT photometry it must be run once on the same
file used for interferometry. The output contains one set of images for the average of all
good input frames, and then a series of such sets made with (5) independent subsets of the
input frames so that the rms variations in calculated parameters can be estimated.
Parameters:
"inFiles" a list of raw MIDI (PRISM/GRISM) input files representing
one exposure. There may be more than one file because they
were broken up into 100MB files by the on-line system.
If there is more than one file they should be separated
by spaces and included between double quotes: "file1 file2 file3"
outFile output file name
nDrop exclude this many frames after a transition from Sky to Target
if chop mirror has not settled (if not specified default = 2)
dSky Normally after Target-Sky subtraction, an additional sky correction is
estimated at the top and bottom of the slit. This is necessary for
faint sources (under 1Jy). The program normally estimates the sky from
rows 7-11 and 23-27 (PRISM) or 4-8 and 27-31 (GRISM). If specified,
dSky indicates that the user wishes to move these rows dSky pixels apart
(dSky positive) or together (dSky negative). If dSky is large
enough so that the sky region moves out of the measured window, no
on-slit sky subtraction is performed (this is indicated in the
text output of the program).
refFile file containing accurate description of curvature of spectra on detector in order
to more accurately position sky subtraction zones. If not present spectra
are assumed to be straight.
Purpose:
Apply a mask to the output(s) of oirChopPhotoImages
and compute all kinds of geometric and arithmetic averages of the AOPEN and BOPEN channels.
They these are summed in the y-direction both with and without Masking
(as described in oir1dCompressData) to produce spectra.
The mask should be IDENTICAL with the one specified there.
Input are two separate (sets of) files containing raw chopping
data for AOPEN and BOPEN shutter positions.
Output is a imaging_data FITS table file with 12 rows, each row
containing a single DATA1 array:
1. Total Flux (ADU/s/channel) for AOPEN shutter position
2. Total Flux (ADU/s/channel) for BOPEN shutter position
3. Total Flux GeometricMean of A&B (i.e. of rows 1 and 2).
4. Masked Flux (ADU/s/channel) for AOPEN shutter position
5. Masked Flux (ADU/s/channel) for BOPEN shutter position
6. Masked Flux GeometricMean of A&B (i.e. of rows 4 and 5).
7-12 are the (poorly) estimated rms of the above quantities.
Parameters:
"AFiles" black separated list of AOPEN chopped photometry files
"BFiles" black separated list of BOPEN chopped photometry files
maskFile mask as described in oirCompressSPec
outFile FITS imaging_data file containing output
nDrop exclude this many frames after a transition from Sky to Target
if chop mirror has not settled (if not specified default = 2)
dSky Normally after Target-Sky subtraction, an additional sky correction is
estimated at the top and bottom of the slit. This is necessary for
faint sources (under 1Jy). The program normally estimates the sky from
rows 7-11 and 23-27 (PRISM) or 4-8 and 27-31 (GRISM). If specified,
dSky indicates that the user wishes to move these rows dSky pixels apart
(dSky positive) or together (dSky negative). If dSky is large
enough so that the sky region moves out of the measured window, no
on-slit sky subtraction is performed (this is indicated in the
text output of the program).
Purpose:
Convert chopped interferometric data in SCI_PHOT format into something that looks like what
you would have gotten if you had run a standard photometric sequence (AOPEN/BOPEN) in SCI_PHOT
mode and which can be digested by oirMakePhotoSpectra.
If you have chopped raw tracking data in SCI_PHOT mode, first run it through
oirChopPhotoImages to subtract sky from target
frames and average everything together. The output is then the input (ABfile)
to the current program. The photometric channels (channels 1 and 4) are bent according
to the information in refCoordFile and rescaled according to the coefficients in
crossCoeffFile to what you would have measured in channels 2 and 3 if you had observed with
only one shutter open. There are then two output files: outFileBase.Aphotometry.fits and
outFileBase.Bphotometry.fits that simulate the result you would have gotten with shutters
AOPEN and BOPEN. These can now be fed into oirMakePhotoSpectra
to produce photometric spectra.
Parameters:
ABfile: output of oirChopPhotoImages for an ABOPEN file containing
chopped data.
outFileBase: prefix for the two output files. These will be named outFileBase.Aphotometry.fits and
outFileBase.Bphotometry.fits
refCoordFile: a "data" file containing an IMAGING_DETECTOR table that describes the curvature of the
spectra on the detector. This information will be used to recurve the photometric channels to
look like the interferometric channels (so that the same masks can be applied). This will
typically be created by a run of oirCurveCal.
crossCoeffFile: a "data" file containing the cross coupling coefficients between the interferometric
photometric channels. This will typically be created by a run of oirCrossCoeff.
Purpose:
This program is somewhat different in purpose than the previous. It
takes the output files from several of the previous steps, as
applied to a calibrator and computes the instrumental visibility
and spectrophotometric calibration.
In this and the programs described below I use "tags" to identify
all the files belonging to one observation set. The full file
names are tag.type.fits where type describes the file type
(see complete listing below). Thus oirRedCal expects all
input files to have this type of name.
The output of oirRedCal is an OI_VIS table with 3 rows:
1. Instrumental visibility (amplitude and phase)
2. Spectrophotometric calibration (ADU/s/Jy)
3. "Strehl Ratio": not really, but the ratio of masked
counts/total counts per spectral channel.
Parameters:
sourceTag: the initial part of input file names, may also include a
directory. e.g. hd10380 or /strw11/jaffe/ngc4151
The type flags actually used are photometry (output of
oirChopPhotometry), and correlated flux (output of oirAverageVis),
thus the real file names would be hd10380.photometry.fits
and hd10380.corr.fits.
output: is not specified. The output is written into a file named
sourceTag.redcal.fits and contains the abovemention OI_VIS table.
This program is obsolete. Use oirSci2HiPhot
Purpose:
Measure source photometry from chopped data in non-interferometric
SCI_PHOT channels. Simultaneously determine the best "mask" for
selecting photometric and interferometric data.
Parameters:
"data": blank-separated list of SCI_PHOT input data files
refCoordDef: FITS file containing "imaging_detector" table specifying
wavelength and curvature date for SCI_PHOT areas on detector
cross: FITS file containing cross-coupling coefficients ("kappa-coefficients")
between photometric and interferometric channels. Produced by
oirCrossCoeff
outFile: file name to receive photometry output. In same format as
output of oirChopPhotometry.
outMaskFile: file name to receive calculated mask. To be used as
input to oirCompressSPData
nDrop: (optional) as in oirCompressSPData, drop this many points
after change of chopping state
sigmaY: (optional) sigma (standard deviation) width of gaussian mask weights
in the Y-direction. Defaults to 2.0
Purpose:
Combine reductions of a target and calibrator observations to produce
calibrated visibilities and plots. The output is three FITS table files.
1. An OI_VIS table file: targetTag.calvis.fits containing one
row per input row in the file targetTag.corr.fits
This contains calibrated visamp and visphi.
2. An OI_VIS table file: targetTag.calcorr.fits containing one
row per input row in the file targetTag.corr.fits
This table contains calibrated correlated fluxes and phases
3. An imaging_data file: targetTag.calphot.fits containing two
rows:
1. calibrated photometry (Jy) total
2. calibrated photometry (Jy) under mask
Parameters:
targetTag: descriptor for science target. The types actually
used are .photometry and .corr
calTag: descriptor for calibrator. The type actually used
is .redcal
output is as described above: targetTag.calvis and .calphot
-sp spTag : if specified use this calibrator tag to define
a spectrophotometric calibrator different from the visibility
calibrator. Use this if the flux/spectrum of the visibility
calibrator are unknown are inappropriate for photometric
calibration.
-calflux calFlux10: specify the flux of the calibrator used
for flux calibration in Jansky at 10 microns. A Rayleigh-
Jeans spectrum is assumed. If not specified 1 Jy is used.
-cald calDiameter: visibility calibrator diameter in milliarcsec.
If not specified 0.0 mas is used.
-nophot: Assume that source photometry is not very accurate.
The effect is that correlated fluxes are computed as:
Jy(calibrator) * raw_correlated_flux(target)/raw_correlated_flux(calibrator)
instead of Jy(calibrator) * visibility(target) raw_total_flux(target)/raw_total_flux(calibrator)
Purpose:
Take a reduced photometric data file (e.g. template.photometry.fits) produced
by dispPhot or midiPhotoPipe from AOPEN/BOPEN
chopped photometry data, and rescale it using photometry data on the same target taken in
SCI_PHOT mode (produced by dispSciPhot or oirSci2HiPhot)
so that output = (A + BX) * template, where A and B are
chosen to give the best least squares fit of output to scale.
Parameters:
templateFile: a nice reduced standard output of chopped photometry data taken
in standard AOPEN/BOPEN mode with the SCI_PHOT beam combiner.
scaleFile: a nice reduced standard output of photometry data taken from
the outer PA+PB channels of interferometry data taken in SCI_PHOT mode.
output: name of output file
Purpose: calculate cross-coupling coefficients (kappa coefficients)
for SCI_PHOT mode data
Parameters:
Afile output of oirChopPhotoImages representing data
taken in chopping photometry mode with the shutter in AOPEN position
Bfile output of oirChopPhotoImages representing data
taken in chopping photometry mode with the shutter in BOPEN position
coordFile: FITS file containing "imaging_detector" table specifying
distortions and wavelength information for this SCI_PHOT detector
mode.
outFile: output file name containing the kappa coefficients in the form of fits
IMAGING_DATA tables.
Purpose:
Read reduced chop imaging files (from oirChopPhotoImages).
Fit gaussians to the spectra at each x-position, then fit power laws to the centers
of these gaussians to represent the curvature of the spectra with x. When you
are done, write the coefficients of these power laws into the dmc/dmp parameters
in the IMAGING_DETECTOR tables in the output file. While you are at it, since
you've done most of the work already, write out an IMAGING_DATA table with
gaussians as specified by the power laws because these make a plausible mask for
extracting photometry and correlated flux from normal measurements.
Parameters:
AFile: output of oirChopPhotoImages for data taken on
a bright star with the shutter in AOPEN position.
BFile: output of oirChopPhotoImages for data taken on
a bright star with the shutter in BOPEN position.
output: name of FITS file to contain output tables
fudge: optional; The half widths of the exponential used to create the masks are increased
by this factor. If omitted, 1.0 is used.
The "tag"."type" fields are used to communicate file names.
Performs in sequence:
oir1dCompressData +
oirFormFringes +
oirRotateInsOpd +
oirGroupDelay +
oirRotateGroupDelay +
oirAutoFlag +
oirAverageVis
Parameters:
"infiles" blank separated list of interferometric files
maskFile as described in oir1dCompressData
outTag first component of file names to use for output;
may include directory path
fringeSmooth high pass filter width used in oirFormFringes
(c.f. above)
delaySmooth smoothing gaussian width used in oirGroupDelay
(c.f. above)
Performs photometry for data taken in HIGH_SENS mode
Runs in sequence:
oirChopPhotoImages on AFiles
oirChopPhotoImages on BFiles
oirMakePhotoSpectra on the output of the above two programs.
Parameters:
"AFiles": input file(s) taken without interferometric beam combiner with shutter = AOPEN
if mutiple files per exposure they should be enclosed in "" and separated by blanks.
"BFiles": input file(s) taken without interferometric beam combiner with shutter = BOPEN
maskFile: file containing mask file as described in oir1dCompressData.
outTag: prefix to output file: outTag.photometry.fits
dSky: sky subtraction position parameter: see "#oirChopPhotoImages">oirChopPhotoImages
curveRefFile: file containing data describing curvature of spectra, c.f. oirSci2HiPhot
if omitted: use straight lines for sky subtraction
Performs equivalent of dispPhot for SCI_PHOT data:
runs in sequence:
oirChopPhotoImages on ABFiles
oirSci2HiPhot on output of above program,
oirMakePhotoSpectra on output of above program,
Parameters:
ABfiles: file(s) containing SCI_PHOT mode data taken with chopping.
maskFile: see oirMakePhotoSpectra
curveFile/crossFile: see oirSci2HiPhot
outTag: prefix added to output file names
dSky: see oirChopPhotoImages
Performs equivalent of dispSciPhot for SCI_PHOT data, but
at the last moment, the PA+PB photometry files created by oirSci2HiPhot are
used only to rescale classical photometry data of the same target contained in photoFile, using
oirRescaleSpectra. This scripts runs in sequence:
oirChopPhotoImages on ABFiles
oirSci2HiPhot on output of above program,
oirMakePhotoSpectra on output of above program,
oirRescaleSpectra on output of above program+ photoFile,
Parameters:
ABfiles: file(s) containing SCI_PHOT mode data taken with chopping.
maskFile: see oirMakePhotoSpectra
curveFile/crossFile: see oirSci2HiPhot
photoFile: file containing fully reduced standard AOPEN/BOPEN photometry of the same target star.
outTag: prefix added to output file names
dSky: see oirChopPhotoImages
Computes cross coupling (kappa) coefficients from input chopped photometry data
Runs in sequence:
oirChopPhotoImages on AFiles
oirChopPhotoImages on BFiles
oirCrossCoeff on the output of the above two programs.
Parameters:
"AFiles": input file(s) taken without interferometric beam combiner with shutter = AOPEN
if mutiple files per exposure they should be enclosed in "" and separated by blanks.
"BFiles": input file(s) taken without interferometric beam combiner with shutter = BOPEN
CurveFile: input file containing IMAGING_DETECTOR table describing curvature of MIDI spectra.
outTag: prefix to output file: outTag.photometry.fits
Purpose: run dispVis, dispPhot, and oirRedCal on 3 files specifying
interferometric and photometric raw data. Effectively a
complete but uncalibrated reduction of this set of data.
Plots tracking versus estimated group delay and estimated
instrumental visibility. After running midiPipe on a science
target and a calibrator you should run
midiCalibrate,tag, caltag
to calibrate the science data.
Optionally: run dispCenter to create
an image of the interferometric signal on the detector, to
check star overlap.
This procedure produces many files, of which the most important
for routine reduction are:
tag.corr.fits
containing the estimated correlated flux (instrumental units)
tag.photometry.fits
containing the estimated photometric fluxes (instrumental units)
tag.redcal.fits
containing the estimated instrumental visibility
For experts:
tag.groupdelay.fits
contains group delay tracking information.
Optionally:
tag.fringeimageave.fits
contains an image of the raw correlated signal on the detector.
Parameters:
tag: output tag to identify files to dispVisPhot
files (optional) a string array of 3 input file names, each of
which may be a blank-separated list of files. The files must
specify one interferometric observation and two (A- and B-
shutter) photometry observations.
If files is not specified, the GORGONZOLA
GUI will come up to allow manual specification of files.
NOTE: files may be specified in at least 3 different ways:
1. specify midiPipe,tag,files = filesarray... where filesarray contains the desired files.
2. specify midiPipe,tag,filesarray... with filesarray as a 2nd argument
3. specify midiPipe,tag... with no filesarray at all. Then the GUI will come up.
mask (optional): mask file name. If not specified a default
depending on the instrumental setup. For old prism high_sens
files (before Dec 2003) use:
mask = getenv('drsRoot')+'/maskfiles/prismmask_before_2003dec01.fits'
smooth:Delay high-pass smoothing width to supress background
fluctuations. Default is 50 but use a smaller value if
your source is very weak.
gsmooth: group delay estimation averageing half-width (frames). Default
is 4 but use larger value for weak sources during stable atmospher,
smaller for strong sources during rapid OPD changes. For weak
sources during rapid OPD changes, give up.
dSky: move on-slit regions for sky estimation (c.f. oirChopPhotometry ).
A large positive number means no on-slit sky estimation.
/dAve: if specified the -removeAverage option is applied in oirFormFringes. See the text there for details.
This removes fringe artifacts near the OPD=zero point which can be useful when you are looking
for weak sources but may reduce correlated flux. If you specify /dAve, then the
-minopd 15 microns option is specified in oirAutoFlag, which
then flags frames within 15 microns of the zero OPD crossings.
minopd: frames where the differences between the estimated OPD and
tracking OPD are less than minopd(microns) are flagged as
unreliable. Defaults: if /dAve is specified, minopd defaults to
20 microns. If /dAve is not specified, minopd defaults to 0. microns
/fringeImage: If specified, an additional set of steps is executed
taking a fairly large amount of time, that generates a file
tag.fringeimageave.fits containing
a (complex) image of the raw correlated flux on the detector.
Purpose: run dispVis and dispSciPhot or dispSciPhotPhot
to reduce a (chopped/interferometric/SCI_PHOT) input file, and optionally some chopped/photometry/SCI_PHOT data.
Parameters: as in midiPipe with the addition of:
file(s): an array of either 1 or 3 character strings specifying (blank separated) sets input
files. If there is only one string, this contains data taken in interferometric mode
with shutter in ABOPEN position. The interferometric signal is extracted with
dispVis. If photoFile is NOT specified, the photometric
signal is extracted from the same input files with dispSciPhot.
If photoFile IS specified, dispSciPhotPhot is
run. This extracts the photometric signal in the same way, but uses it to rescale
previously reduced real photometric observations using oirRescaleSpectra.
If 3 input arrays are given, the reduction proceeds with rescaling as just described, but
the raw photometric observations in AOPEN/BOPEN mode are specified in the
2nd and 3rd strings.
coord: file containing distortion/wavelength information
for individual MIDI channels, not yet included automatically
with data from Paranal. Defaults to either of :
$drsRoot/software/minrts/config/mioSetup_FIELD_PRISM_SCI_PHOT_SLIT.tmp
$drsRoot/software/minrts/config/mioSetup_SPECTRAL_GRISM_SCI_PHOT_SLIT.tmp
cross: file containing kappa/cross-coupling coefficients. Should have been computed
from a previous run of midiCrossCoeff. It is the User's
responsibility to do this.
photoFile: contains the reduced output of standard AOPEN/BOPEN photometric observations of
the same target. Optional. If provided, and file[s] only contains one, interferometric
data set, it will be used as a template for rescaling the photometric data, as described
in oirRescaleSpectra.
run oirCalibrateVis with specified parameters. In addition a whole bunch of plots is produced in a file named sourceTag.ps. It also produces the FITS files: sourceTag.calphot.fits containing calibrated photometry, sourceTag.calvis.fits" containing the calibrated visibility. If /print is specified, a dozen or so sourceTag.xxxx.dat files are produced containing the information that was plotted as ASCII tables. sourceTag.calcorr.fits" containing the calibrated correlated flux (analogous to uncalibrated sourceTag.corr.fits"
Execute dispVis which reduces the interferometric scans specified by fileList and produces Tag.corr.fits, and Tag.groupdelay.fits files. This is same as midiPipe but the photometric reduction is skipped. See midiPipe for details. Note slightly different syntax for fileList.
Execute dispPhot which reduces the photometric scans specified by fileList and produces Tag.photometry.fitsfile.
Similar to midiVisPipe but the fileList should point to data taken in fringe Search mode instead of fringe Tracking mode. The output contains the usual fits files and also a plot of fringe amplitude versus OPD during the search phase.
Calculate the Cross Coupling (Kappa) coefficients between the interferometric and photometric channels for data taken with the SCI_PHOT beam combiner. The input raw data should consist of two file (sets) containing AOPEN and BOPEN data with this combiner for a bright star. This routine runs dispCrossCoeff after digesting the input arguements and filling in defaults. This dechops the input images, straightens out the curved photometry channel images, sums the photometry in the y-direction, and divides the interferometry images by the photometry images to yield the kappa coefficients in the output file. Inputs: tag: output tag to identify files to dispCrossCoeff. fileList: an array of two strings, containing the AOPEN and BOPEN files. As usual if these are multiple files, these should be concatenated and separated by blanks. curve: a file containing descriptions of the curvature of the spectra. You can generate this yourself using oirCurveCal. If not specified this midiCrossCoeff uses the environmental variable prismscurve or grismscurve which are usually to be found in $drsRoot/software/minrts/minrts/config with file names like: minrtsCurve_FIELD_PRISM_SCI_PHOT.fits or minrtsCurve_SPECTRAL_GRISM_SCI_PHOT.fits
Purpose: Make a plot of tracked OPD and estimated Group Delay as a function of time for data which has already been reduced using midiPipe or midiVisPipe. The plot is similar to the one shown by these functions, but by setting !x.range or !y.range before calling the routine, or setting other IDL plot variables, you can influence the appearance of the plot, or direct it to a file, etc. Parameters: tag: as in midiPipe, sets a directory and prefix that determines which data to use. title: a character string to be used at the top of the plot as a title.
corr = midiGetCorr (tag[,wave=wave]) Purpose: return a vector containing the (uncalibrated) Correlated Flux computed by midiPipe, for the data specified by tag. I.e. this is the visamp vector stored in the file tag.corr.fits Parameters: tag: as used everywhere. wave = variable_name: returns into the specified variable a vector of wavelengths (microns) corresponding to the data arrays described below. Returns: a vector of values specifying the uncalibrated correlated flux amplitude (ADU/s) for the data specified by tag
phase = midiGetPhase (tag[,wave=wave]) Purpose: return a vector containing the (uncalibrated) phase of the correlated signal, as computed by midiPipe, for the data specified by tag. I.e. this is the visphi vector stored in the file tag.corr.fits Parameters: tag: as used everywhere. wave = variable_name: returns into the specified variable a vector of wavelengths (microns) corresponding to the data arrays described below. Returns: a vector of values specifying the uncalibrated correlated phase (degrees) the data specified by tag
phot = midiGetPhot(tag) Purpose: return a structure containing the (uncalibrated) photometry for the data specified by tag. Parameters: tag: as used everywhere. Returns: an IDL structure named whatever is on the left-hand side of the = sign containing the uncalibrated photometric data as computed by midiPipe or by midiPhotoPipe. This is the data contained in the file tag.photometry.fits. For example phot[0].data1 contains a vector of the estimated total flux (ADU/s) from telescope A.
opd = midiGetOpd(tag) Purpose: get OPD tracking positions for the data specified by tag. Parameters: tag: as used everywhere. Returns: an vector sign containing the OPD tracking position (microns).
mydelay = midiGetDelay(tag) Purpose: get estimated Group Delay positions for the data specified by tag. Parameters: tag: as used everywhere. Returns: an IDL structure with the group delay information returned by oirGroupDelay The more interesting elements in this structure are: mydelay.delay: the estimated group delay (microns) for each frame mydelay.time: time (MJD days) of each frame mydelay.amplitude: amplitude of fringe signal found at this delay (ADUs)
choppedData = midiChopImage(inputFiles) Purpose: Average all "target" frames in a set of chopped data (e.g. target acquisition or photometry frames) and subtract the average of all "sky" frames. Parameters: inputFiles: name of files, if multiple files this should be a single, blank-separated list: e.g. "file1.fits file2.fits" Returns: an IDL structure with elements like "choppedData.DATA1" "choppedData.DATA1" that represent the signals in each of the MIDI windows for this data. The structure has two rows: choppedData[0] contains Target-Sky while choppedData[1] contains Sky.
keyValue = midiGetKeyword(keyword, inputFiles) Purpose: Get keyword values from primary headers of one or more FITS files. Parameters: keyword: a character string specifying keyword. For ESO-style HIERARCH keywords specify at least two of the keywords and enough to be unique, e.g. "FILT NAME". inputFiles: a single character string or array of character strings specifying a set of files to search. If an array is specified, an array of values will be returned. If the keyword is missing in the header, the program returns 0. If it is present in some files and missing in others the program will probably crash. Returns: an IDL array containing the values of the specified keywords.
p = midiPhase (cArray) Purpose: compute the phase of an array of complex numbers. Parameters: cArray: an IDL complex array Returns: An array of the phases of each element of cArray, i.e. arctan(Im(cArray)/Re(cArray)), in the range -pi <-> pi.
c = pseudoComplex (pcArray) Purpose: convert a 2-dimensional array of pseudoComplex numbers into IDL complex numbers and transpose the array. Several of the c-programs, such as oirGroupDelay produce output data arrays that are in fact complex. These are stored in the FITS files as pseudoComplex, i.e. as pairs of real numbers. Inside IDL it is easier to deal with true complex numbers, so this routine converts the FITS pairs into IDL complex numbers. In addition, most of the FITS files are stored with channel number as the X-axis and frame number, or time as the Y-axis. For visualization it us usually more convenient the other way around, so pseudoComplextransposes the array. Parameters: pcArray: a 2-d array of real numbers, each two values representing one complex number, arranged in channel-time order. Returns: An array of the complex numbers arranged in time-channel order.
EWS includes a data selection GUI, somewhat like GASGANO. To browse your raw data, and select some of it type:
FileList = midiGui()There are alternate versions described below. There is a faster version that caches the contents of the file headers on disk, but it only works if you have write permission on the data disks:
FileList = midiGuis()Note that IDL is indifferent to upper/lower case letters in commands and variable names (but not indifferent to case in FileNames)
It brings up a GUI and returns a list of files selected.
The gui is pretty spartan. Up at the top there are 10 buttons that let you select files, modify the display a bit, and quit.
Below this there is a scrollable area that shows information on each of the available files and on the left a column of ASTERISKs (*) showing which files are currently selected, or blanks if they are not. Initially files are brought up all un-selected.
In the next column you see the file name. NOTE: Due to ESO limitations on file lengths, MIDI files cannot be longer than 100 MB or so, while a single observations may generate Gigabytes of data. The on-line data system breaks the data up into 100 MB files. Gorgonzola recognizes this process, and the displayed file name is only the FIRST file of all the files that constitute an observation. When Gorgonzola returns, all the sub-files for an observations are concatenated into a single character string with spaces between the file names. Most of the EWS programs can recognize such a string and process the files sequentially. NOTE further, however, that if you use such a string to call a C-program directly (e.g. oir1dCompressData), you have to enclose the concatenated string in double quotes "...".
The other columns show keyword values for each observations, gleaned from the file primary FITS headers.
Two things:
We start with the easy ones:
QUIT causes the GUI to leave without making a selection
SELECT causes the GUI to leave and returns the selected files
UP/DOWN IDL and X-windows crash if thousands of widgets are put
on the screen, so I have limited the display to 100 rows.
If the rows you want to see are outside this range you
can push UP or DOWN to scroll. NOTE that the scroll bar
on the right only allows scrolling within the 100 displayed
rows.
HIDE Remove all currently unselected rows from the display. This
can considerably simplify things if you've already de-selected
a lot of files.
SHOW Reverse of HIDE: show all files, selected or not. This gives
you the opportunity to selectively re-select unselected files.
Sometimes the first few non-selected files are not displayed
after hitting the SHOW button. Try clicking the UP button.
To the right of this row the current HIDE/SHOW mode is displayed
Then the real meat: the BOOLEAN buttons.
These are not strictly boolean but sort of.
When you click one of them, the "caught" value
is compared to all other values in the same column
and the currently list of selected files is modified:
In the second row down on the GUI are six comparison operators:
= Equal != Not Equal > Greater than < Less than <= Greater than or equal > Less than or equalTo the right of this row the currently selected comparison is shown.
In the third row two combination operators:
AND The results of the comparison are
logically ANDed with the current selection
OR The results of the comparison are
logicall ORed with the current selection
Confused? It actually works fairly intuitively.
Suppose you have made a hundred or so exposures during a day, and suppose you are looking for a group of exposures with the PRISM in HIGH_SENS mode. The fifth column of selction buttons is labelled "INSGRIS" (shortend from INS GRIS NAME). Look down this column, with the slider if necessary until you find a button with PRISM displayed. Click on this and the words INGRIS and PRISM should appear below the AND/OR buttons to indicate your selection. Now click on the OR button. All observations with GRIS=PRISM are ORed with the initial selection (i.e. nothing), and asterisks should appear in the left hand column of all PRISM observations.
Now click on HIDE and all non-PRISM observations should disappear.
Now go down the INSOPT1 column and find an observations with OPT1=HIGH_SENS. Click on this, and then on AND. This will AND the set of HIGH_SENS observations with the current PRISM selection and any SCI_PHOT or OPEN observations should disappear. You can continue by selecting on OBSTARG (target name as given by the observer) or NRTSMODE (what the online program thought it was doing), or FILTER or SHUTTER position.
Using the other relational buttons, you can also select by time, because the file names are ordered by time. For example you can click on the file name of first observation you want, then click on the >= button and then on AND. All the earlier files will be deselected. Then you can click on the last observation, the <= button and AND, and the later observations will disappear.
When you have made the selection you want, click on SELECT.
At the top right the total number of files in the list, and the total number currently selected, are displayed.
If you click on a data item and then click HIDE, not much will happen (except that previously de-selected rows may disappear). You first have to press on of the boolean buttons to make something change.
files = midiGuiS()
Look for a file in the search directory called midiGui.SAV, and if you find it, assume that it contains the keyword values for all the MIDI files in this directory. This speeds things up a lot. If you don't find it, parse all the headers and store the summary in the abovenamed file upon exit. This will crash if you don't have write privliges.
If you can added/deleted files in the directory, so that the cached version is not up to date, just delete it and try midiGuiS again.
files = midiGui(dir=datadir) or midiGuiS(dir=datadir)
gorgonzola assumes that idl is running in the directory where the data is. If this is not the case you can use the above forms to specify a different data directory. You can also use the IDL utility pushd:
pushd, datadir files = midigui() popd
The following standard file types are produced by the specified programs.
Some notes indicate how to access the contents of the programs:
Some of the lower level programs to access this data are described below
but almost all are listed here:
oirGetData(dataFileName [,rows] [,col=col],[ierr=ierr])
Get raw or semi-reduced detector data
oirGetVis(visFileName [,wave=wave])
Get reduced visibility data and wavelength information
oirGetDelay(groupDelayFileName)
Get estimated group delay for each reduced frame
oirGetOpd(fileName)
From raw or partially reduced data get piezo+VLTI delay
line OPD position
oirGetMeanRMS(inputfile)
Take pixel by pixel mean and rms of all frames in an observation
oirGetDetector(fileName)
Get data from IMAGING_DETECTOR table, specifying detector layout
oirGetWaveNo(detectorFile, region)
Get wavenumbers (here defined as 2*!pi/lambda(micron)) for
each pixel in specified detector region (1-relative)
midChopImage (datafile)
Return 2-row structure with mean of Target- mean of Sky
detector images in first row, and mean of Sky in 2nd row.
File types:
.compressed
Produced by oir1dCompressData, contains a header, an
imaging_detector table and a floating point imaging_data table.
The last has a 1-dimensional DATAn array for each input frame
for each detector window.
This can be accessed with oirGetData(dataFileName).
dataArray = oirGetData(dataFileName)
oirGetData works on raw and partially reduced detector data.
It returns an array of structures. Each structure corresponds to
one data frame, and corresponds to ESO FITS interferometry
data tables. Each row of the table contains information on
the time of exposure (MJD days), exposure time (sec), OPD positions,
chopping mirror positions, and finally the data itself.
dataArray[15].time contains the MJD time of the 16th frame
(IDL is 0-relative).
dataArray.data1 is in general a 3-dimensional data cube, where
each plane represents the detector data from the 1st specified
detector region, and the 3rd dimension is frame number.
The data arrays are INT for raw data and FLOAT for partially
reduced data. Note that the raw data (16-bit integers) is
offset by -32768 from the true detector zero values.
.fringes
Produced by oirFormFringes, contains a header, an imaging_detector table
and a imaging_data table with a single DATA1 array for each input frame.
This can be accessed with
table = oirGetData(dataFileName)
data = table.data1
.insopd
Produced by oirRotateInsOpd. Contains a header and imaging_detector table,
and a pseudo-complex imaging_data table with a single DATA1 array per row.
Header contains keyword OPD0 describing offset subtracted from all
OPDs before rotation (=mode of tracking OPDs).
This can be accessed with
table = oirGetData(dataFileName).
data = pseudocomplex(table.data1)
data is them COMPLEX(nFrames, nFreq)
.groupdelay
Produced by oirGroupDelay. Contains a header and imaging_detector table,
and a pseudo-complex imaging_data table with the FFT of .insopd. The
header contains some special keywords:
OPD0: as in .insopd
OPD1,OPD2: coordinates of delay direction in output FFT. The delays
= OPD1 + OPD2*(xpix-1).
This table can be accessed with oirGetData(dataFileName). The
data section can be accessed using a call to pseudoComplex.
E.g.
gd = oirGetData('mydata.groupdelay.fits')
gdc = pseudocomplex(gd.data1)
gdc is COMPLEX(nFrames, 512), with the y-axis representing
delay.
Additionally there is a 3rd table DELAY containing (per frame) the
time (MJD days), telescope numbers and estimated OPD (seconds) and fringe amplitude. This
can be accessed with oirGetDelay(fileName) or midiGetDelay(tag).
.ungroupdelay
Produced by oirRotateGroupDelay . Contains the usual header and
detector tables and a imaging_data table in pseudo-complex format.
Can be accessed with
ugd = oirGetData(filename)
ugdc = pseudocomplex(ugd.data1) which is COMPLEX(nFrame, nFreq)
Additionally there is a 3rd table DISPERSION containing (per frame)
the value of the average phase subtracted from each row. The
table has three columns "TIME" "TELESCOPE" and "DISPERSION", which
for the purpose at hand, is the subtracted mean phase in degrees.
This is accessed by the generic fitstable routines:
t = obj_new('fitstable', filename, extname='DISPERSION')
dispData = t->readrows()
obj_destroy,t
.flag
Produced by oirAutoFlag. Contains a header and a FLAG table with the
time intervals that have been flagged. There is no oirGetFlag routine.
You can use:
t = obj_new('fitstable', filename, extname='FLAG')
flagData = t->readrows()
obj_destroy,t
.corr
Produced by oirAverageVis. Contains a header, an OI_WAVELENGTH table
specifying the wavelength for each channel, and an OI_VIS table
containing averaged correlated fluxes.
Can be accessed with
visTable = oirGetVis(filename, wave=wave) with wave optional
visTable.visamp is an array of (nWave) correlated fluxes (Jy)
visTable.visphi is an array of (nWave) differential phases (degree)
visTable.visamperr contains the estimated rms of visamp
visTable.visphierr contains the estimated rms of visphi
wave contains nWave values of the wavelength (microns)
.photometry
Produced by oirChopPhotometry. Header, detector tables, and
imaging_data table in float format
Can be accessed with photData = oirGetData(filename)
photData[0].data1 contains the photometry data for telescope A
photData[1].data1 contains the photometry data for telescope B
and so forth (see heading oirChopPhotometry above)
.redcal
Produced by oirRedCal. Header, OI_WAVELENGTH and OI_VIS tables
Can be accessed with oirGetVis(filename, wave=wave)
.calphot
Produced by oirCalibrateVis. Header, detector, imaging_data table
Can be accessed with oirGetData(filename)
.calvis
Produced by oirCalibrateVis. Header, OI_WAVELENGTH and OI_VIS tables
Can be accessed with oirGetVis(filename, wave=wave)
.calcorr
Produced by oirCalibrateVis. Header, OI_WAVELENGTH and OI_VIS tables
Can be accessed with oirGetVis(filename, wave=wave)