[Initial Reductions (ccdproc)]
[Cross Talk Correction (ccdproc, xtalkcoeff)]
[Bias Generation]
[Dark Frame Generation]
[Dome Flat Generation]
In this section we will describe how to generate your "Bias" or "zero" correction frames; your combined "dark" frame; and your dome flats. This will be done using the tasks ccdproc, zerocombine, flatcombine, and for some dome flats taken with Mosaic-1 at the KPNO 4m, rmpupil.
One of the IRAF tasks that you will use frequently is ccdproc, which is a superscript that calls numerous other IRAF tasks. The task ccdproc can be used to perform many basic reduction steps all at once -- or you can use it to call the individual steps one at a time. Combined with ccdlist, they are two useful tasks for working through the early steps of data reduction.
We will introduce ccdproc by discussing how to use it to perform the "cross-talk" correction, required for Mosaic-1 and Mosaic-2 images. The "cross-talk" occurs between readout electronics for CCDs sharing ARCON controller boxes (four boxes per MOSAIC, two CCDs per box -- CCD1 and CCD2 share a control box, etc.). I might add a more verbose description of the cause as time permits, but the end effect is that a stored image for a "victim" CCD (or readout amplifier) will have both the true counts that should be stored for a given pixel in that CCD plus some small fraction of the counts read out by another amplifier(s) at the same time as the readout of the victim pixel. In a raw image the effect is most easily seen as "echos" or copies of saturated stars in one image appearing in the paired CCD (or amplifier). To the best of our ability to measure the effect, it is additive and proportional to the number of counts in the source CCD (or amplifier). The proportionality coefficients appear to be relatively stable (e.g., throughout a run, perhaps even for several months), but they have been observed to change when significant work has occured on the instrument. The most conservative observer should verify for their own data that the values we are providing are appropriate. The coefficients used for the correction can be determined from your own exposures (ideally exposures with many bright stars exposed to near - and past - saturation, but without a dense background of objects or spatially extended emission) using the task xtcoeff or by using the data files provided by NOAO. Normally the headers of NOAO Mosaic images will include the name of the correct file (in the mscdb$ directory of IRAF) to use -- but you should check the "Calibration Files for the Mosaic Imagers" web page for updates (note that that page is still under construction), and edit your headers accordingly if necessary with hedit. Note that the values of the cross-talk coefficients are very small (this is a .1 to .2% effect), so the "echos" of even the saturated stars are only 20-30 ADU -- but still large enough that most observers will want to remove the signal.
We will use ccdproc to perform the initial processing steps of our bias frames ("zeros"), dark frames (if you have them), and dome flats. First, epar ccdproc and turn off all corrections other than the cross-talk, trim, and over-scan corrections and run it on all of your bias, dark, and dome-flat images. Your parameter file might look like this:
PACKAGE = mscred TASK = ccdproc images = dflat*.fits,zero*.fits, dark*.fits List of Mosaic CCD images to process (output = ) List of output processed images (bpmasks= ) List of output bad pixel masks (ccdtype= ) CCD image type to process (noproc = no) List processing steps only? (xtalkco= yes) Apply crosstalk correction? (fixpix = no) Apply bad pixel mask correction? (oversca= yes) Apply overscan strip correction? (trim = yes) Trim the image? (zerocor= no) Apply zero level correction? (darkcor= no) Apply dark count correction? (flatcor= no) Apply flat field correction? (sflatco= no) Apply sky flat field correction? (merge = no) Merge amplifiers from same CCD? (xtalkfi= !xtalkfil) Crosstalk file (fixfile= BPM) List of bad pixel masks (saturat= INDEF) Saturated pixel threshold (sgrow = 0) Saturated pixel grow radius (bleed = INDEF) Bleed pixel threshold (btrail = 20) Bleed trail minimum length (bgrow = 0) Bleed pixel grow radius (biassec= !biassec) Overscan strip image section (trimsec= !trimsec) Trim data section (zero = Zero) List of zero level calibration images (dark = Dark) List of dark count calibration images (flat = Flat*) List of flat field images (sflat = Sflat*) List of secondary flat field images (minrepl= 1.) Minimum flat field value (interac= no) Fit overscan interactively? (functio= legendre) Fitting function (order = 1) Number of polynomial terms or spline pieces (sample = *) Sample points to fit (naverag= 1) Number of sample points to combine (niterat= 1) Number of rejection iterations (low_rej= 3.) Low sigma rejection factor (high_re= 3.) High sigma rejection factor (grow = 0.) Rejection growing radius (fd = ) (fd2 = ) (mode = ql)
Alternatively, you can just move ahead and use zerocombine, darkcombine, and flatcombine to process, respectively, your zero, dark, and dflat exposures. Note that you will need to have set the parameters for ccdproc as shown above before running these combine tasks and you will need in each of these tasks to set process=yes.
Note that having set the xtalkfil, biassec, and trimsec parameters as above (with the "!" followed by the image header keyword) tells ccdproc to use the header keywords indicated as pointers to the proper values for these parameters. For KPNO and CTIO Mosaic data the bias and trim section regions will generally have been set to something reasonable -- and we try to have the xtalk-file set as well -- but verify this with an imheader and a comparison to the calibration web page. If not sure, contact one of the Mosaic instrument scientists.
[Initial Reductions (ccdproc)]
[Bias Generation]
[Dark Frame Generation]
[Dome Flat Generation]
In order to correct for any spatial variation in the bias structure of each of the Mosaic CCDs you need to construct a reference bias or Zero frame that will be subtracted from all of your images. I use zerocombine to generate this file and usually name the file as ZeroUTDATE.fits with UTDATE equal to the date of the observations. If the individual zero.fits files have not been processed, zerocombine will automatically call ccdproc to perform the basic processing (correction for cross-talk, trim, overscan) and will do this correctly provided you have the parameter process=yes. Below is a copy of the parameter file for zerocombine. The file "zero.list" (asci format) would include a list, one image per line, of the input files. If the directory only contained one set of zero frames (7-11 images for one night) you could replace @zero.list with *.fits or zero*.fits and zerocombine would find the correct images. Note that we have had success using crreject as the rejection algorithm, but you might need to choose a different option for your own data.
PACKAGE = mscred TASK = zerocombine input = @zero.list List of zero level images to combine (output = Zero990327) Output zero level name (combine= average) Type of combine operation (reject = crreject) Type of rejection (ccdtype= zero) CCD image type to combine (process= yes) Process images before combining? (delete = no) Delete input images after combining? (scale = none) Image scaling (statsec= ) Image section for computing statistics (nlow = 1) minmax: Number of low pixels to reject (nhigh = 1) minmax: Number of high pixels to reject (nkeep = 1) Minimum to keep (pos) or maximum to reject (neg) (mclip = yes) Use median in sigma clipping algorithms? (lsigma = 3.) Lower sigma clipping factor (hsigma = 3.) Upper sigma clipping factor (rdnoise= RDNOISE) ccdclip: CCD readout noise (electrons) (gain = GAIN) ccdclip: CCD gain (electrons/DN) (snoise = 0.) ccdclip: Sensitivity noise (fraction) (pclip = -0.5) pclip: Percentile clipping parameter (blank = 0.) Value if there are no pixels (mode = ql)
This completes the generation of the bias frame. This file file will be used in further passes of ccdproc on calibration and object-field frames.
[Initial Reductions (ccdproc)]
[Bias Generation]
[Dark Frame Generation]
[Dome Flat Generation]
We have not seen a significant dark current (in up to 40 minute integrations) with the science grade CCD's of MOSAIC-1 or MOSAIC-2, but the earlier edition of MOSAIC-1 with its engineering grade CCD's did have a measurable dark current that some researchers might wish to remove. The task darkcombine can be used to produce your Dark.fits frame, which can then be applied to other calibration and object frames. Again, I generally name the file DarkUTDATE.fits. The setting of the parameters rdnoise= rdnoise and gain= gain instructs the task to use the image header values assigned to the keywords rdnoise and gain (different for each CCD). You can determine these values from your own data (using the mscred task mscfindgain or use the values placed in the header at the time the images were obtained. We try to keep the values placed in the headers current based on measurements made during engineering runs, but it never hurts for you to verify the values yourself).
PACKAGE = mscred TASK = darkcombine input = dark* List of dark images to combine (output = Dark980404) Output dark image root name (combine= average) Type of combine operation (reject = ccdclip) Type of rejection (ccdtype= dark) CCD image type to combine (process= yes) Process images before combining? (delete = no) Delete input images after combining? (scale = exposure) Image scaling (statsec= ) Image section for computing statistics (nlow = 0) minmax: Number of low pixels to reject (nhigh = 1) minmax: Number of high pixels to reject (nkeep = 1) Minimum to keep (pos) or maximum to reject (neg) (mclip = yes) Use median in sigma clipping algorithms? (lsigma = 3.) Lower sigma clipping factor (hsigma = 3.) Upper sigma clipping factor (rdnoise= rdnoise) ccdclip: CCD readout noise (electrons) (gain = gain) ccdclip: CCD gain (electrons/DN) (snoise = 0.) ccdclip: Sensitivity noise (fraction) (pclip = -0.5) pclip: Percentile clipping parameter (blank = 0.) Value if there are no pixels (mode = q)
[Top of Page]
[Initial Reductions (ccdproc)]
[Bias Generation]
[Dark Frame Generation]
[Dome Flat Generation]
While the MOSAIC CCDs are very sensitive, there are significant pixel to pixel variations in their response and these variations must be corrected using "flat field" exposures. Perfect correction is not possible and what is proper for your program will depend on your science goals. For the NDWFS we are interested in photometric accuracy to about 5 percent and to remove artifacts or image defects that might be misidentified as objects. Most of the objects we are trying to study are at or below the brightness of the night-sky. We therefore use a series of two flats to correct our images. We start with a dome-flat, which is constructed from a series of exposures (one set per filter) taken with the telescope pointed at the white-spot and the flat-field lamps turned on. We usually are working with 7 to 11 "dflat???.fits" images. After the dome-flat has been made using flatcombine, it might have to be further corrected for any pupil-ghost image that might be present. This pupil-ghost correction step is currently only necessary for the KP4m plus MOSAIC-1 and is detailed further below.
Remember that before you construct your dome-flat with flatcombine, you need to update the parameters of ccdproc with the name of your ZeroUTDATE.fits bias frame. Also note that in the example below there is no input list for ccdproc. This is because flatcombine will automatically pass the correct images on to ccdproc (assuming that there is just one set of flats per filter in the directory -- if for some reason you want to restrict the input file list you can use an input file list as shown above for the zerocombine step.
PACKAGE = mscred TASK = ccdproc images = List of Mosaic CCD images to process (output = ) List of output processed images (bpmasks= ) List of output bad pixel masks (ccdtype= flat) CCD image type to process (noproc = no) List processing steps only? (xtalkco= yes) Apply crosstalk correction? (fixpix = yes) Apply bad pixel mask correction? (oversca= yes) Apply overscan strip correction? (trim = yes) Trim the image? (zerocor= yes) Apply zero level correction? (darkcor= no) Apply dark count correction? (flatcor= no) Apply flat field correction? (sflatco= no) Apply sky flat field correction? (merge = no) Merge amplifiers from same CCD? (xtalkfi= mscdb$noao/CCDMosaThin1/xtalk.dat) Crosstalk file (fixfile= BPM) List of bad pixel masks (saturat= INDEF) Saturated pixel threshold (sgrow = 0) Saturated pixel grow radius (bleed = INDEF) Bleed pixel threshold (btrail = 20) Bleed trail minimum length (bgrow = 0) Bleed pixel grow radius (biassec= !biassec) Overscan strip image section (trimsec= !trimsec) Trim data section (zero = Zero990327.fits) List of zero level calibration images (dark = Dark) List of dark count calibration images (flat = Flat*) List of flat field images (sflat = Sflat*) List of secondary flat field images (minrepl= 1.) Minimum flat field value (interac= no) Fit overscan interactively? (functio= legendre) Fitting function (order = 1) Number of polynomial terms or spline pieces (sample = *) Sample points to fit (naverag= 1) Number of sample points to combine (niterat= 1) Number of rejection iterations (low_rej= 3.) Low sigma rejection factor (high_re= 3.) High sigma rejection factor (grow = 0.) Rejection growing radius (fd = ) (fd2 = ) (mode = ql)
Here is an example of flatcombine setup to generate initial R and I-band dome-flats. The input list contains all the R and I-band dflat???.fits images. The output rootname has been specified to indicate the UT date of the observations. The task flatcombine will automatically append the value of the image-header keyword filter to the end of the combined dome-flat, and the task will properly handle the mixed input filter list, combining only the R-band images to produce Dflat990327R.fits and only the I-band images to produce Dflat990327I.fits.
PACKAGE = mscred TASK = flatcombine input = dflat*.fits List of flat field images to combine (output = Dflat990327) Output flat field root name (combine= average) Type of combine operation (reject = ccdclip) Type of rejection (ccdtype= flat) CCD image type to combine (process= yes) Process images before combining? (subsets= yes) Combine images by subset parameter? (delete = no) Delete input images after combining? (scale = mode) Image scaling (statsec= ) Image section for computing statistics (nlow = 1) minmax: Number of low pixels to reject (nhigh = 1) minmax: Number of high pixels to reject (nkeep = 1) Minimum to keep (pos) or maximum to reject (neg) (mclip = no) Use median in sigma clipping algorithms? (lsigma = 3.) Lower sigma clipping factor (hsigma = 3.) Upper sigma clipping factor (rdnoise= rdnoise) ccdclip: CCD readout noise (electrons) (gain = gain) ccdclip: CCD gain (electrons/DN) (snoise = 0.) ccdclip: Sensitivity noise (fraction) (pclip = -0.5) pclip: Percentile clipping parameter (blank = 1.) Value if there are no pixels (mode = ql)
Both the CTIO and KPNO 4m prime-focus correctors produce pupil ghosts from light that reflects off of the filters, off the back surface of the corrector, and returns through the filters to your science detectors. In the case of the CTIO 4m this added light is diffuse over the entire field and at an unknown, but believed to be small, level. In the case of the KPNO 4m, the reflected light is evident in a frame as an image of the telescope pupil. The amplitude of this "ghost-image" is largest where the Sol-gel coating of the corrector is least effective (in the red and blue). For the NDWFS we correct for the pupil ghost in our I-band and Bw-band images. Most observers are likely to want to make this correction in the U,I, and z bands, as well as most narrow-band filters. We correct our image frames by subtracting off a template pupil image using the task rmpupil. The template pupil image is made from an image produced by combining 30 to 40 object frames (of the same filter) and the task mscpupil. These steps will be detailed further below. However, before the object frames can be combined to produce a high SNR image of the pupil, we must first apply a dome-flat that will not hide the pupil. This means correcting our dome-flats for the pupil-ghost present in the dome-flat. (If we do not make the correction at this point, the pupil-ghost appears to be corrected because it will be divided-out by the dome-flat, but this is not the correct way to fix what is an additive error). To correct the dome-flat we also use the task mscpupil. We will be dividing the pupil ghost out of the dome-flat. The choice of dividing (as indicated by the choice type=ratio in the parameter file below) the ghost out of the dome-flat vs. subtracting it has confused MANY of the researchers working with KPNO Mosaic data -- and in fact I (BTJ) have failed to convince some of them that "ratio" is the correct choice -- but it is! One way to think about this is that because of the pupil ghost the CCD was not illuminated by the light from a uniform flat field screen, but rather an image that had a brighter "circle" (the image of the pupil) around the center of the field of view. We need to take out the large scale illumination pattern from the dome flat, leaving only the map of the pixel to pixel variations in the combined dome flat.
Note that while fitting the pupil-ghost and removing its effect from our dome-flat we do not want to remove real flat-field variations in the CCDS -- which we need to have the dome-flat correct! In the red the central chips have sensitivity variations near the edges of the CCDs that would be mistakenly fit as part of the pupil-ghost if care is not taken. The edges of the CCDs (a few rows on each) show up as brighter than the rest of the CCD in the I and z-band dome flats. This flat field structure will be mistakenly fit as part of the pupil ghost unless the following fix is applied. For I-band and z-band dome-flats, as well as other filters in this wavelength range, set the parameter lmedian=yes. For data in the U- or Bw-bands this parameter should be set to no. Dome flats in the V, R, etc., bands do not need the mscpupil task applied to them.
Here is a sample parameter file for mscpupil when the task is being used to correct our initial combined I-band dome-flat into an image suitable for use in the initial correction of our object frames.
PACKAGE = mscred TASK = mscpupil input = Dflat990327I.fits List of input images output = Dflat990327Iwop.fits List of output images (masks = BPM) List of masks (type = ratio) Output type (lmedian= yes) Subtract line-by-line median? (xc = 27.) Pattern center offset (pixels) (yc = 9.) Pattern center offset (pixels) (rin = 300.) Radius of inner background ring (pixels) (drin = 20.) Width of inner background ring (pixels) (rout = 1500.) Radius of outer background ring (pixels) (drout = 20.) Width of outer background ring (pixels) (funcin = chebyshev) Inner azimuthal background fitting function (orderin= 2) Inner azimuthal background fitting order (funcout= spline3) Outer azimuthal background fitting function (orderou= 5) Outer azimuthal background fitting order (rfuncti= spline3) Radial profile fitting function (rorder = 5) Radial profile fitting order (abin = 45.) Azimuthal bin (deg) (astep = 10.) Azimuthal step (deg) (niterat= 3) Number of rejection iterations (lreject= 3.) Low rejection rms factor (hreject= 3.) High rejection rms factor (datamin= INDEF) Minimum good data value (datamax= INDEF) Maximum good data value (verbose= yes) Print information? (fd = ) (mode = ql)For this example,Dflat990327Iwop.fits would be the image to use for ccdproc on object images (the "wop" is just my notation to indicate "without pupil-ghost"). Now you are almost ready to do the initial processing on the actual object frames. Please move to III.B..