Preparations

Before you can begin the reduction of your MOSAIC images you need to bring your raw data to the necessary computing and software resources. This means gathering enough computing power, the proper software, your raw images with complete and accurate image headers, and the proper data calibration files. The topics below should help you gather together the necessary components that will enable you to perform your reductions.

  1. Computing Power
  2. Software
    1. IRAF
    2. mscred, nproto, xdwred
    3. setinstrument
  3. Load Raw Data with Complete Headers
    1. msrfits
    2. mscsetwcs
    3. msccmd and hedit
  4. Examining Your Images
    1. ccdlist
    2. mscdisplay, mscexamine
    3. ximtool-alt
  5. Calibration Data
    1. Calibrations You Should Have Obtained During Your Run.
    2. Getting NOAO Provided Calibration Files

Computing Power:
To reduce MOSAIC images you need a fast computer (we started in 1998 using a Sun Sparc Ultra-2, found that to be too slow, moved on to an Ultra-60, wich is ok, but we want more speed and are moving on to fast PC Linux boxes) with plenty of memory (the more the better) and a significant amount of available hard-disk space to store the raw and intermediate images while working toward your final data products. We find that approximately 100 GB is enough ``working-space'' to allow us to process a couple of night's of data -- but how much space is required will depend on your specific program (you could easily end up needing more!). Some groups processing NOAO Mosaic images report buying a Tera-byte of diskspace for their reduction machines.

NDWFS Computers

Software:
We currently process the NDWFS MOSAIC images with IRAF tasks and scripts. After starting IRAF, verify that you are using the same versions of IRAF and mscred used during the construction of these notes. This can be done by loading the mscred package and/or the command at the command line prompt of "epar mscred".

NDWFS: IRAF on our Computers

IMPORTANT: As of May 2002 we are using IRAF V2.12 and mscred V4.7. You will also need the package nproto, version distributed with IRAF V2.12, in order for mscred to be able to make use of the objmasks task, so important to several of our improved reduction steps. (If you want to use the new version of mscred with IRAF V2.11.3, a version is available at fttp://iraf.noao.edu/iraf/extern-v211/mscred/, but as mentioned, several of the best new features will require V2.12). The NDWFS occasionally makes use of development and/or test versions of new IRAF tasks that live in the NOAO local package xdwred, which is not generally available -- but these tasks, after testing and demonstrated utility, will make it into the general release of mscred and/or IRAF. Those reducing NDWFS images should load xdwred and then mscred. You will also need to load nproto.

Initialize the Software

After starting IRAF and loading the mscred package (and nproto), you should generally use the setinstrument task to populate various parameters of several IRAF tasks with the correct values for the version of MOSAIC camera that produced the data you are reducing. Currently data from three major versions of the NOAO MOSAIC cameras exhist. MOSAIC-1 has so far had two incarnations. Initially MOSAIC-1 was deployed with engineering grade CCDs and used at the Kitt Peak (KP) 4m and 0.9m telescopes from 1997 through the first semester of 1998. Starting with the second semester of 1998, MOSAIC-1 has benefited from thinned SITE CCDs. An attempted upgrade of MOSAIC-1 in August of 2000 did result in some changes to the instrument parameters. The second MOSAIC camera (MOSAIC-2), used at the CTIO Blanco 4m telescope (CT4m), was commissioned with science grade detectors in 1999 and first used by visiting observers in semester 99B. MOSAIC-2 has three observing modes (8A-amp, 8B-amp, and 16-amp). A web page that will try to track available calibration files is under construction and can be found at this link, Calibration Files For the Mosaic Imagers. Desired calibration data is discussed further below. An example of the proper parameters for setinstrument when reducing KP4m MOSAIC data follows.

Parameters for setinstrument:

PACKAGE = mscred
TASK = setinstrument

  site    =                 kpno  Site (? for menu)
  telescop=               4meter  Telescope (? for menu)
  instrume=         CCDMosaThin1  Instrument (? for a list)
  (directo=          mscdb$noao/) Instrument directory
  (review =                  yes) Review instrument parameters?
  query_si=                 kpno  Site (? for menu or q to quit)
  query_te=                       Telescope (? for menu or q to quit)
  query_in=         CCDMosaThin1  Instrument (? for menu or q to quit)
  (mode   =                   ql)

Currently available combination of choices for site/telescope/instrument are

site = kpno
telescope = 36inch or 4meter [Mayall 4m] or wiyn
instrument = mosaic1 [for Engineering CCD version of instrument]
instrument =CCDMosaThin1 [for Science grade CCD version]

OR

site = ctio
telescope = 4meter [Blanco 4m]
instrument = Mosaic2 [Mosaic 2 with 16 amplifiers]
instrument = Mosaic2A [Mosaic 2 with A amplifiers]
instrument = Mosaic2B [Mosaic 2 with B amplifiers]

When you run setinstrument you will also be prompted to edit the parameters for mscred and ccdproc. At this time (or if you want to make a change at a later time, you can do so by epar mscred), set the path to the "back-up" directory. The default is /Raw. This is where ccdproc will copy your raw data after it has started to generate a reduced version. You can also adjust whether the data is backed up once, on every change, or not at all. Remember that if you do save a backed up version this will take up more diskspace, but it will save you time restoring things from tape. If you are just starting to learn to reduce Mosaic images, I strongly recommend you use the back-up option.

Here is what mscred parameters will look like for reducing CCD MOSAIC data from the KPNO 4m telescope. Note that with the parameters below the raw images will be backed-up to a subdirectory named "Raw".

	

TASK = mscred

(pixelty=            real real) Output and calculation pixel datatypes
(verbose=                  yes) Print log information to the standard output?
(logfile=              logfile) Text log file
(plotfil=                     ) Log metacode plot file
(backup =                 once) Backup data (none|once|all)?
(bkuproo=                 Raw/) Backup root (directory or prefix)
(instrum= mscdb$noao/kpno/4meter/CCDMosaThin1.dat) CCD instrument file
(ampfile=                 amps) Amplifier translation file
(ssfile =              subsets) Subset translation file
(im_bufs=                   4.) Image I/O buffer size (in Mbytes)
(graphic=             stdgraph) Interactive graphics output device
(cursor =                     ) Graphics cursor input
(version= V4.7: April 11, 2002)
(mode   =                   ql)
($nargs =                    0)

Load Raw Data with Complete Headers:

NOAO currently recommends that mscwfits be used to write raw data to tape. If this is how you stored your data at the telescope you should restore it to your computer using the task mscrfits in the mscred package. The raw data files will be multiextension fits files (MEF) and roughly 135MB's in size. For NOAO MOSAIC data the MEF files will include nine extensions numbered 0-8. The "0" extension contains header information common to all of the eight individual CCD images that comprise a single exposure. The 1-8 extensions are the fits images of respectively CCD1 through CCD8. Each of these extensions contains both CCD specific header information (e.g., gain and read noise values) as well as the data themselves.

Here is an example of the mscrfits parameter file set-up to read MOSAIC images back from a tape written with mscwfits. Remember that prior to running mscrfits you should (although it does not seem to always be required) allocate the tape drive via the command cl> allocate mtm , where you substitute your correct device name for mtm. After reading in your data you should deallocate mtm.
PACKAGE = mscred
   TASK = mscrfits

input   =                  mtm  Input tape
output  =                 image Output file(s)
(tapefil=                 1-84) Tape file list
(listonl=                   no) List only?
(shortli=                  yes) Short listing?
(longlis=                   no) Long listing?
(offset =                    0) Offset for numbering of output disk filenames
(origina=                  yes) Restore original file name?
(mode   =                   ql)

With the above options I've chosen to read the images stored in slots 1 through 84 on the tape and to restore the original file names.

If you used the unix tar command to write your tape, you will be restoring the data with a command like tar -xvf /dev/rmt/???, where ??? will be the device name. To restore only selected files from a tar file,


setenv TAPE /dev/rmt/???
tar -xv -T file.list

Where file.list is the name of an asci file listing the names of the files you would like to extract from the tar file.

NDWFS: Tape Drive names.

Occasionally a data set may have incomplete/incorrect image headers due to either problems with the TCS (telescope control system) or DCA (Data Capture Agent) while the observations were made or incomplete databases being accessed at the time the headers were populated (for example, incomplete information on the world coordinate system for each of the CCDs). Possible problems include missing "gain'' and "rdnoise'' keywords and/or missing WCS information. I will explain how to attempt to fix such problems as part of my discussion of the general data reduction procedures, but the IRAF tasks msccmd and hedit can be used to update header parameters, and mscsetwcs can be used to update/correct the WCS information in the headers.

Examining Your Images:

Once your data has been loaded on to your computer's harddisk it is a good idea to verify that the files are intact/complete. Tasks that can help you examine your data include ccdlist, mscdisplay, mscexamine, and imheader. The task ccdlist provides you with a listing of some basic header information for your images as well as a coded summary of the current reduction status of your images. Typing at the cl> prompt help ccdlist will provide you more information about this task. Here is an example of the parameter file. Note you must specify im1 (or some other extension) or it will generate a status report for all eight CCDs in each image, which is often more information than you really want.


PACKAGE = mscred
TASK = ccdlist

images  =        @object.list   CCD images to listed
(ccdtype=                 ) CCD image type to be listed
(extname=                  im1) Extension name pattern
(names  =                   no) List image names only?
(long   =                   no) Long format listing?
(mode   =                   ql)

Here is an example of the output generated by ccdlist when run on raw and reduced images.

For raw images:


obj030.fits[im1][2136,4096][ushort][object][1][I]:31q1 First Pass - DelRA =   0.0, DelDec =   0.0
obj031.fits[im1][2136,4096][ushort][object][1][I]:31q1 First Pass - DelRA =  41.5, DelDec = -62.3
obj033.fits[im1][2136,4096][ushort][object][1][I]:31q1 First Pass - DelRA = -41.5, DelDec =  62.3
obj034.fits[im1][2136,4096][ushort][object][1][I]:31q1 First Pass - DelRA =  20.8, DelDec =  31.1
obj035.fits[im1][2136,4096][ushort][object][1][I]:31q1 First Pass - DelRA = -20.8, DelDec = -31.1

For reduced images:


obj030.fits[im1][2048,4096][real][object][1][I][XBOTZF]:31q1 First Pass - DelRA =   0.0, DelDec =   0.0
obj031.fits[im1][2048,4096][real][object][1][I][XBOTZF]:31q1 First Pass - DelRA =  41.5, DelDec = -62.3
obj033.fits[im1][2048,4096][real][object][1][I][XBOTZF]:31q1 First Pass - DelRA = -41.5, DelDec =  62.3
obj034.fits[im1][2048,4096][real][object][1][I][XBOTZF]:31q1 First Pass - DelRA =  20.8, DelDec =  31.1
obj035.fits[im1][2048,4096][real][object][1][I][XBOTZF]:31q1 First Pass - DelRA = -20.8, DelDec = -31.1

The flags identifying the processing operations already performed on the image are given by the following single letter codes:

    
 X - Cross talk correction
 B - Bad pixel replacement
 O - Overscan bias subtraction
 T - Trimming
 Z - Zero level subtraction
 D - Dark count subtraction
 F - Flat field calibration

The task mscdisplay allows you to display a MEF file to your image display window. Mscexamine works in a manner very similar to the older task imexamine, but is able to handle the MEF format files. If you are using IRAV V2.12, mscred 4.7, and XIMTOOL V1.3, then imexamine should work on the MEF files as well, with the added benefit that it should handle binned data properly (which mscexamine does not fully handle).

You will be using an image display window a LOT. I use ximtool-alt, and start it with the command:

csh% ximtool-alt -unix_only &
The -unix_only tag ensures that other tasks/users do not send their images to my display. As of May 2002 I am using the most recent version of ximtool-alt, which provides real pixel value read-out in realtime, realtime display of RA and DEC (for images with a valid WCS), up to 16 images held in the image buffers at one time, and many other cool features. See the web page about X11IRAFV1.3 for more information about the new version of ximtool-alt.

Calibration Data:

This section of the guide will be completed as soon as I can, but for now here is a brief list of the kinds of calibration files you will need. Obtained during your run: bias frames (zero???.fits files), dark frames (dark???.fits, for most applications not needed with current versions of the instruments), dome flats (dflat???.fits) for each filter used, and twilight flats (sflat???.fits; there is not a consensus view on whether twilight flats, dome-flats, dark-sky flats, or a combination of the above work best; it is most certainly a function of the science you are trying to do as well as the wavelength(s) at which you are observing and I will try to address this topic at greater length in the near future). You might also obtain observations of an astrometric standard field as well as photometric standards. The astrometric standard field can be used to improve the quality of default WCS loaded into your image headers. We will cover this in more detail later in this guide, but a complete discussion of how to use astrometric standard star field observations to generate high-order term astrometric corrections for your images can be found on the web page "Creating a Mosaic World Coordinate System" by Frank Valdes. While you can determine the following information from your own data, NOAO does provide some basic information about the MOSAIC camera CCD's via header keywords and reference files. A new webpage with links to calibration files is in preparation, Calibration Files For the Mosaic Imagers. This page includes some information on the cross-talk correction files and will eventually include some default bad-pixel masks (which are also included in the mscdb database in IRAF).

Now that the computer, software, raw data, and basic calibration files are all ready, lets review the steps that we will go through to generate calibrated images. Please move on to section III. Reducing Your Data.