SIMPLE Imaging and
(National Radio Astronomy Observatory)
SIMPLE-MOIRCS version: 1.1
Document version: 1.2
bug report and feedback: whwang at aoc.
Imaging and Mosaicking Pipeline (SIMPLE) is an IDL based system
designed for galactic/extragalactic imaging where there are no extended
objects (i.e., blank fields). It generates reasonably good
photometry. It is also capable of making large mosaics of many
sensor areas, while still maintaining the astrometry
accuracy. SIMPLE can be used on different data after simple
particular distribution (SIMPLE-MOIRCS) is optimized for the MOIRCS
camera on the Subaru Telescope.
descriptions about the algorithms used in SIMPLE can be found in this paper.
To run this
package, the most recent version of the IDL Astronomy User's Library
and the package SExtractor
are required to be pre-installed. The SExtractor will be called
from command line with the command "sex." Recent x86 and SPARC
GHz speeds are all
fast enough. However, this package uses lot of memory to speed up
the processing. At least 2 GB of RAM is recommended for basic
reduction. For making large mosaics and cleanly removing cosmic
rays, more than 4 GB of RAM and a 64 bit system are required. The
32 bit IDL cannot locate more than 2 GB of RAM, so the 64-bit
requirement is not just to
speed up the processing.
Flat Field and Background Subtraction
Sky flat is
generated from dithered broad-band images. By masking detected
with a better treatment to frame-to-frame fluctuation of sky
background, artifacts caused by bright objects is minimized. It
is also possible to use dome flat or twilight
flat. The standard background subtraction in SIMPLE-MOIRCS is to
subtract a smooth surface in each quadrant of the HAWAII array.
However, SIMPLE-MOIRCS is also equipped with median sky background
Distortion Correction and Absolute Astrometry
are warped and resampled to project onto a sky plane with sub-pixel
accuracy. Distortion correction uses all detected objects in the
dithered images, and does
not rely on any external information. This is important for
since its small FOV does not always have enough cataloged stars
uniformly distributed across the FOV, especially at high galactic
latitude. Absolute astrometry is obtained with objects in the NOMAD catalog or any
source catalog, and only a
relatively small number of objects are required. For data taken
under <0".4 seeing and with a dense source catalog of good
accuracy (such as the GOODS ACS catalog), the final astrometric
across one MOIRCS FOV can be as good as 0".03 rms relative
input catalog. With NOMAD
catalogs, the typical astrometric
accuracy is 0".06 to 0".1 rms, which is the limit of the NOMAD catalog
itself. Throughout the
reduction, astrometry in the header follows the FITS standard described
in Calabretta & Creisen (2002, A&A, 395, 1077).
procedure can be used to generate
zero points from standard star observations. The MOIRCS
photometry seems consistent with independently reduced WIRCAM
photometry across 200 arcmin2 in the
The current version does not keep records of saturated pixels, so
aperture photometry on bright stars are not reliable. The current
version has functions for handling data taken under
specify the coordinate system (projection center on the sky, rotation
angle, and plate scale) for the reduced/combined images. A
tangential projection is adopted, but other projections can be
implemented by the users easily. Images
with the same coordinate system can easily be combined to form
larger mosaics accurately. The final size of the mosaic is only
limited by available RAM. Exposure time on each pixel is always
traced and recorded so the map can be deepened or widened easily.
1.5 Cosmic Ray Removal
Two methods are
provided for cosmic ray removal. The brightest
and most obvious cosmic ray hits are removed from individual exposures
with a sigma filter of 5x5 pixels. Fainter cosmic ray hits are
removed by applying a sigma filter to pixels that have the same sky
position in a dither set.
1.6 Image Weighting
Basic image weighting in SIMPLE-MOIRCS is to weight each image
according to its exposure time. An optional weighting method is
provided to weight each pixel
according to sky transparency (cloud extinction and airmass effect),
pixel quantum efficiency, sky brightness, and exposure time.
highlights the main procedures in this package that are required to
complete a full reduction, several procedures that may need some manual
tweak, and other useful procedures. Please see the
explanations in the procedure files for details.
procedure in SIMPLE-MOIRCS is reduce_moircs.
It deals with images within a dither set from the same chip. It
produces a flattened, sky subtracted, defringed, cosmic ray removed,
and distortion and astrometrically corrected mosaic image, as well as
an exposure time map. The reduced images and the exposure time
maps can be used later to form large mosaics. This procedure is
controlled by several keywords that need to be provided in the IDL
command line, and by external parameters that are stored in an ASCII
parameter file (APF). An example of APF is provided as the file GOODSN.para.
derives the sensitivity offset between the two
chips. It is recommended to run this procedure at least once per
observing run, although the chip offset does not
change over the past 1.5 year (of course only before the scientific
grade chip 1 breaks). See field #9 in the APF
observations are reduced with this procedure. Zero points will be
automatically calculated if the star is a UKIRT Faint Standard (FS)
Star. Zero points can be applied to images to change the image
the mosaic_wide procedure.
Images reduced by reduce_moircs
can be combined to form large mosaics or deeper images with mosaic_wide.
In order to make the images combinable by mosaic_wide, they have to have
the same astrometry system. This is achieved by using identical
projection in fields #4 through #7 in the APF of reduce_moircs. Photometric
calibration can be applied at this stage, using zero points provided by
It handles images taken under non-photometric conditions.
SIMPLE-MOIRCS uses Jy (or
linear units) as default zero point. For example, a zero point of
1 uJy per data unit corresponds to 23.9 magnitude in AB system.
This is the
subroutine called by the main procedure that reads in a set
of dithered MOIRCS images. When there are problems with the
file/header format, or telescope pointing offsets, or change in
platescale, modification to this
procedure is needed. Sometimes this procedure is useful for
shooting as well.
is called by the main procedure. It removes circular fringe
patterns in MOIRCS images. For chip 2, the center of
the circles always moves around and the user needs to find out its
coordinates. The method of determining the coordinates is
explained in the procedure. An example script is provided in the
for this purpose.
This is usually the most time consuming part of the entire SIMPLE
reduction. After MOIRCS upgrades its filter
(hopefully soon), this procedure may not be needed any more.
If you are too
lazy to keep logs during the observations, or if you are using images
from the archive that were not taken by yourself, this procedure may
tells you the accuracy of astrometry in a FITS image, useful for
compares fluxes of detected objects in two FITS images, useful for
If there are
satellite tracks, this procedure can remove them from the raw
This can rewrite a
NOMAD catalog into the format required by SIMPLE.
This produces dome
flats or twilight flats.
This produces dark
3. Reduction Steps and Trouble Shooting
necessary preparation, the reduction can be subdivided into three
major steps: reduction of the target images, reduction of the standard
and calculating zeropoints, and combining multiple reduced images into
large mosaic. An example script for a set of standard reduction
provided in the file script.example.
If successfully executed, the example script should generate an image
Below I describe some details and the associated trouble shooting.
In order to
obtain absolute astrometry, one needs to prepare a reference catalog
contains enough objects with good astrometry. Usually this is the
NOMAD catalog (or USNO-B1
and equivalents), which
provides minimum astrometric accuracies and numbers of stars. The
procedure recompile_nomad.pro (§2.11) can be
used to convert a NOMAD catalog into the required format. See recompile_nomad.pro for details.
Rather than using a NOMAD catalog, it will be much
better if there is already a deep and astrometrically correct
image/catalog in the observing fields (such as the ACS catalogs of the
GOODS fields). One can use that image/catalog to generate a
deeper reference catalog. The reduced image will be registered to
that image/catalog with high accuracies. See fit_distortion.pro
for the format of the reference catalog.
preparation will be an ASCII file that stores the reduction
parameters (the APF), such as coordinate frame, methods of
background subtraction, treatment of cosmic ray, and many others.
An example and explanations are provided in the file GOODSN.para.
Note that if you wish to combine images from different reductions to
form a big mosaic, the astrometry related fields (4 through 7) should
be kept identical in all reductions.
have to be ASCII files that tell the reduction
program what files to process. The reduction procedure is
designed to process images taken by the same chip within one dither
sequence at once. Therefore two file lists per dither sequence
are needed, one for chip 1 and the other for chip 2. Each file
contains the filename of the first exposure and the last
exposure. See read_moircs.pro
for more details. As for reducing standard stars, only one file
list is required for a dither sequence rather than two files, and all
should be included rather than just the first one and the last
one. Example file lists are provided under the directory lists.
The procedure summary_moircs
(§2.7) may help
you preparing the file lists.
Forth, for SExtractor to work, you will need to modify the file sexfind.default.
Change the file paths to match your system.
sensitivity offset between the two chips has to be derived
first before photometrically accurate images can be produced. To
do this, put 0 in field 9 in the APF, and reduce a full set of dithered
images for both chips. Then use the procedure chip_offset (§2.2) to derive the
sensitivity offset between the two chips, and put back the chip offset
filename in field 9 in the
Now you are ready to rock.
Once the above
preparations are done, reducing the images should be
fairly easy and automatic, using the main procedure reduce_moircs (§2.1). Examples
are provided in the script script.example. Occasionally
(or quite often?) there are errors. The common errors at this
stage are: wrong astrometry, imperfect background subtraction,
circular fringes, and flat-field artifacts caused by bright
stars. It is always good to set the keyword individual=1
in the main procedure and this should help
the diagnosis in most cases. Below describes each case.
3.2.1 Problem in Astrometry
At this moment,
the procedure reduce_std_moircs (§2.3) is
specifically designed for my standard star observations. In my
case, I obtain at least two (usually three or more) dithered exposures
standard star on each of the chip. I can easily imagine different
people do this differently (for example, using one chip only), and the
procedure may not work. In case it doesn't, you need to reduce
the standard star observations manually. Fortunately this is not
difficult. A very simple example in script.standard
shows you how to do this. If you want, you can
even use defringe_moircs
and §2.6) to remove fringes,
and/or use subtract_median_sky
to remove residual background pattern. If you do follow the
example script and use read_moircs and flat-field
images generated in the target reduction, both the image counts and the
flat-field image will be normalized properly. The zeropoint you
derived from such standard-star frames can be directly applied to any
reduced images, without correcting for the sensitivity difference
between the two chips.
When reduce_std_moircs works, things are
much easier. If the standard star is a UKIRT faint standard, it
will automatically calculate the zeropoint for you. Good Luck.
Calibration and Mosaicking
The final step
is to put all single-chip, SIMPLY reduced images together, to form a
deeper mosaic image, using the procedure mosaic_wide
(§2.3). To achieve this, there are two requirements.
First, all images to be combined have to have identical world
coordinate systems, including center of projection, platescale, and
rotation. This can be done by keeping fields 4 through 7
APF throughout the SIMPLE reduction. Second, the zeropoint
difference between the images have to
be taken into account. This is done automatically by mosaic_wide if you can provide
it the zeropoint of each image through the flux_conv
keyword. In the example script, I choose to use uJy as the final
map unit. This corresponds to an AB mag zeropoint of 23.9.
How do we know
there is a problem in astrometry? During the
reduction, the main procedure will tell you what is the rms astrometry
error relative to the reference catalog. It is a function of
seeing and brightness of
objects in the field. Usually the error should be sub-pixel,
sometimes around one pixel if the seeing is bad or the quality of the
reference catalog is not too great. Any rms error significantly
larger than that indicates something might be wrong in
astrometry. If the reduced image is warped into a very strange
shape, something is definitely wrong. You can also write the
reference catalog into a ds9 reg file and overlay the catalog objects
on the reduced image in ds9. (This could be done with the
provided with this package. See details therein.) If the
catalog objects do not match the objects in the reduced images,
something is wrong.
In a few cases,
astrometry problems are caused by the low quality of
the reference catalog, such as not having enough stars. This can
be solved by decreasing the degree of fitting in determining the
absolute astrometry, by setting the keyword fit_order=2 or
(default is 3)
One should try this first. If it does not work, then try what is
in the next paragraph.
In many cases,
especially the worst cases, errors are caused by image
platescales that do not match what is in the header. This
reduction package is quite tolerant to pointing errors. Even if
the Subaru pointing is off by several arcsec, it still works. On
the other hand, it is not tolerant to even 2% of platescale
errors. One should use the individual=1 keyword in the main
procedure to look at individual images. Identify pair of objects
in the image and in the catalog, and compare the angular separation
between the objects suggested by the header and that in the
catalog. This should tell you what the real platescale is, to
within 1% or so. In the source code of the procedure read_moircs,
identify a line that contains "; ps_factor = xxxx."
Uncomment this line, put the relative platescale offset in it, and
this procedure. If you do this correctly, it should solve the
There may be
other problems. Always use the individual=1 keyword in the main
procedure and look at the reduction of individual exposures.
Chance is that you will spot a problem there.
3.2.2 Imperfect Background Subtraction
will see residual backgrounds in the reduced image.
Changing field 10 in the APF from 0 to 1 should solve
most problems. (This is my default mode.) Occasionally
there are still residual backgrounds, especially around the image
corners. In this case, use the individual=1 keyword to inspect
individual images and identify the frames with background subtraction
problems. Then use the keyword more_bg_subtract
in the main procedure to specify these frames and to perform a more
aggressive background subtraction. This should
give you a very flat image background. This will also produce
artifacts on very bright and saturated stars. (Who care about
these stars anyway?) The real caveat is, the entire package is
designed to reduce images that only contain faint galaxies. So
are the background subtraction routines. The more_bg_subtract
is very aggressive and is very likely to produce inaccurate photometry
on large galaxies (larger than ~5"). This will be a
problem for observations of low-redshift clusters, for example.
For high-redshift blank-field imaging, there is less to worry about.
sometimes produces nearly circular fringes. To remove the
circular fringes, an image is first transformed to a polar coordinate
a model of the fringes is derived there. Then the model is
transformed back to Cartesian coordinate and is subtracted from the
image. The only question here is that the defringe procedure
cannot automatically determine the center of the circular fringes, and
for chip 2 the center of
the fringes moves slowly, by a time scale of roughly an hour.
Thus, images taken in different dither sets may have different fringe
centers and the user needs to find out where the center is.
Fortunately this is not too difficult. The first step is to use
the individual=1 keyword in the main
defringing. Then pick up a reduced
individual-frame image that has strong fringes, and use the write_transform=1
keyword in the defringe_moircs
procedure to check if the fringes in polar coordinate are perfectly
vertical straight lines. If not, change the center of fringe
until they are vertical and straight (as much as possible). An
example script of how to do this is provided in the file scripts.fringe.
This is only necessary for chip 2 for most cases.
The fringe center on chip 1 rarely moves. Hopefully after MOIRCS
upgrades its filter, there won't be any fringe problems.
3.2.4 Artifacts around Bright Stars
When there are
very bright stars in the field, the flat-fielding can be
screwed and there will be artifacts around the bright stars with a
shape identical to the dither pattern. When this happens,
changing the flat method in field 2 in the APF from 1 to 0 usually
reduces the problem. If your computer is fast enough, always
0 is not a bad idea. Also see field 14 in the APF.
3.5 New Detector in Channel 1
Between Oct 2007 and June 2008, chip 1 is replaced with an engineer
grade one. After that, a new science grade chip was
installed. At this moment, SIMPLE can only deal with data taken
with the engineer grade chip (and the old sceince grade one, of
course), but not yet the new science grade chip. SIMPLE can tell
whether the data were taken with the engineer grade chip by looking at
the date in the header. Users do not need to do anything. A
revision that accounts for the new science grade chip will be provided
4. Limits and
Possible Future Improvements
version of SIMPLE-MOIRCS
tangential projection for the reduced maps. However,
it should be fairly easy to include other kinds of projection, if
procedure is specifically designed for MOIRCS in many ways.
However, many SIMPLE subroutines are written as
general as possible. It is possible to modify this package for
other near-IR mosaic cameras. With appropriate treatment for flat
fielding and defringing, I believe it can be used on CCD cameras as
well. SIMPLE has not been tested for cameras with large
distortions, such as the SuprimeCam on Subaru. This will be the
next SIMPLE development.
SIMPLE cannot drizzle images. A drizzle function can be
easily written for SIMPLE but I do not feel a need for this. If I
do receive considerable amount of request from users, I may provide
this function in the future.
When publishing data processed by SIMPLE, please refer to SIMPLE's
homepage http://www.aoc.nrao.edu/~whwang/idl/SIMPLE .