Hectospec Reduction Pipeline Bill Wyatt, 2006-Jul-14

[Definitions] [Preparation] [Observation] [Reduction (Pipeline) (Software)] [Distribution] [Archiving]

---

SPECROAD

The specroad script is the master program for the Hectospec reduction pipeline. It itself runs some of the processing, then calls specroadcal, which later calls specroadobj. At every stage, where processing of several images can be done in parallel, the scripts multiprocess. The default number of parallel processes is twice the number of cpus in the system, but can be explicitly set by the -j option.

specroad, specroadcal and specroadobj all take options:

-u   <UPARM>  - the IRAF parameter directory

-j   <# of parallel-jobs>

also,

specroad -e cal     - skips to the specroadcal script

specroad -e obj     - skips to the specroadobj script

Preparation

1. Get software installed or updated:
   • execute: ~wyatt/mine/Hectospec/Hectoproc/Pkg.install ~/bin/
   • execute: rehash

2. UPARM must be set in the environment or on the command line!

   setenv UPARM ~/iraf2.12.2/uparm/   or
   specroad -u ~/iraf2.12.2/uparm/

   Note that if the command line option is used, the UPARM is passed to the follow-on scripts specroadcal and specroadobj, but UPARM is not set in the current environment. (This is the norm for all scripts & programs - they do not alter their parents' environment.)

3. The PATH environment variable must include the directories /data/oiropt/starbase and /data/oiropt/bin.

4. cd to the data directory (but not in the archive hierarchy!).
5. Execute   specroad

Processing summary

SPECROAD

1. hscreen
   • Puts up a list of files with "IMAGETYP" = "test" or "PROPID" != "SAO", queries for deletion
   • File-by-file deletion is an option
   • Deletes *.cfg, *.cat files
   • If in one of the ARCH* trees, hscreen refuses to do any deletion

2. mapcheck
   • checks that the *_map file is the concatenation of *_map1 and *_map2
   • checks if the calibration maps are identical. If so, makes "generic_map"
   • checks that the maps for multiple exposures of a single object field are identical.

3. hcalibproc
   • runs HPROC on all BIAS files (zero files)
   • checks statistics on bias files, combines to Zero.fits
   • prompts for zerocorr flag setting for subsequent HPROCs
   • runs HPROC on all DARK files
   • checks statistics on dark files, combines to Dark.fits
   • prompts for darkcorr flag flag setting for subsequent HPROCs

4. multihproc (calibration phase)
   • runs HPROC on all remaining non-object files, usually comps, sflats, and domeflats.

5. multihmerge (calibration phase)
   • Runs HMERGE to merge the 4 amps into 1 image for all non-object, non-bias, non-dark fits files

6. multicombine
   - combines domeflats -> domeflat.fits, skyflats -> sflat.fits, comps -> comp.fits

SPECROADCAL

1. apflatten
   • creates aperture mask, flat.fits
   • Forks xterm to run interactive plotting

2. multihflat (calibration phase)
   • Runs HFLAT to flatten all dome, comp, sflat calibrations

3. multihextract (calibration phase)
   • Run HEXTRACT to extract spectra from domes, comps, sflats; makes *.ms files
   • forks xterm to run interactive domeflat extraction

4. hcal
   • create dispersion function
   • needs idCOMP.ms in ./database/
   • needs COMP.ms.fits and idhenear.dat in ./
   • forks xterm to run interactive dispersion fitting
   • makes summary table for each dispersion fit in database/idcomp.db as 300 starbase tables, each named APn, then a summary table for all 300 in database/idcomsumm.db

5. hlin sflat.ms.fits
   • linearize the skyflat in wavelength

6. maketran sflat.ms.fits
   • make throughput correction file tpc_sflat.ms.fits

SPECROADOBJ

1. multihproc (object phase)
   • Runs HPROC on all object files

2. multihmerge (object phase)
   • Runs HMERGE to merge all object files' extensions

3. multihflat (object phase)
   • flatten all object files

4. multihextract (object phase)
   • extract object exposures to .ms files

5. multihlin (object phase)
   • linearize object .ms files

6. multihscombine
   • combines each field's .ms files to a single one

7. multihtran
   • Does geometric correction on each field's combined .ms file
   • Does throughput correction on each field's combined .ms file

8. multihabsky
   • remove selected sky absorption features

9. multidoredleak
   • fit and remove "red leak" from all .ms files -> rename to *_r.ms

10. multihsplit
    • Split the _r.ms files to 300 spectra, into 1d_ directories

11. multiskyproc
    • Run SKYPROC on each 1d_* subdirectory, making skysub_* directories for the output spectra
    • Moves final sky spectra files into subdir "skydir" of each skysub_*, since they usually are not cross-correlated

12. xcsao   and/or   multixcsao
    • Run cross-correlations on skysub_* directories
    • If xcsao.displot is "yes", query before each subdirectory as to whether to do output plots on each one.
    • generates zout_<fieldname$gt; velocity table
    • generates general correlation summary table