Note: CalFUSE v3.1 employed a new scheme to encode the data quality of the jitter files. CalFUSE v3.2 uses yet another scheme. As a result, old jitter files will not work properly with the new pipeline. It's easy to generate new ones; just follow the instructions below.
Introduction
CalFUSE v3.2 is the latest version of the FUSE data-reduction pipeline. The new code incorporates dozens of tweaks and bug fixes relative to CalFUSE v3.1, including a correction to the background scattered-light model, a new grating-motion correction algorithm, and a new set of routines to create and employ the jitter-correction files.
Recent Improvements
New in CalFUSE v3.1.5
Bad-Pixel Correction: A new routine, cf_screen_bad_pixels, flags photon events falling in known bad-pixel regions.
Histogram Mode: For data obtained in HIST mode, cf_count_rate_y_distort must use a single count-rate value for the entire exposure. To better estimate it, we replace the time-dependent FEC-rate array with its weighted mean value. Rather than using different HIST_PAD values for each aperture, cf_extraction_limits now pads apertures by 8 pixels (top and bottom).
Jitter Correction To improve the jitter-correction algorithm, we removed the data-screening functions from cf_satellite_jitter and moved them to a new routine, cf_screen_jitter.
New in CalFUSE v3.1.6
Burst Correction: Because cf_screen_bursts attempts to exclude airglow photons from the calculated background, these photons must be flagged before the routine is called. To this end, cf_screen_airglow is now the first of the screening routines.
Background Models: The model background consists of a spatially-uniform component and a scattered-light image. As a result of changes introduced in CalFUSE v3.1.3, the scattered-light component was over-estimated, because the data were summed over a larger region of the detector than the scattered-light image. Now, the same limits are applied to both data and models.
New in CalFUSE v3.1.7
Spectral Centroids: We tightened the limits over which cf_find_spectra computes the spectral centroid to prevent a bright star in a nearby aperture or the enhanced background near the detector edge from pulling the extraction window off center. In both cf_compute_y_centroid and cf_find_spectra, we returned the parameter-file keywords EMAX_LIF and EMAX_SIC to their original behavior: if the centroid differs from the default value by more than this limit, the default value is used.
New in CalFUSE v3.1.8
Astigmatism and Dispersion: The routine cf_astigmatism_and_dispersion was divided into two routines, cf_astigmatism and cf_dispersion.
Fewer Warnings: The pipeline no longer issues WARNING messages when a raw or intermediate data file is empty, when the jitter file is missing, when the detector voltage is low, or when it cannot perform optimal extraction. It issues level 1 verbose messages instead.
Fewer Bad Spectra: The program cf_extract_spectra no longer discards (that is, sets to zero the flux and error arrays of) spectra with non-zero values of EXP_STAT.
New in CalFUSE v3.2.0
Time-Dependent X and Y Distortions: It turns out that the X and Y coordinates computed by the detector electronics for a particular spectral feature drift slowly with time. The new routine cf_time_xy_distort corrects for this effect.
Grating Motion: The grating-motion correction has been revised to better correct the zero point of the wavelength scale.
Jitter Files: Over the years, FUSE engineers have repeatedly modified the spacecraft-control software, but the routine that generates jitter files has not been kept up to date. As a result, the program was not using all available information to determine the quality and value of pointing errors. We have modified the code to make use of this information. As a result, the meaning of the jitter-file TRKFLG array has changed: values between 1 and 5 reflect increasing levels of confidence in the quality of reported pointing errors, a value of 0 indicates that pointing telemetry is missing, and a value of -1 says that the pointing is unknown but likely to be bad. Jitter files generated with the new routine receive a version number of 3.0. The use of old jitter files with the new software is likely to yield unsatisfactory results. If you have the housekeeping files, you can generate jitter files that are consistent with the new pipeline. To do it, delete (or hide) the old jitter files, then type
prompt> cf_jitter P9990101001hskpf.fit P9990101001jitrf.fit
for each exposure. Run the pipeline as usual. Note that the following warning message may be safely ignored:
cf_jitter-3.3: Housekeeping file format is out of date.
Jitter Correction: The jitter routines cf_screen_jitter and cf_satellite_jitter have been updated to take advantage of the information present in the new jitter files, and three new keywords have been added to the parameter file. Times for which the X or Y pointing errors are greater than DX_MAX or DY_MAX (default is 30") are flagged as bad. Only pointing errors with associated tracking flag values >= TRKFLG (default is 1) are considered reliable.
FIFO Overflow: Before, the screening routines flagged times when the detector count rate went to zero because of FIFO overflows. This technique underestimated the target flux. Now, we modify the IDS dead-time correction to account for photon losses due to FIFO overflows.
New in CalFUSE v3.2.1
Jitter Correction: We have modified the program cf_jitter, which constructs jitter files, to be more robust when presented with weird pointing data.
SAA Contours: The size of the South Atlantic Anomaly varies on timescales of weeks. If you find that the default contours (in the file saac004.fit) are too small, causing times with elevated background rates to be flagged as good, you can switch to the smaller set of contours in the file saac005.fit by modifying the file master_calib_file.dat.
New in CalFUSE v3.2.2
New Platforms: CalFUSE now runs on 64-bit machines running Linux and Sun workstations running Solaris v10. Please see the document INSTALLING_CalFUSEv3.2.2 for installation instructions.
New Calibration Files: We have used data from the last few months of the mission to produce final versions of the PHAH_CAL, STIM_CAL, and PHAX_CAL files. New effective-area and background files for 2005-2007 are also provided.
Overview of the Pipeline
The following modules are called by the shell script calfuse.csh:
cf_ttag_init, cf_hist_init: Converts raw-data file into an IDF containing photon-event list, GTI's, and timeline table. Histogram data are converted to a pseudo-time-tag format.
cf_convert_to_farf: Corrects for detector deadtime. Transforms photon coordinates into the FARF (flight alignment reference frame).
cf_screen_photons: Checks data quality. Assigns status flag to each photon.
cf_remove_motions: Corrects for mirror, grating, FPA, and spacecraft motions.
cf_assign_wavelength: Applies astigmatism and Doppler corrections and assigns a wavelength to each photon.
cf_flux_calibrate: Converts WEIGHT to ERG/CM2 for each photon.
cf_bad_pixels: Applies image-motion corrections to bad-pixel map.
cf_extract_spectra: Extracts LiF and SiC spectra only for target aperture. Wavelength array is user defined; default spacing is 0.013 Å.
Installing and Running the Pipeline
CalFUSE is available for machines running Solaris, Linux, and Mac OS X (10.2 and higher). We no longer have access to a Dec Alpha machine, so cannot support that platform.
Please see the document INSTALLING_CalFUSEv3.2.2 for installation instructions.
To run the code, the command is the same for both TTAG and HIST data:
prompt> calfuse P99901010011attagfraw.fit prompt> calfuse P99901010011ahistfraw.fit
The pipeline expects to find all data files (*raw.fit, *jitrf.fit, *hskpf.fit) in the current directory. It operates on one detector segment at a time.
CalFUSE v3 produces the following output files: an intermediate data file (IDF), discussed below; a bad-pixel map (BPM), with a format similar to the IDF; two extracted spectral files, one for each of the LiF and SiC channels; and (if IDL is installed on your machine) a pair of GIF or JPEG files, one an image of the detector and the other a count-rate plot.
The Intermediate Data File (IDF)
The photon-list files consist of three FITS binary tables. The first contains the photon events themselves. The pipeline does not discard any events, but uses a series of flags to indicate whether particular photons violate pulse-height limits, limb-angle constraints, etc. The IDL tool cf_edit (discussed below) allows users to modify these flags and combine IDF files from multiple exposures.
The second extension to the data files is a list of good-time intervals (GTI's). They are not used by the extraction routine, but may prove helpful to pipeline users.
The third extension is called a timeline table. For each second during the exposure, it lists a dozen parameters, including count rate, day-night status, and detector voltage. The pipeline uses this list to set the status flags for each photon.
Note: because of the file format employed by IDF files, extensions 1 and 3 must be read using the /fscale keyword.
idl> a=mrdfits('P99901010011attagfidf.fit',1,/fscale)
idl> help,a,/str
Elements of individual arrays must be addressed using the syntax
idl> print,a.time[3:30] -- not a[3:30].time
For details about the format and contents of the IDF, please see the document IDF_Format.txt.
CF_EDIT is an Interactive Data Language (IDL) visualization tool for the examination and modification of FUSE IDF files. Users without IDL licenses may use the IDL "Virtual Machine" version of the tool for free. Complete instructions for installing and using cf_edit are available from the FUSE IDL Tools web page. More information on the IDL Virtual Machine is available from ITT.
The Output Spectral File
The format of the output spectral files is somewhat changed from earlier versions of CalFUSE.
float WAVE Wavelength (Angstroms) float FLUX Flux (erg/cm2/s/A) float ERROR Gaussian errors (erg/cm2/s/A) int COUNTS Raw counts in extraction window float WEIGHTS Raw counts corrected for deadtime float BKGD Estimated background in extraction window short QUALITY Percentage of window used for extraction
Note that the QUALITY array no longer lists the number of Y pixels that contribute to each output X pixel. Instead, it is the fraction of the extraction window considered to contain valid data, expressed as a percentage (0 is all bad; 100 is all good).
Do-it-yourself-ers can use the following recipe to generate a flux-calibrated spectrum:
TARGET_COUNTS = WEIGHTS - BKGD TARGET_FLUX = TARGET_COUNTS * HC / LAMBDA / AEFF / EXPTIME /WPC
where AEFF = effective area in cm2, and WPC = width of output pixel in Ångstroms.
Additional Files Available from MAST
CalFUSE produces eight extracted spectral (*fcal.fit) files for each exposure. We combine them into a set of three observation-level files for submission to the MAST archive. Depending on the target and the questions that you are trying to answer, you may find that these files are of sufficient fidelity for scientific investigation. Here's a brief description of their contents:
ALL: For each channel (LiF 1A, SiC 1A, etc.), we combine data from all exposures in the observation into a single spectrum. If the individual spectra are bright enough, we cross correlate and shift before combining them. If the spectra are too faint, we combine the individual IDF files and extract a single spectrum to optimize the background model. The combined spectrum for each channel is stored in a separate extension.
Note: It turns out that the cataloging software used by MAST requires the presence of an ALL file for each exposure, not just for the entire observation. We now generate such a file, but it contains no data, only a FITS file header. The observation-level ALL files discussed above can be identified by the string "00000all" in their names.
ANO (all, night-only): Same format as the ALL files, but using only data obtained during the night-time portion of each exposure. These files are generated only for TTAG data, and only if EXPNIGHT > 0.
NVO (National Virtual Observatory): Contains a single spectrum spanning the entire FUSE wavelength range. The spectrum is assembled by cutting and pasting segments from the most sensitive channel at each wavelength. Segments are shifted to match LiF 1A between 1045 and 1070 Å. Columns are WAVE, FLUX, and ERROR and are stored in a single binary table extension.
Data-Analysis Tools
A variety of IDL routines to display and manipulate FUSE data are available from the FUSE IDL Tools Reference Page. Data-analysis tools distributed with the CalFUSE pipeline are reviewed in the document FUSE Tools in C. A subset of these programs, designed specifically for the manipulation of IDF files, is described in the IDF Cookbook, available in both PDF and HTML formats.
Bug Reports
Please report any anomalies to
fuse_support@pha.jhu.edu.
WebMasters: