Introduction to CalFUSE v2.4 General Release Version 2.4.0 7 May 2003 Van Dixon

Principal changes from CalFUSE Version 1.8.7

Principal changes from CalFUSE Version 2.0.1

Principal changes from CalFUSE Version 2.0.5

Principal changes from CalFUSE Version 2.1.6

Principal changes from CalFUSE Version 2.2.1

Principal changes from CalFUSE Version 2.2.3

1. Corrected heliocentric wavelength scale

In previous versions of CalFUSE, the file-header keywords V_HELIO, V_LSRDYN, and V_LSRSTD had the wrong sign. The pipeline attempts to correct for the Doppler shift introduced by both the spacecraft's motion about the Earth and the Earth's motion about the sun. (V_LSRDYN and V_LSRSTD, corrections to the dynamical and standard-solar local standard of rest, respectively, are not used by the calibration software.) Because of the sign error in V_HELIO, the heliocentric correction was applied incorrectly, enhancing, rather than reducing, the velocity shift. This error has been corrected in CalFUSE v2.0.

2. Detection and removal of event bursts

Occasionally, the FUSE detectors register large count rates for short periods of time. These event bursts occur in fairly well-defined regions of one or more detectors and often have a complex structure, including scalloping and sharp edges. CalFUSE v2.0 includes a module that screens the data to identify and exclude bursts. The data are binned, then median filtered to reject time intervals whose count rates differ by more than N standard deviations from the mean. The algorithm also rejects time intervals in which there is no data or in which the background rate rises rapidly, as when an observation extends into an SAA. Burst removal is possible only for ttag data.

3. Walk correction (for low-pulse-height events)

The FUSE detectors convert each ultraviolet photon into a shower of electrons, for which the detector electronics calculate the X and Y positions and the intensity, or pulse height. Prolonged exposure to photons causes the detectors become less efficient (a phenomenon called "gain sag"), and the mean pulse height slowly decreases. Unfortunately, the X location of low-pulse-height photon events is systematically miscalculated by the detector electronics. As the number of low-pulse-height events increases, spectral features appear to "walk" across the detector. (The amount of walk varies with location on the detector.) CalFUSE v2.0 incorporates a module to reposition low-pulse-height events, correcting for this effect. Histogram data, for which pulse-height information is unavailable, are only partially corrected for walk.

4. Improved scattered-light model

The background has two components, detector dark count and scattered light. The dark count is spatially uniform but varies in intensity with time. The scattered light has considerable spatial structure and differs between day and night-time observations. Previous versions of the pipeline simply scaled a flat background image by the exposure time. CalFUSE v2.0 scales separate day- and night-time scattered-light images and combines them with a uniform dark-count component to produce a model of the scattered light for a given observation. The details differ for TTAG and HIST data. For more information, see the CalFUSE Pipeline Reference Guide.

5. Astigmatism correction

The astigmatic height of FUSE spectra perpendicular to the dispersion direction is significant and varies as a function of wavelength. Moreover, spectral absorption features show considerable curvature, especially near the ends of the detectors, where the astigmatism is greatest. CalFUSE v2.0 shifts the data to correct for the spectral- line curvature introduced by the FUSE optics, providing a noticeable improvement in spectral resolution for point sources. Please note: because the effects of instrumental astigmatism are different in point-source and diffuse spectra, airglow lines may be distorted by the astigmatism correction applied to point-source spectra. Care should be taken when using airglow lines to fix the absolute wavelength scale of point-source spectra. At present, no astigmatism correction is defined for diffuse sources, and none is applied.

6. Optimal (weighted) spectral extraction

The extraction algorithms used in CalFUSE v2.0 provide two important improvements over earlier versions of the pipeline. First, the software determines the Y location of the target spectrum by performing a cross-correlation between the data (in the form of a detector image) and the known distribution of flux on the detector. This routine greatly improves our ability to center the extraction window on the target spectrum.

Second, spectra are extracted via an optimal-extraction algorithm, which produces a weighted sum of the pixels falling within the extraction window. Pixels having a quality flag of BAD_PIXEL do not contribute to the sum. The corresponding variances are weighted and summed to produce an error array. Optimal extraction should improve the signal-to-noise ratio of spectra extracted for faint point sources.

7. Improved wavelength calibration

Previous versions of CalFUSE employed a single wavelength calibration for all three apertures (LWRS, MDRS, HIRS). In fact, the image distortion and wavelength dispersion vary slightly with aperture. For v2.0, we provide a separate wavelength calibration for each aperture. These calibration files are derived from measurements of some 14,000 molecular hydrogen absorption lines. FUSE relative wavelengths are accurate to 3-4 pixels across most of the spectrum and 6-8 pixels in certain regions, as discussed in the White Paper The FUSE Wavelength Calibration.

The new FUSE wavelength scale is derived from astigmatism-corrected, point-source spectra. No astigmatism correction is applied to extended-source spectra, raising the possibility that these wavelength files may be inappropriate for diffuse sources. To find out, we ran the pipeline on a point-source spectrum twice, once with astigmatism correction and once without. The resulting calibrated spectra differed in wavelength by less than four pixels, consistent with the uncertainties in the wavelength scale. Thus, the new FUSE wavelength scale should work well for diffuse spectra, especially given their lower resolution.

8. Improved flux calibration

The changes for the LWRS aperture are minor, but the MDRS and HIRS calibrations are much improved over previous versions of CalFUSE. The MDRS and HIRS curves are derived from spectra obtained through those apertures, rather than simply being copies of the LWRS curve scaled by their relative areas.

9. How to turn things off

Some of the pipeline steps introduced in v2.0 may not be appropriate for all data sets. To provide an easy way for users to disable individual pipeline modules, we have added the following lines to each of the parm*.fit files:

RUN_BRST= 'YES'                / Burst rejection
RUN_WALK= 'YES'                / Walk correction
RUN_MKBK= 'YES'                / Create background image
RUN_BKGD= 'YES'                / Subtract background image
RUN_ASTG= 'YES'                / Astigmatism correction

To turn off background subtraction, set both RUN_MKBK and RUN_BKGD to 'NO'. Though not used by the background-subtraction module, the zero-value background image is needed by the both the distortion-correction and optimal-extraction routines. If you wish to substitue your own background file for the default pipeline version, be sure to edit the calling sequences for all three of the following modules: cf_bkgd, cf_ttag_geodopp, and cf_extract.

To turn off optimal extraction, set the keyword OPT_EXTR = 0 in each of the parm*.fit files. The Y location of the spectrum will be determined as usual (via a cross-correlation between the data and the weights file), but the resulting spectrum will be a simple, unweighted sum of all good pixels in the extraction window. Bad pixels and those for which the corresponding weight is 0 are not included in the sum.

10. Improvements to ttag_combine

For some time now, we have been telling people that the best way to model the background for long observations is to run CalFUSE through the screening step (cf_ttag_screen) for each exposure, combine the resulting *_scrn.fit files, then run the rest of the pipeline on the combined file. This is the best approach, but we lacked a tool to combine the *_scrn.fit files, because ttag_combine did not propagate the DX,DY,and DNFLAG columns. Now, it does. The program also propagates a number of new counters and issues a warning if you try to combine different segments of an FP-split observation.

11. Bug fix in walk correction

In v2.0.1, the walk correction was erroneously applied to the stim pulses, causing the drift correction to be miscalculated. We have corrected this problem and revised our wavelength calibration. As a result, you may see small differences in the wavelength scales of spectra processed through CalFUSE versions 2.0.1 and 2.0.5, especially in detector segments 1B and 2B, for which the stim-pulse shifts were significant.

12. Screening module employs new logic

Previous versions of CalFUSE treated user-defined good-time intervals with a bit too much respect: all other screening (e.g., for SAA's) was turned off, and the exposure time was set equal to the sum of the good-time intervals whether or not they all contained data.  For CalFUSE v2.1, we have modified the logic of cf_ttag_screen as follows: the program reads both the user-defined intervals and the intervals contained in the data file (from OPUS or cf_ttag_combine) and uses the intersection of the two (user wants data and data were taken) to screen photon events and compute the total exposure time.  All other screening (for SAA crossings, limb angle, day/night, and pulse height) is performed as usual.  Screening can be turned off by editing the scrn*.fit file.

13. Bad-pixel maps reflect changes in detector Y scale

The detector Y scale is not constant, but changes both with time through the mission and as a function of detector count rate.  As the Y scale changes, dead spots on the detector move relative to our bad-pixel maps, preventing the pipeline from properly excluding them during spectral extraction.  To account for these effects, we have developed a series of bad-pixel maps that span the entire mission, reflecting the change in Y scale with time.  In addition, we have modified the pipeline module cf_update_qual to stretch the bad-pixel map to account for the count-rate-dependent change in the Y scale.

14. New module corrects for motion of the focal-plane assembly

For various reasons, we often move the focal-plane assemblies (FPA's), which contain the spectrograph apertures, from their nominal positions, causing the target spectra to move on the detectors. To allow wavelength calibration of the resulting spectra, we must shift them back to the standard FPA position. In previous versions of CalFUSE, this shift was performed within the wavelength-calibration module. We have moved it to a separate module, cf_dpix, which immediately follows the astigmatism correction in the pipeline. The advantage is two-fold: first, the FPA shift is included with the other shifts as a floating point, rather than requiring a separate floating-point-to-integer conversion; second, separating this function from the wavelength calibration will simplify future efforts to produce spectra with a linear wavelength scale.

15. Bug fix in dead-time correction

We recently discovered that the dead-time correction (TOT_DEAD) was being calculated incorrectly for TTAG exposures. In all previous versions of CalFUSE, the module cf_dtcorr reads the exposure time from the header keyword EXPTIME. Various engineering keywords record the total number of counts seen by each detector both before and after an observation. From these, cf_dtcorr determines first the count rate, then the dead-time correction. This works if EXPTIME is roughly equal to the time period covered by the engineering data, but if CalFUSE has rejected significant chunks of data due to bursts, screening, etc., then EXPTIME will be too small, and the dead-time correction will be overestimated. Fortunately, the effect is negligible unless the count rate is very high (thousands of counts per second), so reducing EXPTIME by even a factor of two will not change TOT_DEAD significantly. The problem came to light only when a user attempted to dice her observation into 50-second chunks: as EXPTIME got very small, TOT_DEAD began to grow.

The solution is simple: the times associated with the various counters are also recorded in the file headers, so we can determine the actual time intervals that they sample. The program cf_dtcor now performs this calculation. Note that, even with this change, the dead time may be overestimated, because the calculated deadtime is an average over the entire exposure, which may include periods of high count rate (bursts and SAA passes) that are excluded by CalFUSE. Here's a test: if the source count rate is less than 4000 counts per sec (LiF+SiC), then TOT_DEAD should be less than 1.05. (At the low count rates seen in time-tagged data, TOT_DEAD should be close to unity.) If bursts have been removed, then the deadtime correction is probably spurious and the spectra should be divided by TOT_DEAD.

16. Bug fix to APER_PA keyword

We have discovered an error in the keyword APER_PA, which contains the position angle of the Y axis (in degrees E of N). Future versions of OPUS (2.6 or greater) will report the correct value; in the mean time, it is

Aperture Position Angle = APER_PA - ROLLOFF - 2.5,

17. Bug fix in burst-rejection routine

In CalFUSE v2.0, the burst-rejection routine cf_ttag_burst did not update the header keyword EXPNIGHT to account for night time lost to bursts. That keyword is now properly modified.

18. Bug fixes in error propagation

We've corrected a couple of bugs in the way that CalFUSE treats error bars. In each case, one or more programs assumed that the error bars stored in intermediate pipeline files were values of sigma rather than variance (= sigma²). The programs cf_make_ttag_bkgd and cf_make_hist_bkgd produce background models with an uncertainty assumed to be 10% of the model intensity. The third HDU for each output file was set to 10% of the model background, rather than (0.1 × background)². This bug was introduced in v2.0 of the pipeline.

When creating the final, corrected detector image, cf_ttag_geodopp and cf_hist_geodopp multiplied both individual photons (or pixels) and their associated uncertainties by the same scale factor, which would be correct if the errors were sigma values. Because they are variances, however, the correct scaling is (scale factor)². This bug has been around for a while; it is present in v1.8.7 of CalFUSE and thus in all calibrated data from the archive.

How these two bugs interacted is difficult to determine, but preliminary results from v2.1 suggest that error bars in the final, fully-calibrated spectra produced with CalFUSE v2.0 were slightly over-estimated, especially for moderately faint continuum targets.

19. Improved background models

We have developed a new, time-dependent set of background calibration files. To conserve disk space and speed i/o, the files are binned by 16 pixels in X. We have modified the background modules to read the compressed files and implemented a new fitting algorithm to make the programs run faster.

20. New parameter files contain background information

We received several complaints that the scrn*.fit parameter files were getting unwieldy, so have moved all of the binary extensions to a new set of bchr*.fit files. BCHR stands for ``background characterization,'' and the files contain information used to scale the background models. Both the SCRN and PARM parameter files are now pure ASCII files, though still in FITS format.

21. Jitter-Correction Module

We can now correct FUSE data for slow drifts resulting from a loss of pointing control during an observation. For time-tagged data, we reposition individual photon events to correct for spacecraft motion. For histogram data, we determine when the target was out of the (LiF 1) aperture and scale the flux accordingly. Both routines use information from a new file, *jitrf.fit, created by OPUS during level-zero processing (LZP). Because this file does not exist for any data processed before August 2002, the jitter-correction routines cannot be applied to older data sets. For details, please see the FUSE white paper Jitter Correction in CalFUSE.

22. Time-dependent Flux Calibration Files

Since the middle of 2001, we have noticed a slow degradation in the effective area of the FUSE spectrograph, more or less independent of wavelength. We have produced a new set of flux calibration files that account for this effect. For details, please see The FUSE Flux Calibration.

23. Improved Background Fits

The models have not changed, but we have modified our scaling algorithm to better match the observed background. We've also corrected some bugs that troubled Linux users.

24. Count-rate Plots for Histogram Data

Our new jitter-correction files are derived from housekeeping data files that contain, among other things, counter information for each of the FUSE detectors. We use these counters to derive the count rates of the LiF and SiC channels for each detector during histogram observations. The resolution is only 16 seconds, but the resulting plots are good enough to reveal bursts or periods when the target was out of the aperture.

25. Improved Calibration Files

Besides the time-dependent flux-calibration files, we have new QUAL, YSTR, and SPEX files. The YSTR files, which correct for the expansion of the detector Y scale as a function of count rate, are now defined relative to a count rate of zero, rather than infinity. The QUAL files, which list the positions of known detector artifacts, and the SPEX files, which contain extraction limits for the various apertures, now use the same zero-count-rate standard.

26. New High-Voltage Calibration Files

In the header of each data file is recorded the detector high voltage setting during that exposure. If this value falls outside the nominal range, CalFUSE writes warning messages to both the file header and the trailer file. We change the detector high voltage every six to twelve months and record the nominal values in a set of volt*.fit files, one for each detector segment. Version 006 of the volt*.fit files reflects the most recent of these changes, which occurred in February 2003. For more information on time-dependent changes to the detectors, see the CalFUSE White Paper Time-Dependent FUSE Calibration Effects.

27. Date Change for Initial Flux Calibration File

When a set of time-dependent flux-calibration files was released (see Section 22 above), the MJD associated with the first of these files was set to 50000, our short-hand for ``a long time ago.'' In fact, FUSE was launched on MJD 51403. Because we interpolate linearly between pairs of flux-calibration files, use of this early date for the first file could result in a slight under-estimate of the instrument sensitivity early in the mission. We have modified the date associated with the first set of flux-calibration files in master_calib_file.dat.

28. New Meaning for Keyword EXP_BAD

We have modified the burst and jitter modules to update the keyword EXP_BAD when they reject some fraction of the exposure time. EXP_BAD now represent the total exposure time rejected by the pipeline, rather than just the time rejected by the screening module.

29. Various Modifications and Bug Fixes

Over the past six months, we have made a number of modifications to the pipeline to make it more robust. For example, the jitter modules now issue a warning, but continue running, if the keyword JIT_STAT is not present in the file header. A number of bugs that showed up only on Dec Alpha compilers have also been repaired.

30. Error Bars in Time-Tag Mode

One of the changes included in CalFUSE v2.1.6 was a modification of the way that error bars for individual pixels are calculated in the final, motion-corrected detector image (see Item 18, above). It was recently pointed out that this new algorithm does not yield SQRT(N) error bars for bright sources (fluxes > 1E-12 erg/cm2/s/A). The trouble arises when we normalize the error for each detector pixel by 1/N^2. For most targets, the number of counts per detector pixel is zero or one, so this normalization has no effect. For bright targets, N can be as large as 3 or 4, so the error bars are greatly reduced. After considerable debate, we have decided to return to the 1/N normalization originally used by the CalFUSE pipeline.

31. An Error in the Application of the FUSE Flux Calibration

In the course of our ongoing monitoring of the FUSE sensitivity, we have discovered a bug in the way that CalFUSE applies the flux calibration. For data obtained between the dates of two flux-calibration files, the pipeline interpolates between the two calibration files, as it should. For data obtained after the most recent calibration file, however, the pipeline derives a flux- calibration curve by linearly extrapolating from the last two calibration files. This is an error; the program should simply use the most recent flux-calibration file. For more information on this bug, together with tools for correcting it, please see the document An Error in the Application of the FUSE Flux Calibration.

Please report any anomalies to fuse_support@pha.jhu.edu.

Last modified Wednesday, May 7, 2003.