Detector Scaling Product¶
Metadata¶
Data product name |
DpdSirDetectorScaling |
Data product custodian |
SIR |
Name of the Schema file |
|
Processing function using the data product |
SIR |
Proposed for inclusion in EAS/SAS¶
This product is proposed for inclusion in the SAS: no
This is an internal Data Product.
Data product elements¶
Header |
object of type sys:genericHeader |
Data |
object of type sir:detectorScaling |
QualityFlags |
object of type dqc:sqfPlaceHolder |
Parameters |
object of type ppr:genericKeyValueParameters |
Detailed description of the data product¶
The sirDetScaling describes the steps to flat-field each detector at the pixel level.
Description of Product:¶
The detector scaling product is a set of 2-dimensional flat-field images (one per detector) representing the average of the pre-launch quantum efficiency (QE) maps weighted over wavelength. The weighting of the original chromatic QE cube (obtained from the MDB) is performed assuming a simple power-law spectral shape derived from the zodiacal light. Thus the product acts as both a small-scale flat field (correcting for small-scale pixel-to-pixel variations), but also includes the average QE conversion factor for each pixel. Thus after applying this product in the apply-step of the SIR_Preprocessing science pipeline, we are effectively converting from electron number per pixel, to photon counts per pixel. Although we are aware that weighting the QE maps by a Zody spectrum is an approximation, the detectors are relatively achromatic, and so this approach seems justified as a first approximation. Furthermore, many sources in the survey will be detected close to the background signal which is dominated by the Zody. Local variations in the QE maps averaged over several hundreds of pixels are typically correcting the input signal at the 1% level, whereas for standard integration times used in the survey, the shot-noise in the detectors is typically 3-4% in a blank region of the detector. This means that application of the flat for a well-behaved part of the image is relatively benign. Despite this, the flat does have a significant net positive effect on the output images, because it also corrects for discontinuities that occur at the boundaries of known detector artifacts (e. g. the famous “fish” and “duck” shaped regions) where regions of the detector show rapid drops in QE (at 3-5% level) at the edges of larger islands of lowers QE (sometimes hundreds of pixels in length) resulting from manufacturing issues with the detector substrate. These effects are well corrected by the detector scaling product and lead to a significant flattening of the input images, after application.
Description of the input files:¶
This process makes use of the quantum efficiencies (QEs), for each pixel, measured at the NASA Goddard at 40 different wavelengths (\(\lambda\)) between 600 nm and 2550 nm, for each detector. These QE maps are saved in one file per detector, the naming pattern being qe_SCA_18XXX_mdb_v1.fits, with 18XXX the SCA id of an individual detector. These fits filenames are picked up from the MDB XML files as a list. In each fits file, the primary hdu contains no data but the header provides the number of wavelength (NUMWAVES=40), the wavelength range (WAVRANGE = 600-2550 nm) and the unit of QE values. Each following extension contains a map of QE (in percentage) for a given wavelength. The wavelength is saved in the header keyword WAVELEN. The extension name pattern is WAVELENXXXnm (or WAVELENXXXXnm, depending on the number of digits in the wavelength), with XXX (or XXXX) being the corresponding wavelength in nm. QE value can exceed 100% or be negative due to experimental errors. Each extension header has additional keywords to match the different detector ids, such as SCA_ID, SCS_ID, SCE_POS and SCS_NUM.
This process also uses the NISP total transmission for grism saved in a fits file. The corresponding fits filename is also read up from the MDB XML file. The fits file contains several BinTableHDUs, one per grism order transmission. The names of the HDUs are RGS000_ORDER_0, RGS000_ORDER_1, RGS000_ORDER_2, RGS180_ORDER_0, RGS180_ORDER_1, RGS180_ORDER_2, RGS270_ORDER_0, RGS270_ORDER_1, RGS270_ORDER_2, BGS000_ORDER_0, BGS000_ORDER_1, BGS000_ORDER_2; with RGS and BGS standing for Red and Blue Grism respectively. Each of these tables have two columns, representing the values of the wavelength and the corresponding response curve, WAVELENGTH and TRANSMISSION respectively. The units of the WAVELENGTH column are specified by the TUNIT1 header entry and is in nm. For our module, we used only the extensions RGS000_ORDER_1 and BGS000_ORDER_1 for Red and Blue grism input transmission curves respectively.
Structure of the Calibration Product:¶
We save each detector’s 2-dimensional flat-field image in separate fits extensions, having all 16 detectors’ images in one single fits file. The primary header has detector-specific information. For example, DETNAME as DET_11, DET_21 etc; SCAID as 18453 etc; CalibID as RGS or BGS; EXTNAME as DET_11.FLAT, DET_21.FLAT etc. We produce two separate files for RGS and BGS. In the end, we wrap these fits files around with the detector-scaling product XML file, each corresponding to the appropriate blue or red grism.
How the 2-D Product is Created from the 3-D QE cube:¶
During the calibration process, we use an effective wavelength range, for each grism, for which the input transmission curve throughput is greater than 10% of the maximum value of the transmission curve. The rest of the wavelength range, if QE maps are available, are ignored for this purpose. In this effective wavelength range, we interpolate the transmission values, \(Tr\), at the specific wavelengths (specified by \(\lambda\) vector, in nm) onto the same wavelength scale and range for which we have available QE values from the QE maps. Thus, now we have QE maps at an array of \(\lambda`s, while we also have an array of interpolated transmission values at the same :math:\)lambda` s. Please note that we used some Python notations, such as \(len(array)\) to describe the size of that array, in our documentation. We define Median zody spectrum intensity for all wavelengths \(\lambda\), as an array,
\(I_{\lambda, \text{ zody}} \sim {(\lambda/1000)^\alpha} \text{, with } \alpha = -0.9 \text{ & } -0.75 \text{ for RGS and BGS respectively}\)
We also define,
\(\Delta \lambda = \lambda_0/(2 \times resolution), \text{where } \lambda_0 \text{ is the first element of the } \lambda \text{ array and } resolution = 500\)
Next, we define \(\lambda_s\), \(\lambda_{lo}\) and \(\lambda_{hi}\) such that,
\(\lambda_{s, \, index} = \lambda_{s, \, index-1} + \Delta \lambda\) as an array where \(\lambda_{s, \, index = 0} = \lambda_0\) and \(\lambda_{s, \, index = len(\lambda)-1} = \lambda_{len(\lambda)-1}\)
\(\lambda_{lo} = \lambda_s - {\Delta \lambda}/2\)
and
\(\lambda_{hi} = \lambda_s + {\Delta \lambda}/2\)
Then we define,
\(\Delta \nu = h \, c \times 10^9 \, \left(\frac{ 1}{\lambda_{lo}} - \frac {1}{\lambda_{hi}} \right) \text{ , where } h \text{ is Plank's constant & } c \text{ is speed of light in nm/s}\)
At this point, we set up a cubic interpolation function with Python function \(interp1d\) having arguments \(\lambda_s \text{ and } \Delta \nu\). We use this function to interpolate for \(\Delta \nu \text{, defined as } \Delta \nu_{\text{ interpolated}} \text{, at various values of } \lambda\)
Subsequently set, \(\Delta \nu_{\text{ interpolated}} = 10^{-11} \, \Delta \nu_{\text{ interpolated}}\)
Now, we estimate number of Zody photons at each wavelengths of \({\lambda}\), for width \(\Delta \nu_{\text{ interpolated}}\) as,
\(\text{photons}_{\, {\text zody}} = I_{\lambda, \text{ zody}} \; \Delta \nu_{\text{ interpolated}} \, Tr /(h \, c)/(\lambda/1000)) \text{ , where } h \text{ is Plank's constant & } c \text{ is speed of light in {\mu}m/s}\)
With this, we define weight vector, at each element of the \({\lambda}\) vector, as,
\(w = \frac{\text{photons}_{\,\text{ zody}}}{\sum \; \text{photons}_{\, \text{ zody}}}\)
We apply the \(w\) vector to all pixels, within the useful range of QE values between 5.0% to 120%, for QE transmission values, for each element of the \(\lambda\) vector, as follows,
\(\text{flat-fielded pixel map for a detector} = \sum_{\text{over all }\lambda} \;\; w_{\text{ for each }\lambda} \times {\text{input QE map for the detector }}_{\text{for each }\lambda}\)