.. _NIRStackedFrame: NIR Stacked Frame ================= Metadata ______________________ +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Data product name | DpdNirStackedFrame | +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Data product custodian | NIR | +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Name of the Schema file | .. raw:: html | | | | | | `euc-nir-StackedFrame.xsd `_ | +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Last Edited for DPDD Version | 1.1 | +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Processing Element(s) creating/using the data | **Creators:** | | product | | | | * NIR | | | | | | The **DpdNirStackedFrame** data product components are created by different tasks during the | | | execution of the *NIR_ProcessField_Pipeline*: | | | | | | * The :term:`MEF` file is created by the **NIR_DostackProgram** task of the *NIR_Stacking* | | | project | | | * The PSF model and image are created by the **NIR_PSFModelling** task of the | | | *NIR_PointSpreadFunction* project | | | * The data product xml is written by the final *NIR_ProcessField_Pipeline* step that is | | | **NIR_CreateScientificDpds** of *NIR_Utilities* project | | | | | | **Consumers:** | | | | | | * NIR | | | | | | Into the *NIR_ProcessField_Pipeline* the :term:`MEF` file is used as input for the | | | **NIR_CatalogExtraction** PE to create the :ref:`NIRStackedFrameCatalog` containing source | | | extracted. | +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Processing function using the data product | .. | | | Indicate here the PF you are aware of using the product. Can be the custodian PF itself for | | | internal product. Can also be Science Working Group for final products. The list does not | | | need to be exhaustive. | | | | | | This is one of the output of the *NIR_ProcessField_Pipeline*. | | | | | | In the pipeline the :term:`MEF` file is used as input for the **NIR_CatalogExtraction** PE | | | to create the :ref:`NIRStackedFrameCatalog` | | | with source extracted. | | | | | | * PF-NIR | +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Proposed for inclusion in EAS/SAS | This product is proposed for inclusion in the SAS: **yes** | +------------------------------------------------+----------------------------------------------------------------------------------------------+ Data Product Elements _____________________ +--------------+----------------------------------------------+ | Header | object of type sys:genericHeader | +--------------+----------------------------------------------+ | Data | object of type nir:nirStackedFrame | +--------------+----------------------------------------------+ | QualityFlags | object of type dqc:sqfPlaceHolder | +--------------+----------------------------------------------+ | Parameters | object of type ppr:genericKeyValueParameters | +--------------+----------------------------------------------+ Detailed Description of the Data Product _________________________________________ .. DetailedDescStart The **DpdNirStackedFrame** is one of the output of the *NIR_ProcessField_Pipeline* applied to wide survey observations and in general to "scientific" pointings (i.e. Self Calibration). It is the result of the stacking process of a set of input images, 4 in case of wide, the nominal survey strategy. It could be created also in some specific cases with more exposures. A single stacked image is provided at the end of each *NIR_ProcessField_Pipeline* run. .. _NIR_stacked_frame_catalog: .. figure:: ../figures/nirStackedFrame.png :scale: 100 % :align: center :figclass: align-center : Graphical representaton of ``Data`` section structure for **DpdNirStackedFrame** The main elements inside the *Data* section are: * **Filter**: The filter name (e.g. NIR_Y, NIR_J, NIR_H) * **WCS**: The description of the WCS for the image. This field mirrors values in the FITS header * **ZeroPoint**: The effective photometric zero point of this stack of single images * **ImgSpatialFootprint**: This spatial area defines the spatial footprint of the image in a series of vertex * **ObservationId**: Image Observation Id * **TargetPointing**: Coordinates of target pointing * **DataStorage**: FITS file name containing the Stacked image FITS file * **PsfStorage**: Reference to the PSF files (Model and FITS) * **QualityParams** (optional):Data Quality Parameters * **Min**: Minimum image pixel value * **Max**: Maximum image pixel value * **Mean**: Average image level (mean of all image pixel values) * **Median**: Median image level (median of all image pixel values) * **StDev**: Sample standard deviation of the image pixel values * **MaskedPixelFraction**: The fraction of masked pixels The **DpdNirStackedFrame** data product contains the link to 3 files: * a :term:`MEF` file with the science image * a PSF model measured from science frames * a PSF image of the model Scientific image FITS file ^^^^^^^^^^^^^^^^^^^^^^^^^^ The main result of the stacking process is a :term:`MEF` file stored into the **DpdNirStackedFrame** in the ``Data.DataStorage`` field. The file is composed by four layers: ============ ================= =============== ======================== Layer name Data Format Units Content ============ ================= =============== ======================== **PRIMARY** Main headers **SCI** *float32* microJy Scientific data **RMS** *float32* microJy RMS data **DQ** *int32* Data Quality mask ============ ================= =============== ======================== .. _NIR_stacked_frame_image: .. figure:: ../figures/stacked_frame.png :scale: 100 % :align: center :figclass: align-center : Graphical representation of 3-layer structure of NIR stacked frames Primary layer """""""""""""" The primary layer is an header dedicated to store global infomation on pointings and parameters used by SWarp for resampling and stacking (i.e. starting from *COMBINET*, *RESAMPT1* up to *PSCALET2* in the next table) together with the inverse sensitivity value *PHOTFNU* needed to convert pixels in [microJy]. .. table:: :width: 100% +---------------------------------------------------------------------------------------------------------------------------------------+ | **HDU** *PRIMARY*: header | +==========+==========+=====================================================================================+===========================+ | | **Name** | **Description** | **Value** | +----------+----------+-------------------------------------------------------------------------------------+---------------------------+ | Keywords | FITS_DEF | FITS name definition | string = nir.stackedImage | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | FITS_VER | FITS version | string = 0.3 | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | TELESCOP | | string = Euclid | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | INSTRUME | or NISPsim for simulations | string = NISP | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | VERSION | Data Release version | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | DATE | UT date when this file was created | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | IMG_CAT | ImgType:Category DP category (SCIENCE, CALIB, ...) | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | IMG_T1 | ImgType:FirstType DP type (OBJ, SRD, DARK, ...) | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | IMG_T2 | ImgType:SecondType DP type (LAMP, SKY, ...) | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | OBSMODE | ImgType:ThirdType DP type (WIDE, DEEP, CALIBRATION) | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | OBSTYPE | ImgType:Technique DP technique (IMAGE, SPECTROIMAGE) | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | OBS_ID | eso:observationSequence:ObservationId Running Sequence number of the Observed block | integer | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | FILTER | Filter name, e.g. NIR_J | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | DATE_STA | DateRange:TimestampStart Observation Date first dither | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | DATE_END | DateRange:TimestampEnd Observation Date last dither | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | RA | TargetPointing:RA Center of field of view right ascension (deg) | double | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | DEC | TargetPointing:DEC Center of field of view right declination (deg) | double | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | PA | TargetPointing:Orientation Position angle (deg) | double | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | EQUINOX | Standard FK5 (years), e.g. 2000. | double | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | RADESYS | Coordinate reference frame, e.g. FK5 | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | EXPTIME | ExposureTime Effective integration time in seconds, e.g. 60.0005 | double | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | COMBINET | COMBINE_TYPE config parameter for SWarp | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | RESAMPT1 | RESAMPLING_TYPE config parameter | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | CENTERT1 | CENTER_TYPE config parameter | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | PSCALET1 | PIXELSCALE_TYPE config parameter | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | RESAMPT2 | RESAMPLING_TYPE config parameter | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | CENTERT2 | CENTER_TYPE config parameter | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | PSCALET2 | PIXELSCALE_TYPE config parameter | string | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | CAL_VER | version of calibration factors | double | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | WAVE_CAL | wavelength used for calibration microns | double | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | WAVELEN | central wavelength of bandpass microns | double | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | WAVEHPBW | half power bandwidth microns | double | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | PHOTFNU | inverse sensitivity Jy s / e | double | + +----------+-------------------------------------------------------------------------------------+---------------------------+ | | PHOTFNUE | error in PHOTNU Jy s / e | double | +----------+----------+-------------------------------------------------------------------------------------+---------------------------+ Scientific Layer """""""""""""""" The scientific layer, named **SCI**, stores the data obtained combining togheter all images used as input to the NIR Pipeline. The header contains some basic image definition keywords together with the *GAIN* and the *SATURATE* keywords derived by SWarp for resampling and stacking. .. table:: :width: 100% +---------------------------------------------------------------------------------------------------------------------------------------+ | **HDU** *DETn.SCI*: header | +==========+==========+=====================================================================================================+===========+ | | **Name** | **Description** | **Value** | +----------+----------+-----------------------------------------------------------------------------------------------------+-----------+ | Keywords | EXTNAME | e.g. DET11.SCI, meaning Detector number 0, Science data | string | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | DET_ID | Detector:DetectorId e.g. Detector ID, e.g. 11 | string | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | BITPIX | array data type | integer | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | NAXIS | e.g. number of array dimensions | integer | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | NAXIS1 | e.g. pixel numbers on first axis | integer | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | NAXIS2 | e.g. pixel numbers on second axis | integer | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | BUNIT | physical units of the array values | string | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | RPIX_PRC | Reference Pixel processing mode (0=no_px_correction,1=horiz_px_correction,2=vertical_px_correction) | integer | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | GAIN | Detector:Gain e- to ADU conversion factor, e.g. 1.5 | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | EQUINOX | Standard FK5 (years), e.g. 2000. | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | RADESYS | Coordinate reference frame, e.g. FK5 | string | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CRVAL1 | Right ascension at ref pixel | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CRVAL2 | Declination at ref pixel | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CRPIX1 | Reference pixel x coordinate | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CRPIX2 | Reference pixel y coordinate | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CTYPE1 | Coordinate 1 type | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CTYPE2 | Coordinate 2 type | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CUNIT1 | Axis 1 unit | string | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CUNIT2 | Axis 2 unit | string | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CD1_1 | Translation matrix element | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CD1_2 | Translation matrix element | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CD2_1 | Translation matrix element | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CD2_2 | Translation matrix element | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV1_0 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV1_1 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV1_2 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV1_4 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV1_5 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV1_6 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV1_7 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV1_8 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV1_9 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV1_10 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV2_0 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV2_1 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV2_2 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV2_4 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV2_5 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV2_6 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV2_7 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV2_8 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV2_9 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PV2_10 | Projection distortion parameter | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | ASTIRMS1 | Astrom. dispersion RMS (intern., high S/N) | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | ASTIRMS2 | Astrom. dispersion RMS (intern., high S/N) | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | ASTRRMS1 | Astrom. dispersion RMS (ref., high S/N) | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | ASTRRMS2 | Astrom. dispersion RMS (ref., high S/N) | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | ZPAB | zero point for conversion to AB magnitude | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | ZPABE | error in ZPAB determination | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | ZPVEGA | zero point for conversion to Vega magnitude | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | ZPVEGAE | error in ZPVEGA determination | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PHRELDT | Correction factor for relative photometric calibration for detector | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | PHRELDTE | Uncertainty in PHRELDT | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CRSNGALG | Algorithm used in finding cosmic rays | string | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | CRMULALG | Algorithm used in finding cosmic rays | string | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | DARKFILL | Dark fill value in e- | double | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | NBADPIXT | Number of bad pixels for detector | integer | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | NREJNL | Number of high noise pixels not corrected for non-linearity | integer | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | NSATPIX | Number of pixels flagged for saturation | integer | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | NDFILL | Number of high noise pixels with filled dark values | integer | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | NCRPIXS | Number of pixels masked as CR pixels in the single frame rejection | integer | + +----------+-----------------------------------------------------------------------------------------------------+-----------+ | | NCRPIXM | Number of pixels masked as CR pixels in the multi frame rejection | integer | +----------+----------+-----------------------------------------------------------------------------------------------------+-----------+ RMS Layer """"""""" The RMS layer is DQ Layer """""""" The DQ layer is PSF files description ^^^^^^^^^^^^^^^^^^^^^ The Point Spread Function (PSF) is measured from the image. It is provided into two formats: a `.psf` file containing model parameters and a `.fits` file with an instance of model. A PSF model is extracted from the scientific image by PSFEx software. This solution is provided both as **model** and as **image**. The PSF **model** is produced as *.psf* file, an Astromatic format that can be directly provided to SExtractor for catalogue extraction. The model can track wavelength and focal plane position dependencies. The PSF **image** is a FITS file with a primary and a single data layer containing a PSF image. PSF Model file """""""""""""" This is a model file PSF image file """""""""""""" The PSF primary header contains information on Observation Id in *OBS_ID*. Because of the FITS structure is in common with the one used on :ref:`NIRCalibratedFrame` definition some keywords are meaning-less in case of stacked frame. As in example *DITHOBS* could be ignored and *PSFTYPE* is fixed to *"Inflight"* value because the PSF is always extracted from images.