.. _VisAnalysisResults: ============================ Vis Analysis Results Product ============================ Metadata ______________________ +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Data product name | DpdVisAnalysisResults | +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Data product custodian | VIS | +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Name of the Schema file | .. raw:: html | | | | | | `euc-vis-AnalysisResults.xsd `_ | +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Processing Element(s) creating/using the data | VIS_science_xml_out | | product | | +------------------------------------------------+----------------------------------------------------------------------------------------------+ | Processing function using the data product | VIS | +------------------------------------------------+----------------------------------------------------------------------------------------------+ Proposed for inclusion in EAS/SAS ========================================== .. PFUsingStart Yes .. PFUsingEnd Data product elements ===================== +--------------+----------------------------------------------+ | Header | object of type sys:genericHeader | +--------------+----------------------------------------------+ | Data | object of type vis:visAnalysisResults | +--------------+----------------------------------------------+ | QualityFlags | object of type dqc:sqfPlaceHolder | +--------------+----------------------------------------------+ | Parameters | object of type ppr:genericKeyValueParameters | +--------------+----------------------------------------------+ Detailed description of the data product ======================================== .. DetailedDescStart The DpdVisAnalysisResults is a general product that can be filled by different pipelines. It is designed to store data quality control during the production of product. VISProcessField DQC description ------------------------------- The analysis results is a .json file combining all the data quality control created during the processing of the calibrated(Quad)Frame exposure. This is very useful for determining which data is really usable for science and which not. **Name of file:** ``EUC_VIS_QC_PERI-[ObsId]-[Dither]-[exposure]-0000000__[YYYYMMDDThhmmss.s]Z.json`` where the date is the production date. Cosmic Rays *********** To check if the data is contaminated by X-rays, we compute the ratio of cosmic rays detection by `LA Cosmicx `_. For the moment, we have determined that data with a number of cosmic rays above 5.0% for a nominal exposition is data that cannot be used. To normalise to the exposure time, we also need to add the readout time (70 s) to take into account the fact that cosmics rays can also arrive during this time. In fact, this type of data can be used with caution due to the fact that the X-rays only affect some quadrants and not all the FPA. Above this threshold, a lot of cosmic rays can be missed by LA Cosmicx. In PF-13 this DQC was saved in ratio. In PF-14 and above it was saved in percent. We save the ratio of pixel with the flag COSMIC per detector in **quadrant.cosmic_pixel_percent**. We calculate the maximum and the mean of this ratio per detector and store it in these 2 indicators : **FPA.max_cosmic_percent:** This corresponds to the values **quadrant.cosmic_pixel_percent** of the quadrant with the maximum percentage of pixels with the COSMIC flag in the FPA. If this indicator is high it may indicate an X-Rays pattern. **FPA.cosmic_pixel_percent:** This corresponds to the mean percentage of pixels with the COSMIC flag in the FPA. This may indicate a proton shower or a realy high event on one or a few quadrant. We also used the flags CR_REGION (added in PF 14.0) to indicate if the quadrant is above a threshold defined in the config file of the VIS-PF. In PF-14 the thredsold was 3.0%. It was increased up to 5.0% in PF-15. The number of quadrants masked by CR_REGION is stored in **FPA.cr_regions_count**. RA/DEC residuals **************** The requirement R-VIS-CAL-F-010 is defined as 'astrometric residual :math:`< 3` mas'. The mean residual for each exposure is saved in **vis_res_mean**. The RA/DEC mean and std is stored individualy in: **ra_res_final_mean** ; **ra_res_final_sdr** ; **dec_res_final_mean** ; **dec_res_final_sdr**. Spread model ************ **FPA.spreadmodel_peak:** It is the peak of the spread model corresponding to the star in the flux_auto vs spread_model for :math:`1.5 <` log10( FLUX_AUTO/FLUXERR_AUTO) :math:`< 2.5`. It is a good indicator of the quality of the guidance. We have estimated that spread model peak above 0.002 corresponds to data with guiding errors. The **FPA.spreadmodel_sigma** is standard deviation of the gaussian fit of the histogram of SExtractor spread model values. Moment ****** This DQC was added in PF-14. During the data processing we calculate the median and the standard deviation of e1, e2, r2 of the stars based on Konrad Kuijken methode. PF-14.0.4 add the calculation of t1, t2, c1, c2. All this DQC are saved in the format **FPA.moment_{r2/e1/e2/t1/t2/c1/c2}_{med/std}** From this measure, we can derive a score with : .. math:: & \frac{(e2_{med}-e2m)^2}{vare2} + (r2_{med}-r2m)^2*covinv[0,0]+ \\ & 2*(r2_{med}-r2m)*(e1_{med}-e1m)*covinv[1,0]+(e1_{med}-e1m)^2*covinv[1,1] Where:: r2m=1.9233812949640285 e1m=-0.023129496402877696 e2m=0.002733812949640288 cov00=0.0005422666041080192 cov01=-0.00011841403399019925 cov11=4.4881659889479756e-05 vare2=5.332022152062523e-06 covinv=np.linalg.inv([[cov00,cov01],[cov01,cov11]]) calculated from a reference field. By statistical analysis, we defined that a score higher than 60 is bad. Zero point ********** For quality control we calculate a zero point by comparing with Gaia. **FPA.zero_point:** VIS zero point calculated by comparison with Gaia data. **FPA.zp_rms:** Standard deviation of the VIS zero point. A zero point for each quadrant is also available in **{quadrant}.zp_mean** and **{quadrant}.zp_rms**. .. Check straylight .. **************** .. .. To check if the data are affected by the straylight, you can compute the standard deviation of the mean background in the FPA. .. This is a good proxy but notice that a high standard deviation can be caused by a large object in the data. .. .. Data is rejected if the standard deviation of the background in second is above 0.004. .. .. The mean background in ADU of each CCD is reported in **ccd.avg_bkg**. .. .. DetailedDescEnd