Interpret DQ flags
The reftools.interpretdq module contains functions for
interpreting DQ flags.
Examples
>>> from reftools.interpretdq import ImageDQ, DQParser
Create a class instance for HST/ACS image using pre-defined DQ definitions. Then, display the associated metadata and translation table:
>>> acsdq = ImageDQ.from_fits('j12345678q_flt.fits', ext=('DQ', 2))
>>> acsdq.parser.metadata
<Table length=1>
key val
str10 str3
---------- ----
INSTRUMENT ACS
>>> acsdq.parser.tab
<Table length=17>
DQFLAG ... LONG_DESCRIPTION
uint16 ... str74
------ ... ---------------------------------------------------------------
0 ... Good pixel
1 ... Lost during compression
2 ... Replaced by fill value
4 ... Bad detector pixel or beyond aperture or HRC upper-right defect
8 ... Masked by aperture feature or HRC occulting finger
16 ... Hot pixel
... ... ...
512 ... Bad pixel in reference file (FLAT, polarizer, or dust mote)
1024 ... Charge trap
2048 ... A-to-D saturated pixel
4096 ... Cosmic ray and detector artifact (AstroDrizzle, CR-SPLIT)
8192 ... Cosmic ray (ACSREJ)
16384 ... Manually flagged by user
32768 ... Not used
Interpret DQ value for a single pixel at IRAF-style coordinate
X=1457 and Y=170. Also display the original pixel value for comparison:
>>> acsdq.interpret_pixel(1457, 170)
<Table length=3>
DQFLAG SHORT_DESCRIPTION LONG_DESCRIPTION
uint16 str9 str74
------ ----------------- ----------------
16 HOT Hot pixel
32 CTE CTE tail
1024 TRAP Charge trap
>>> acsdq.data[169, 1456]
1072
This is the same as above, except pixel is given as 0-indexed values:
>>> acsdq.interpret_pixel(1456, 169, origin=0)
Interpret DQ values for all pixels. Then, extract mask for interpreted DQ value of 16 (hot pixel) and display pixel values for that mask:
>>> acsdq.interpret_all()
Parsing DQ flag(s)...
Done!
Run time: 2.822 s
N_FLAGGED: 550656/8388608 (6.564%)
FLAG=1 : 0 (0.000%)
FLAG=2 : 0 (0.000%)
FLAG=4 : 0 (0.000%)
FLAG=8 : 0 (0.000%)
FLAG=16 : 78901 (0.941%)
FLAG=32 : 78915 (0.941%)
FLAG=64 : 343917 (4.100%)
FLAG=128 : 20373 (0.243%)
FLAG=256 : 15202 (0.181%)
FLAG=512 : 26623 (0.317%)
FLAG=1024 : 5128 (0.061%)
FLAG=2048 : 15117 (0.180%)
FLAG=4096 : 0 (0.000%)
FLAG=8192 : 0 (0.000%)
FLAG=16384: 0 (0.000%)
FLAG=32768: 0 (0.000%)
>>> hotmask = acsdq.dq_mask(16)
>>> acsdq.data[hotmask]
array([528, 528, 528, ..., 560, 560, 560], dtype=int16)
If you are only interested in a subset of the data, e.g., the 100x100 corner,
you can also pass the subarray of interest into ImageDQ like this:
>>> from astropy.io import fits
>>> dqdata = fits.getdata('jbt7a3k7q_flc.fits', 3) # DQ, 1
>>> dqparser = DQParser.from_instrument('ACS')
>>> acsdq = ImageDQ(dqdata[:100, :100], dqparser=dqparser)
>>> acsdq.interpret_all()
Parsing DQ flag(s)...
Done!
Run time: 0.006 s
N_FLAGGED: 366/10000 (3.660%)
FLAG=1 : 0 (0.000%)
FLAG=2 : 0 (0.000%)
FLAG=4 : 0 (0.000%)
FLAG=8 : 0 (0.000%)
FLAG=16 : 111 (1.110%)
FLAG=32 : 21 (0.210%)
FLAG=64 : 115 (1.150%)
FLAG=128 : 0 (0.000%)
FLAG=256 : 0 (0.000%)
FLAG=512 : 100 (1.000%)
FLAG=1024 : 0 (0.000%)
FLAG=2048 : 0 (0.000%)
FLAG=4096 : 49 (0.490%)
FLAG=8192 : 0 (0.000%)
FLAG=16384: 0 (0.000%)
FLAG=32768: 0 (0.000%)
>>> acsdq.data.shape
(100, 100)
Create a class instance for HST/WFC3 DQ parser (without image). Then, interpret a given DQ value:
>>> wfc3dq = DQParser.from_instrument('WFC3')
>>> wfc3dq.interpret_dqval(16658)
<Table length=4>
DQFLAG SHORT_DESCRIPTION LONG_DESCRIPTION
uint16 str9 str69
------ ----------------- ------------------------------------------------
2 FILLED Replaced by fill value
16 HOT Hot pixel
256 SATURATED Full-well or A-to-D saturated pixel
16384 CRMAX Pixel has more than max CRs, ghost, or crosstalk
- class reftools.interpretdq.DQParser(definition_file)
Class to handle parsing of DQ flags.
Definition Table
A “definition table” is an ASCII table that defines each DQ flag and its short and long descriptions. It can have optional comment line(s) for metadata, e.g.:
# INSTRUMENT = HSTGENERICIt must have three columns:
DQFLAGcontains the flag value (uint16).SHORT_DESCRIPTION(string).LONG_DESCRIPTION(string).
Example file contents:
# INSTRUMENT = HSTGENERIC DQFLAG SHORT_DESCRIPTION LONG_DESCRIPTION 0 "OK" "Good pixel" 1 "LOST" "Lost during compression" ... ... ...
The table format must be readable by
astropy.io.ascii.- Parameters:
- definition_filestr
ASCII table that defines the DQ flags (see above).
- Attributes:
- tab
astropy.table.Table Table object from given definition file.
- metadata
astropy.table.Table Table object from file metadata.
- tab
Methods
from_instrument(instrument)Use pre-defined DQ flags for the given instrument (HST only for now).
interpret_array(data[, verbose])Interpret DQ values for an array.
interpret_dqval(dqval)Interpret DQ values for a single pixel.
- classmethod from_instrument(instrument)
Use pre-defined DQ flags for the given instrument (HST only for now). The data files are distributed with this module.
- Parameters:
- instrument{‘ACS’, ‘WFC3’}
Currently not all instruments are supported. If the instrument is not found, a generic definition is used.
- Returns:
- cls
DQParser An instance of this class.
- cls
- interpret_array(data, verbose=True)
Interpret DQ values for an array.
Warning
If the array is large and has a lot of flagged elements, this can be resource intensive.
- Parameters:
- datandarray
DQ values.
- verbosebool
Print info to screen.
- Returns:
- dqs_by_flagdict
Dictionary mapping each interpreted DQ value to indices of affected array elements.
- interpret_dqval(dqval)
Interpret DQ values for a single pixel.
- Parameters:
- dqvalint
DQ value.
- Returns:
- dqs
astropy.table.Table Table object containing a list of interpreted DQ values and their meanings.
- dqs
- class reftools.interpretdq.ImageDQ(data, dqparser=None)
Class to handle DQ flags in an image.
- Parameters:
- Raises:
- ValueError
Invalid image dimension.
- Attributes:
- datandarray
Same as input.
- parser
DQParser DQ parser for interpretation.
Methods
dq_mask(dqval)Generate mask for the given interpreted DQ value.
from_fits(filename[, ext, inskey])Use data from given FITS image and create DQ parser based on header value.
interpret_all([verbose])Interpret DQ values for all flagged pixels.
interpret_pixel(x, y[, origin])Construct a user-friendly table object with interpreted DQ values for a given pixel location.
pixlist([origin])Convert cached results to a list of
(X, Y)pixel coordinates.- dq_mask(dqval)
Generate mask for the given interpreted DQ value.
- Parameters:
- dqvalint
Interpreted DQ value.
- Returns:
- maskndarray
Boolean mask where the affected pixels are marked
True.
- Raises:
- ValueError
Missing interpreter result or invalid DQ value.
- classmethod from_fits(filename, ext=('DQ', 1), inskey='INSTRUME')
Use data from given FITS image and create DQ parser based on header value.
- Parameters:
- filenamestr
Image filename.
- extstr or int or tuple
Extension identifier, as accepted by
astropy.io.fits.- inskeystr
Keyword value in
PRIMARYheader that defines the instrument.
- Returns:
- cls
ImageDQ An instance of this class.
- cls
- interpret_all(verbose=True)
Interpret DQ values for all flagged pixels. Results are stored in
self._dqs_by_flaginternal cache.Warning
If the image is large and has a lot of flagged pixels, this can be resource intensive.
- Parameters:
- verbosebool
Print info to screen.
- interpret_pixel(x, y, origin=1)
Construct a user-friendly table object with interpreted DQ values for a given pixel location.
- Parameters:
- x, yint
X and Y pixel coordinates. Also see
origin.- origin{0, 1}
Indexing system for the pixel values given. Use 0 for Python 0-indexed system and 1 for IRAF 1-indexed system.
- Returns:
- dqs
astropy.table.Table Table object containing a list of interpreted DQ values and their meanings.
- dqs
- pixlist(origin=1)
Convert cached results to a list of
(X, Y)pixel coordinates.- Parameters:
- origin{0, 1}
Indexing system for the pixel values returned. Use 0 for Python 0-indexed system and 1 for IRAF 1-indexed system.
- Returns:
- pixlist_by_flagdict
Dictionary mapping each interpreted DQ value to a list of coordinates.