Using aoquality
¶
The aoquality
program is a command-line tool to query or modify the quality statistics in the table. It is more or less a command-line counterpart to aoqplot.
What quality statistics are¶
Quality statistics are optional tables in a measurement set that describe statistics of the visibility in the measurement set. These can be computed from the visibilities and are therefore redundant, but they are much smaller in size, and therefore alleviate the need to read an entire set just to get some first idea of the quality of the visibilities. They can be calculated during flagging (using the lua collect_statistics()
function), which avoids reading the data again just for getting the statistics.
Note
If the visibilities are changed, the quality statistics become out of date. This can be desirable in some cases, e.g. to keep the statistics of the raw, high-resolution data before averaging.
(Re)collecting statistics¶
To collect statistics, the following command can be used:
aoquality collect [-d <column name>] observation.ms
This will read all the visibilities and add the quality statistics tables to the measurement set afterwards. If quality tables already exist, they are overwritten. If no column name is specified, the DATA
column is used.
Extracting and combining statistics¶
Normally, statistics are written inside a measurement set. The aoquality
tool allows the extraction of statistics and to write these into separate (small) files, so that it becomes possible to store statistics separately from the measurement set. The stored statistics have the form of an empty measurement set with quality tables and just enough metadata to interpret the statistics. The aoquality combine
option can be used to create such separate statistic sets. The recommended extension for extracted statistics is .qs
. For example, to extract statistics from a measurement set and write them in a separate, use:
aoquality combine mystatistics.qs input.ms
This would create mystatistics.qs
and fill it with the quality statistics from input.ms
. It is also possible to specify multiple measurement sets. In this case, the statistics are combined and then written to the output file, for example:
aoquality combine mystatistics.qs \
observation1.ms observation2.ms observation3.ms
This would combine the statistics from the three observations and write these to mystatistics.qs
. For quality statistics operations, the .qs
statistics can be used any time that a .ms
can be used, so aoquality query
commands can be applied to it, they can be further combined with aoquality combine
, and aoqplot can be used to view the contents of the .qs
statistics.
Querying statistics¶
There are several dimensions over which statistics, such as the standard deviation, are stored and can be queried: over time (integrated over all other dimensions), frequency or baseline. To query these, the aoquality
tool accepts these commands:
aoquality query_b
: Query the statistics per baseline (integrated over all other dimensions)aoquality query_f
: Query the statistics per channel (integrated over all other dimensions)aoquality query_t
: Query the statistics over time (integrated over all other dimensions)aoquality query_g
: Query a statistic integrated over all dimensions
Each of these commands uses the same syntax. We will use query_f
as an example. To query the standard deviation per channel, one would issue this command:
aoquality query_f StandardDeviation observation.ms
As shown, after query_f
the type of statistic is given, followed by the name of the measurement set. A few other common statistics are Mean
, Variance
, Count
(=nr of visibilities), RFIRatio
and DStandardDeviation
(=frequency-differenced standard deviation).
Here is an example output:
This shows the frequency in the first column, the standard deviation of the real values of XX in the second column, followed by imaginary, then real of XY, etc.
Other options¶
The aoquality
command has a few more options. These can be queried by running aoquality
without parameters.