Functions

This section lists all the Lua functions provided by AOFlagger.

Summary

The aoflagger module provides the following functions:

Detailed descriptions

aoflagger.apply_bandpass(data, filename)

Apply a bandpass file to the data. The data is changed in place. Each line in the file contains <antenna name> <X/Y polarization> <channel index> <gain>, separated by spaces, for example:

RT2 X 0 0.7022
RT2 X 1 0.7371
RT2 X 2 0.8092
Parameters:
  • data (Data) – Data to which the bandpass is applied.

  • filename (string) – Path to bandpass textfile.

aoflagger.collect_statistics(after_data, before_data)

Calculate statistics, such as visibility standard deviation and flag percentages. When running the strategy on a measurement set, the statistics are stored inside the measurement set after finishing all baselines. These can be inspected by the aoqplot tool.

The function takes the data after and before flagging. Any data that are flagged in before_data will not contribute to the statistics. This avoids counting e.g. correlator faults or shadowing as interference.

Parameters:
  • after_data (Data) – Flagged data.

  • before_data (Data) – Unflagged data.

aoflagger.copy_to_channel(destination_data, source_data, channel)

Copy the data (visibilities & flags) from the source data to the destination data with a specified channel offset. This function can be used together with trim_channels() to flag a subset of the data and copy the result back to the full data.

In that scenario, the start channel for trim_channels() equals the channel parameter in this call.

When the source does not fit into the destination at the specified offset, only the part that fits is copied.

Available since v3.1.

Parameters:
  • destination_data (Data) – Destination data

  • source_data (Data) – Data to be copied. These data are unchanged.

  • channel – A channel index that specifies the offset to which the data is copied in the destination data.

aoflagger.copy_to_frequency(destination_data, source_data, frequency)

This function is similar to copy_to_channel(), but instead of specifying the target channel offset as an index, it is specified as a frequency. This can be used as counterpart to trim_frequencies().

See copy_to_channel() for further info.

Available since v3.1.

Parameters:
  • destination_data (Data) – Destination data

  • source_data (Data) – Data to be copied. These data are unchanged.

  • channel – A frequency in MHz that specifies the offset to which the data is copied in the destination data.

aoflagger.downsample(data, xfactor, yfactor, masked)

Decrease the resolution of the data using simple linear binning. This can be effective to increase the speed of data smoothing, for example when using high_pass_filter(). At the function end of execute(), the data should have the original size. Therefore, a call to downsample should normally be followed by a call to upsample_image() or upsample_mask() to restore the visibilities and flags, respectively, to their original resolution.

When the input data is not exactly divisable by the downsampling factors, fewer samples will be averaged into the last bins.

Parameters:
  • data (Data) – Input data (not modified).

  • xfactor (integer) – Downsampling factor in time direction.

  • yfactor (integer) – Downsampling factor in frequency direction.

  • masked (boolean) – true means take flags into account during averaging

Returns:

Downsampled version of input data.

Return type:

Data

aoflagger.high_pass_filter(data, xsize, ysize, xsigma, ysigma)

Apply a Gaussian high-pass filter to the data. This removes the diffuse ‘background’ in the data. With appropriate settings, it can filter the signal of interest (slow sinusoidal signals), making the interference easier to detect.

The function convolves the data with a 2D “1 minus Gaussian” kernel. The kernel is clipped at the edges. The sigma parameters define the strength (band-limit) of the filter: lower values remove more of the diffuse structure.

Parameters:
  • data (Data) – The data (modified in place).

  • xsize (integer) – Kernel size in time direction

  • ysize (integer) – Kernel size in frequency direction

  • xsigma (number) – Gaussian width in time direction.

  • ysigma (number) – Gaussian width in frequency direction.

aoflagger.low_pass_filter(data, xsize, ysize, xsigma, ysigma)

Apply a Gaussian low-pass filter to the data. It convolves the data with a Gaussian. See high_pass_filter() for further details.

Parameters:
  • data (Data) – The data (modified in place).

  • xsize (integer) – Kernel size in time direction

  • ysize (integer) – Kernel size in frequency direction

  • xsigma (number) – Gaussian width in time direction.

  • ysigma (number) – Gaussian width in frequency direction.

aoflagger.norm(input_data)

Creates a new Data object for which the data values have been replaced by their norm. If the data is complex, a complex norm is performed. In all other cases, a real-valued norm is performed (i.e., the data is squared).

Parameters:

input_data (Data) – Data object (not modified)

Returns:

Norm of input_data, element-wise applied.

Return type:

Data

aoflagger.normalize_bandpass(data)

Normalizes the RMS over frequency. If multiple polarizations are present in the data, the RMS over the combination of all polarizations is calculated and normalized.

Available since v3.1.

Parameters:

data (Data) – The data (modified in place).

aoflagger.normalize_subbands(data, nr_subbands)

Remove jumps between subbands. A subband is in this context a number of adjacent channels, equally spaced over the bandwidth. This function therefore assumes that all subbands have an equal number of channels.

Each subband is scaled such that the standard deviation of the visibilities in a subband is unity. To avoid influence from interference, a stable method is used to estimate the standard deviation (Winsorized standard deviation).

A typical use-case for this function is the MWA phase 1 and 2. The 30 MHz bandwidth of the MWA is split in 24 ‘course channels’, each consisting of 128 channels. Each course channel has an independent gain, and needs normalization before it can be compared with adjacent course channels.

Parameters:
  • data (Data) – The data (modified in place).

  • nr_subbands (integer) – Number of subbands.

aoflagger.print_polarization_statistics(data)

Print RFI percentages per polarization to the command line.

Parameters:

data (Data) – Input data.

aoflagger.require_max_version(version)

Checks if the aoflagger version is lower or equal to the provided version. If the condition is not met, an error is thrown. This function can be used when it is known a strategy is making use of Lua functionality that was changed in newer aoflagger versions.

The version string can be of the form “major”, “major.min” or “major.minor.subminor”. The version is only checked up to the level that is specified: requiring at most version “3.2” will allow version “3.2.1”, but not version “3.3.0” or “4.0”. To disallow version “3.2.1”, a maximum version of “3.2.0” should be specified.

Available since v3.1.

See also require_min_version().

Parameters:

version (string) – Latest version that is allowed, e.g. "3.0.4".

aoflagger.require_min_version(version)

Checks if the aoflagger version is equal to or newer than the provided version string. If the condition is not met, an error is thrown. This is a useful way of notifying users that their version of aoflagger is too old. A version of aoflagger should (only) be considered too old when the strategy requires a function, method or other functionality that is not available in versions before the specified version.

The version string can be of the form “major”, “major.min” or “major.minor.subminor”. The version is only checked up to the level that is specified: requiring version “3.2” will allow versions such as “3.2-alpha” and “3.2.1”.

Available since v3.1.

Parameters:

version (string) – Minimum version that is allowed, e.g. "3.0.4".

aoflagger.save_heat_map(filename, data)

Save the data as a “heat map” image. The type is determined from the extension. Supported extensions are .svg, .png and .pdf.

Parameters:
  • filename (string) – Path to image to be written.

  • data (Data) – Input data.

aoflagger.scale_invariant_rank_operator(data, xlevel, ylevel)

Extend flags in time and frequency direction in a scale-invariant manner. This fills holes in the flag mask and makes flag sequences longer. Details are described in Offringa et al. 2012.

Parameters:
  • data (Data) – The data (modified in place).

  • xlevel (number) – aggressiveness in time-direction

  • ylevel (number) – aggressiveness in frequency-direction

aoflagger.scale_invariant_rank_operator_masked(data, mask_data, xlevel, ylevel, penalty)

Perform the same operation as scale_invariant_rank_operator(), but with an input mask that identifies invalid data. Invalid data is treated differently, and the penalty parameter selects how it is treated. With a penalty of 0, it is as if invalid samples are removed before applying the operator. With a penalty of 1, invalid samples are counted in the same way as unflagged samples (i.e., they penalize their extension). A typical penalty value is 0.1. For backwards compatibility, penalty may be left out, in which case a value of 0.1 is used.

Available since v3.1. The penalty parameter is available since v3.2.

Parameters:
  • data (Data) – The data (modified in place).

  • mask_data (Data) – The data that is used as mask.

  • xlevel (number) – aggressiveness in time-direction

  • ylevel (number) – aggressiveness in frequency-direction

  • penalty (number) – penalty given to the extension through invalid regions.

aoflagger.set_progress(progress, max_progress)

Notify user of the progress of this call. The gui uses this information to show a progress bar to the user. Example: when the execute() function iterates over the polarizations, progress can be reported by calling aoflagger.set_progress(curpol, npol) inside the loop.

Parameters:
  • progress (integer) – current progress

  • max_progress (integer) – value of progress when complete

aoflagger.set_progress_text(task_description)

Notify user of the current task being done. The description can be anything, and can literally be presented to the user.

Parameters:

task_description (string) – Description string.

aoflagger.sqrt(input_data)

Creates a new Data object where all data values are replaced by their square root. If the data is complex, a complex square root is performed. In all other cases, a real-valued square root is performed.

Parameters:

input_data (Data) – Data object (not modified)

Returns:

Square root of input_data, element-wise applied.

Return type:

Data

aoflagger.sumthreshold(data, x_threshold_factor, y_threshold_factor, x_direction, y_direction)

Run the SumThreshold algorithm on the data. This algorithm detects sharp, line-shaped features in the time-frequency domain that are typical for RFI. See Offringa et al. (2010) for details about the algorithm.

The thresholds are relative to a (stable) estimate of the noise in the visibilities. They define the base sensitivity of the algorithm. Lower values will detect more features. A reasonable value for the thresholds is 1.

The x_direction/y_direction parameters turn detection in their particular directions on and off. If a direction is turned off, the threshold factor for that direction is ignored. Note that detection in x-direction (which is the time-direction) means detection of contiguous high-power samples in time, such as transmitters that occupy the same channel continuously. The y-direction detection is sensitive to transient, broadband RFI.

Parameters:
  • data (Data) – The data (modified in place)

  • x_threshold_factor (number) – Threshold factor in time direction

  • y_threshold_factor (number) – Threshold factor in frequency direction

  • x_direction (boolean) – Enable flagging in time direction

  • y_direction (boolean) – Enable flagging in frequency direction

aoflagger.sumthreshold_masked(data, mask_data, x_threshold_factor, y_threshold_factor, x_direction, y_direction)

Same as sumthreshold(), but with a mask. Visibilities that are flagged in the mask are considered to be visibilities that have not been sampled and are removed from the SumThreshold operation. A typical case for this is to make sure that correlator faults, shadowing and band-edges are correctly treated.

Parameters:
  • data (Data) – The data (modified in place).

  • mask_data (Data) – The data that is used as mask

  • x_threshold_factor (number) – Threshold factor in time direction

  • y_threshold_factor (number) – Threshold factor in frequency direction

  • x_direction (boolean) – Enable flagging in time direction

  • y_direction (boolean) – Enable flagging in frequency direction

aoflagger.threshold_channel_rms(data, threshold, flag_low_outliers)

Calculate the root-mean-square (RMS) for each channel and flags channels that have an outlier RMS. The threshold is a “sigma level”. Typical values for the threshold are therefore around 3.

Parameters:
  • data (Data) – The data (modified in place).

  • threshold (number) – Sigma-level of threshold.

  • flag_low_outliers (boolean) – Flag channels with low RMS.

aoflagger.threshold_timestep_rms(data, threshold)

Like threshold_channel_rms(), but thresholds timesteps with outlier RMS. Both timesteps with high and low RMS values are flagged.

Parameters:
  • data (Data) – The data (modified in place).

  • threshold (number) – Sigma-level of threshold.

aoflagger.trim_channels(data, start_channel, end_channel)

Create a new data object from a subset of the input data. This can be used to flag a subset of the data, together with copy_to_channel() to copy the result back. All channels for which start_channel <= channel index < end_channel are copied into the result. All timesteps are copied.

Available since v3.1.

Parameters:
  • data (Data) – Input data (unchanged).

  • start_channel (integer) – Index of first channel

  • end_channel (integer) – Index of end of the channel range. The end range is excluding.

Returns:

A new data object, trimmed as specified.

Return type:

Data

aoflagger.trim_frequencies(data, start_frequency, end_frequency)

This function is equal to trim_channels(), except that the channel range is specified with frequency values. All channels for which start_frequency <= channel frequency < end_frequency are copied into the result. All timesteps are copied.

copy_to_frequency() can be used to copy the result back after processing.

Available since v3.1.

Parameters:
  • data (Data) – Input data (unchanged).

  • start_frequency (number) – Start frequency in MHz of the selected range.

  • end_frequency (number) – End frequency in MHz of the channel range.

Returns:

A new data object, trimmed as specified.

Return type:

Data

aoflagger.upsample_image(input_data, destination_data, xfactor, yfactor)

Increase the resolution of the data. This function is to restore the resolution of the data after having called downsample(). input_data is normally the data that was returned by downsample(), and destination_data is the input object that was specified as parameter. The upsampling is done by nearest neighbour interpolation.

The x and y factors should be the equal to the values specified in the call to downsample. The size of the destination_data is not changed: the input data is stretched by the given factors, and trimmed to the destination size in case the image dimensions were not exactly divisable by the factors.

The function only upsamples the visibilities, not the flags. To upsample the flags, see upsample_mask().

Parameters:
  • input_data (Data) – Input low-resolution data (not modified).

  • destination_data (Data) – Where the result will be stored.

  • xfactor (integer) – Upsampling factor in time direction.

  • yfactor (integer) – Upsampling factor in frequency direction.

aoflagger.upsample_mask(input_data, destination_data, xfactor, yfactor)

Increase the resolution of the mask. It is identical to upsample_image(), but works with the mask (flags) instead of the image (visibilities).

Parameters:
  • input_data (Data) – Input low-resolution data (not modified).

  • destination_data (Data) – Where the result will be stored.

  • xfactor (integer) – Upsampling factor in time direction.

  • yfactor (integer) – Upsampling factor in frequency direction.

aoflagger.visualize(data, label, sorting_index)

Save a visualization of the data for inspection in rfigui. When this strategy runs outside of the rfigui, the call is ignored. Can be used to e.g. inspect partial results.

Parameters:
  • data (Data) – Input data (not modified).

  • label (string) – A short description that is displayed to the user.

  • sorting_index – Where to place this visualization in the list of visualization