Functions ========= This section lists all the Lua functions provided by AOFlagger. .. default-domain:: lua Summary ^^^^^^^ .. module:: aoflagger The ``aoflagger`` module provides the following functions: * Reporting and user interface: - :meth:`print_polarization_statistics` - :meth:`save_heat_map` - :meth:`set_progress` - :meth:`set_progress_text` - :meth:`visualize` * Scaling: - :meth:`apply_bandpass` - :meth:`normalize_subbands` * Filtering and resolution: - :meth:`downsample` - :meth:`high_pass_filter` - :meth:`low_pass_filter` - :meth:`upsample` * Thresholding: - :meth:`sumthreshold` - :meth:`sumthreshold_masked` - :meth:`threshold_channel_rms` - :meth:`threshold_timestep_rms` * Morphological: - :meth:`scale_invariant_rank_operator` - :meth:`scale_invariant_rank_operator_masked` * Other: - :meth:`collect_statistics` Detailed descriptions ^^^^^^^^^^^^^^^^^^^^^ .. function:: apply_bandpass(data, filename) Apply a bandpass file to the data. The data is changed in place. Each line in the file contains , separated by spaces, for example: | RT2 X 0 0.7022 | RT2 X 1 0.7371 | RT2 X 2 0.8092 | ... :param data: Data to which the bandpass is applied. :type data: :class:`Data` :param filename: Path to bandpass textfile. :type filename: string .. function:: collect_statistics(after_data, before_data) Calculate statistics, such as visibility standard deviation and flag percentages. When running the strategy on a measurement set, the statistics are stored inside the measurement set after finishing all baselines. These can be inspected by the ``aoqplot`` tool. The function takes the data after and before flagging. Any data that are flagged in ``before_data`` will not contribute to the statistics. This avoids counting e.g. correlator faults or shadowing as interference. :param after_data: Flagged data. :type after_data: :class:`Data` :param before_data: Unflagged data. :type before_data: :class:`Data` .. function:: downsample(data, xfactor, yfactor, masked) Decrease the resolution of the data using simple linear binning. This can be effective to increase the speed of data smoothing, for example when using :meth:`high_pass_filter`. At the function end of :meth:`execute`, the data should have the original size. Therefore, a call to downsample should normally be followed by a call to :meth:`upsample` to restore the flags and visibilities to their original resolution. When the input data is not exactly divisable by the downsampling factors, fewer samples will be averaged into the last bins. :param data: Input data (not modified). :type data: :class:`Data` :param xfactor: Downsampling factor in time direction. :type xfactor: integer :param yfactor: Downsampling factor in frequency direction. :type yfactor: integer :param masked: ``true`` means take flags into account during averaging :type masked: boolean :return: Downsampled version of input data. :rtype: :class:`Data` .. function:: high_pass_filter(data, xsize, ysize, xsigma, ysigma) Apply a Gaussian high-pass filter to the data. This removes the diffuse 'background' in the data. With appropriate settings, it can filter the signal of interest (slow sinusoidal signals), making the interference easier to detect. The function convolves the data with a 2D "1 minus Gaussian" kernel. The kernel is clipped at the edges. The sigma parameters define the strength (band-limit) of the filter: lower values remove more of the diffuse structure. :param data: The data (modified in place). :type data: :class:`Data` :param xsize: Kernel size in time direction :type xsize: integer :param ysize: Kernel size in frequency direction :type ysize: integer :param xsigma: Gaussian width in time direction. :type xsigma: number :param ysigma: Gaussian width in frequency direction. :type ysigma: number .. function:: low_pass_filter(data, xsize, ysize, xsigma, ysigma) Apply a Gaussian low-pass filter to the data. It convolves the data with a Gaussian. See :meth:`high_pass_filter` for further details. :param data: The data (modified in place). :type data: :class:`Data` :param xsize: Kernel size in time direction :type xsize: integer :param ysize: Kernel size in frequency direction :type ysize: integer :param xsigma: Gaussian width in time direction. :type xsigma: number :param ysigma: Gaussian width in frequency direction. :type ysigma: number .. function:: normalize_subbands(data, nr_subbands) Remove jumps between subbands. A subband is in this context a number of adjacent channels, equally spaced over the bandwidth. This function therefore assumes that all subbands have an equal number of channels. Each subband is scaled such that the standard deviation of the visibilities in a subband is unity. To avoid influence from interference, a stable method is used to estimate the standard deviation (Winsorized standard deviation). A typical use-case for this function is the MWA phase 1 and 2. The 30 MHz bandwidth of the MWA is split in 24 'course channels', each consisting of 128 channels. Each course channel has an independent gain, and needs normalization before it can be compared with adjacent course channels. :param data: The data (modified in place). :type data: :class:`Data` :param nr_subbands: Number of subbands. :type nr_subbands: integer .. function:: print_polarization_statistics(data) :deprecated: Print RFI percentages per polarization to the command line. :param data: Input data. :type data: :class:`Data` .. function:: save_heat_map(filename, data) Save the data as a "heat map" image. The type is determined from the extension. Supported extensions are ``.svg``, ``.png`` and ``.pdf``. :param filename: Path to image to be written. :type filename: string :param data: Input data. :type data: :class:`Data` .. function:: scale_invariant_rank_operator(data, xlevel, ylevel) Extend flags in time and frequency direction in a scale-invariant manner. This fills holes in the flag mask and makes flag sequences longer. Details are described in `Offringa et al. 2012 `_. :param data: The data (modified in place). :type data: :class:`Data` :param xlevel: aggressiveness in time-direction :type xlevel: number :param ylevel: aggressiveness in frequency-direction :type ylevel: number .. function:: scale_invariant_rank_operator_masked(data, mask_data, xlevel, ylevel) Perform the same operation as :meth:`scale_invariant_rank_operator`, but with a mask. Data that is flagged in the mask are removed before applying the operator. :param data: The data (modified in place). :type data: :class:`Data` :param mask_data: The data that is used as mask. :type mask_data: :class:`Data` :param xlevel: aggressiveness in time-direction :type xlevel: number :param ylevel: aggressiveness in frequency-direction :type ylevel: number .. function:: set_progress(progress, max_progress) Notify user of the progress of this call. The gui uses this information to show a progress bar to the user. Example: when the :meth:`execute` function iterates over the polarizations, progress can be reported by calling ``aoflagger.set_progress(curpol, npol)`` inside the loop. :param progress: current progress :type progress: integer :param max_progress: value of progress when complete :type max_progress: integer .. function:: set_progress_text(task_description) Notify user of the current task being done. The description can be anything, and can literally be presented to the user. :param task_description: Description string. :type task_description: string .. function:: sumthreshold(data, xthreshold_f, ythreshold_f, include_x, include_y) Run the SumThreshold algorithm on the data. This algorithm detects sharp, line-shaped features in the time-frequency domain that are typical for RFI. See `Offringa et al. (2010) `_ for details about the algorithm. The thresholds are relative to a (stable) estimate of the noise in the visibilities. They define the base sensitivity of the algorithm. Lower values will detect more features. A reasonable value for the thresholds is 1. The "include" parameters turn detection in their particular directions on and off. Note that detection in "x" (=time) direction means detection of contiguous high-power samples in time, such as transmitters that occupy the same channel continuously. The y-direction detection is sensitive to transient, broadband RFI. :param data: The data (modified in place). :type data: :class:`Data` :param xthreshold_f: Threshold factor in time direction :type xthreshold_f: number :param ythreshold_f: Threshold factor in frequency direction :type ythreshold_f: number :param include_x: The data that is used as mask. :type include_x: :class:`Data` :param include_y: The data that is used as mask. :type include_y: :class:`Data` .. function:: sumthreshold_masked(data, mask_data, x_threshold_f, y_threshold_f, x_direction, y_direction) Same as :meth:`sumthreshold`, but with a mask. Visibilities that are flagged in the mask are considered to be visibilities that have not been sampled and are removed from the SumThreshold operation. A typical case for this is to make sure that correlator faults, shadowing and band-edges are correctly treated. :param data: The data (modified in place). :type data: :class:`Data` :param mask_data: The data that is used as mask. :type mask_data: :class:`Data` :param x_threshold_f: Threshold factor in time direction :type x_threshold_f: number :param y_threshold_f: Threshold factor in frequency direction :type y_threshold_f: number :param x_direction: The data that is used as mask. :type x_direction: boolean :param y_direction: The data that is used as mask. :type y_direction: boolean .. function:: threshold_channel_rms(data, threshold, flag_low_outliers) Calculate the root-mean-square (RMS) for each channel and flags channels that have an outlier RMS. The threshold is a "sigma level". Typical values for the threshold are therefore around 3. :param data: The data (modified in place). :type data: :class:`Data` :param threshold: Sigma-level of threshold :type threshold: number :param flag_low_outliers: Flag channels with low RMS :type flag_low_outliers: boolean .. function:: threshold_timestep_rms(data, threshold) Like :meth:`threshold_channel_rms`, but thresholds *timesteps* with outlier RMS. Both timesteps with high and low RMS values are flagged. :param data: The data (modified in place). :type data: :class:`Data` :param threshold: Sigma-level of threshold :type threshold: number .. function:: upsample(input_data, destination_data, xfactor, yfactor) Increase the resolution of the data. This function is to restore the resolution of the data after having called :meth:`downsample`. ``input_data`` is normally the data that was returned by :meth:`downsample`, and ``destination_data`` is the input object that was specified as parameter. The upsampling is done by nearest neighbour interpolation. The x and y factors should be the equal to the values specified in the call to `downsample`. The size of the ``destination_data`` is not changed: the input data is stretched by the given factors, and trimmed to the destination size in case the image dimensions were not exactly divisable by the factors. :param input_data: Input low-resolution data (not modified). :type input_data: :class:`Data` :param destination_data: Where the result will be stored. :type destination_data: :class:`Data` :param xfactor: Upsampling factor in time direction. :type xfactor: integer :param yfactor: Upsampling factor in frequency direction. :type yfactor: integer .. function:: visualize(data, label, sorting_index) Save a visualization of the data for inspection in ``rfigui``. When this strategy runs outside of the ``rfigui``, the call is ignored. Can be used to e.g. inspect partial results. :param data: Input data (not modified). :type data: :class:`Data` :param label: A short description that is displayed to the user. :type label: string :param sorting_index: Where to place this visualization in the list of visualization