Data class

class Data

The Data class contains the visibility data, flag masks and meta data.

It may hold information for multiple polarizations, channels and timesteps, such that it contains the data for a particular observing field, spectral window (band) and antenna-pair (or single dish). The data objects thus holds a dynamic spectrum for each polarization.

The visibility data can consist of complex values, or single-float values representing the real, imaginary, amplitude or phase of the visibilities (see get_complex_state()).

The Data class is a Lua userdata object. Assigning one Data variable to another will therefore cause both variables to point to the same object, and changing one will also change the other. To create an independent object, copy() can be used.

Internally, AOFlagger uses a copy-on-write-like mechanism for the visibility sets, flag masks and meta data. It is not required for users of the class to know about these implementation details, but the relevant thing to know is that copy or assignment operations are generally fast and don’t increase memory usage.

Detailed descriptions

Data.clear_mask(data)

Clear the flag mask. Unflags all visibilities (sets all flags to false).

Parameters:data (Data) – Data for which the mask is cleared.
Data.convert_to_complex(data, new_state)

Make a new Data object with a different complex-value state. Complex input data (data with get_complex_state() == "complex") can be converted to real, imaginary, amplitude or phase values. Amplitude data (get_complex_state() == "amplitude") can also be converted back to complex. In that case the phases become zero. Other conversions are not implemented and will cause an error.

The complex state of an Data object is stored internally and can be acquired by calling get_complex_state().

Parameters:
  • data (Data) – Input data (unchanged).
  • new_state (string) – "complex", "real", "imaginary", "amplitude" or "phase".
Returns:

New object with get_complex_state() == new_state
Return type:

Data
Data.convert_to_polarization(data, new_polarization)

Make a new Data object by converting the polarization. If the input data does not hold the polarimetric data to convert to the requested polarization, an error is thrown. For example, converting to "i" from data for which get_polarizations() == {"xx","yy"} is possible, but converting to "q" from data with get_polarizations() == {"ll"} is not.

This method can also be used to extract a single polarization from the set of available polarizations, e.g.

xxdata = data:convert_to_polarization("xx")

for data with get_polarizations() == {"xx", "xy", "yx", "yy"}.

Parameters:
  • data (Data) – input data (unchanged).
  • new_polarization (string) – "i", "q", "u", "v", "xx", "xy", "yx", "yy", "rr", "rl", "lr" or "ll".
Data.copy(data)

Make a value copy of the data.

Parameters:data (Data) – Source data.
Returns:Value copy of input data.
Return type:Data
Data.flag_nans(data)

Flag visibilities that are ‘not a number’ (nan) or hold overflow. Each polarization is independently searched for nans, and its mask is updated for that polarization (this is different from Data.flag_zeros()).

Since:AOFlagger 3.1.
Data.flag_zeros(data)

Flag visibilities that are exactly zero. This corrects for correlators that output zeros during faults, such as network problems.

It flags samples when the sum of visibilities over polarizations is zero. When it is necessary to flag the polarizations independently, the statement should be placed inside a loop, e.g.:

for _,polarization in ipairs(data.get_polarizations()) do
  pol_data = data:convert_to_polarization(polarization)
  flag_zeros(pol_data)
  data:set_polarization_data(polarization, pol_data)
end
Parameters:data (Data) – Data (modified inplace).
Data.get_antenna1_index(data)

Get first antenna index of the two correlated antennas. Throws an error of the antenna metadata is not available (see has_metadata()).

Parameters:data (Data) – Input data (unchanged).
Returns:Index of first antenna
Return type:integer
Data.get_antenna1_name(data)

Get name of first antenna of the two correlated antennas. Throws an error of the antenna metadata is not available (see has_metadata()).

Parameters:data (Data) – Input data (unchanged).
Returns:Name of first antenna
Return type:string
Data.get_antenna2_index(data)

Get second antenna index of the two correlated antennas. Throws an error of the antenna metadata is not available (see has_metadata()).

Parameters:data (Data) – Input data (unchanged).
Returns:Index of second antenna
Return type:integer
Data.get_antenna2_name(data)

Get name of second antenna of the two correlated antennas. Throws an error of the antenna metadata is not available (see has_metadata()).

Parameters:data (Data) – Input data (unchanged).
Returns:Name of second antenna
Return type:string
Data.get_baseline_angle(data)

Get angle of this baseline. This is that angle between the line from antenna2 to antenna1 and North. Throws an error of the antenna metadata is not available (see has_metadata()).

Parameters:data (Data) – Input data (unchanged)
Returns:Baseline angle in radians
Return type:number
Data.get_baseline_distance(data)

Get distance of the antenna1-antenna2 baseline in meters. Throws an error of the antenna metadata is not available (see has_metadata()).

Parameters:data (Data) – Input data (unchanged).
Returns:Baseline distance in meters
Return type:number
Data.get_baseline_vector(data)

Get a table with items x, y and z that form the three- dimensional vector between antennas 1 and 2. Throws an error of the antenna metadata is not available (see has_metadata()).

Parameters:data (Data) – Input data (unchanged).
Returns:Baseline vector in meters
Return type:table
Data.get_complex_state(data)

Get the state that the visibilities represent. This can be "phase", "amplitude", "real", "imaginary" or "complex". When the data is complex, each visibility consists of two number. Conversions can be performed with convert_to_complex().

Parameters:data (Data) – Input data (unchanged).
Returns:Complex state of data.
Return type:string
Data.get_frequencies(data)

Get the frequencies of the channels. Throws an error of the spectral window metadata is not available (see has_metadata()).

Parameters:data (Data) – Input data (unchanged).
Returns:List that maps channel nr to frequency in Hz.
Return type:table
Data.get_polarizations(data)

Get the list of polarizations provided by the data. See convert_to_polarization() for the list of possible polarization names.

Parameters:data (Data) – Input data (unchanged).
Returns:List that maps polarization nr to a string.
Return type:table
Data.get_times(data)

Get the time of each timestep in these data. Throws an error of the time metadata is not available (see has_metadata()).

Parameters:data (Data) – Input data (unchanged).
Returns:List that maps timestep nr to MJD time in s.
Return type:table
Data.has_metadata(data)

Returns whether metadata is completely present. Not all data formats (or simulations) provide all metadata items and some of the other methods (e.g. get_times()) may throw an error because of this. If this function returns true, all these functions will succeed.

Parameters:data (Data) – Input data (unchanged).
Returns:true in case all metadata is available, false otherwise.
Return type:boolean
Data.is_auto_correlation(data)

Determine whether this baseline is an auto-correlation. This is the case if get_antenna1_index() == get_antenna2_index(). Unlike the get_antenna* functions, this method won’t throw an error when no meta-data is available. false is returned in that case.

Parameters:data (Data) – Input data (unchanged).
Returns:true when this is an auto-correlation.
Type:boolean
Data.is_complex(data)
Parameters:data (Data) – Input data (unchanged).
Returns:true when get_complex_state() == "complex"
Type:boolean
Data.join_mask(first_data, second_data)

Join two masks together. A flag will be set when it is set in either or both of the input data sets.

Parameters:
  • first_data (Data) – First mask and destination of operation.
  • second_data (Data) – Second mask (unchanged).
Data.set_mask(destination_data, mask_data)

Assign the mask of one Data object to another.

Parameters:
  • first_data (Data) – Destination data (changed inplace).
  • second_data (Data) – Source data (unchanged).
Data.set_mask_for_channel_range(destination_data, mask_data, freq_start, freq_end)

Partially assign the mask of one Data object to another. The flag mask of channels within the given frequency range are copied from mask_data to destination_data.

This can for example be useful when a certain channel range should not be flagged, by partially copying the initial flags to the mask produced by the RFI detection.

Parameters:
  • first_data (Data) – Destination data (changed inplace).
  • second_data (Data) – Source data (unchanged).
  • freq_start (number) – Frequency range start in MHz
  • freq_end (number) – Frequency range end in MHz
Data.set_polarization_data(destination_data, polarization, source_data)

Replace one polarization of a Data object with some other data. The source_data should have only one polarization. The typical use-case for this method is to loop over polarizations and modify them one by one, e.g.:

for _,polarization in ipairs(data:get_polarizations()) do
  pol_data = input:convert_to_polarization(polarization)
  -- Change pol_data here...
  data:set_polarization_data(polarization, pol_data)
end

Note that the two data sets should have the same complex state (see get_complex_state()). This method copies both the mask and the visibilities. The meta-data is unchanged.

Parameters:
  • destination_data (Data) – Destination data (changed inplace).
  • polarization (string) – Name of polarization to change
  • source_data (Data) – Source data (unchanged).
Data.set_visibilities(destination_data, visibility_data)

Assign the visibility data from one Data object to another. The flagmask and meta-data are unchanged. The two sets should have the same number of polarizations and the same complex state.

Parameters:
  • destination_data (Data) – Destination data (changed inplace).
  • visibility_data (Data) – Source data (unmodified).
Data.__gc(data)

Internal method for garbage collection function of the Data class. This will free the allocated data when those data are no longer used by other Data objects. Note that AOFlagger will immediately clear data when the execute() function is finished, even when the class hasn’t been garbage collected yet.

TODO add a method set_persistent to disable this.

Parameters:destination_data (Data) – Garbage collected data.
Data.__sub(lhs_data, rhs_data)

Lua-special method that makes it possible to subtract Data objects, e.g.:

data = data - filtered_data

The output data will have the same meta-data and masks as lhs_data.

Parameters:
  • lhs_data (Data) – Left-hand side data (not modified)
  • rhs_data (Data) – Right-hand side data (not modified)
Returns:

Left - right
Return type:

Data