aiqclib.nrtqc.step4_compare_flags package

Submodules

aiqclib.nrtqc.step4_compare_flags.compare_base module

This module provides the CompareFlagsBase class, the NRT QC module’s optional step 4: comparing existing NRT QC flags with the newly computed ones.

For every configured variable that carries an existing flag column (the flag entry of its qc_variable_set definition, e.g. temp_qc), a summary report is built with three sections:

  1. contingency β€” the cross-tabulation of existing flag value and new NRT flag value with counts and percentages. Works with any existing flag scheme; no value mapping is required.

  2. agreement β€” binary agreement metrics (accuracy, precision, recall plus the underlying confusion counts), reported only when pos_flag_values / neg_flag_values are configured for the variable, using the same convention as the other modules.

  3. item_breakdown β€” per enabled QC item, how many observations the item flagged within each existing flag value, showing which items drive agreement or disagreement.

Variables without a configured flag are silently skipped; a configured flag column that is absent from the data raises an error (no silent skip on typos).

class aiqclib.nrtqc.step4_compare_flags.compare_base.CompareFlagsBase(config, merged_data=None)[source]

Bases: DataSetBase

Base class for the existing-vs-new flag comparison (step "compare").

Takes the final frame produced by step 3 (merged_data, holding both the existing flag columns from the input and the newly computed {variable}_nrt_flag columns) and builds one report per comparable variable, stored in reports and written as TSV files.

Parameters:

  • config (ConfigBase)

  • merged_data (DataFrame | None)

compare(target_name, flag_column)[source]

Build the comparison report for a single variable.

Parameters:

  • target_name (str) – The variable to compare.

  • flag_column (str) – The variable’s existing flag column.

Returns:

The report frame (see REPORT_SCHEMA).

Return type:

DataFrame

Raises:

polars.exceptions.ColumnNotFoundError – If the existing flag column or the new NRT flag column is absent from the data.

compare_targets()[source]

Build the comparison report for every comparable variable.

Variables without a configured existing flag are skipped silently (there is nothing to compare against).

Raises:

  • ValueError – If merged_data is empty.

  • polars.exceptions.ColumnNotFoundError – If a configured flag column is absent from the data.

Return type:

None

default_file_name: str

The default per-variable report file name pattern.

merged_data: DataFrame | None
output_file_names: Dict[str, str]

A dictionary mapping variable names to report file paths.

reports: Dict[str, DataFrame]

A dictionary mapping variable names to comparison reports.

write_reports()[source]

Write each variable’s comparison report to a TSV file.

Writes one file per entry in reports; when no variable had an existing flag configured there is nothing to write and the method returns without error.

Return type:

None

aiqclib.nrtqc.step4_compare_flags.compare_base.REPORT_SCHEMA: Dict = {'count': Int64, 'existing_flag': Int64, 'item': String, 'metric': String, 'new_flag': Int64, 'percent': Float64, 'section': String, 'value': Float64}

Column schema of the comparison report (long format, one section column).

aiqclib.nrtqc.step4_compare_flags.dataset_all module

This module defines the CompareFlagsAll class, the concrete implementation of the NRT QC module’s step 4 (existing-vs-new flag comparison) for Copernicus CTD data.

class aiqclib.nrtqc.step4_compare_flags.dataset_all.CompareFlagsAll(config, merged_data=None)[source]

Bases: CompareFlagsBase

A specialized implementation of aiqclib.nrtqc.step4_compare_flags.compare_base.CompareFlagsBase that compares the flags of all observations.

Variables:

expected_class_name (str) – The class identifier used for configuration matching.

Parameters:

  • config (ConfigBase)

  • merged_data (DataFrame | None)

expected_class_name: str = 'CompareFlagsAll'