raitap.metrics

RAITAP Metrics Module

Provides performance metric computation for classification and detection tasks using torchmetrics.

Metrics Public Surface

MetricComputer

Protocol defining the interface for all metric computers (reset, update, compute).

MetricResult

Dataclass containing computed metrics (dict[str, float]) and artifacts (dict[str, Any]).

Metric classes

BinaryClassificationMetrics

Accuracy, precision, recall, and F1 for two-class problems.

MulticlassClassificationMetrics

Accuracy, precision, recall, and F1 for multiclass classification.

MultilabelClassificationMetrics

Accuracy, precision, recall, and F1 for multilabel classification.

DetectionMetrics

Computes mean average precision (mAP) and related metrics for object detection tasks.

Module layout (for contributors):

• phase.py — pipeline entry point: MetricsPhase (what the registry assembles) + the evaluate_metrics work fn. Singleton phase (no adapter loop). Start here to follow a run.

• factory.py — metrics_run_enabled + evaluate (instantiation).

• *_metrics.py — the metric adapters (classification, detection).

• inputs.py — target / prediction alignment + label fallbacks.

class raitap.metrics.BaseMetricComputer

Bases: ABC, AdapterMixin

class raitap.metrics.BinaryClassificationMetrics(*, ignore_index=None, threshold=0.5, **kwargs)

Bases: _ClassificationBase

Binary classification metrics (accuracy, precision, recall, F1).

class raitap.metrics.DetectionMetrics(*, box_format='xyxy', iou=None, class_metrics=False, extended_summary=False, average='macro', backend='faster_coco_eval', **kwargs)

Bases: BaseMetricComputer

Calculates and manages detection metrics for evaluating the performance of object detection models.

This class is responsible for computing detection metrics, updating them with predictions and targets, and resetting their state. It uses a MeanAveragePrecision calculator internally to handle the computation logic. It supports a variety of configurations, including box formats, IoU types, thresholds, class-specific metrics, and more.

Variables:

metric -- Instance of the MeanAveragePrecision calculator used to compute metrics.

class raitap.metrics.MetricResult(scalars: 'dict[str, float]', artifacts: 'dict[str, Any]'=<factory>)

Bases: object

class raitap.metrics.Metrics(config, predictions, targets)

Bases: object

Run configured metrics, write JSON under the run metrics/ directory.

class raitap.metrics.MetricsEvaluation(result, run_dir, computer, resolved_target)

Bases: Trackable, Reportable

Outcome of a metrics run (JSON on disk + optional computer handle).

log(tracker, *, prefix='performance', **kwargs)

Log the object's artifacts or metadata to the provided tracker.

report_sections(ctx)

Return this object's contribution as ordered report sections.

class raitap.metrics.MetricsVisualizer

Bases: object

Generate matplotlib figures from MetricResult data.

static create_figures(result)

Generate charts for metrics.

Returns dict with: - "metrics_overview": bar chart of all scalar metrics - "confusion_matrix": if confusion matrix exists in artifacts

class raitap.metrics.MulticlassClassificationMetrics(*, num_classes, average='macro', ignore_index=None, **kwargs)

Bases: _ClassificationBase

Multiclass classification metrics (accuracy, precision, recall, F1).

class raitap.metrics.MultilabelClassificationMetrics(*, num_labels, average='macro', ignore_index=None, threshold=0.5, **kwargs)

Bases: _ClassificationBase

Multilabel classification metrics (accuracy, precision, recall, F1).

raitap.metrics.create_metric(metrics_config)

Instantiate a metric computer from Hydra-style config (_target_ + kwargs).

raitap.metrics.evaluate(config, predictions, targets)

Compute metrics, persist JSON outputs, and return a MetricsEvaluation.

raitap.metrics.metrics_prediction_pair(output)

Build a placeholder (predictions, targets) pair for metrics when no labels exist.

For multiclass logits (N, C) with C > 1, uses argmax for both (trivial self-consistency). For other shapes, passes output through unchanged so users can pair metrics configs with regression / detection / etc.

raitap.metrics.metrics_run_enabled(config)

True when metrics is present and _target_ is a non-empty string.

raitap.metrics.resolve_metric_targets(predictions, labels)

Use ground truth labels when available, else warn and fall back to predictions.

raitap.metrics.scalar_metrics_for_tracking(result)

Keep only JSON-friendly scalars suitable for tracker log_metrics.