Research evaluation catalog

Evaluators compare an algorithm result with reference evidence. They are kept outside Applications and estimators so the implementation under test cannot silently choose its own oracle, validity convention, or held-out split. The NumPy-canonical region, feature, pose and radial-range evaluator surface is Stable as panorai-spherical-evaluation/v1. Surface-reconstruction evaluation is a separately inventoried Experimental component.

Registered evaluator surfaces

Surface	Stability	Canonical import	Contract	Public components or entries
spherical-evaluation	Stable	panorai.evaluation	panorai-spherical-evaluation/v1	BinaryRegionEvaluation, DepthEvaluation, EvaluationProtocol, EvaluationRecord, FeatureEvaluation, PoseEvaluation, evaluate_binary_regions, evaluate_features, evaluate_pose, evaluate_radial_range, solid_angle_weights
spherical-surface-reconstruction-evaluation	Experimental	panorai.evaluation	panorai-surface-reconstruction-evaluation/v1-experimental	SurfaceReconstructionEvaluation, SurfaceReconstructionEvaluationReport, aggregate_surface_reconstruction, evaluate_surface_reconstruction

Available evaluators

Geometry-aware evaluation components

Component	Input	Output	Spherical contract
evaluate_binary_regions	two boolean ERP regions plus support and validity	BinaryRegionEvaluation	solid-angle overlap and great-circle boundary matching
evaluate_features	predicted/reference unit bearings and optional scales	FeatureEvaluation	one-to-one angular assignment
evaluate_pose	two rotations and optional translation directions	PoseEvaluation	geodesic SO(3) and S2 errors
evaluate_radial_range	two radial-range ERPs plus support, validity and confidence	DepthEvaluation	area-weighted accuracy, coverage and confidence ordering
evaluate_surface_reconstruction	predicted/reference SurfaceMesh plus a frozen distance threshold	SurfaceReconstructionEvaluation	Experimental; exact bidirectional vertex-to-triangle distance, F-score and oriented normals
aggregate_surface_reconstruction	named scene evaluations from one frozen cohort	SurfaceReconstructionEvaluationReport	Experimental; unweighted per-scene medians; common units and threshold required
EvaluationProtocol	evaluator identity, metrics and frozen parameters	fingerprinted protocol	thresholds are fixed before evaluation
EvaluationRecord	protocol, result metrics and provenance	canonical JSON evidence	fail-closed, finite and pickle-free

The exact signatures are in the evaluation API. Protocol design and interpretation are in the evaluation concept.