Choose modality, layout, and interpolation

PanorAi separates three concepts:

Surface	Stability	Canonical import	Contract	Public components or entries
spherical-region-analysis-data	Stable	panorai.data	panorai-spherical-region-analysis/v1	EnclosingSphericalCap, RegionAdjacencyEdge, RegionAdjacencyGraph, RegionAnalysisResult, RegionMeasurement, RegionMeasurements, SphericalBoundary, SphericalBoundarySet, SphericalContour, SphericalContourSet, panorai.data.regions.EnclosingSphericalCap, panorai.data.regions.RegionAdjacencyEdge, panorai.data.regions.RegionAdjacencyGraph, panorai.data.regions.RegionAnalysisResult, panorai.data.regions.RegionMeasurement, panorai.data.regions.RegionMeasurements, panorai.data.regions.SphericalBoundary, panorai.data.regions.SphericalBoundarySet, panorai.data.regions.SphericalContour, panorai.data.regions.SphericalContourSet
spherical-observations	Stable	panorai.data	panorai-spherical-observations/v1	AggregatedViewEvidence, DataFactory, EquirectangularImage, FacePointBatch, FaceRegionBatch, FaceViewOutput, GnomonicFace, GnomonicFaceSet, ObservationFusionResult, RasterFieldResult, RasterFieldSpec, SphericalBinaryMask, SphericalPointObservation, SphericalPointSet, SphericalRegionObservation, SphericalRegionSet, SphericalTranspose, SphericalViewObservation, SphericalViewSet, XYZImageDataset, panorai.data.utils.PanoraiError, panorai.data.utils.InvalidDataError, panorai.data.utils.ChannelMismatchError, panorai.data.utils.MetadataValidationError, panorai.data.utils.DataConversionError, panorai.data.utils.MissingChannelError, panorai.data.utils.ensure_type_consistency, panorai.data.utils.ShapeManager, panorai.data.xyz_image_dataset.XYZImageDataset, panorai.data.xyz_image_dataset.extract_hw_from_ply_filename, panorai.data.xyz_image_dataset.read_encrypted_ply_xyz_image, panorai.data.xyz_image_dataset.read_ply_xyz_image
spherical-core-contracts	Experimental	panorai.core	panorai-core-operations/v1	SPHERICAL_CORE_INTERFACE, SphericalConv2dSpec, SphericalExecutionSpec, SphericalGridSpec, SphericalNeighborhoodSpec, SphericalPool2dSpec, SphericalResizeSpec, panorai.core.specs.BackendName, panorai.core.specs.Interpolation, panorai.core.specs.Layout, panorai.core.specs.SphericalConv2dSpec, panorai.core.specs.SphericalExecutionSpec, panorai.core.specs.SphericalGridSpec, panorai.core.specs.SphericalNeighborhoodSpec, panorai.core.specs.SphericalPool2dSpec, panorai.core.specs.SphericalResizeSpec
panorama-stitching-data	Experimental	panorai.stitching	panorai-panorama-stitching/v1-experimental	ImagePairMatches, ImagePairRegistration, PanoramaStitchingResult, RegisteredImageSet, StitchingDiagnostics, StitchingFeatureSet, StitchingFeatures, StitchingImage, StitchingInput, StitchingMatches, StitchingOptions, WarpedImageSet

support_mask

Whether projection geometry covers an output pixel.

data validity

Whether a supported sample is meaningful, such as a finite radial range.

numeric value

The sample itself. Zero and black are ordinary values, not missingness.

RGB

Use floating HWC RGB for bilinear interpolation. Convert integer RGB to a floating range deliberately, or select nearest when class-like integer values must remain exact.

Labels and masks

Integer labels and boolean masks require nearest. Bilinear interpolation of these modalities fails explicitly rather than creating fractional labels or soft booleans.

Depth

Depth means radial range in the canonical frame. State units alongside the array (metres in this example). Floating NaN stays invalid data and must not silently become a valid zero. Support remains a separate boolean mask.

The default bilinear policy, propagate, keeps strict IEEE NaN propagation. For sparse measurements, opt into renormalize with an explicit boolean validity mask and a scientifically chosen min_valid_weight. The result separately reports geometric support_mask, thresholded validity_mask, and the contributing valid_weight. PanorAi does not choose one threshold for every sensor or modality.

Dataset-neutral XYZ images

XYZImageDataset loads organized PLY rasters without embedding dataset names or split rules in PanorAi. The consumer supplies an explicit directory or ordered file list, coordinate frame, units, and optional shadow angle:

from panorai.data import XYZImageDataset

samples = XYZImageDataset(
    files=authorized_files,
    coordinate_frame="scanner-local-right-up-forward",
    units="metres",
    shadow_angle=0.0,
)

Each filename declares _HxW before the optional _encrypted suffix. Each sample returns rgb_image, the original xyz_image, derived radial_depth, validity_mask, frame, units, shadow angle, and source path. Split construction and licensed Matterport3D or Stanford2D3D access remain application responsibilities. Importing panorai.data does not import Open3D; PLY decoding loads that optional dependency only when needed.

Stable object workflow

The Stable panorai-object-workflow/v1 surface records these semantics without a user-authored channel dictionary:

import panorai as pa

pano = (
    pa.EquirectangularImage(rgb)
    .with_depth(radial_range_m, valid=depth_is_valid, units="m")
    .with_labels(labels)
)
views = pano.views("cube")

views projects each modality separately: image and depth use bilinear, while labels use nearest. views.describe() reports the resolved policy. Legacy dictionaries remain accepted by the existing methods, but the typed workflow rejects an untyped dictionary rather than guessing its semantics.

Application raster fields

Use RasterFieldSpec for CAM, logits, confidence, dense embeddings, or application categories:

from panorai.data import RasterFieldSpec

pano = pano.with_field(
    "cam",
    cam,
    spec=RasterFieldSpec.continuous(component_ids=("pipe", "vessel")),
    valid=cam_valid,
)

New fields passed to GnomonicFaceSet.map() require the same explicit spec=. See Spherical raster fields and observations for point, region, mask, and whole-view outputs.

from panorai.data import (
    FacePointBatch,
    FaceRegionBatch,
    FaceViewOutput,
    RasterFieldSpec,
)

cam = np.stack((rgb[..., 0], rgb[..., 1]), axis=-1)
semantic = workflow_panorama.with_field(
    "cam",
    cam,
    spec=RasterFieldSpec.continuous(component_ids=("class-a", "class-b")),
)
semantic_views = semantic.views("cube", size=8)
cam_erp = semantic_views.reconstruct(modalities="cam")
assert cam_erp.field("cam").shape == cam.shape

point_observations = semantic_views.map_points(
    lambda face: FacePointBatch(np.array([[3.5, 3.5]])),
    source_id="example-points",
)
assert len(point_observations.angular_nms(0.1).observations) == 6

region_observations = semantic_views.map_regions(
    lambda face: FaceRegionBatch.from_boxes(np.array([[1.0, 1.0, 6.0, 6.0]])),
    source_id="example-boxes",
)
assert len(region_observations.nms(0.8).observations) == 6

view_observations = semantic_views.map_views(
    lambda face: FaceViewOutput(
        scores=np.array([0.25, 0.75]), class_ids=np.array([0, 1])
    ),
    source_id="example-classifier",
)
view_summary = view_observations.aggregate("coverage_mean")
assert np.allclose(view_summary.scores, [0.25, 0.75])

labels = (np.arange(height * width) % 7).reshape(height, width).astype(np.int16)
label_view = equirectangular_to_gnomonic(labels, view_spec, interpolation="nearest")
assert label_view.data.dtype == np.int16

valid = np.ones((height, width), dtype=bool)
valid[:, :4] = False
mask_view = equirectangular_to_gnomonic(valid, view_spec, interpolation="nearest")
assert mask_view.data.dtype == np.bool_

radial_range_m = np.full((height, width), 2.5, dtype=np.float32)
radial_range_m[0, 0] = np.nan  # invalid data stays distinct from support
depth_is_valid = np.isfinite(radial_range_m)
depth_view = equirectangular_to_gnomonic(
    radial_range_m,
    view_spec,
    interpolation="bilinear",
    invalid_policy="renormalize",
    validity_mask=depth_is_valid,
    min_valid_weight=0.5,
)
assert depth_view.data.dtype == np.float32
assert depth_view.support_mask.all()
assert depth_view.validity_mask.shape == view_spec.output_shape_hw
assert depth_view.valid_weight.shape == view_spec.output_shape_hw

Mask-aware blending

from panorai.blenders import AverageBlender

black = np.zeros((4, 6, 3), dtype=np.float32)
white = np.ones((4, 6, 3), dtype=np.float32)
black_is_valid = np.ones((4, 6), dtype=bool)
white_is_invalid = np.zeros((4, 6), dtype=bool)
blended, blend_support = AverageBlender().blend(
    [black, white],
    [black_is_valid, white_is_invalid],
    return_mask=True,
)
assert np.array_equal(blended, black)  # black is data, not missingness
assert blend_support.all()