How PanorAi executes computer vision on spherical images

Surface	Stability	Canonical import	Contract	Public components or entries
spherical-features-core	Stable	panorai.features	panorai-spherical-features/v1	DeduplicationResult, FaceSetSpec, FeatureExtractor, FeatureExtractorConfig, FeatureMatcher, FeatureMatcherConfig, FeatureProvenance, FaceDetectionDiagnostics, MatchProvenance, MultifaceDetectionDiagnostics, OpenCVFeatureBackend, SphericalBearingCorrespondences, SphericalFeature, SphericalFeatureMatches, SphericalFeaturePipeline, SphericalFeaturePipelineConfig, SphericalFeatureSet, available_presets, deduplicate_spherical_keypoints, extract_opencv_features, gnomonic_feature_mask, match_opencv_features, panorai.features.backends.OpenCVFeatureBackend
spherical-features-pycolmap	Experimental	panorai.features	panorai-spherical-features-pycolmap/v1	GnomonicRig, GnomonicRigCamera, PyCOLMAPExportResult
spherical-features-multiscale	Experimental	panorai.features	panorai-spherical-features-multiscale/v1	MultiscaleEmbeddingConfig, MultiscaleFeatureMatches, MultiscaleFeatureSet, MultiscaleSphericalFeaturePipeline, OpenCVContextEmbedding, VisualContextNode, VisualEmbeddingProvider
spherical-features-dog-sift	Experimental	panorai.features	panorai-spherical-dog-sift/v2	SphericalDoGSIFTConfig, SphericalDoGSIFTExtractor, SphericalDoGSIFTPipeline
spherical-features-dog-detector	Experimental	panorai.features	panorai-spherical-dog-detector/v2	SphericalDetectionDiagnostics, SphericalDoGDetector, SphericalDoGDetectorConfig, SphericalKeypoint, SphericalKeypointSet, SphericalOctaveDetectionDiagnostics, SphericalRefinedKeypoint
spherical-features-coarse-dog-detector	Experimental	panorai.features	panorai-spherical-coarse-dog-detector/v1	SphericalCoarseDoGDetector, SphericalCoarseDoGDetectorConfig
spherical-features-tangent-patches	Experimental	panorai.features	panorai-tangent-patches/v1	OpenCVTangentDescriptor, OpenCVTangentDescriptorConfig, TangentDescriptorSet, TangentPatch, TangentPatchGeometry, TangentPatchProvider, TangentPatchRequest, TangentPatchSet
spherical-feature-resolution-selection	Experimental	panorai.features	panorai-resolution-selection/v1	ResolutionLevelReport, ResolutionSelectionObservation, ResolutionSelectionPolicy, ResolutionSelectionReport, ResolutionTransitionReport, select_feature_resolution
spherical-features-tangent-descriptor-v2	Experimental	panorai.features	panorai-tangent-opencv-descriptor/v2	OpenCVTangentDescriptorV2, OpenCVTangentDescriptorV2Config, TangentDescriptorHypothesisSet
spherical-relative-pose-pipeline	Experimental	panorai.features	panorai-spherical-relative-pose-pipeline/v1-experimental	ExtractSphericalFeaturePair, MatchSphericalFeatures, SphericalFeaturePair, SphericalImagePair, ToBearingCorrespondences
spherical-deep-learning-depth	Experimental	panorai.experimental.deep_learning	panorai-spherical-metric-depth/v1-experimental	SphericalMetric3D, acquire_metric3d_convnext_large_v1, acquire_metric3d_convnext_tiny_v1, load_spherical_metric3d_convnext_large_v1, load_spherical_metric3d_convnext_tiny_v1, panorai.depth.DepthAdapterUnavailableError, panorai.depth.ModelRegistry, panorai.depth.load_dav2_model, panorai.depth.load_dust3r_model, panorai.depth.load_m3dv2_model, panorai.depth.load_zoe_model, panorai.depth.zoe.load_zoe_model, panorai.experimental.deep_learning.depth.SPHERICAL_METRIC_DEPTH_INTERFACE, panorai.experimental.deep_learning.depth.METRIC3D_CHECKPOINT_REVISION, panorai.experimental.deep_learning.depth.METRIC3D_CONVNEXT_LARGE_V1, panorai.experimental.deep_learning.depth.METRIC3D_CONVNEXT_TINY_V1, panorai.experimental.deep_learning.depth.METRIC3D_SOURCE, panorai.experimental.deep_learning.depth.METRIC3D_SOURCE_COMMIT, panorai.experimental.deep_learning.depth.ExternalArtifactRecord, panorai.experimental.deep_learning.depth.ExternalArtifactSpec, panorai.experimental.deep_learning.depth.LoadedSphericalDepthModel, panorai.experimental.deep_learning.depth.Metric3DAssets, panorai.experimental.deep_learning.depth.SphericalMetric3D, panorai.experimental.deep_learning.depth.UpstreamTermsNotAcceptedError, panorai.experimental.deep_learning.depth.acquire_metric3d_convnext_large_v1, panorai.experimental.deep_learning.depth.acquire_metric3d_convnext_tiny_v1, panorai.experimental.deep_learning.depth.load_spherical_metric3d_convnext_large_v1, panorai.experimental.deep_learning.depth.load_spherical_metric3d_convnext_tiny_v1
tangent-da3-metric-depth	Experimental	panorai.experimental.deep_learning	panorai-depth-anything-3-metric-tangent/v1-experimental	DA3MetricTangent, acquire_da3metric_large, load_da3metric_large, panorai.experimental.deep_learning.da3.DA3Assets, panorai.experimental.deep_learning.da3.DA3MetricTangent, panorai.experimental.deep_learning.da3.DA3TermsNotAcceptedError, panorai.experimental.deep_learning.da3.DA3_CANONICAL_FOCAL_PX, panorai.experimental.deep_learning.da3.DA3_CHECKPOINT_REVISION, panorai.experimental.deep_learning.da3.DA3_MODEL_ID, panorai.experimental.deep_learning.da3.DA3_PATCH_SIZE, panorai.experimental.deep_learning.da3.DA3_SOURCE, panorai.experimental.deep_learning.da3.DA3_SOURCE_COMMIT, panorai.experimental.deep_learning.da3.DA3METRIC_LARGE, panorai.experimental.deep_learning.da3.LoadedDA3MetricModel, panorai.experimental.deep_learning.da3.SPHERICAL_DA3_METRIC_DEPTH_INTERFACE, panorai.experimental.deep_learning.da3.acquire_da3metric_large, panorai.experimental.deep_learning.da3.load_da3metric_large
spherical-semantic-segmentation	Experimental	panorai.experimental.deep_learning.segmentation	panorai-spherical-semantic-segmentation/v1	SPHERICAL_MASK_PROPOSALS_INTERFACE, SPHERICAL_PROXY_SEGMENTATION_INTERFACE, SPHERICAL_SEMANTIC_SEGMENTATION_INTERFACE, SAM21_HIERA_LARGE_LICENSE, SAM21_HIERA_LARGE_REPOSITORY, SAM21_HIERA_LARGE_REVISION, IMAGENET_PROXY_VOCABULARY, DenseSemanticEvidence, Sam21Assets, Sam21FileRecord, Sam21HieraLargeSegmenter, Sam21TermsNotAcceptedError, SamMultimaskOutput, SegmentPrompt, SemanticProxyConcept, SphericalBinaryMask, SphericalProxySegmenter, SphericalMaskProposal, SphericalMaskProposalSet, SphericalSeed, SphericalSegment, SphericalSegmentationConfig, SphericalSegmentationResult, SphericalSemanticSegmenter, acquire_sam21_hiera_large, advance_chart, backproject_face_mask_sets, angular_distance_degrees, backproject_face_masks, concept_display_names, concept_evidence_from_imagenet, concept_proxy_names, connected_component_at, consensus_frontiers, extract_mask_face, extract_rgb_face, face_point_direction, fibonacci_coverage_seeds, load_sam21_hiera_large, majority_and_envelope, mask_stability_score, maximum_pairwise_iou, merge_proposals, non_maximum_suppression, partition_proposals_by_origin, pairwise_mask_iou, prompt_from_mask, prompt_grid, proposal_is_accepted, resolve_panoptic_map, select_semantic_seeds, semantic_support_for_mask, solid_angle_weights, weighted_containment, weighted_intersection_over_union
spherical-relative-pose	Experimental	panorai.estimators	panorai-spherical-relative-pose/v1	CalibrationEvaluation, DecoupledPoseReport, EstimateRelativePose, FivePointSample, FivePointSampler, FivePointSamplingDiagnostics, ModelCompetitionReport, ModelEvidence, PoseStabilityReport, RelativePoseAcceptancePolicy, RelativePoseConfidenceCalibrator, RelativePoseOptions, RelativePoseQualityReport, RelativePoseResult, TranslationOrientationReport, SpatiallyWeightedFivePointSampler, SphericalRelativePoseEstimator, UniformFivePointSampler, estimate_relative_pose, native_kernels_available, solve_five_point_essential, spherical_tangent_sampson_error
semantic-object-localization	Experimental	panorai.object_localization	panorai-object-localization/v1	JOINT_MATCH_CLUSTERS_INTERFACE, JointMatchCluster, JointMatchClusterConfig, JointMatchClusterResult, MULTIVIEW_OBJECT_HYPOTHESES_INTERFACE, BearingTrackTriangulation, DirectionalRegionTransfer, HypothesisAffinity, HypothesisAffinityConfig, HypothesisGnomonicRegion, HypothesisPairCues, HypothesisReductionCandidate, HypothesisReductionResult, HypothesisRegionEvaluation, HypothesisTransferEvaluation, HypothesisViewConsistency, HypothesisViewEmbedding, HypothesisViewPairEvaluation, MultiviewObjectHypothesis, MultiviewObjectHypothesisResult, MultiviewSpatialRegion, OBJECT_LOCALIZATION_INTERFACE, OBJECT_LOCALIZATION_STABILITY, ObjectHypothesis, ObjectLocalizationConfig, ObjectLocalizationPipeline, ObjectLocalizationResult, PairObjectLocalizationInput, RegionAssociation, ReducedHypothesisRecord, RegisteredSemanticEvidence, RegisteredBearingObservation, RegisteredBearingTrack, RegisteredPoseInconsistencyError, RegisteredRadialDepth, SEMANTIC_MATCH_PRIOR_INTERFACE, SEMANTIC_REGION_PROPOSALS_INTERFACE, SPHERICAL_ANCHOR_FUSION_INTERFACE, SemanticMatchPrior, SemanticMatchPriorConfig, SemanticQuery, SemanticConcept, SemanticRegionProposalConfig, SemanticRegionProposalResult, SemanticRegionObservation, SemanticTrackObservation, SphericalObjectHypothesisConfig, SphericalObjectHypothesisGenerator, SphericalObjectHypothesisMethod, SphericalObjectHypothesisMethodConfig, SphericalObjectHypothesisMethodResult, SphericalAnchorAffinity, SphericalAnchorCluster, SphericalAnchorClusterDecision, SphericalAnchorDecision, SphericalAnchorFusionConfig, SphericalAnchorFusionResult, SphericalAnchorPairCues, SphericalFOVAnchor, SphericalRegionMetrics, SpatialLocationHypothesis, SymmetricRegionTransfer, assign_region_associations, build_semantic_match_prior, cluster_region_association_matches, evaluate_spherical_region, fuse_registered_spherical_anchors, generate_spherical_fov_anchors, hypothesis_pair_affinity, hypothesis_view_consistency, triangulate_registered_bearing_track, localize_region_association, object_hypothesis_id, propose_region_associations, reduce_object_hypotheses, semantic_region_from_map, semantic_regions_from_map, spherical_region_mask, spherical_anchor_pair_affinity, spherical_solid_angle_weights, symmetric_spherical_region_transfer, transfer_spherical_region
spherical-reconstruction	Experimental	panorai.reconstruction	panorai-spherical-reconstruction/v1	BuildRegisteredReferenceSets, ColorizeSurface, DenseReconstructionWorkspace, EstimateMultiViewRanges, ExtractSurface, FuseTSDF, MultiViewRangeMap, MultiViewRangeSet, RegisteredReferenceSets, RegisteredSphericalScene, RegisteredSphericalView, SphericalBaselinePrior, SphericalCameraPose, SphericalDenseReconstructionResult, SphericalGlobalMapper, SphericalGlobalMapperOptions, SphericalPairwisePoseEdge, SphericalRangePrior, SphericalReconstructionDiagnostics, SphericalReconstructionResult, SphericalTrack, SphericalTrackObservation, SurfaceMesh, TSDFSpec, TSDFVolume, ValidateDenseReconstruction, colorize_surface, extract_surface, fuse_tsdf
metric-spherical-landmark-ba	Experimental	panorai.reconstruction.metric_landmark_ba	panorai-metric-spherical-landmark-ba/v1-experimental	METRIC_SPHERICAL_LANDMARK_BA_INTERFACE, MetricBearingObservation, MetricLandmarkBAOptions, MetricLandmarkBAReport, MetricLandmarkBAResult, MetricRadialRangePrior, MetricScaleGauge, MetricSphericalCamera, MetricSphericalLandmark, refine_metric_spherical_landmarks
spherical-dense-stereo	Experimental	panorai.stereo	panorai-spherical-dense-stereo/v1-experimental	SphericalDenseStereo, SphericalStereoOptions, SphericalStereoResult, classical, panoramic, colorize_spherical_range, estimate_spherical_range, render_spherical_stereo_result
classical-rectified-stereo	Experimental	panorai.stereo.classical	panorai-classical-dense-stereo/v1-experimental	ADCensusConfig, ADCensusCost, AbsoluteDifferenceCost, BirchfieldTomasiCost, BoxCostAggregation, BuildStereoCost, CensusCost, CensusMGMConfig, ClassicalStereoResult, CrossBasedCostAggregation, ELASConfig, ELASSupportPrior, MoreGlobalMatching, OpenCVStereoMatcher, PatchMatchConfig, PatchMatchStereo, RectifiedStereoGeometry, RectifiedStereoPair, RefineStereoDisparity, SelectStereoDisparity, SemiGlobalMatching, StereoBMConfig, StereoCostVolume, StereoDisparityCandidate, StereoSGBMConfig, StereoValidationConfig, TransformStereoCost, ad_census_pipeline, census_mgm_pipeline, disparity_labels, elas_pipeline, patchmatch_pipeline, stereo_bm_pipeline, stereo_sgbm_pipeline
registered-full-sphere-stereo-core	Experimental	panorai.stereo.panoramic	panorai-registered-full-sphere-stereo/v1-experimental	FullSphereMultiViewStereo, FullSphereStereoOptions, FullSphereStereoResult, OverlappingCubemapPlanner, PanoramicStereoPose, RegisteredPanoramaSet, StereoSourceView
panoramic-dense-stereo-workflow	Experimental	panorai.stereo.panoramic	panorai-panoramic-dense-stereo/v1-experimental	BackProjectStereoRange, DenseStereoPerPair, FuseStereoRange, PanoramicStereoInput, PanoramicStereoResult, PlanRectifiedStereoViews, ProjectStereoViewPairs, RectifiedStereoViewPlanner, RectifiedStereoViewSpec, ResolveStereoPose, ResolveStereoScale, SelectStereoViewPairs, SphereNativeStereo, SphericalStereoViewPairer, StereoPairDiagnostic, StereoViewDescriptor, StereoViewPairScore, StereoViewSelectionDiagnostic, StereoViewPlan, ValidatePanoramicStereo, spec_from_axes
spherical-slam	Experimental	panorai.slam	panorai-spherical-slam/v1	EquidistantFisheyeCamera, FisheyeRayProjection, SphericalImageFrame, SphericalIncrementalSLAM, SphericalIncrementalSLAMDiagnostics, SphericalIncrementalSLAMOptions, SphericalIncrementalSLAMResult, SphericalKeyframeSummary, SphericalLocalBAReport, SphericalMapObservation, SphericalMapPoint, SphericalSLAMDiagnostics, SphericalSLAMFrameSummary, SphericalSLAMOptions, SphericalSLAMPose, SphericalSLAMResult, SphericalTrackingResult, SphericalVisualSLAM
rotation-pair-calibration	Experimental	panorai.calibration	panorai-rotation-pair-calibration/v1-experimental	RotationCalibrationOptions, RotationCalibrationResult, calibrate_rotation_pair
robust-model-fitting	Experimental	panorai.model_fitting	panorai-robust-model-fitting/v1-experimental	HomographyModel, ModelFamily, RobustFitOptions, RobustFitResult, RotationModel, fit_homography, fit_rotation, robust_fit
spherical-estimators-auto-experimental	Experimental	panorai.estimators	panorai-estimator/auto-experimental	—

Surface	Stability	Canonical import	Contract	Public components or entries
public-catalog	Stable	panorai.catalog	panorai-catalog/v1	CatalogSurface, DocumentationContext, DocumentationLink, DocumentationSite, PublicCatalog, public_catalog
object-workflow	Stable	panorai	panorai-object-workflow/v1	EquirectangularImage, GnomonicFace, GnomonicFaceSet, SphericalTranspose
spatial-semantic-graph	Experimental	panorai.graph	panorai-spatial-semantic-graph/v1	GRAPH_INTERFACE, GRAPH_STABILITY, CallablePanorAiProvider, DepthProvider, DisabledProvider, EvidenceMode, EvidenceProvider, EvidenceSource, ExternalProvider, GraphArchiveError, GraphBuilderConfig, GraphEvent, GraphQueryFilters, GraphQueryResult, MultiViewPoseProvider, ObjectHypothesisNode, OracleProvider, PairFeatureEvidence, PairwiseLocalizationConfig, PairwiseLocalizationInput, PairwiseLocalizationResult, PoseProvider, PromotionPolicy, RegionAssociationConfig, RegionAssociationInput, RegionAssociationResult, RegionCorrespondenceCandidate, RegionCorrespondenceEdge, RelativePoseEdge, SemanticProvider, SemanticRegionNode, SpatialSemanticGraph, SpatialSemanticGraphBuilder, SpatialSemanticGraphQuery, SpatialSemanticPosterior, SphericalViewNode, TextImageEncoder, assign_region_correspondences, associate_regions, load_graph_archive, localize_region_pair, neighbors, propose_region_correspondences, query_embedding, query_spatial, query_text, replay_graph_archive, save_graph_archive, panorai.graph.association.RegionAssociationConfig, panorai.graph.association.RegionAssociationInput, panorai.graph.association.RegionAssociationResult, panorai.graph.association.assign_region_correspondences, panorai.graph.association.associate_regions, panorai.graph.association.propose_region_correspondences, panorai.graph.builder.GraphBuilderConfig, panorai.graph.builder.SpatialSemanticGraphBuilder, panorai.graph.localization.PairwiseLocalizationConfig, panorai.graph.localization.PairwiseLocalizationInput, panorai.graph.localization.PairwiseLocalizationResult, panorai.graph.localization.localize_region_pair, panorai.graph.observations.deterministic_id, panorai.graph.observations.feature_evidence_from_panorai, panorai.graph.observations.pose_edge_from_panorai, panorai.graph.observations.semantic_region_from_observation, panorai.graph.providers.CallablePanorAiProvider, panorai.graph.providers.DepthProvider, panorai.graph.providers.DisabledProvider, panorai.graph.providers.EvidenceProvider, panorai.graph.providers.ExternalProvider, panorai.graph.providers.MultiViewPoseProvider, panorai.graph.providers.OracleProvider, panorai.graph.providers.PoseProvider, panorai.graph.providers.SemanticProvider, panorai.graph.query.GraphQueryFilters, panorai.graph.query.GraphQueryResult, panorai.graph.query.SpatialSemanticGraphQuery, panorai.graph.query.TextImageEncoder, panorai.graph.query.neighbors, panorai.graph.query.query_embedding, panorai.graph.query.query_spatial, panorai.graph.query.query_text, panorai.graph.serialization.GraphArchiveError, panorai.graph.serialization.load_graph_archive, panorai.graph.serialization.replay_graph_archive, panorai.graph.serialization.save_graph_archive
spherical-ops-algorithms-gateway	Experimental	panorai.ops.spherical	panorai-spherical-algorithms/v1-experimental	algorithms
spherical-mask-rcnn-taxonomy	Experimental	panorai.experimental.deep_learning.mask_rcnn	panorai-spherical-mask-rcnn/v1-experimental	Backbone, BoxHead, MaskHead, MaskRCNNArchitecture, ProposalGenerator, RegionProposalNetwork, RoIAlign, RoIHeads, SphericalMaskRCNN, SphericalMaskRCNNPaper, SphericalMaskRCNNWorkflow, UnboundMaskRCNNError, paper, workflow
spherical-surface-open3d-adapter	Experimental	panorai.pcd	panorai-surface-open3d-adapter/v1-experimental	surface_mesh_to_open3d

Field	Contract
Application	Cross-application execution-strategy guide
Input	A spherical computer-vision task
Output	A justified sphere-native, projection-domain, or hybrid route
Strategy	Comparison of all three execution domains
API	Links to the existing application APIs; no new abstraction
Stability	Mixed; each linked capability states its own level
Dependencies	Core, with optional dependencies named by capability
Evidence	Contract map and links to real outputs, analytic fixtures, and method diagrams

How this application is composed

Every route begins with canonical geometry and explicit support. A sphere-native route composes panorai.core operations; a projection-domain route composes projectors with face-local methods and workflow fusion; a hybrid route uses both. Application wrappers package one validated composition and its diagnostics. They are conveniences over shared representations, not independent geometry stacks.

The README and tutorial index start from applications. This linked guide gives the complementary strategy view: it explains how the existing PanorAi APIs preserve spherical geometry while producing those application results.

The library uses two complementary strategies. Some operations evaluate rays, angular neighbourhoods, or residuals directly on the sphere. Others create gnomonic or cubemap images so mature planar algorithms can be reused, then map their results back to panorama pixels, unit bearings, or an ERP image. Many complete workflows are deliberately hybrid.

Choose a computer-vision task

Theme	What you can do	Typical input → result	Start here	Stability
Projection	Convert among ERP, gnomonic views, cubemaps, and unit rays; process and reconstruct view sets	panorama or rays → projected images, rays, or reconstructed ERP	Projection foundations: sphere, rays, views, samplers, and blenders	Stable geometry; Compatibility samplers
Image processing	Convolve, smooth, equalize, transform, detect edges, and build pyramids without treating the ERP seam as an image border	ERP → enhanced or transformed ERP	Tutorial: image processing on the sphere	Experimental
Deep learning depth	Acquire one pinned external Metric3D CNN and port its learned spatial layers without resizing the ERP	one native 2:1 ERP → radial range on the same lattice	Tutorial: resize-free spherical monocular depth	Experimental
Features and matching	Detect SIFT/ORB/AKAZE features in overlapping views, or detect DoG keypoints directly on the sphere; match panoramas	one or two ERPs → descriptors and matched unit bearings	Tutorial: spherical features and matching	Stable face-based core; spherical DoG Experimental
Registration and stitching	Recover a central rotation-only camera set from overlapping planar pinhole images and build one partial ERP	2+ overlapping images → typed panorama, camera geometry, contribution maps, diagnostics	Registration and panorama stitching	Experimental
Two-view geometry	Estimate relative rotation and translation direction and triangulate correspondences	matched bearings → relative pose and sparse points	Tutorial: spherical Essential matrix and two-view triangulation	Experimental
Multiview reconstruction	Build pair graphs and tracks, initialize cameras and points, and run spherical bundle adjustment	3+ panoramas/features → arbitrary-scale camera poses and sparse 3D map	Tutorial: from matched panoramas to a multiview reconstruction	Experimental
Dense stereo	Estimate radial range from two posed panoramas with spherical epipolar search and angular local evidence	two ERPs plus metric pose → dense radial range, validity, and confidence	Tutorial: dense stereo from two spherical images	Experimental
SLAM	Track an ERP or calibrated central-fisheye sequence, create keyframes, map, relocalize, and correct loops	ordered frames → online trajectory, keyframes, and map	Tutorial: spherical visual SLAM	Experimental
Spatial-semantic graph	Associate prepared semantic observations, preserve spatial uncertainty, maintain entity lineage, archive, and query	explicit multimodal evidence → replayable graph snapshot	Build and query a spatial-semantic graph	Experimental

These themes form an end-to-end path, but each can also be used independently:

How spherical images are handled

The execution domain is a mathematical choice; it is independent from whether the implementation is Python, NumPy, OpenCV, or first-party C++.

Execution domain	What happens	Best suited to	Output
Sphere-native	PanorAi works with unit rays, angular neighbourhoods, tangent frames, or spherical residuals without materializing a full planar view	convolution, gradients, equalization, rotations, bearing geometry, bundle adjustment	ERP-valued result, bearings, pose, or 3D structure
Projection-domain	PanorAi materializes gnomonic/cubemap images and applies an operation in those images	established planar detectors, descriptors, neural models, and view-local processing	projected images, or results mapped to ERP/bearings
Hybrid	A workflow combines sphere-native stages with small or full projected views	spherical DoG with tangent SIFT descriptors, face-based features followed by bearing geometry, projection/process/reconstruction	depends on the workflow

Neither strategy is universally better. Sphere-native processing preserves a constant angular interpretation across latitude and the longitude seam. Projection-domain processing makes well-tested planar algorithms available in locally low-distortion views. PanorAi’s job is to make the chosen route and its coordinate transformations explicit.

Projection

What you can do with projection

Use canonical pixel-centre geometry to convert ERP pixels, gnomonic pixels, cubemap pixels, and unit rays. Materialize one view or a batch, process views, and reconstruct an ERP with explicit support and validity. The result is useful on its own and is also the foundation for features, learned models, and camera geometry.

Typical input → result: ERP/rays/projected images → views, bearings, or ERP.

Start here: Projection foundations: sphere, rays, views, samplers, and blenders; use Quick start for the shortest path and Cubemap, batch, and autograd for batched processing.

Capability	Public entry point	Execution domain	What PanorAi adds
ERP pixels ↔ rays	erp_pixels_to_rays, rays_to_erp_pixels	Sphere-native coordinates	Canonical +X right, +Y up, +Z forward frame, pixel centres, seam and pole rules
ERP ↔ one tangent view	equirectangular_to_gnomonic, gnomonic_to_equirectangular, GnomonicProjector	Projection-domain	Forward/inverse geometry, FOV and roll, seam-aware sampling, support and validity
ERP ↔ cubemap	equirectangular_to_cubemap, cubemap_to_equirectangular	Projection-domain	Fixed face order/bases and deterministic edge/vertex ties
ERP ↔ arbitrary view set	EquirectangularImage.views, GnomonicFaceSet.reconstruct	Projection-domain	Face provenance, modality-aware sampling, inverse projection, explicit-support blending
Process and reconstruct	EquirectangularImage.process_views	Projection-domain	ERP → views → caller operation → backprojection → blend orchestration; the caller owns the view-local operation
Lift point detections	GnomonicFaceSet.map_points	Projection-domain + spherical mapping	Face pixels → ERP pixels and unit bearings, with provenance and deterministic angular NMS
Lift regions	GnomonicFaceSet.map_regions	Projection-domain + spherical mapping	Boxes/polygons → bearing boundaries; masks → compact ERP masks; solid-angle NMS and merge
Aggregate whole-view evidence	GnomonicFaceSet.map_views	Hybrid	Face footprints, overlap-normalized solid-angle scores, and explicitly normalized embeddings
Batched differentiable projection	Torch gnomonic/cubemap APIs	Projection-domain	NCHW batch/channel/dtype/device preservation and gradients with respect to input values
Sphere sampling layouts	cube, icosahedron, Fibonacci, and Compatibility samplers	Sphere-native planning	View centres/FOVs in the canonical spherical frame; a raster exists only when projection is requested
Blending	average, feather, Gaussian, closest, Huber, and research blenders	Hybrid reconstruction	Combines already backprojected ERP layers using explicit masks; zero/black remains valid data
RGB, range, labels, masks, CAM/logits	typed raster-field workflow	Projection-domain	Radial range stays radial; categories use nearest; continuous channels declare components; support and validity remain separate

Quick start and object workflow

The functional and object APIs share the same geometry. project materializes a gnomonic image; back_project reverse-maps it onto explicit ERP support. process_views is therefore a projection-domain workflow, even if the caller uses a filter whose name resembles a spherical operation. Use a panorai.image_processing spherical function instead when the desired neighbourhood must remain angularly constant everywhere.

Cubemap, batch, and learned-model workflow

Cubemap and arbitrary-N views are first-class outputs. A model can operate on those images and PanorAi can reconstruct its predictions into ERP. PanorAi owns the projection, geometry, masks, and fusion; the caller or model owns the semantics of the projected-image operation.

Image processing

What you can do with image processing

Apply familiar OpenCV-style filtering concepts with angular rather than raw ERP-pixel neighbourhoods. The current image-processing API is sphere-native: it does not split the panorama into gnomonic tiles and stitch the result.

Typical input → result: ERP image → filtered, enhanced, transformed, edge, or pyramid ERP data.

Start here: Tutorial: image processing on the sphere.

Capability	Public entry point	Execution domain	What PanorAi adds
General 2D correlation	spherical_filter2d	Sphere-native spherical convolution	Kernel taps live in every ray’s local east/north tangent frame, use constant angular spacing, and cross the ERP seam
Averaging and Gaussian smoothing	spherical_box_blur, spherical_gaussian_blur	Sphere-native spherical convolution	Angularly consistent support instead of latitude-dependent ERP-pixel support
First/second derivatives	spherical_sobel, spherical_gradient, spherical_laplacian	Sphere-native spherical convolution	East/north tangent derivatives, magnitude/orientation, and seam-consistent second derivatives
Median smoothing	spherical_median_blur	Sphere-native nonlinear neighbourhood	Median over a sampled geodesic tangent neighbourhood
Bilateral smoothing	spherical_bilateral_filter	Sphere-native nonlinear neighbourhood	Angular spatial weights plus radiometric weights
Edge detection	spherical_canny	Sphere-native hybrid of convolution and nonlinear stages	Spherical Gaussian/gradients followed by geodesic suppression and seam-connected hysteresis
Histogram equalization	spherical_equalize_histogram	Sphere-native global statistics	Optional solid-angle weighting avoids over-counting the dense ERP polar rows
Superpixels	spherical_slic	Sphere-native iterative clustering	Concentrated-local geodesic/Lloyd seeds, rotation-covariant textured seeds, an equal-area uniform fallback, great-circle spatial distance, solid-angle centroid updates, and seam-periodic connectivity
Rotation	spherical_rotate	Sphere-native direct ray sampling	Inverse maps output rays through a proper 3×3 rotation, with no artificial image border
Resize	spherical_resize	Sphere-native direct ray sampling	Samples exact output ERP pixel-centre rays and treats longitude as periodic
Gaussian pyramid	spherical Gaussian pyramid API	Sphere-native convolution plus direct ray resizing	Angular prefiltering before each scale change
Laplacian pyramid	spherical Laplacian pyramid/reconstruction API	Sphere-native convolution plus direct ray resizing	Sphere-aware residual levels with tested reconstruction

This distinction matters: linear convolution, nonlinear neighbourhood filters, global histogram statistics, and geometric transforms are all sphere-native, but only the linear kernel stages are spherical convolution.

Features and matching

What you can do with features and matching

Extract local features from a panorama, deduplicate overlap geometrically, match descriptors, and obtain correspondence pairs as unit bearings. Choose a stable projection-domain frontend or the Experimental sphere-native DoG detector according to where keypoint selection must happen.

Typical input → result: one/two ERPs → spherical feature sets, descriptors, matches, and paired unit bearings.

Start here: Tutorial: spherical features and matching; see Spherical features and matching for task-oriented recipes.

Capability	Public entry point	Execution domain	What PanorAi adds
Stable SIFT/ORB/AKAZE frontend	spherical feature façade with OpenCVFeatureBackend	Projection-domain detection and description	Overlapping gnomonic views, masks, mapping back to ERP pixels/unit bearings, angular deduplication, provenance
Spherical DoG + SIFT	SphericalDoGSIFTPipeline	Hybrid: sphere-native Gaussian/DoG detection, one local tangent patch per descriptor	Seam-aware keypoint selection, angular scale, tangent orientation, and standard spherical result objects
Descriptor matching	BF/FLANN through the spherical matcher	Descriptor-domain, then sphere-native correspondence handling	Match provenance, angular deduplication, and aligned bearing pairs
Multiscale visual context	Experimental wide/local view routing	Projection-domain frontend, sphere-native deduplication	Explicit scale/view provenance and spherical NMS
COLMAP export	PyCOLMAP export adapter	Projection-domain virtual-camera representation	Consistent gnomonic camera rigs and feature/match export

PanorAi does not reimplement the SIFT descriptor. In the Experimental route, PanorAi detects DoG extrema on a spherical scale space and materializes only a small tangent patch at each selected bearing; OpenCV computes the SIFT descriptor on that patch. In the stable route, both detection and description happen in overlapping gnomonic views.

Registration and stitching

What you can do with overlapping planar captures

Use PanoramaStitcher to detect and match evidence, estimate missing shared horizontal FOV when observable, recover a connected set of SO(3) camera orientations, warp every source to ERP and fuse valid contributions. The Application is a convenience over public calibration, model-fitting and Pipeline components.

Typical input → result: two or more overlapping planar pinhole images → partial typed ERP, recovered camera geometry and diagnostics.

Start here: Registration and panorama stitching.

Stage	Public entry point	Domain	Current boundary
Feature evidence	DetectStitchingFeatures	planar sources	masked SIFT or ORB
Overlap discovery	DiscoverOverlaps	descriptor pairs	symmetric ratio-tested matching
Calibration and registration	CalibrateAndRegister	pixels → pinhole rays → SO(3)	centred pinhole model; known or observable shared horizontal FOV
ERP warp	WarpRegisteredImages	camera rays → ERP	values, support and validity transformed independently
Photometric compensation	CompensateExposure	valid ERP overlap	none or one positive gain per image
Fusion	BlendStitchingPanorama	aligned ERP contributions	average or feather; no seam finder or multiband blend

The first image defines the world gauge. Translation, strong parallax, rolling shutter, disconnected overlap, fisheye distortion and unobservable FOV must fail rather than acquiring plausible but unsupported geometry.

Two-view geometry

What you can do with two views

Turn matched panorama features into a relative camera pose without pretending that ERP pixel coordinates belong to a pinhole camera. The estimator consumes unit bearings, scores hypotheses with spherical residuals, checks cheirality, and reports degeneracy diagnostics.

Typical input → result: paired unit bearings → panorama-frame rotation, unit translation direction, inliers, quality diagnostics, and arbitrary-scale triangulated points.

Start here: Tutorial: spherical Essential matrix and two-view triangulation.

Stage	Public entry point	Execution domain	What PanorAi adds
Correspondence adapter	matched spherical feature rows → bearing pairs	Sphere-native coordinates	Removes all virtual-face pixel frames before geometry
Essential pose	relative-pose estimator	Sphere-native bearing geometry	Five-ray hypotheses, tangent-Sampson scoring, refinement, cheirality, quality and degeneracy reporting
Triangulation example	tutorial helper, not public API	Sphere-native bearing geometry	Demonstrates the canonical frame, positive depth, and arbitrary-scale convention

No projected image is required after the frontend has produced unit bearings. If features came from gnomonic views, the complete workflow is hybrid: view projection for local evidence, then sphere-native two-view geometry.

Multiview reconstruction

What you can do with multiple views

Connect pairwise matches into a view graph and conflict-free tracks, initialize camera rotations/positions and 3D points, then jointly optimize spherical reprojection residuals. Failures are represented explicitly instead of being hidden behind partial maps.

Typical input → result: features and matches from 3+ panoramas → optimized panorama poses, sparse 3D points, observations, and diagnostics at arbitrary global scale.

Start here: Tutorial: from matched panoramas to a multiview reconstruction; see Estimate spherical pose and reconstruct multiple panoramas for operational recipes.

Stage	Public entry point	Execution domain	What PanorAi adds
Pair graph	reconstruction builder/mapper	Sphere-native bearing geometry	Quality-gated pose edges with accepted/rejected reasons and panorama identity
Tracks	multiview track construction	Sphere-native observation graph	Conflict-free tracks with at most one observation per panorama
Initialization	reconstruction mapper	Sphere-native bearing geometry	Panorama-frame pose initialization, point triangulation, and explicit gauge conventions
Bundle adjustment	spherical reconstruction options/mapper	Sphere-native optimization	Tangent-plane log-map residuals between predicted and measured bearings, filtering, Jacobians, and diagnostics
COLMAP alternative	PyCOLMAP export	Projection-domain virtual cameras	PanorAi owns export geometry; PyCOLMAP/COLMAP owns the downstream reconstruction

The native PanorAi mapper never reconstructs ERP images during optimization. Its output is geometry. The COLMAP alternative intentionally uses a projected virtual-camera rig because that is the external tool’s camera model.

Spherical dense stereo

What you can do with two posed panoramas

Estimate radial range directly on the ERP lattice from two central panoramas and a metric relative pose. Inverse-range hypotheses follow spherical epipolar curves; no full gnomonic image set is materialized.

Typical input → result: two aligned ERP panoramas plus metric $R,t$ → radial range, validity, confidence, cost, and hypothesis-index maps.

Start here: Tutorial: dense stereo from two spherical images; see Direct spherical dense stereo: geometry and optimization for the objective and limitations.

Stage	Public entry point	Execution domain	What PanorAi adds
Plane sweep	estimate_spherical_range	Sphere-native bearing/range geometry	Pose-driven spherical epipolar hypotheses and seam-safe ERP sampling
Adaptive range pyramid	SphericalStereoOptions.pyramid_levels	Sphere-native coarse-to-fine range search	Broad absolute inverse-range sweep followed by uncertainty-aware normalized local offsets; supplied high-resolution $R,t$ is unchanged
Appearance normalization and gradients	SphericalStereoOptions.filter_backend	Sphere-native tangent neighbourhoods	Angular box support and east/north derivatives instead of rectangular ERP filters
Local cost filtering	dense stereo estimator	Sphere-native spherical convolution	Constant angular support across latitude; C++/NumPy parity through the image-processing contract
Path aggregation and consistency	dense stereo estimator	ERP optimization over spherical evidence	Four edge-aware paths, sub-hypothesis refinement, explicit validity and bidirectional checks

OpenCV supplies the bilinear and nearest remapping primitive. The local filters use PanorAi spherical convolution directly; they do not project to gnomonic views and reconstruct an ERP.

SLAM

What you can do with SLAM

Estimate an online trajectory from a central ERP or calibrated central-fisheye sequence, select keyframes, create and optimize a local map, relocalize, and apply loop corrections. PanorAi exposes status and diagnostics alongside the pose so tracking failures are inspectable.

Typical input → result: ordered central-camera frames or caller-provided spherical feature sets → per-frame pose/status, keyframes, map observations, loop information, and an arbitrary-scale trajectory.

Start here: Tutorial: spherical visual SLAM; see Run spherical visual SLAM for configuration and lifecycle details.

Workflow	Public entry point	Execution domain	What PanorAi adds
ERP incremental SLAM	SphericalIncrementalSLAM.add_frame	Hybrid: default gnomonic feature frontend, then sphere-native tracking/mapping	Central panorama bearings, immediate status/pose, keyframes, spherical 2D–3D refinement, local BA, relocalization and loop correction
Precomputed feature route	SphericalIncrementalSLAM.add_features	Sphere-native after caller-owned features	Skips image projection while retaining feature, match, pose, and map contracts
Calibrated central-fisheye SLAM	SphericalVisualSLAM	Hybrid: OpenCV features on the physical fisheye image, calibrated ray inversion, then sphere-native graph/reconstruction	Equidistant calibration, central-camera bearings, graph diagnostics, and arbitrary-scale trajectory
Dual-fisheye input	explicit validation	Not modeled as one central sphere	Rejects the invalid assumption that two physical camera centres are one central camera

SLAM produces camera and map geometry, not a filtered or reconstructed ERP. Its spherical specificity comes from calibrated unit bearings and tangent-space residuals after the image frontend has produced observations.

Spatial-semantic graph

What you can do with prepared multimodal evidence

Associate feature-indexed semantic regions across views, estimate angular, scale-free, or metric spatial support, maintain conservative entity lifecycle and lineage, persist a deterministic snapshot, and query it with decomposed scores and evidence provenance.

Typical input → result: explicit views, regions, matches/inliers, pose, and optional depth/scale → replayable spatial-semantic graph.

Start here: Build and query a spatial-semantic graph.

Stage	Public entry point	Boundary
Evidence declaration	EvidenceSource, provider protocols, observation models	Records modality, frame, units, split, access role, calibration, provenance, and correlation group
Pair association	panorai.graph.association	Reuses pose-supporting features and inliers; optional rematching cannot rewrite pose
Pair localization	panorai.graph.localization	Localizes exactly one region pair and creates no persistent identity
Entity lifecycle	SpatialSemanticGraphBuilder	Consumes results only; one view cannot confirm an entity; merge/split retain lineage
Persistence and query	JSONL/NPZ archive and SpatialSemanticGraphQuery	Content-addressed arrays, checksum validation, explicit encoder/frame/unit compatibility

The graph is not an end-to-end perception facade. Applications explicitly invoke their providers, inspect the results, and decide which method-usable evidence enters the builder. Evaluation-only evidence fails closed.

C++ acceleration is a separate axis

Sphere-native versus projection-domain describes the mathematics. C++ versus NumPy/Python/OpenCV describes execution. A native kernel does not turn a projected algorithm into a spherical one, and a NumPy implementation can still be fully sphere-native.

Area	Mathematical route	C++ acceleration	Reference/fallback	Effect on semantics
Projection	Projection-domain	First-party fused arbitrary-N ERP → gnomonic sampling, selective bilinear cubemap → ERP, and compatible Gaussian view reconstruction	NumPy engine; Torch has its own differentiable route	None: same coordinates, interpolation, masks, and result contract
Image processing	Sphere-native	First-party spherical_filter2d kernel for compatible float32/float64 NumPy HW/HWC inputs; linear filters and pyramids can reuse it	NumPy spherical sampler selected with backend="numpy" or when native is unavailable	None: same angular taps and output layout
Experimental deep learning	Sphere-native learned neighbourhoods	Differentiable Torch spherical Conv2d, exact-adjoint ConvTranspose2d, and explicit resize-convolution decoder policy	Direct ports preserve external parameters; transformed upsamplers and planar/cubemap controls remain explicit	Porting preserves declared provenance and output lattice, not perspective-domain calibration or spherical invariance
Dense stereo local evidence	Sphere-native	The same first-party spherical-convolution kernel accelerates local normalization, east/north gradients, and batched cost-volume filtering with hypotheses as channels	NumPy route via filter_backend="numpy"; OpenCV remains the remapping primitive	None: the range search, angular support, and result contract are unchanged
Features/descriptors	Projection-domain or hybrid	OpenCV’s C++ implementation provides SIFT/ORB/AKAZE, BF, and FLANN; it is not a PanorAi native kernel	PanorAi orchestrates geometry and spherical metadata	OpenCV owns descriptor/detector semantics; PanorAi owns mapping and provenance
Two-view geometry	Sphere-native	First-party five-point polynomial coefficients and tangent-Sampson residual kernels	NumPy path via compute_backend="numpy"	None: same estimator policy and result model
Multiview reconstruction	Sphere-native	First-party spherical-BA residual and local Jacobian blocks	NumPy/SciPy path via bundle_compute_backend="numpy"	None: same tangent residual and optimization contract
SLAM	Hybrid	Reuses accelerated frontend, estimator, and bundle-adjustment components when their conditions apply; no separate native SLAM engine	Python orchestration over the same component contracts	None: backend choice does not redefine the SLAM model

"auto" selects an available compatible first-party native path where that public option exists. Explicit NumPy modes are useful for reference testing; native and reference paths are expected to satisfy the same public numerical contract.

Responsibility and stability boundary

PanorAi owns spherical coordinates, projection, sampling, support/validity, angular neighbourhoods, view orchestration, unit-bearing conversion, spherical residuals, provenance, and result objects. OpenCV owns its detector, descriptor, and matcher implementations. A caller-supplied model owns its inference semantics. PyCOLMAP/COLMAP owns reconstruction after export.

The geometry and face-based feature core are Stable unless the API reference says otherwise. Spherical image processing, spherical DoG, relative pose, deep-learning portability, native multiview reconstruction, multiscale routing, SLAM, and the spatial-semantic graph are currently Experimental. Consult API stability in 3.x before treating an Experimental signature as a long-term compatibility contract.