Tutorial: spherical visual SLAM¶

Field	Contract
Application	Reconstruct and navigate
Input	Ordered ERP or calibrated central-fisheye frames
Output	Arbitrary-scale trajectory, keyframes, and map diagnostics
Strategy	Hybrid frontend and sphere-native geometry
API	`panorai.slam`
Stability	Experimental
Dependencies	`panorai[slam]`
Evidence	Integration fixtures and explicit tracking states

How this application is composed¶

SLAM reuses the feature, matched-bearing, relative-pose, triangulation and spherical bundle-adjustment components, then adds temporal state, keyframes, local-map culling, relocalization and loop correction. Use the session wrapper: the composition is shown here for traceability, but manually reproducing its state machine would bypass its failure and provenance contracts.

This tutorial closes the sequence from panorama pixels to an incremental camera trajectory. PanorAi exposes two Experimental central-camera routes: one for already stitched equirectangular panoramas and one for a calibrated equidistant fisheye lens. Both return arbitrary-scale visual trajectories.

See the SLAM theme in the spherical computer-vision guide. In particular, SLAM does not filter an ERP or reconstruct one: the default ERP frontend materializes gnomonic faces for feature extraction, maps detections to unit bearings, and performs tracking and mapping in bearing space.

PanorAi-specific: central-camera ray conversion, panorama-frame tracking, spherical 2D–3D refinement and bundle adjustment, explicit tracking states, relocalization/loop evidence and arbitrary-scale trajectory diagnostics.

1. Incremental tracking from central ERP frames¶

from panorai.slam import SphericalIncrementalSLAM

session = SphericalIncrementalSLAM.from_preset(
    "sift-flann",
    face_sampler="cube",
    face_fov_deg=100.0,
    face_shape_hw=(320, 320),
    max_features=2048,
)

for frame_id, timestamp_s, panorama_rgb in decoded_erp_frames:
    tracking = session.add_frame(
        panorama_rgb,
        timestamp_s=timestamp_s,
        frame_id=frame_id,
    )
    if tracking.pose is not None:
        R_world_to_panorama = tracking.pose.R
        center_world = tracking.pose.center
    else:
        log_tracking_failure(frame_id, tracking.reasons)

result = session.finish()

The frontend follows the Stable feature route from Tutorial: spherical features and matching: ERP → overlapping gnomonic views → OpenCV descriptor extraction → panorama bearings. No projected image is blended back into ERP. Accepted geometric inliers seed the local map; spherical 2D–3D refinement and local bundle adjustment minimize angular tangent residuals.

Never invent a pose for tracking.state == "lost". Inspect correspondence count, parallax, active local points and reasons on every frame. The final result can contain a useful partial map while still declaring that the full trajectory was not completed.

2. Supply precomputed spherical features¶

Use add_features() when another component already owns a SphericalFeatureSet. This route starts directly from unit bearings and avoids re-projecting an ERP inside SLAM:

features = feature_pipeline.extract(panorama_rgb, panorama_id=frame_id)
tracking = session.add_features(features, timestamp_s=timestamp_s)

The public feature objects are copied. A reset begins a new session with the same configuration.

The CI-sized lifecycle example is executable:

from panorai.slam import (
    SphericalIncrementalSLAM,
    SphericalIncrementalSLAMOptions,
)

first_features = feature_pipeline.extract(feature_rgb, panorama_id="frame-000")
slam = SphericalIncrementalSLAM(
    feature_pipeline=feature_pipeline,
    relative_pose_estimator=pose_estimator,
    options=SphericalIncrementalSLAMOptions(
        min_frame_features=1,
        global_refine_on_finish=False,
    ),
)
first_tracking = slam.add_features(first_features, timestamp_s=0.0)
assert first_tracking.state == "initializing"
assert first_tracking.pose is not None
partial_trajectory = slam.finish()
assert partial_trajectory.scale == "arbitrary"
assert not partial_trajectory.success  # one frame cannot define a trajectory

3. Track one calibrated fisheye lens¶

from panorai.slam import EquidistantFisheyeCamera, SphericalVisualSLAM

camera = EquidistantFisheyeCamera.from_kalibr_yaml(
    "kalibr_imucam_chain.yaml",
    camera_id="cam0",
)
session = SphericalVisualSLAM(camera)

for frame_id, timestamp_s, image in decoded_frames:
    session.add_frame(image, timestamp_s=timestamp_s, frame_id=frame_id)

result = session.finish()

Here PanorAi inverts calibrated equidistant fisheye pixels directly into unit bearings. It does not pretend that two non-coincident fisheye lenses form one central camera. Use a single calibrated lens, or a stitched ERP only when its central-camera approximation is justified.

The fisheye frontend builds a temporal graph as frames arrive. Rotation averaging, tracks, positioning, triangulation and spherical bundle adjustment run when finish() freezes the graph.

4. Understand the output boundary¶

Output	Meaning
`pose.R`	world-to-panorama or world-to-camera rotation under the documented frame
`pose.center`	camera centre in the arbitrary-scale world frame
tracking state	explicit initializing/tracking/keyframe/relocalized/lost status
map observations	bearing-linked visual landmarks, never reconstructed ERP pixels
diagnostics	accepted/rejected edges, inliers, parallax, reprojection and optimization evidence

Visual-only translation scale is unobservable. Metric evaluation requires an external scale or similarity alignment. Current limits include rolling shutter, IMU fusion, metric scale, dynamic-scene modelling, exact generalized dual- fisheye geometry and bounded-memory marginalization.

Continue with Run spherical visual SLAM for operational guidance and Spherical visual SLAM for the complete Experimental API and limitations.