Zero-shot satellite video object tracking

SatSAM2

Motion-constrained tracking with promptable SAM2 and Kalman priors.

Ruijie Fan1,2,*, Junyan Ye2,*, Huan Chen2, Zilong Huang2, Xiaolei Wang2, Weijia Li1,†

1Tsinghua Shenzhen International Graduate School, Tsinghua University 2School of Geospatial Engineering and Science, Sun Yat-sen University

*Equal contribution   †Corresponding author

Paper Dataset Explore the method Code coming soon
Zero-shot No fine-tuning or weight updates
1,500+ MVOT video sequences
157,900 Annotated frames
67.23% AUC on OOTB

Why SatSAM2

Motion and state reasoning for satellite targets.

Directly applying SAM2 remains difficult under severe occlusion, subtle appearance changes, and multiple visually similar objects.

Existing satellite video tracking methods often generalize poorly across scenes, require scenario-specific training, and lose targets under occlusion or appearance ambiguity. SatSAM2 adapts the promptable foundation model SAM2 to satellite video object tracking in a strict zero-shot regime.

The Kalman Filter-based Constrained Motion Module (KFCMM) combines temporal motion cues with historical trajectories to stabilize localization and suppress drift. The Motion-Constrained State Machine (MCSM) regulates tracking states using segmentation confidence and motion consistency.

SatSAM2 was evaluated on OOTB, MVOT, SatSOT, and SAT-MTB using one fixed hyperparameter set. It achieved the best AUC on OOTB and SatSOT, while the new MatrixCity Video Object Tracking (MVOT) benchmark provides more than 1,500 sequences and 157,900 annotated frames.

Method

A three-part tracking loop

The framework encodes approximate scale invariance, smooth near-linear target motion, and the absence of abrupt camera-induced transformations.

Framework The SAM2 Observer generates candidate masks, KFCMM provides constrained motion estimates, and MCSM selects an estimation strategy according to the current tracking state.
01

SAM2 Observer

Fuses frame features with long-term memory and generates three candidate masks conditioned on the current box prompt.

02

KFCMM

Updates target dynamics from high-confidence boxes and preserves target area under the near-orthographic imaging assumption.

03

MCSM

Governs five states: Uninitialized, Initialized, Stabilizing, Stable, and ResetStable for conservative recovery after degradation.

Results

Best AUC on OOTB and SatSOT in a strict zero-shot regime.

All four benchmarks use the same hyperparameters, so the measured gains reflect framework design rather than dataset-specific tuning.

Qualitative comparison On sequences with visually similar targets and temporary occlusions, SatSAM2 maintains identity and re-acquires targets after they pass under bridges or elevated structures.

67.23% AUC on OOTB. SatSAM2 improves by 3.71 points over SAM2 and by 7.77 points over the strongest supervised competitor in the comparison.

68.17% AUC on MVOT. Vanilla SAM2 and SAMURAI remain below 53% AUC, isolating the value of motion-aware and state-aware integration.

54.72% AUC on SAT-MTB. With no hyperparameter changes, SatSAM2 outperforms the supervised RTS baseline and both zero-shot baselines on all three metrics.

Motion analysis

State dynamics explain occlusion recovery.

On sequence car_30, complete occlusion between frames 436 and 496 produces an uncertainty spike that switches the tracker from Update to pure Prediction.

KFCMM state dynamics During complete occlusion, position variance rises and MCSM prioritizes the motion prior. The accumulated velocity preserves a smooth path, while the geometric constraint prevents bounding-box collapse.
Hyperparameter sensitivity on OOTB Performance peaks at αkf = 0.2 and remains robust for αkf between 0.0 and 0.3. The stability threshold has a broad plateau for τh between 0.1 and 0.4, peaking at 0.3.

Tracking demos

Tracking under scale change, clutter, and occlusion.

Vehicle 01Dense urban road network
Vehicle 02Small target under scale change
Vehicle 03Wide-area satellite view
Vehicle 04Curved trajectory
Rail 01Long target and linear motion
Rail 02Complex rail-side background

New benchmark

MVOT

MatrixCity Video Object Tracking

MVOT provides controlled variation in viewing angle, illumination, and occlusion. It complements rather than replaces real satellite datasets, whose sensor artifacts and atmospheric effects are difficult to reproduce synthetically.
Download MVOT on Hugging Face

1,500+
Sequences
157,900
Frames
3-10×
More frames

Citation

Cite SatSAM2

SatSAM2 is available as arXiv:2511.18264. Version 3 was posted on April 23, 2026.

BibTeX 
@misc{fan2026satsam2,
  title        = {SatSAM2: Motion-Constrained Video Object Tracking
                  in Satellite Imagery using Promptable SAM2 and Kalman Priors},
  author       = {Fan, Ruijie and Ye, Junyan and Chen, Huan and
                  Huang, Zilong and Wang, Xiaolei and Li, Weijia},
  year         = {2026},
  eprint       = {2511.18264},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url          = {https://arxiv.org/abs/2511.18264}
}