RPP

RPP: Rover Photogrammetry Pipeline

tests docs PyPI License DOI

Mars rover images to true-scale 3D, across many rover stops.

Every time a Mars rover stops, it sees the terrain from a new viewpoint. The mission’s own stereo products use one stop at a time. RPP combines the images of nearby stops: it turns the public images of NASA’s Planetary Data System (PDS) into an aligned structure-from-motion project, in meters, with north and up known.

What RPP does

RPP takes a block, a set of rover stations within a few tens of meters of each other, through five steps:

  1. Images. PDS radiance products (*.IMG) become 16-bit linear images that keep their PDS names, with the colors of the cameras brought into one color space.
  2. Masks. A neural network marks the rover hardware and the rover’s shadow in every image, so that the matcher ties images together on the terrain. The sky is not masked.
  3. Cameras and stations. RPP applies its own in-flight camera models, which vary with temperature, focus, zoom and sol, or converts the mission’s CAHV, CAHVOR and CAHVORE models. Each station is placed from the rover’s localization in a local east-north-up frame.
  4. Alignment. COLMAP features and matching, then RPP’s weighted bundle adjustment with position and attitude priors and the stereo rigs.
  5. Health report and exports. A report says how well the block aligned. The block opens in COLMAP, Agisoft Metashape and Nerfstudio.

From an aligned block, RPP can also make station products: a terrain mesh around one rover station, depth and normal maps for every image, a stereo 360° panorama and a 3D station model. RPV shows them.

Supported cameras

Rover Camera Status in 1.0
Mars 2020 Perseverance Navcam (left, right) calibrated in flight: temperature, sol, stereo rig
Mars 2020 Perseverance Mastcam-Z 34 mm and 110 mm calibrated in flight: focus, focus-motor backlash state
Mars 2020 Perseverance Mastcam-Z 26, 48, 63, 79, 100 mm first models, from one to a few blocks each
Mars 2020 Perseverance Front and rear Hazcam first models
MSL Curiosity Navcam, front Hazcam experimental (a research study in the source repository)

What is validated and what is experimental has the details and the numbers.

Install

RPP needs Python 3.10 or later. Install it into a conda environment of its own (Miniforge, Miniconda or Anaconda). The commands are the same in a Windows Anaconda Prompt, PowerShell, and a Linux or macOS shell.

conda create -n rpp -c conda-forge python=3.12 -y
conda activate rpp
pip install "rover-photogrammetry[pipeline]"
rpp doctor

The distribution is rover-photogrammetry; the import name and the command are rpp. rpp doctor reports what is installed and what is missing.

To work on the code, or to run the notebooks and the research studies, install from the repository instead:

conda create -n rpp -c conda-forge python=3.12 git -y
conda activate rpp
git clone --depth 1 https://github.com/ctate7163/rpp.git
cd rpp
pip install -e ".[all]"
rpp doctor

Notes:

Installation covers the environment variables, offline use and the GPU setup on Windows.

First run

This downloads a small Mars 2020 example (Rockytop, sols 461-464: 36 PDS products) and aligns it.

rpp example fetch
rpp run rockytop_example --zcam

The results are in ~/rpp_scapes/. The block folder holds the processed images, the COLMAP project (colmap/sparse/cahv_ba/), the health report (colmap/health/) and files that open the block in the COLMAP GUI.

Next steps:

The repository

Path Content
src/rpp/ the installed package: the pipeline, the camera models, the station products, the viewer page
src/rpp_studies/ research studies that use the pipeline: camera calibration, reconstruction error, mask training, site maps, Curiosity. Not in the PyPI package
notebooks/ two examples of the pipeline, and one notebook per study
scripts/ batch runs over many sites, and Windows batch files
docs/ the documentation site
paper/ the software paper
tests/ the test suite (pytest -q)

Roadmap

1.0 is the validated pipeline with an experimental viewer. The roadmap lists what comes next: measurement and source look-up in the viewer, downloads with a DOI, more Curiosity support, and a catalog of stations along the traverse.

Contributing and help

Questions, bug reports and requests are welcome in the issue tracker. See CONTRIBUTING.md for how to set up a development environment and run the tests.

Citing

If you use RPP, please cite it as CITATION.cff describes. GitHub shows the citation under “Cite this repository”.

Credits and license

RPP is written by Christian Tate at Malin Space Science Systems and released as open source, under the Apache-2.0 license, with the company’s approval.

All images RPP uses are public PDS products. Image credits: Navcam and Hazcam NASA/JPL-Caltech; Mastcam-Z NASA/JPL-Caltech/ASU/MSSS. The rover model is derived from NASA’s “Mars Perseverance Rover, 3D Model” (NASA/JPL-Caltech). The viewer includes three.js (MIT license). See NOTICE.

Until version 0.98 the package was called MPPP (Mars Photogrammetry Preprocessing Pipeline). Work folders made by MPPP are read as they are.