Fine-tuning Segement Anything (SAM) for Mastcam-Z Mission Operations

Fine-tuning Segement Anything (SAM) for Mastcam-Z Mission Operations

In Fall 2023, I fine-tuned the Segment Anything Model (SAM) to produce robust terrain segmentation masks for Mars 2020 Rover imagery in Mastcam-Z and Navcam images to support enhanced image stitching and spectral contrast in a Unity based observation planning tool used by the Mastcam-Z Operations Engineers and Scientists called Viewpoint. The fine-tuned model has been deployed since fall 2023 without requiring retraining to date (July 2025), producing segmentation masks upon image downlink to support daily operations shifts. Fine-tuning used pytorch and lightning on a A-100 80GB gpu with the huggingface implementation of SAM and Weights and Biases for experiment tracking.

Avatar
Andrew M. Annex
Senior Science Systems Engineer

My research interests include the applications of deep learning to planetary geology.