The Moon's many faces: a single unified transformer for multimodal lunar reconstruction
Tom Sander · Moritz Tenthoff · Kay Wohlfarth · Christian Wöhler
TU Dortmund University, Image Analysis Group · ISPRS Journal of Photogrammetry and Remote Sensing, 2026
Abstract
Multimodal learning is an emerging research topic across multiple disciplines but has rarely been applied to planetary science. We identify that reflectance-parameter estimation and image-based 3D reconstruction of lunar images can be formulated as a multimodal learning problem.
We propose a single, unified transformer trained to learn shared representations between multiple sources — grayscale images, digital elevation models, surface normals and albedo maps — supporting flexible translation from any input modality to any target. Predicting DEMs and albedo maps from grayscale images simultaneously solves 3D reconstruction of planetary surfaces and disentangles photometric parameters from height information.
Our results demonstrate that the foundation model learns physically plausible relations across these four modalities. Adding more input modalities in the future will enable tasks such as photometric normalization and co-registration.