More research

Learning a Generalist Cloud Removal Prior from Heterogeneous Observations

Jeonghyeok Do Munchurl Kim†

Korea Advanced Institute of Science and Technology (KAIST), South Korea

† Corresponding authorehwjdgur0913@kaist.ac.krmkimee@kaist.ac.kr

arXiv preprint, 2026

Abstract

Cloud removal methods are typically specialized to individual datasets and input configurations, limiting reuse across sensors, spectral bands, and observation settings. We introduce GeoCR, a generalist model that unifies RGB-only-based CR and multispectral-based CR from single- or multi-temporal cloudy observations, with optional SAR guidance, within a single network.

To accommodate different spectral and sensing domains, compact input and output stems extend a pretrained RGB autoencoder while keeping its encoder and decoder trunks frozen. This shared latent interface enables a single flow transformer to jointly model clean RGB and non-RGB latents, conditioned on separate cloudy-observation streams and optional SAR tokens. Through joint pretraining on the training splits of ten datasets comprising 883,331 cloud-free target images, GeoCR learns a shared cloud removal prior across these heterogeneous configurations.

The same pretrained checkpoint supports direct inference without dataset-specific fine-tuning and efficient adaptation through low-rank adaptation (LoRA). We evaluate GeoCR against general image restoration and cloud removal methods on test splits of the contributing datasets under full-band and RGB-only settings. GeoCR achieves the best FID and DISTS on full-band SEN12MS-CR and Sen2_MTC_New and RGB-only CUHK-CR2, outperforming existing models and demonstrating the effectiveness of a reusable generative model across diverse settings.

One model, ten datasets

A single network removes clouds from RGB or multispectral, single- or multi-temporal inputs, with or without SAR.

Grid of five cloud-removal benchmarks, one per row (SEN12MS-CR, Sen2_MTC_New, CUHK-CR2, WHUS2-CRv, T-CLOUD). Columns: input cloudy image, UnCRtainTS, EMRDM, GACR, GeoCR without fine-tuning, GeoCR with LoRA, and the cloud-free ground truth.
Qualitative comparison on cloud-removal benchmarks. (e) GeoCR without fine-tuning and (f) with LoRA adaptation.

Quantitative results

GeoCR achieves the best FID and DISTS on full-band Sen2_MTC_New and SEN12MS-CR and RGB-only CUHK-CR2.

Radar chart of normalized FID and DISTS on CUHK-CR2, WHUS2-CRv, T-CLOUD, SEN12MS-CR and Sen2_MTC_New for UnCRtainTS, EMRDM, GACR, GeoCR (w/o FT) and GeoCR (LoRA); larger is better.
Cross-dataset comparison. FID and DISTS on five benchmarks, normalized as 100 × best / value (larger is better).

Main comparison tables

Table 2. Quantitative comparison on CUHK-CR2. RGB evaluation setting, all ten baselines.
Bold bestUnderline second best↓ lower is better · ↑ higher is better
Scroll for more columns
Method Venue FID↓ DISTS↓ KID↓ DINO↑ LPIPS↓ SSIM↑ PSNR↑
General image translation/restoration methods
pix2pixCVPR’17194.80.2410.12010.5380.3190.45219.46
pix2pixHDCVPR’18257.10.2620.21300.3790.3040.40119.34
BBDMCVPR’23514.60.5420.58270.1310.7370.32617.68
HI-DiffNeurIPS’23165.70.1920.07690.6640.2530.63823.54
Cloud removal methods
UnCRtainTSCVPRW’23165.60.2650.07610.5290.3450.58622.12
DiffCRTGRS’24245.20.2850.18740.6150.3530.58222.85
IDF-CRTGRS’24167.00.2140.08730.6450.2700.64123.18
ThiefCloudTCSVT’25136.30.2260.05930.6680.2320.63823.88
EMRDMCVPR’25104.40.1670.02590.7270.2080.65423.61
GACRECCV’26125.00.1780.04960.7390.2170.62023.45
Ours
GeoCR (w/o FT)–93.60.1530.02000.7840.1940.60723.21
GeoCR (LoRA)–94.70.1520.01940.7790.1960.60123.11

FID and DISTS are the primary metrics. T-CLOUD, CUHK-CR1 and WHUS2-CRv: Table 10.

Pretraining corpus Table 1 · ten datasets
Table 1. Overview of the GeoCR pretraining corpus. Counts refer to cloud-free target images in the training splits. Spectral bands describe the targets unless otherwise noted; cloudy conditions indicate the number of optical input frames. Tile sizes are in pixels. L1C: Level-1C; TOA/BOA: top-/bottom-of-atmosphere reflectance; RGB: red, green, and blue; NIR: near-infrared; SAR: synthetic aperture radar; GSD: ground sampling distance. B8 and B10 denote Sentinel-2 band identifiers.
Scroll for more columns
Source Imagery source Spectral bands Delivered
tile
GSD
(m)
Cloudy
cond.
SAR
cond.
Clean
targets
AllClearSentinel-2 + Sentinel-113 (L1C, TOA)2562101–3✓662,022
SEN12MS-CRSentinel-2 + Sentinel-113 (L1C, TOA)2562101✓107,143
WHUS2-CRvSentinel-213 (BOA; B10: L1C, TOA)3842/1922/642101✗18,816
Sen2_MTC_OldSentinel-23 (RGB; +NIR for cloudy inputs)2562103✗88,874
Sen2_MTC_NewSentinel-24 (RGB+B8, BOA)2562103✗2,380
T-CLOUDLandsat-83 (RGB, 8-bit)2562301✗2,234
RICE2Landsat-83 (RGB, 8-bit)5122301✗553
CUHK-CR1Jilin-1 KF01B4 (RGB+NIR, 8-bit)51220.51✗508
CUHK-CR2Jilin-1 KF01B4 (RGB+NIR, 8-bit)51220.51✗426
RICE1Google Earth3 (RGB, 8-bit)5122–1✗375
Ablations Tables 5–7 · stems, temporal and SAR conditioning
Table 5. Effect of Stage 1 stem training. Reconstruction PSNR (dB) on AllClear validation samples before and after stem training, with the pretrained encoder and decoder frozen. VV and VH denote SAR polarizations.
Scroll for more columns
Recon.
PSNR
RGB (w/o training) Non-RGB (stem training) SAR (stem training)
AllClearCloudy AllClearCloudy AllVVVH
Before45.7846.7745.1118.8420.6018.2018.4018.5718.24
After45.7846.7745.1140.0841.4138.3429.1428.5430.21
Gain+0.00+0.00+0.00+21.24+20.81+20.14+10.74+9.97+11.98
Reconstruction grid for the RGB, single-band NIR, 10-band TOA, 10-band BOA and SAR routes (clear and cloudy observations of the same place; SAR clear only): input, reconstruction before stem training (trained routes only), and reconstruction with the final stem, labelled with PSNR in dB.
Shared autoencoder. Reconstruction PSNR (dB) through the frozen trunks. The RGB and initialized NIR routes stay fixed; the ten-band optical and SAR stems are trained in Stage 1.
Table 6. Temporal conditioning on Sen2_MTC_New. GeoCR uses the shared checkpoint without fine-tuning.
Scroll for more columns
MethodCloudyPSNR↑SSIM↑
GeoCR116.3900.4595
GeoCR219.3190.5963
GeoCR320.5220.6568

Fixed test subset of Sen2_MTC_New; GeoCR (w/o FT).

Table 7. SAR conditioning on SEN12MS-CR. GeoCR uses the shared checkpoint without fine-tuning.
Scroll for more columns
MethodSARPSNR↑SSIM↑
GeoCR✗28.5960.8572
GeoCR✓29.4420.8703

Fixed test subset of SEN12MS-CR; GeoCR (w/o FT).

Appendix tables Tables 9–12 · more benchmarks and settings
Table 9. Reconstruction by configuration. PSNR (dB) is computed from pooled reconstruction error on 64 training images per source, without clipping. These diagnostics differ from the held-out before/after comparison in the main paper and do not measure cloud removal.
Scroll for more columns
RouteStem parametersTrainedReconstruction PSNR
10-band TOA23,178YesAllClear 38.23; SEN12MS-CR 37.69
10-band BOA23,178YesWHUS2-CRv 37.89
Single-band NIR2,433NoSen2_MTC_New 42.06; CUHK-CR 40.41; Sen2_MTC_Old 49.51
SAR VV/VH4,738YesAllClear 28.51; SEN12MS-CR 31.19
Table 10. Additional quantitative comparisons. Feature metrics use RGB views; PSNR/SSIM follow Table 12.
Bold bestUnderline second best↓ lower is better · ↑ higher is better
Scroll for more columns
MethodFID↓DISTS↓KID↓DINO↑LPIPS↓SSIM↑PSNR↑
(a) T-CLOUD: RGB display imagery
pix2pix107.60.2540.05090.4500.3300.63520.13
pix2pixHD60.80.1690.01730.6160.1770.74124.75
BBDM186.60.4220.11790.2590.5460.54122.69
HI-Diff40.20.1240.00700.7260.1140.87430.54
UnCRtainTS63.30.1800.01880.6400.1780.80926.28
DiffCR139.60.3580.07320.3760.4820.40220.67
IDF-CR84.40.2200.03040.4670.2360.78725.99
ThiefCloud40.40.1230.00580.7260.1090.86129.36
EMRDM36.50.1210.00450.7590.1100.86728.25
GACR39.20.1200.00580.7470.1130.85829.56
GeoCR (w/o FT)36.50.1260.00240.7630.1220.78527.15
GeoCR (LoRA)36.30.1250.00220.7510.1220.78627.23
(b) CUHK-CR1: RGB+NIR setting
UnCRtainTS134.80.2040.02830.7120.3110.67923.82
DiffCR237.60.2910.13600.6490.3180.57322.75
EMRDM77.80.115-0.00060.8450.1460.76025.64
GACR97.10.1330.00990.8250.1600.72625.02
GeoCR (w/o FT)80.40.125-0.00250.8250.1650.68023.88
GeoCR (LoRA)81.50.125-0.00270.8210.1670.67723.83
(c) WHUS2-CRv: native multispectral setting
UnCRtainTS26.90.1420.00620.8080.1390.92631.14
IDF-CR40.00.1790.01150.6910.1880.85829.00
EMRDM16.90.1010.00090.8750.0970.93732.55
GACR22.60.2300.00330.7890.1380.88231.09
GeoCR (w/o FT)18.90.1110.00270.8500.1060.90532.29
GeoCR (LoRA)18.50.1070.00250.8580.1030.90332.15
(d) WHUS2-CRv: RGB-only setting
pix2pix95.10.2380.05230.4320.2740.74623.75
pix2pixHD27.90.1480.00570.7510.1450.82826.77
BBDM50.40.2130.01710.4130.2910.63925.19
HI-Diff17.80.1070.00180.8530.0960.89429.91
GeoCR (w/o FT)21.60.1370.00320.7860.1330.83327.16
GeoCR (LoRA)20.70.1380.00390.8110.1340.80425.35
Table 11. Dataset representations and pretraining mixture. Bands describe optical targets; K is the number of cloudy observations. Mix denotes the source sampling probability.
Scroll for more columns
SourceOptical representationSARKMix (%)Training targets
AllClear13 bands, L1C TOAVV/VH1–353.14662,022
SEN12MS-CR13 bands, L1C TOAVV/VH132.93107,143
WHUS2-CRv13 bands, BOA; B10 TOA–18.8118,816
Sen2_MTC_OldRGB; NIR in cloudy inputs only–32.0088,874
Sen2_MTC_NewRGB+B8, BOA–31.102,380
T-CLOUDRGB, 8-bit–11.102,234
RICE2RGB, 8-bit–10.27553
CUHK-CR1RGB+NIR, 8-bit–10.25508
CUHK-CR2RGB+NIR, 8-bit–10.21426
RICE1RGB, 8-bit–10.19375
Total100.00883,331
Table 12. Evaluation settings. Ntest denotes the size of each evaluated test split. CUHK-CR1 counts reflect the duplicate exclusions described in the Appendix (Datasets and Preprocessing).
Scroll for more columns
SettingPSNR/SSIM bandsNtest
SEN12MS-CR13 bands7,899
Sen2_MTC_NewRGB+B8687
WHUS2-CRv13 bands3,746
T-CLOUDRGB588
CUHK-CR1RGB+NIR98
CUHK-CR2RGB111
SEN12MS-CR, RGB-onlyRGB7,899
Sen2_MTC_New, RGB-onlyRGB687
WHUS2-CRv, RGB-onlyRGB3,746

One latent interface, one shared prior

GeoCR framework diagram. Top: the multi-source cloud removal pretraining corpus, with cloudy and clear examples from ten datasets. Left, Stage 1: input and output stems (3x3 convolutions) around the shared frozen RGB encoder and decoder trunks; the RGB stems stay frozen, while the non-RGB and SAR stems are trained with reconstruction losses. Right, Stage 2: the frozen stems encode clean and cloudy RGB, non-RGB and SAR inputs into latents, and a DiT initialized from a pretrained generator, with multi-stream and single-stream blocks, is trained with a flow-matching loss on the joint target latent.
GeoCR framework. A shared latent interface enables joint cloud removal across heterogeneous spectral, temporal, and SAR configurations.
  1. 01Shared latent interface. Compact stems map non-RGB and SAR inputs into a frozen pretrained RGB autoencoder.
  2. 02One generalist prior. A single flow transformer is jointly pretrained on 10 datasets (883,331 cloud-free targets).
  3. 03Two modes. The same checkpoint is used directly (w/o FT) or adapted with LoRA.

BibTeX

@article{do2026geocr,
  title={GeoCR: Learning a Generalist Cloud Removal Prior from Heterogeneous Observations},
  author={Do, Jeonghyeok and Kim, Munchurl},
  journal={arXiv preprint arXiv:2609.32510},
  year={2026}
}