DAKD: Data Augmentation and Knowledge Distillation using Diffusion Models for SAR Oil Spill Segmentation

Jaeho Moon* 1     Jeonghwan Yun* 1     Jaehyun Kim* 1     Jaehyup Lee† 2     Munchurl Kim† 1
*Co-first authors (equal contribution)
†Co-Corresponding authors
1Korea Advanced Institute of Science and Technology, South Korea       
2Kyungpook National University, South Korea
Paper arXiv Code (To be released)

Overview of our Data Augmentation and Knowledge Distillation (DAKD) pipeline. The diffusion-based SAR-JointNet learns to generate SAR images and their soft labels for data augmentation and knowledge distillation to our student SAROSS-Net.

Abstract

Oil spills in the ocean pose severe environmental risks, making early detection essential. Synthetic aperture radar (SAR) based oil spill segmentation offers robust monitoring under various conditions but faces challenges due to the limited labeled data and inherent speckle noise in SAR imagery. To address these issues, we propose (i) a diffusion-based Data Augmentation and Knowledge Distillation (DAKD) pipeline and (ii) a novel SAR oil spill segmentation network, called SAROSS-Net.

In our DAKD pipeline, we present a diffusion-based SAR-JointNet that learns to generate realistic SAR images and their labels for segmentation, by effectively modeling joint distribution with balancing two modalities. The DAKD pipeline augments the training dataset and distills knowledge from SAR-JointNet by utilizing generated soft labels (pixel-wise probability maps) to supervise our SAROSS-Net. The SAROSS-Net is designed to selectively transfer high-frequency features from noisy SAR images, by employing novel Context-Aware Feature Transfer blocks along skip connections.

We demonstrate our SAR-JointNet can generate realistic SAR images and well-aligned segmentation labels, providing the augmented data to train SAROSS-Net with enhanced generalizability. Our SAROSS-Net trained with the DAKD pipeline significantly outperforms existing SAR oil spill segmentation methods with large margins.

Network Architecture



Network Architecture: The left figure shows the overall architecture of our SAR-JointNet, which is designed for joint modality fusion and SAR image generation. The right figure illustrates the SAR Oil Spill Segmentation Network (SAROSS-Net), optimized for precise oil spill segmentation tasks using SAR images.

Quantitative comparison of generated SAR images


Quantiity comparison of generated SAR images between our SAR-JointNet and other diffusion-based generative models such as DDPM and Joint-Net.

Quantitative comparison of segmentation model



TOP: Oil spill segmentation performance comparison on OSD dataset and SOS dataset (ALOS & Sentinel-1A satellites) with other segmentation model such as CBD-Net, DeepLabV3+, and Segformer.
BOTTOM: Oil spill segmentation performance comparison on OSD dataset showing IoU (%) for 5 classes and mIoU (%) with other segmentation model such as CBD-Net, DeepLabV3+, and Segformer.

Generation result in SAR images


Joint generation results: Qualitative comparison of SAR images and corresponding segmentation masks generated by: (a) DDPM, (b) JointNet, and (c) SAR-JointNet (ours) to the samples from (d) the Original OSD dataset

Qualitative Comparison of SAR images segmentation

  • Satellite Image
  • CBD-Net
  • SegFormer
  • DeepLabV3+
  • SAROSS-Net(Ours)
  • Ground Truth

Qualitative comparison of segmentation results on the OSD dataset. Each row shows a SAR input, four segmentation results by CBD-Net, SegFormer, DeepLabV3+ and our SAROSS-Net, and a ground truth segmentation mask, respectively.

BibTeX

@misc{moon2024dakddataaugmentationknowledge,
            title={DAKD: Data Augmentation and Knowledge Distillation using Diffusion Models for SAR Oil Spill Segmentation}, 
            author={Jaeho Moon and Jeonghwan Yun and Jaehyun Kim and Jaehyup Lee and Munchurl Kim},
            year={2024},
            eprint={2412.08116},
            archivePrefix={arXiv},
            primaryClass={cs.CV},
            url={https://arxiv.org/abs/2412.08116}, 
      }