Existing Video Frame interpolation (VFI) models tend to suffer from time-to-location ambiguity when trained with video of non-uniform motions, such as accelerating, decelerating, and changing directions, which often yield blurred interpolated frames. In this paper, we propose (i) a novel motion description map, Bidirectional Motion field (BiM), to effectively describe non-uniform motions; (ii) a BiMguided Flow Net (BiMFN) with Content-Aware Upsampling Network (CAUN) for precise optical flow estimation; and (iii) Knowledge Distillation for VFI-centric Flow supervision (KDVCF) to supervise the motion estimation of VFI model with VFI-centric teacher flows. The proposed VFI is called a Bidirectional Motion field-guided VFI (BiMVFI) model. Extensive experiments show that our BiMVFI model significantly surpasses the recent state-of-the-art VFI methods by 26% and 45% improvements in LPIPS and STLPIPS respectively, yielding interpolated frames with much fewer blurs at arbitrary time instances.
BiM-VFI: Bidirectional Motion
Fields-Guided Frame Interpolation for Video with Non-uniform Motions
@misc{seo2024bimvfidirectionalmotionfieldguided,
title={BiM-VFI: directional Motion Field-Guided Frame Interpolation for Video with Non-uniform Motions},
author={Wonyong Seo and Jihyong Oh and Munchurl Kim},
year={2024},
eprint={2412.11365},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2412.11365},
}