Transflow: Transformer as flow learner

Y Lu, Q Wang, S Ma, T Geng… - Proceedings of the …, 2023 - openaccess.thecvf.com
Y Lu, Q Wang, S Ma, T Geng, YV Chen, H Chen, D Liu
Proceedings of the IEEE/CVF conference on computer vision and …, 2023openaccess.thecvf.com
Optical flow is an indispensable building block for various important computer vision tasks,
including motion estimation, object tracking, and disparity measurement. In this work, we
propose TransFlow, a pure transformer architecture for optical flow estimation. Compared to
dominant CNN-based methods, TransFlow demonstrates three advantages. First, it provides
more accurate correlation and trustworthy matching in flow estimation by utilizing spatial self-
attention and cross-attention mechanisms between adjacent frames to effectively capture …
Abstract
Optical flow is an indispensable building block for various important computer vision tasks, including motion estimation, object tracking, and disparity measurement. In this work, we propose TransFlow, a pure transformer architecture for optical flow estimation. Compared to dominant CNN-based methods, TransFlow demonstrates three advantages. First, it provides more accurate correlation and trustworthy matching in flow estimation by utilizing spatial self-attention and cross-attention mechanisms between adjacent frames to effectively capture global dependencies; Second, it recovers more compromised information (eg, occlusion and motion blur) in flow estimation through long-range temporal association in dynamic scenes; Third, it enables a concise self-learning paradigm and effectively eliminate the complex and laborious multi-stage pre-training procedures. We achieve the state-of-the-art results on the Sintel, KITTI-15, as well as several downstream tasks, including video object detection, interpolation and stabilization. For its efficacy, we hope TransFlow could serve as a flexible baseline for optical flow estimation.
openaccess.thecvf.com
Showing the best result for this search. See all results