GraphI2P: Image-to-Point Cloud Registration with Exploring Pattern of Correspondence via Graph Learning

Although the fusion of images and LiDAR point clouds is crucial to many applications in computer vision, the relative poses of cameras and LiDAR scanners are often unknown. However, due to the modality and domain gap between images and LiDAR point clouds, Image-to-Point Cloud Registration is a significant challenge, especially when the image and point cloud come from non-synchronized frames. To tackle these issues, we introduce the virtual point cloud as a bridge to alleviate the cross-modality gap between images and LiDAR point clouds. In this way, the modality gap is converted to the domain gap of point clouds. Moreover, we introduce a virtual-spherical representation achieving orthogonal decoupling between pixel location and predicted depth. As for the domain gap, we propose a distribution-based adaptive sample module to generate a unified distribution of two types of point clouds. Then, we explore the correct correspondence pattern consistency and prune the false correspondences through a graph-based selection process. Experimental results demonstrate that our method outperforms the state-of-the-art methods by more than 10.77% and 12.53% performance on the KITTI Odometry and nuScenes datasets, respectively. The results demonstrate that our method can effectively solve non-synchronized random-frame registration.