MIRaSeg: Exploring mmWave Radar and Low Resolution Infrared Sensor Fusion for Robust Human Semantic Segmentation

We proposed a human semantic segmentation system based on mmWave radar point cloud with low-resolution infrared sensor fusion, called MIRaSeg. Our method first fuses mmWave radar point cloud with low-resolution infrared sensor images for human sematic segmentation. We have designed unique feature extraction modules for mmWave radar point clouds and infrared heatmaps respectively, and developed an NLN with self-attention and mutual-attention for the fusion of features from the two modalities. We conduct extensive experiments on MIRaSeg using our self-constructed multi-modal dataset and ultimately achieve superior performance compared to current advanced single millimeter-wave radar modality methods under multiple human semantic segmentation standards.

Recommended citation: Tang, X., Shi, R., Wang, S., Zhang, Z., et al. (2025). MIRaSeg: Exploring mmWave Radar and Low Resolution Infrared Sensor Fusion for Robust Human Semantic Segmentation. In Proceedings of the Chinese Conference on Pattern Recognition and Computer Vision (PRCV).