Face recognition has always been courted in computer vision and is especially amenable to situations with significant variations between frontal and profile faces. Traditional techniques make great strides either by synthesizing frontal faces from sizable datasets or by empirical pose invariant learning. In this paper, we propose a completely integrated embedded end-to-end Lie algebra residual architecture (LARNeXt) to achieve pose robust face recognition. First, we explore how the face rotation in the 3D space affects the deep feature generation process of convolutional neural networks (CNNs), and prove that face rotation in the image space is equivalent to an additive residual component in the feature space of CNNs, which is determined solely by the rotation. Second, on the basis of this theoretical finding, we further design three critical subnets to leverage a soft regression subnet with novel multi-fusion attention feature aggregation for efficient pose estimation, a residual subnet for decoding rotation information from input face images, and a gating subnet to learn rotation magnitude for controlling the strength of the residual component that contributes to the feature learning process. Finally, we conduct a large number of ablation experiments, and our quantitative and visualization results both corroborate the credibility of our theory and corresponding network designs. Our comprehensive experimental evaluations on frontal-profile face datasets, general unconstrained face recognition datasets, and industrial-grade tasks demonstrate that our method consistently outperforms the state-of-the-art ones.  

　　Publication:  

　　IEEE Transactions on Pattern Analysis and Machine Intelligence ( Volume: 45, Issue: 10, October 2023)  

　　http://dx.doi.org/10.1109/TPAMI.2023.3279378  

　　Author:  

　　Xiaolong Yang  

　　Academy of Mathematics and Systems Science of the Chinese Academy of Sciences, University of Chinese Academy of Sciences, Huairou 101408, China  

　　Xiaohong Jia  

　　Academy of Mathematics and Systems Science of the Chinese Academy of Sciences, University of Chinese Academy of Sciences, Huairou 101408, China  

　　Email: xhjia@amss.ac.cn  

　　Dihong Gong  

　　Tencent Data P Platform, ShenZhen 518054, China  

　　Zhifeng Li  

　　Tencent Data P Platform, ShenZhen 518054, China  

　　Wei Liu  

　　Tencent Data P Platform, ShenZhen 518054, China  

　　Dong-Ming Yan  

　　State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS)  

　　NLPR, Institute of Automation of the Chinese Academy of Sciences  

　　School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 101408, China