Environmental perception system is an important part of unmanned driving technology, which is the premise of the safety and stability of the unmanned vehicle system. At present, the environmental perception technology is mainly focused on the environmental information acquisition under ideal environment, the high-precision recognition of semantic information, and the multi-sensor fusion etc. A comprehensive attack detection and defense system for the unmanned driving systems is still not available. In this paper, the correlation of multi-sensor perception signals in the perception system in both temporal and spatial domains are considered to establish a mathematical model, which is used to connect the information among different sensors, and to detect the attacked sensors. More important, it can be also used to recover the distorted data based on the matrix completion method. The experimental results showed that, the proposed method can detect the attacked sensor effectively and recover the missed information in the attached perception systems.