Automatic drive is an important application field of artificial intelligence. In this paper, a novel training strategy for self-driving vehicles was investigated based on the deep reinforcement learning model. The proposed method involves a Q-learning algorithm with filtered experience replay and pre-training with experiences from professional drivers, which accelerates the training process due to reduced exploration spaces. By resampling the input state after clustering, generalization ability of the strategy can be improved due to the individual and independent distribution of the samples. Experimental results show that, in comparison with conventional neural fitted Q-iteration algorithm, the training efficiency and controlling stability can be improved more than 90% and 30% respectively by the proposed approach. Experimental results with more complex testing tracks show that, average travel distance can be improved more than 70% in comparison with the Q-learning algorithm by the proposed method.