With the regulations of wearing helmets while driving the electric bicycle, it is urgent to develop a detection algorithm that can accurately detect whether the drivers are wearing helmets. This paper introduces a novel method to detect the helmets based on the YOLO framework. The branch absorption module is proposed to improve the residual backbone network, then the feature fusion is improved through the channel recombination. Finally, the designed structural fusion pruning is applied to further compress the hyper-parameters of the model. The experimental results showed that, the proposed algorithm has higher accuracy and faster speed. Performance of small targets detection also can be improved, with the average accuracy of multiple classification up to 88.8% and detection speed of 29.5fps, which can meet the demand of video surveillance in real applications.