To solve the problem of serious imbalance between the foreground and background in medical images and small objects segmentation, we propose an attention network based on Gaussian image pyramid to fuse spatial information and abstract information in the feature decoding stage. In addition, a feature recaller is designed to force the encoder to avoid missing features of the region of interest. Finally, a hybrid loss function composed of classification accuracy and global overlapping terms is employed to deal with the serious imbalance between the foreground and background. The proposed method was validated on a knee articular cartilage dataset and the COVOID-19 chest CT dataset where the foreground proportions are 2.08% and 10.73%, respectively. The proposed method achieves the highest Dice coefficients on both datasets as compared with U-Net and its state-of-the-art variants, which are 0.884±0.032 and 0.831±0.072, respectively.