To effectively diagnose Alzheimer’s disease (AD) and mild cognitive impairment (MCI), a multimodal ensemble support vector machine (SVM) based on multi-modality data was proposed and used for the classification of AD and MCI. The ensemble learning was employed and the discrimination information of classification was extracted from different multiple modalities data, then the SVM was used for classification of AD and MCI. In order to validate the efficacy of proposed method, a 10-fold cross-validation was used and tested on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. Experimental results show that the proposed method is better than multi-modality linear multiple kernel learning and single-modality method.