In recent years, machine learning techniques have been rapidly developed and widely applied to many industrial and academic fields. Moreover, as an important part of machine learning, ensemble techniques have drawn significant attention in both academic researches and practical applications, which make use of multiple single models to construct a hybrid model. Usually, compared to each individual model, a better performance can be achieved by ensemble methods. In this thesis, a novel ensemble method is proposed to improve the performance for binary classification. The proposed method can non-linearly combine the base models by adaptively selecting the most suitable one for each data instance. The new approach has been validated on two datasets, and the experiments results show an up to 18.5% improvement on F1 score compared to the best individual model. In addition, the proposed method outperforms two other commonly used ensemble methods (Averaging and Stacking) in improving F1 score.