Consensus-Based Combining Method for

Classifier Ensembles

Omar Alzubi¹, Jafar Alzubi², Sara Tedmori³, Hasan Rashaideh⁴, and Omar Almomani⁴

¹Computer and Network Security, Al-Balqa Applied University, Jordan

²Computer Engineering Department, Al-Balqa Applied University, Jordan

³Computer Science Department, Princess Sumaya University, Jordan

⁴Information Technology, Al-Balqa Applied University, Jordan

Abstract: In this paper, a new method for combining an ensemble of classifiers, called Consensus-based Combining Method (CCM) is proposed and evaluated. As in most other combination methods, the outputs of multiple classifiers are weighted and summed together into a single final classification decision. However, unlike the other methods, CCM adjusts the weights iteratively after comparing all of the classifiers’ outputs. Ultimately, all the weights converge to a final set of weights, and the combined output reaches a consensus. The effectiveness of CCM is evaluated by comparing it with popular linear combination methods (majority voting, product, and average method). Experiments are conducted on 14 public data sets, and on a blog spam data set created by the authors. Experimental results show that CCM provides a significant improvement in classification accuracy over the product and average methods. Moreover, results show that the CCM’s classification accuracy is better than or comparable to that of majority voting.

Keywords: Artificial intelligence, classification, machine learning, pattern recognition, classifier ensembles, consensus theory, combining methods, majority voting, mean method, product method.

Received June 3, 2015; accept January 13, 2016