Kernel Logistic Regression Algorithm for Large-Scale Data Classification

Murtada Elbashir2 and Jianxin Wang1
1School of Information Science and Engineering, Central South University, China
2Faculty of Mathematical and Computer Sciences, University of Gezira, Sudan

Abstract: Kernel Logistic Regression (KLR) is a powerful classification technique that has been applied successfully in many classification problems. However, it is often not found in large-scale data classification problems, and this is mainly because it is computationally expensive. In this paper, we present a new KLR algorithm based on Truncated Regularized Iteratively Re-Weighted Least Squares(TR-IRLS) algorithm to obtain sparse large-scale data classification in short evolution time. This new algorithm is called Nystrom Truncated Kernel Logistic Regression (NTR-KLR). The performance achieved using NTR-KLR algorithm is comparable to that of Support Vector Machines (SVMs) methods. The advantage is NTR-KLR can yield probabilistic outputs, and its extension to the multi-class case is well-defined. In addition, its computational complexity is lower than that of SVMs methods, and it is easy to implement.

Keywords: KLR, iteratively reweighted least squares, nystrom method, newton's method.

 Received November 18, 2012; accepted July 2, 2013