Linear classifier design for heteroscedastic LDA under class imbalance

Linear Discriminant Analysis (LDA) yields the optimal Bayes classifier for binary classification for normally distributed classes with equal covariance. To improve the performance of LDA, heteroscedastic LDA (HLDA) that removes the equal covariance assumption has been developed. In this paper, we show that the existing approaches either have no principled computational procedure for optimal parameter selection, or underperform in terms of the accuracy of classification and the area under the receiver operating characteristics curve (AUC) under class imbalance. We then derive a Bayes optimal linear classifier for heteroscedastic LDA that is robust against class imbalance and is obtained via an efficient gradient descent optimisation procedure. Our experimental work on one artificial dataset shows that our proposed algorithm achieves the minimum misclassification rate as compared to existing HLDA approaches if the errors in both the minority and majority classes are of equal importance. In the scenario where the errors in the minority class may be of more importance, further experiments on five real-world datasets show the superiority of our algorithm in terms of the AUC as compared to the original LDA procedure, existing HLDA algorithms, and the linear support vector machine (SVM).