Linear classifier design for heteroscedastic LDA under class imbalance

Sarfo Gyamfi, James Brusey, Andrew Hunt, Elena Gaura

Research output: Chapter in Book/Report/Conference proceedingConference proceeding

Abstract

Linear Discriminant Analysis (LDA) yields the optimal Bayes classifier for binary classification for normally distributed classes with equal covariance. To improve the performance of LDA, heteroscedastic LDA (HLDA) that removes the equal covariance assumption has been developed. In this paper, we show that the existing approaches either have no principled computational procedure for optimal parameter selection, or underperform in terms of the accuracy of classification and the area under the receiver operating characteristics curve (AUC) under class imbalance. We then derive a Bayes optimal linear classifier for heteroscedastic LDA that is robust against class imbalance and is obtained via an efficient gradient descent optimisation procedure. Our experimental work on one artificial dataset shows that our proposed algorithm achieves the minimum misclassification rate as compared to existing HLDA approaches if the errors in both the minority and majority classes are of equal importance. In the scenario where the errors in the minority class may be of more importance, further experiments on five real-world datasets show the superiority of our algorithm in terms of the AUC as compared to the original LDA procedure, existing HLDA algorithms, and the linear support vector machine (SVM).
Original languageEnglish
Title of host publicationProceedings of the IJCAI 2017 Workshop on Learning in the Presence of Class Imbalance and Concept Drift (LPCICD'17)
Subtitle of host publicationInternational Joint Conference on Artificial Intelligence 2017
Pages8-15
Number of pages8
Publication statusPublished - 2017
EventInternational Joint Conference on Artificial Intelligence 2017: Workshop on Learning in the Presence of Class Imbalance and Concept Drift - Melbourne, Australia
Duration: 19 Aug 201721 Aug 2017

Conference

ConferenceInternational Joint Conference on Artificial Intelligence 2017
Abbreviated titleLPCICD'17
CountryAustralia
CityMelbourne
Period19/08/1721/08/17

    Fingerprint

Cite this

Gyamfi, S., Brusey, J., Hunt, A., & Gaura, E. (2017). Linear classifier design for heteroscedastic LDA under class imbalance. In Proceedings of the IJCAI 2017 Workshop on Learning in the Presence of Class Imbalance and Concept Drift (LPCICD'17): International Joint Conference on Artificial Intelligence 2017 (pp. 8-15)