Linear classifier design for heteroscedastic LDA under class imbalance

Sarfo Gyamfi, James Brusey, Andrew Hunt, Elena Gaura

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding

Abstract

Linear Discriminant Analysis (LDA) yields the optimal Bayes classifier for binary classification for normally distributed classes with equal covariance. To improve the performance of LDA, heteroscedastic LDA (HLDA) that removes the equal covariance assumption has been developed. In this paper, we show that the existing approaches either have no principled computational procedure for optimal parameter selection, or underperform in terms of the accuracy of classification and the area under the receiver operating characteristics curve (AUC) under class imbalance. We then derive a Bayes optimal linear classifier for heteroscedastic LDA that is robust against class imbalance and is obtained via an efficient gradient descent optimisation procedure. Our experimental work on one artificial dataset shows that our proposed algorithm achieves the minimum misclassification rate as compared to existing HLDA approaches if the errors in both the minority and majority classes are of equal importance. In the scenario where the errors in the minority class may be of more importance, further experiments on five real-world datasets show the superiority of our algorithm in terms of the AUC as compared to the original LDA procedure, existing HLDA algorithms, and the linear support vector machine (SVM).
Original languageEnglish
Title of host publicationProceedings of the IJCAI 2017 Workshop on Learning in the Presence of Class Imbalance and Concept Drift (LPCICD'17)
Subtitle of host publicationInternational Joint Conference on Artificial Intelligence 2017
Pages8-15
Number of pages8
Publication statusPublished - 2017
EventInternational Joint Conference on Artificial Intelligence 2017: Workshop on Learning in the Presence of Class Imbalance and Concept Drift - Melbourne, Australia
Duration: 19 Aug 2017 → 21 Aug 2017

Conference

ConferenceInternational Joint Conference on Artificial Intelligence 2017
Abbreviated titleLPCICD'17
CountryAustralia
CityMelbourne
Period19/08/17 → 21/08/17

Fingerprint

Discriminant analysis
Classifiers
Support vector machines
Experiments

Cite this

Gyamfi, S., Brusey, J., Hunt, A., & Gaura, E. (2017). Linear classifier design for heteroscedastic LDA under class imbalance. In Proceedings of the IJCAI 2017 Workshop on Learning in the Presence of Class Imbalance and Concept Drift (LPCICD'17): International Joint Conference on Artificial Intelligence 2017 (pp. 8-15)

Linear classifier design for heteroscedastic LDA under class imbalance. / Gyamfi, Sarfo; Brusey, James; Hunt, Andrew; Gaura, Elena.

Proceedings of the IJCAI 2017 Workshop on Learning in the Presence of Class Imbalance and Concept Drift (LPCICD'17): International Joint Conference on Artificial Intelligence 2017. 2017. p. 8-15.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding

Gyamfi, S, Brusey, J, Hunt, A & Gaura, E 2017, Linear classifier design for heteroscedastic LDA under class imbalance. in Proceedings of the IJCAI 2017 Workshop on Learning in the Presence of Class Imbalance and Concept Drift (LPCICD'17): International Joint Conference on Artificial Intelligence 2017. pp. 8-15, International Joint Conference on Artificial Intelligence 2017, Melbourne, Australia, 19/08/17.
Gyamfi S, Brusey J, Hunt A, Gaura E. Linear classifier design for heteroscedastic LDA under class imbalance. In Proceedings of the IJCAI 2017 Workshop on Learning in the Presence of Class Imbalance and Concept Drift (LPCICD'17): International Joint Conference on Artificial Intelligence 2017. 2017. p. 8-15
Gyamfi, Sarfo ; Brusey, James ; Hunt, Andrew ; Gaura, Elena. / Linear classifier design for heteroscedastic LDA under class imbalance. Proceedings of the IJCAI 2017 Workshop on Learning in the Presence of Class Imbalance and Concept Drift (LPCICD'17): International Joint Conference on Artificial Intelligence 2017. 2017. pp. 8-15
@inproceedings{ae5ee1f419d84d4fa77d03a32cbf1a47,
title = "Linear classifier design for heteroscedastic LDA under class imbalance",
abstract = "Linear Discriminant Analysis (LDA) yields the optimal Bayes classifier for binary classification for normally distributed classes with equal covariance. To improve the performance of LDA, heteroscedastic LDA (HLDA) that removes the equal covariance assumption has been developed. In this paper, we show that the existing approaches either have no principled computational procedure for optimal parameter selection, or underperform in terms of the accuracy of classification and the area under the receiver operating characteristics curve (AUC) under class imbalance. We then derive a Bayes optimal linear classifier for heteroscedastic LDA that is robust against class imbalance and is obtained via an efficient gradient descent optimisation procedure. Our experimental work on one artificial dataset shows that our proposed algorithm achieves the minimum misclassification rate as compared to existing HLDA approaches if the errors in both the minority and majority classes are of equal importance. In the scenario where the errors in the minority class may be of more importance, further experiments on five real-world datasets show the superiority of our algorithm in terms of the AUC as compared to the original LDA procedure, existing HLDA algorithms, and the linear support vector machine (SVM).",
author = "Sarfo Gyamfi and James Brusey and Andrew Hunt and Elena Gaura",
year = "2017",
language = "English",
pages = "8--15",
booktitle = "Proceedings of the IJCAI 2017 Workshop on Learning in the Presence of Class Imbalance and Concept Drift (LPCICD'17)",

}

TY - GEN

T1 - Linear classifier design for heteroscedastic LDA under class imbalance

AU - Gyamfi, Sarfo

AU - Brusey, James

AU - Hunt, Andrew

AU - Gaura, Elena

PY - 2017

Y1 - 2017

N2 - Linear Discriminant Analysis (LDA) yields the optimal Bayes classifier for binary classification for normally distributed classes with equal covariance. To improve the performance of LDA, heteroscedastic LDA (HLDA) that removes the equal covariance assumption has been developed. In this paper, we show that the existing approaches either have no principled computational procedure for optimal parameter selection, or underperform in terms of the accuracy of classification and the area under the receiver operating characteristics curve (AUC) under class imbalance. We then derive a Bayes optimal linear classifier for heteroscedastic LDA that is robust against class imbalance and is obtained via an efficient gradient descent optimisation procedure. Our experimental work on one artificial dataset shows that our proposed algorithm achieves the minimum misclassification rate as compared to existing HLDA approaches if the errors in both the minority and majority classes are of equal importance. In the scenario where the errors in the minority class may be of more importance, further experiments on five real-world datasets show the superiority of our algorithm in terms of the AUC as compared to the original LDA procedure, existing HLDA algorithms, and the linear support vector machine (SVM).

AB - Linear Discriminant Analysis (LDA) yields the optimal Bayes classifier for binary classification for normally distributed classes with equal covariance. To improve the performance of LDA, heteroscedastic LDA (HLDA) that removes the equal covariance assumption has been developed. In this paper, we show that the existing approaches either have no principled computational procedure for optimal parameter selection, or underperform in terms of the accuracy of classification and the area under the receiver operating characteristics curve (AUC) under class imbalance. We then derive a Bayes optimal linear classifier for heteroscedastic LDA that is robust against class imbalance and is obtained via an efficient gradient descent optimisation procedure. Our experimental work on one artificial dataset shows that our proposed algorithm achieves the minimum misclassification rate as compared to existing HLDA approaches if the errors in both the minority and majority classes are of equal importance. In the scenario where the errors in the minority class may be of more importance, further experiments on five real-world datasets show the superiority of our algorithm in terms of the AUC as compared to the original LDA procedure, existing HLDA algorithms, and the linear support vector machine (SVM).

M3 - Conference proceeding

SP - 8

EP - 15

BT - Proceedings of the IJCAI 2017 Workshop on Learning in the Presence of Class Imbalance and Concept Drift (LPCICD'17)

ER -

Access to Document