A dynamic linear model for heteroscedastic LDA under class imbalance

Research output: Contribution to journalArticle

Abstract

Linear Discriminant Analysis (LDA) yields the optimal Bayes classifier for binary classification for normally distributed classes with equal covariance. To improve the performance of LDA, heteroscedastic LDA (HLDA) that removes the equal covariance assumption has been developed. In this paper, we show using first and second-order optimality conditions that the existing approaches either have no principled computational procedure for optimal parameter selection, or underperform in terms of the accuracy of classification and the area under the receiver operating characteristics curve (AUC) under class imbalance. Using the same optimality conditions, we then derive a dynamic Bayes optimal linear classifier for heteroscedastic LDA that is optimised via an efficient iterative procedure, which is robust against class imbalance. Experimental work is conducted on two artificial and eight real-world datasets. Our results show that the proposed algorithm compares favourably with the existing heteroscedastic LDA procedures as well as the linear support vector machine (SVM) in terms of the error rate, but is superior to all the algorithms in terms of the AUC under class imbalance. The fast training time of the proposed algorithm also encourages its use for large-data applications that show high incidence of class imbalance, such as in human activity recognition.

LanguageEnglish
Pages65-75
Number of pages11
JournalNeurocomputing
Volume343
Early online date4 Feb 2019
DOIs
Publication statusPublished - 28 May 2019

Fingerprint

Discriminant Analysis
Discriminant analysis
Linear Models
Area Under Curve
Classifiers
Human Activities
ROC Curve
Support vector machines
Incidence

Keywords

  • AUC
  • Class imbalance
  • Heteroscedasticity
  • LDA

ASJC Scopus subject areas

  • Computer Science Applications
  • Cognitive Neuroscience
  • Artificial Intelligence

Cite this

A dynamic linear model for heteroscedastic LDA under class imbalance. / Gyamfi, Sarfo; Brusey, James; Hunt, Andrew; Gaura, Elena.

In: Neurocomputing, Vol. 343, 28.05.2019, p. 65-75.

Research output: Contribution to journalArticle

@article{24c59796410c45a28e88b2e4e41f475f,
title = "A dynamic linear model for heteroscedastic LDA under class imbalance",
abstract = "Linear Discriminant Analysis (LDA) yields the optimal Bayes classifier for binary classification for normally distributed classes with equal covariance. To improve the performance of LDA, heteroscedastic LDA (HLDA) that removes the equal covariance assumption has been developed. In this paper, we show using first and second-order optimality conditions that the existing approaches either have no principled computational procedure for optimal parameter selection, or underperform in terms of the accuracy of classification and the area under the receiver operating characteristics curve (AUC) under class imbalance. Using the same optimality conditions, we then derive a dynamic Bayes optimal linear classifier for heteroscedastic LDA that is optimised via an efficient iterative procedure, which is robust against class imbalance. Experimental work is conducted on two artificial and eight real-world datasets. Our results show that the proposed algorithm compares favourably with the existing heteroscedastic LDA procedures as well as the linear support vector machine (SVM) in terms of the error rate, but is superior to all the algorithms in terms of the AUC under class imbalance. The fast training time of the proposed algorithm also encourages its use for large-data applications that show high incidence of class imbalance, such as in human activity recognition.",
keywords = "AUC, Class imbalance, Heteroscedasticity, LDA",
author = "Sarfo Gyamfi and James Brusey and Andrew Hunt and Elena Gaura",
year = "2019",
month = "5",
day = "28",
doi = "10.1016/j.neucom.2018.07.090",
language = "English",
volume = "343",
pages = "65--75",
journal = "Neurocomputing",
issn = "0925-2312",
publisher = "Elsevier",

}

TY - JOUR

T1 - A dynamic linear model for heteroscedastic LDA under class imbalance

AU - Gyamfi, Sarfo

AU - Brusey, James

AU - Hunt, Andrew

AU - Gaura, Elena

PY - 2019/5/28

Y1 - 2019/5/28

N2 - Linear Discriminant Analysis (LDA) yields the optimal Bayes classifier for binary classification for normally distributed classes with equal covariance. To improve the performance of LDA, heteroscedastic LDA (HLDA) that removes the equal covariance assumption has been developed. In this paper, we show using first and second-order optimality conditions that the existing approaches either have no principled computational procedure for optimal parameter selection, or underperform in terms of the accuracy of classification and the area under the receiver operating characteristics curve (AUC) under class imbalance. Using the same optimality conditions, we then derive a dynamic Bayes optimal linear classifier for heteroscedastic LDA that is optimised via an efficient iterative procedure, which is robust against class imbalance. Experimental work is conducted on two artificial and eight real-world datasets. Our results show that the proposed algorithm compares favourably with the existing heteroscedastic LDA procedures as well as the linear support vector machine (SVM) in terms of the error rate, but is superior to all the algorithms in terms of the AUC under class imbalance. The fast training time of the proposed algorithm also encourages its use for large-data applications that show high incidence of class imbalance, such as in human activity recognition.

AB - Linear Discriminant Analysis (LDA) yields the optimal Bayes classifier for binary classification for normally distributed classes with equal covariance. To improve the performance of LDA, heteroscedastic LDA (HLDA) that removes the equal covariance assumption has been developed. In this paper, we show using first and second-order optimality conditions that the existing approaches either have no principled computational procedure for optimal parameter selection, or underperform in terms of the accuracy of classification and the area under the receiver operating characteristics curve (AUC) under class imbalance. Using the same optimality conditions, we then derive a dynamic Bayes optimal linear classifier for heteroscedastic LDA that is optimised via an efficient iterative procedure, which is robust against class imbalance. Experimental work is conducted on two artificial and eight real-world datasets. Our results show that the proposed algorithm compares favourably with the existing heteroscedastic LDA procedures as well as the linear support vector machine (SVM) in terms of the error rate, but is superior to all the algorithms in terms of the AUC under class imbalance. The fast training time of the proposed algorithm also encourages its use for large-data applications that show high incidence of class imbalance, such as in human activity recognition.

KW - AUC

KW - Class imbalance

KW - Heteroscedasticity

KW - LDA

UR - http://www.scopus.com/inward/record.url?scp=85061076167&partnerID=8YFLogxK

U2 - 10.1016/j.neucom.2018.07.090

DO - 10.1016/j.neucom.2018.07.090

M3 - Article

VL - 343

SP - 65

EP - 75

JO - Neurocomputing

T2 - Neurocomputing

JF - Neurocomputing

SN - 0925-2312

ER -