Optimized precision - A new measure for classifier performance evaluation

Romesh Ranawana; Vasile Palade

doi:10.1109/CEC.2006.1688586

Optimized precision - A new measure for classifier performance evaluation

Romesh Ranawana, Vasile Palade

University of Oxford

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding › peer-review

67 Citations (Scopus)

Abstract

All learning algorithms attempt to improve the accuracy of a classification system. However, the effectiveness of such a system is dependent on the heuristic used by the learning paradigm to measure performance. This paper demonstrates that the use of Precision (P) for performance evaluation of unbalanced data sets could lead the solution towards sub-optimal answers. We move onto present a novel performance heuristic, the 'Optimized Precision (OP)', to negate these detrimental effects. We also analyze the impact of these observations on the training performance of ensemble learners and Multi-Classifier Systems (MCS), and provide guidelines for the proper training of multi-classifier systems.

Original language	English
Title of host publication	2006 IEEE Congress on Evolutionary Computation, CEC 2006
Publisher	IEEE
Pages	2254-2261
Number of pages	8
ISBN (Print)	0780394879, 9780780394872
DOIs	https://doi.org/10.1109/CEC.2006.1688586
Publication status	Published - 11 Sept 2006
Externally published	Yes
Event	2006 IEEE Congress on Evolutionary Computation, CEC 2006 - Vancouver, BC, Canada Duration: 16 Jul 2006 → 21 Jul 2006

Conference

Conference	2006 IEEE Congress on Evolutionary Computation, CEC 2006
Country/Territory	Canada
City	Vancouver, BC
Period	16/07/06 → 21/07/06

Keywords

Computational intelligence
Laboratories
Performance analysis
Guidelines
Wikipedia
Classification algorithms
Competitive intelligence
Measurement
Multilayer perceptrons
Mean square error methods

ASJC Scopus subject areas

Artificial Intelligence
Software
Theoretical Computer Science

Access to Document

10.1109/CEC.2006.1688586

Cite this

@inproceedings{19b7aa7fa6e0418fbf99c5f86aeb0d22,

title = "Optimized precision - A new measure for classifier performance evaluation",

abstract = "All learning algorithms attempt to improve the accuracy of a classification system. However, the effectiveness of such a system is dependent on the heuristic used by the learning paradigm to measure performance. This paper demonstrates that the use of Precision (P) for performance evaluation of unbalanced data sets could lead the solution towards sub-optimal answers. We move onto present a novel performance heuristic, the 'Optimized Precision (OP)', to negate these detrimental effects. We also analyze the impact of these observations on the training performance of ensemble learners and Multi-Classifier Systems (MCS), and provide guidelines for the proper training of multi-classifier systems.",

keywords = "Computational intelligence, Laboratories, Performance analysis, Guidelines, Wikipedia, Classification algorithms, Competitive intelligence, Measurement, Multilayer perceptrons, Mean square error methods",

author = "Romesh Ranawana and Vasile Palade",

year = "2006",

month = sep,

day = "11",

doi = "10.1109/CEC.2006.1688586",

language = "English",

isbn = "0780394879",

pages = "2254--2261",

booktitle = "2006 IEEE Congress on Evolutionary Computation, CEC 2006",

publisher = "IEEE",

address = "United States",

note = "2006 IEEE Congress on Evolutionary Computation, CEC 2006 ; Conference date: 16-07-2006 Through 21-07-2006",

}

TY - GEN

T1 - Optimized precision - A new measure for classifier performance evaluation

AU - Ranawana, Romesh

AU - Palade, Vasile

PY - 2006/9/11

Y1 - 2006/9/11

N2 - All learning algorithms attempt to improve the accuracy of a classification system. However, the effectiveness of such a system is dependent on the heuristic used by the learning paradigm to measure performance. This paper demonstrates that the use of Precision (P) for performance evaluation of unbalanced data sets could lead the solution towards sub-optimal answers. We move onto present a novel performance heuristic, the 'Optimized Precision (OP)', to negate these detrimental effects. We also analyze the impact of these observations on the training performance of ensemble learners and Multi-Classifier Systems (MCS), and provide guidelines for the proper training of multi-classifier systems.

AB - All learning algorithms attempt to improve the accuracy of a classification system. However, the effectiveness of such a system is dependent on the heuristic used by the learning paradigm to measure performance. This paper demonstrates that the use of Precision (P) for performance evaluation of unbalanced data sets could lead the solution towards sub-optimal answers. We move onto present a novel performance heuristic, the 'Optimized Precision (OP)', to negate these detrimental effects. We also analyze the impact of these observations on the training performance of ensemble learners and Multi-Classifier Systems (MCS), and provide guidelines for the proper training of multi-classifier systems.

KW - Computational intelligence

KW - Laboratories

KW - Performance analysis

KW - Guidelines

KW - Wikipedia

KW - Classification algorithms

KW - Competitive intelligence

KW - Measurement

KW - Multilayer perceptrons

KW - Mean square error methods

UR - http://www.scopus.com/inward/record.url?scp=34547372256&partnerID=8YFLogxK

U2 - 10.1109/CEC.2006.1688586

DO - 10.1109/CEC.2006.1688586

M3 - Conference proceeding

AN - SCOPUS:34547372256

SN - 0780394879

SN - 9780780394872

SP - 2254

EP - 2261

BT - 2006 IEEE Congress on Evolutionary Computation, CEC 2006

PB - IEEE

T2 - 2006 IEEE Congress on Evolutionary Computation, CEC 2006

Y2 - 16 July 2006 through 21 July 2006

ER -

Optimized precision - A new measure for classifier performance evaluation

Abstract

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this