A visual interpretation rating scale for validation of numerical models

D. E. Coleby, A. P. Duffy

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

Purpose – The comparison of large volumes of complex data resulting from numerical modelling in computational electromagnetics is a demanding task, especially when validating the performance of numerical models against experimental results and testing experimental repeatability. “Byeye” comparisons can lead to inconsistencies and inherent subjectivity. This paper establishes a “visual” benchmark by which comparisons can be made and therefore used to assist in the development of an algorithmic approach to data comparison. Design/methodology/approach – This new method presented here is based on the CooperHarper Rating Scale, which is a test pilot's evaluationrating instrument. This has been modified through qualitative research. The assertion that the rating scale will leave the group mean response unaltered but will reduce the variance has been statistically tested. Findings – The proposed rating scale provides a calibration technique by which to benchmark comparisons. The scale also reduces subjectivity by producing an overall quantitative measure of similarity. The paper concludes with an application of the rating scale to assessment of a candidate algorithmic approach against correlation. Research limitations/implications – The research findings are based on small data sets, which is a limit imposed by the industrial environment in which this scale will be used. Practical implications – This paper provides a tool to overcome some of the key substantial difficulties in communicating similarity or difference, namely that “similarity” and “difference” have no standalone definition, there is a lack of a shared language for the comparisons and little commonality for a decisionmaking framework. Originality/value – This paper provides modellers and experimentalists in computational electromagnetics (particularly electromagnetic compatibility) with a structured approach to quantifying the quality of comparative results.

Original languageEnglish
Pages (from-to)1078-1092
Number of pages15
JournalCOMPEL - The International Journal for Computation and Mathematics in Electrical and Electronic Engineering
Volume24
Issue number4
DOIs
Publication statusPublished - 1 Dec 2005
Externally publishedYes

Fingerprint

Computational electromagnetics
Numerical models
Computational Electromagnetics
Electromagnetic compatibility
Electromagnetic Compatibility
Calibration
Benchmark
Qualitative Research
Repeatability
Testing
Numerical Modeling
Assertion
Inconsistency
Design Methodology
Vision
Interpretation
Decision Making
Experimental Results
Similarity

Keywords

  • Electromagnetism
  • Numerical analysis
  • Simulation

ASJC Scopus subject areas

  • Computer Science Applications
  • Computational Theory and Mathematics
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

A visual interpretation rating scale for validation of numerical models. / Coleby, D. E.; Duffy, A. P.

In: COMPEL - The International Journal for Computation and Mathematics in Electrical and Electronic Engineering, Vol. 24, No. 4, 01.12.2005, p. 1078-1092.

Research output: Contribution to journalArticle

@article{1559d646e6d64176a8d837ab4cb9a3c2,
title = "A visual interpretation rating scale for validation of numerical models",
abstract = "Purpose – The comparison of large volumes of complex data resulting from numerical modelling in computational electromagnetics is a demanding task, especially when validating the performance of numerical models against experimental results and testing experimental repeatability. “Byeye” comparisons can lead to inconsistencies and inherent subjectivity. This paper establishes a “visual” benchmark by which comparisons can be made and therefore used to assist in the development of an algorithmic approach to data comparison. Design/methodology/approach – This new method presented here is based on the CooperHarper Rating Scale, which is a test pilot's evaluationrating instrument. This has been modified through qualitative research. The assertion that the rating scale will leave the group mean response unaltered but will reduce the variance has been statistically tested. Findings – The proposed rating scale provides a calibration technique by which to benchmark comparisons. The scale also reduces subjectivity by producing an overall quantitative measure of similarity. The paper concludes with an application of the rating scale to assessment of a candidate algorithmic approach against correlation. Research limitations/implications – The research findings are based on small data sets, which is a limit imposed by the industrial environment in which this scale will be used. Practical implications – This paper provides a tool to overcome some of the key substantial difficulties in communicating similarity or difference, namely that “similarity” and “difference” have no standalone definition, there is a lack of a shared language for the comparisons and little commonality for a decisionmaking framework. Originality/value – This paper provides modellers and experimentalists in computational electromagnetics (particularly electromagnetic compatibility) with a structured approach to quantifying the quality of comparative results.",
keywords = "Electromagnetism, Numerical analysis, Simulation",
author = "Coleby, {D. E.} and Duffy, {A. P.}",
year = "2005",
month = "12",
day = "1",
doi = "10.1108/03321640510615472",
language = "English",
volume = "24",
pages = "1078--1092",
journal = "COMPEL - The International Journal for Computation and Mathematics in Electrical and Electronic Engineering",
issn = "0332-1649",
publisher = "Emerald",
number = "4",

}

TY - JOUR

T1 - A visual interpretation rating scale for validation of numerical models

AU - Coleby, D. E.

AU - Duffy, A. P.

PY - 2005/12/1

Y1 - 2005/12/1

N2 - Purpose – The comparison of large volumes of complex data resulting from numerical modelling in computational electromagnetics is a demanding task, especially when validating the performance of numerical models against experimental results and testing experimental repeatability. “Byeye” comparisons can lead to inconsistencies and inherent subjectivity. This paper establishes a “visual” benchmark by which comparisons can be made and therefore used to assist in the development of an algorithmic approach to data comparison. Design/methodology/approach – This new method presented here is based on the CooperHarper Rating Scale, which is a test pilot's evaluationrating instrument. This has been modified through qualitative research. The assertion that the rating scale will leave the group mean response unaltered but will reduce the variance has been statistically tested. Findings – The proposed rating scale provides a calibration technique by which to benchmark comparisons. The scale also reduces subjectivity by producing an overall quantitative measure of similarity. The paper concludes with an application of the rating scale to assessment of a candidate algorithmic approach against correlation. Research limitations/implications – The research findings are based on small data sets, which is a limit imposed by the industrial environment in which this scale will be used. Practical implications – This paper provides a tool to overcome some of the key substantial difficulties in communicating similarity or difference, namely that “similarity” and “difference” have no standalone definition, there is a lack of a shared language for the comparisons and little commonality for a decisionmaking framework. Originality/value – This paper provides modellers and experimentalists in computational electromagnetics (particularly electromagnetic compatibility) with a structured approach to quantifying the quality of comparative results.

AB - Purpose – The comparison of large volumes of complex data resulting from numerical modelling in computational electromagnetics is a demanding task, especially when validating the performance of numerical models against experimental results and testing experimental repeatability. “Byeye” comparisons can lead to inconsistencies and inherent subjectivity. This paper establishes a “visual” benchmark by which comparisons can be made and therefore used to assist in the development of an algorithmic approach to data comparison. Design/methodology/approach – This new method presented here is based on the CooperHarper Rating Scale, which is a test pilot's evaluationrating instrument. This has been modified through qualitative research. The assertion that the rating scale will leave the group mean response unaltered but will reduce the variance has been statistically tested. Findings – The proposed rating scale provides a calibration technique by which to benchmark comparisons. The scale also reduces subjectivity by producing an overall quantitative measure of similarity. The paper concludes with an application of the rating scale to assessment of a candidate algorithmic approach against correlation. Research limitations/implications – The research findings are based on small data sets, which is a limit imposed by the industrial environment in which this scale will be used. Practical implications – This paper provides a tool to overcome some of the key substantial difficulties in communicating similarity or difference, namely that “similarity” and “difference” have no standalone definition, there is a lack of a shared language for the comparisons and little commonality for a decisionmaking framework. Originality/value – This paper provides modellers and experimentalists in computational electromagnetics (particularly electromagnetic compatibility) with a structured approach to quantifying the quality of comparative results.

KW - Electromagnetism

KW - Numerical analysis

KW - Simulation

U2 - 10.1108/03321640510615472

DO - 10.1108/03321640510615472

M3 - Article

VL - 24

SP - 1078

EP - 1092

JO - COMPEL - The International Journal for Computation and Mathematics in Electrical and Electronic Engineering

JF - COMPEL - The International Journal for Computation and Mathematics in Electrical and Electronic Engineering

SN - 0332-1649

IS - 4

ER -