The Q-norm complexity measure and the minimum gradient method: A novel approach to the machine learning structural risk minimization problem

Douglas A G Vieira, Ricardo H C Takahashi, Vasile Palade, J. A. Vasconcelos, W. M. Caminhas

Research output: Contribution to journalArticle

21 Citations (Scopus)

Abstract

This paper presents a novel approach for dealing with the structural risk minimization (SRM) applied to a general setting of the machine learning problem. The formulation is based on the fundamental concept that supervised learning is a bi-objective optimization problem in which two conflicting objectives should be minimized. The objectives are related to the empirical training error and the machine complexity. In this paper, one general Q-norm method to compute the machine complexity is presented, and, as a particular practical case, the minimum gradient method (MGM) is derived relying on the definition of the fat-shattering dimension. A practical mechanism for parallel layer perceptron (PLP) network training, involving only quasi-convex functions, is generated using the aforementioned definitions. Experimental results on 15 different benchmarks are presented, which show the potential of the proposed ideas.

Original languageEnglish
Pages (from-to)1415-1430
Number of pages16
JournalIEEE Transactions on Neural Networks
Volume19
Issue number8
DOIs
Publication statusPublished - 2008
Externally publishedYes

Fingerprint

Gradient methods
Learning systems
Benchmarking
Neural Networks (Computer)
Supervised learning
Oils and fats
Fats
Learning
Neural networks
Machine Learning

Keywords

  • Complexity measure
  • Multiobjective training algorithms
  • Neural networks
  • Parallel layer perceptron (PLP)
  • Regularization methods
  • Structural risk minimization (SRM)

ASJC Scopus subject areas

  • Software
  • Medicine(all)
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this

The Q-norm complexity measure and the minimum gradient method : A novel approach to the machine learning structural risk minimization problem. / Vieira, Douglas A G; Takahashi, Ricardo H C; Palade, Vasile; Vasconcelos, J. A.; Caminhas, W. M.

In: IEEE Transactions on Neural Networks, Vol. 19, No. 8, 2008, p. 1415-1430.

Research output: Contribution to journalArticle

Vieira, Douglas A G ; Takahashi, Ricardo H C ; Palade, Vasile ; Vasconcelos, J. A. ; Caminhas, W. M. / The Q-norm complexity measure and the minimum gradient method : A novel approach to the machine learning structural risk minimization problem. In: IEEE Transactions on Neural Networks. 2008 ; Vol. 19, No. 8. pp. 1415-1430.
@article{a2535f51b47c47b69084b3c93bdd88ef,
title = "The Q-norm complexity measure and the minimum gradient method: A novel approach to the machine learning structural risk minimization problem",
abstract = "This paper presents a novel approach for dealing with the structural risk minimization (SRM) applied to a general setting of the machine learning problem. The formulation is based on the fundamental concept that supervised learning is a bi-objective optimization problem in which two conflicting objectives should be minimized. The objectives are related to the empirical training error and the machine complexity. In this paper, one general Q-norm method to compute the machine complexity is presented, and, as a particular practical case, the minimum gradient method (MGM) is derived relying on the definition of the fat-shattering dimension. A practical mechanism for parallel layer perceptron (PLP) network training, involving only quasi-convex functions, is generated using the aforementioned definitions. Experimental results on 15 different benchmarks are presented, which show the potential of the proposed ideas.",
keywords = "Complexity measure, Multiobjective training algorithms, Neural networks, Parallel layer perceptron (PLP), Regularization methods, Structural risk minimization (SRM)",
author = "Vieira, {Douglas A G} and Takahashi, {Ricardo H C} and Vasile Palade and Vasconcelos, {J. A.} and Caminhas, {W. M.}",
year = "2008",
doi = "10.1109/TNN.2008.2000442",
language = "English",
volume = "19",
pages = "1415--1430",
journal = "IEEE Transactions on Neural Networks",
issn = "1045-9227",
publisher = "Institute of Electrical and Electronics Engineers",
number = "8",

}

TY - JOUR

T1 - The Q-norm complexity measure and the minimum gradient method

T2 - A novel approach to the machine learning structural risk minimization problem

AU - Vieira, Douglas A G

AU - Takahashi, Ricardo H C

AU - Palade, Vasile

AU - Vasconcelos, J. A.

AU - Caminhas, W. M.

PY - 2008

Y1 - 2008

N2 - This paper presents a novel approach for dealing with the structural risk minimization (SRM) applied to a general setting of the machine learning problem. The formulation is based on the fundamental concept that supervised learning is a bi-objective optimization problem in which two conflicting objectives should be minimized. The objectives are related to the empirical training error and the machine complexity. In this paper, one general Q-norm method to compute the machine complexity is presented, and, as a particular practical case, the minimum gradient method (MGM) is derived relying on the definition of the fat-shattering dimension. A practical mechanism for parallel layer perceptron (PLP) network training, involving only quasi-convex functions, is generated using the aforementioned definitions. Experimental results on 15 different benchmarks are presented, which show the potential of the proposed ideas.

AB - This paper presents a novel approach for dealing with the structural risk minimization (SRM) applied to a general setting of the machine learning problem. The formulation is based on the fundamental concept that supervised learning is a bi-objective optimization problem in which two conflicting objectives should be minimized. The objectives are related to the empirical training error and the machine complexity. In this paper, one general Q-norm method to compute the machine complexity is presented, and, as a particular practical case, the minimum gradient method (MGM) is derived relying on the definition of the fat-shattering dimension. A practical mechanism for parallel layer perceptron (PLP) network training, involving only quasi-convex functions, is generated using the aforementioned definitions. Experimental results on 15 different benchmarks are presented, which show the potential of the proposed ideas.

KW - Complexity measure

KW - Multiobjective training algorithms

KW - Neural networks

KW - Parallel layer perceptron (PLP)

KW - Regularization methods

KW - Structural risk minimization (SRM)

UR - http://www.scopus.com/inward/record.url?scp=49649091536&partnerID=8YFLogxK

U2 - 10.1109/TNN.2008.2000442

DO - 10.1109/TNN.2008.2000442

M3 - Article

VL - 19

SP - 1415

EP - 1430

JO - IEEE Transactions on Neural Networks

JF - IEEE Transactions on Neural Networks

SN - 1045-9227

IS - 8

ER -