The Q-norm complexity measure and the minimum gradient method: A novel approach to the machine learning structural risk minimization problem

Douglas A G Vieira, Ricardo H C Takahashi, Vasile Palade, J. A. Vasconcelos, W. M. Caminhas

Research output: Contribution to journalArticle

22 Citations (Scopus)

Abstract

This paper presents a novel approach for dealing with the structural risk minimization (SRM) applied to a general setting of the machine learning problem. The formulation is based on the fundamental concept that supervised learning is a bi-objective optimization problem in which two conflicting objectives should be minimized. The objectives are related to the empirical training error and the machine complexity. In this paper, one general Q-norm method to compute the machine complexity is presented, and, as a particular practical case, the minimum gradient method (MGM) is derived relying on the definition of the fat-shattering dimension. A practical mechanism for parallel layer perceptron (PLP) network training, involving only quasi-convex functions, is generated using the aforementioned definitions. Experimental results on 15 different benchmarks are presented, which show the potential of the proposed ideas.

Original languageEnglish
Pages (from-to)1415-1430
Number of pages16
JournalIEEE Transactions on Neural Networks
Volume19
Issue number8
DOIs
Publication statusPublished - 2008
Externally publishedYes

Keywords

  • Complexity measure
  • Multiobjective training algorithms
  • Neural networks
  • Parallel layer perceptron (PLP)
  • Regularization methods
  • Structural risk minimization (SRM)

ASJC Scopus subject areas

  • Software
  • Medicine(all)
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'The Q-norm complexity measure and the minimum gradient method: A novel approach to the machine learning structural risk minimization problem'. Together they form a unique fingerprint.

Cite this