Comparing Information Metrics for a Coupled Ornstein–Uhlenbeck Process

James Heseltine, Eun-jin Kim

Research output: Contribution to journalArticle

1 Downloads (Pure)

Abstract

It is often the case when studying complex dynamical systems that a statistical formulation can provide the greatest insight into the underlying dynamics. When discussing the behavior of such a system which is evolving in time, it is useful to have the notion of a metric between two given states. A popular measure of information change in a system under perturbation has been the relative entropy of the states, as this notion allows us to quantify the difference between states of a system at different times. In this paper, we investigate the relaxation problem given by a single and coupled Ornstein–Uhlenbeck (O-U) process and compare the information length with entropy-based metrics (relative entropy, Jensen divergence) as well as others. By measuring the total information length in the long time limit, we show that it is only the information length that preserves the linear geometry of the O-U process. In the coupled O-U process, the information length is shown to be capable of detecting changes in both components of the system even when other metrics would detect almost nothing in one of the components. We show in detail that the information length is sensitive to the evolution of subsystems.
Original languageEnglish
Article number775
Number of pages16
JournalEntropy
Volume21
Issue number8
DOIs
Publication statusPublished - 8 Aug 2019
Externally publishedYes

Fingerprint

entropy
dynamical systems
divergence
formulations
perturbation
geometry

Bibliographical note

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Keywords

  • Fisher information
  • Fokker-Planck equation
  • Information length
  • Langevin equation
  • Metrics
  • O-U process
  • Probability density function
  • Stochastic processes

ASJC Scopus subject areas

  • Physics and Astronomy(all)

Cite this

Comparing Information Metrics for a Coupled Ornstein–Uhlenbeck Process. / Heseltine, James; Kim, Eun-jin.

In: Entropy, Vol. 21, No. 8, 775, 08.08.2019.

Research output: Contribution to journalArticle

@article{7666110dda234e54915765c71212f490,
title = "Comparing Information Metrics for a Coupled Ornstein–Uhlenbeck Process",
abstract = "It is often the case when studying complex dynamical systems that a statistical formulation can provide the greatest insight into the underlying dynamics. When discussing the behavior of such a system which is evolving in time, it is useful to have the notion of a metric between two given states. A popular measure of information change in a system under perturbation has been the relative entropy of the states, as this notion allows us to quantify the difference between states of a system at different times. In this paper, we investigate the relaxation problem given by a single and coupled Ornstein–Uhlenbeck (O-U) process and compare the information length with entropy-based metrics (relative entropy, Jensen divergence) as well as others. By measuring the total information length in the long time limit, we show that it is only the information length that preserves the linear geometry of the O-U process. In the coupled O-U process, the information length is shown to be capable of detecting changes in both components of the system even when other metrics would detect almost nothing in one of the components. We show in detail that the information length is sensitive to the evolution of subsystems.",
keywords = "Fisher information, Fokker-Planck equation, Information length, Langevin equation, Metrics, O-U process, Probability density function, Stochastic processes",
author = "James Heseltine and Eun-jin Kim",
note = "{\circledC} 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).",
year = "2019",
month = "8",
day = "8",
doi = "10.3390/e21080775",
language = "English",
volume = "21",
journal = "Entropy",
issn = "1099-4300",
publisher = "MDPI",
number = "8",

}

TY - JOUR

T1 - Comparing Information Metrics for a Coupled Ornstein–Uhlenbeck Process

AU - Heseltine, James

AU - Kim, Eun-jin

N1 - © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

PY - 2019/8/8

Y1 - 2019/8/8

N2 - It is often the case when studying complex dynamical systems that a statistical formulation can provide the greatest insight into the underlying dynamics. When discussing the behavior of such a system which is evolving in time, it is useful to have the notion of a metric between two given states. A popular measure of information change in a system under perturbation has been the relative entropy of the states, as this notion allows us to quantify the difference between states of a system at different times. In this paper, we investigate the relaxation problem given by a single and coupled Ornstein–Uhlenbeck (O-U) process and compare the information length with entropy-based metrics (relative entropy, Jensen divergence) as well as others. By measuring the total information length in the long time limit, we show that it is only the information length that preserves the linear geometry of the O-U process. In the coupled O-U process, the information length is shown to be capable of detecting changes in both components of the system even when other metrics would detect almost nothing in one of the components. We show in detail that the information length is sensitive to the evolution of subsystems.

AB - It is often the case when studying complex dynamical systems that a statistical formulation can provide the greatest insight into the underlying dynamics. When discussing the behavior of such a system which is evolving in time, it is useful to have the notion of a metric between two given states. A popular measure of information change in a system under perturbation has been the relative entropy of the states, as this notion allows us to quantify the difference between states of a system at different times. In this paper, we investigate the relaxation problem given by a single and coupled Ornstein–Uhlenbeck (O-U) process and compare the information length with entropy-based metrics (relative entropy, Jensen divergence) as well as others. By measuring the total information length in the long time limit, we show that it is only the information length that preserves the linear geometry of the O-U process. In the coupled O-U process, the information length is shown to be capable of detecting changes in both components of the system even when other metrics would detect almost nothing in one of the components. We show in detail that the information length is sensitive to the evolution of subsystems.

KW - Fisher information

KW - Fokker-Planck equation

KW - Information length

KW - Langevin equation

KW - Metrics

KW - O-U process

KW - Probability density function

KW - Stochastic processes

UR - http://www.scopus.com/inward/record.url?scp=85070445031&partnerID=8YFLogxK

U2 - 10.3390/e21080775

DO - 10.3390/e21080775

M3 - Article

VL - 21

JO - Entropy

JF - Entropy

SN - 1099-4300

IS - 8

M1 - 775

ER -