On the robustness of Bayesian networks to learning from non-conjugate sampling

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Recent results concerning the instability of Bayes Factor search over Bayesian Networks (BN’s) lead us to ask whether learning the parameters of a selected BN might also depend heavily on the often rather arbitrary choice of prior density. Robustness of inferences to misspecification of the prior density would at least ensure that a selected candidate model would give similar predictions of future data points given somewhat different priors and a given large training data set. In this paper we derive new explicit total variation bounds on the calculated posterior density as the function of the closeness of the genuine prior to the approximating one used and certain summary statistics of the calculated posterior density. We show that the approximating posterior density often converges to the genuine one as the number of sample point increases and our bounds allow us to identify when the posterior approximation might not. To prove our general results we needed to develop a new family of distance measures called local DeRobertis distances. These provide coarse non-parametric neighbourhoods and allowed us to derive elegant explicit posterior bounds in total variation. The bounds can be routinely calculated for BNs even when the sample has systematically missing observations and no conjugate analyses are possible.
Original languageEnglish
Pages (from-to)558-572
Number of pages15
JournalInternational Journal of Approximate Reasoning
Volume51
Issue number5
DOIs
Publication statusPublished - 2010

Fingerprint

Bayesian networks
Bayesian Networks
Sampling
Robustness
Total Variation
Missing Observations
Bayes Factor
Statistics
Sample point
Misspecification
Distance Measure
Learning
Converge
Prediction
Arbitrary
Approximation

Keywords

  • bayesian network
  • Bayesian robustness
  • Isoseparation property
  • Local DeRobertis distance
  • Total variation distance

Cite this

On the robustness of Bayesian networks to learning from non-conjugate sampling. / Smith, Jim; Daneshkhah, Alireza.

In: International Journal of Approximate Reasoning, Vol. 51, No. 5, 2010, p. 558-572.

Research output: Contribution to journalArticle

@article{55f75eb6f71d4a75870bb99a037b8809,
title = "On the robustness of Bayesian networks to learning from non-conjugate sampling",
abstract = "Recent results concerning the instability of Bayes Factor search over Bayesian Networks (BN’s) lead us to ask whether learning the parameters of a selected BN might also depend heavily on the often rather arbitrary choice of prior density. Robustness of inferences to misspecification of the prior density would at least ensure that a selected candidate model would give similar predictions of future data points given somewhat different priors and a given large training data set. In this paper we derive new explicit total variation bounds on the calculated posterior density as the function of the closeness of the genuine prior to the approximating one used and certain summary statistics of the calculated posterior density. We show that the approximating posterior density often converges to the genuine one as the number of sample point increases and our bounds allow us to identify when the posterior approximation might not. To prove our general results we needed to develop a new family of distance measures called local DeRobertis distances. These provide coarse non-parametric neighbourhoods and allowed us to derive elegant explicit posterior bounds in total variation. The bounds can be routinely calculated for BNs even when the sample has systematically missing observations and no conjugate analyses are possible.",
keywords = "bayesian network, Bayesian robustness, Isoseparation property, Local DeRobertis distance, Total variation distance",
author = "Jim Smith and Alireza Daneshkhah",
year = "2010",
doi = "10.1016/j.ijar.2010.01.013",
language = "English",
volume = "51",
pages = "558--572",
journal = "International Journal of Approximate Reasoning",
issn = "0888-613X",
publisher = "Elsevier",
number = "5",

}

TY - JOUR

T1 - On the robustness of Bayesian networks to learning from non-conjugate sampling

AU - Smith, Jim

AU - Daneshkhah, Alireza

PY - 2010

Y1 - 2010

N2 - Recent results concerning the instability of Bayes Factor search over Bayesian Networks (BN’s) lead us to ask whether learning the parameters of a selected BN might also depend heavily on the often rather arbitrary choice of prior density. Robustness of inferences to misspecification of the prior density would at least ensure that a selected candidate model would give similar predictions of future data points given somewhat different priors and a given large training data set. In this paper we derive new explicit total variation bounds on the calculated posterior density as the function of the closeness of the genuine prior to the approximating one used and certain summary statistics of the calculated posterior density. We show that the approximating posterior density often converges to the genuine one as the number of sample point increases and our bounds allow us to identify when the posterior approximation might not. To prove our general results we needed to develop a new family of distance measures called local DeRobertis distances. These provide coarse non-parametric neighbourhoods and allowed us to derive elegant explicit posterior bounds in total variation. The bounds can be routinely calculated for BNs even when the sample has systematically missing observations and no conjugate analyses are possible.

AB - Recent results concerning the instability of Bayes Factor search over Bayesian Networks (BN’s) lead us to ask whether learning the parameters of a selected BN might also depend heavily on the often rather arbitrary choice of prior density. Robustness of inferences to misspecification of the prior density would at least ensure that a selected candidate model would give similar predictions of future data points given somewhat different priors and a given large training data set. In this paper we derive new explicit total variation bounds on the calculated posterior density as the function of the closeness of the genuine prior to the approximating one used and certain summary statistics of the calculated posterior density. We show that the approximating posterior density often converges to the genuine one as the number of sample point increases and our bounds allow us to identify when the posterior approximation might not. To prove our general results we needed to develop a new family of distance measures called local DeRobertis distances. These provide coarse non-parametric neighbourhoods and allowed us to derive elegant explicit posterior bounds in total variation. The bounds can be routinely calculated for BNs even when the sample has systematically missing observations and no conjugate analyses are possible.

KW - bayesian network

KW - Bayesian robustness

KW - Isoseparation property

KW - Local DeRobertis distance

KW - Total variation distance

U2 - 10.1016/j.ijar.2010.01.013

DO - 10.1016/j.ijar.2010.01.013

M3 - Article

VL - 51

SP - 558

EP - 572

JO - International Journal of Approximate Reasoning

JF - International Journal of Approximate Reasoning

SN - 0888-613X

IS - 5

ER -