Bayesian Treatment of Incomplete Discrete Data applied to Mutual Information and Feature Selection

Simple item page
dc.creatorHutter, Marcus
dc.creatorZaffalon, Marco
dc.date2003-06-24
dc.date.accessioned2026-07-07T03:19:58Z
dc.date.available2026-07-07T03:19:58Z
dc.descriptionGiven the joint chances of a pair of random variables one can compute quantities of interest, like the mutual information. The Bayesian treatment of unknown chances involves computing, from a second order prior distribution and the data likelihood, a posterior distribution of the chances. A common treatment of incomplete data is to assume ignorability and determine the chances by the expectation maximization (EM) algorithm. The two different methods above are well established but typically separated. This paper joins the two approaches in the case of Dirichlet priors, and derives efficient approximations for the mean, mode and the (co)variance of the chances and the mutual information. Furthermore, we prove the unimodality of the posterior distribution, whence the important property of convergence of EM to the global maximum in the chosen framework. These results are applied to the problem of selecting features for incremental learning and naive Bayes classification. A fast filter based on the distribution of mutual information is shown to outperform the traditional filter based on empirical mutual information on a number of incomplete real data sets.
dc.description11 pages, 1 figure
dc.identifierhttps://arxiv.org/abs/cs/0306126
dc.identifierhttp://arxiv.org/abs/cs/0306126
dc.identifierProceedings of the 26th German Conference on Artificial Intelligence (KI-2003) 396-406
dc.identifier.urihttp://salesiana.dossiersoluciones.com/handle/123456789/31671
dc.subjectMachine Learning
dc.subjectArtificial Intelligence
dc.subjectProbability
dc.subjectG.3; G.1.2; I.2
dc.titleBayesian Treatment of Incomplete Discrete Data applied to Mutual Information and Feature Selection
dc.typetext

Files

Collections

arXiv