Machine Learning and Natural Language Processing Lab

Student's Project

EM and Gradient-Based Learning of Bayesian Networks: A Case Study

Niels Landwehr, 2002

---

To learn Bayesian networks, one must estimate the parameters of the network from the data. EM (Expectation-Maximization) and gradient-based algorithms are the two best known techniques to estimate these parameters. Although the theoretical properties of these two frameworks are well studied, it remains an open question as to when and whether EM is to be preferred over gradients. In this paper, we answer this question empirically. More specifically, we first adapt scaled conjugate gradients well-known from neural network learning. This accelerated conjugate gradient avoids the time consuming line search of more traditional methods. Secondly, we empirically compare scaled conjugate gradients with EM. The experiments show that accelerated conjugate gradients are competitive with EM. Although in general EM is the domain indepedent method of choice, gradient-based methods can be superior.