TY - GEN
T1 - Severity of local maxima for the em algorithm
T2 - 3rd European Workshop on Probabilistic Graphical Models, PGM 2006
AU - Wang, Yi
AU - Zhang, Nevin L.
PY - 2006
Y1 - 2006
N2 - It is common knowledge that the EM algorithm can be trapped at local maxima and consequently fails to reach global maxima. We empirically investigate the severity of this problem in the context of hierarchical latent class (HLC) models. Our experiments were run on HLC models where dependency between neighboring variables is strong. (The reason for focusing on this class of models will be made clear in the main text.) We first ran EM from randomly generated single starting points, and observed that (1) the probability of hitting global maxima is generally high, (2) it increases with the strength of dependency and sample sizes, and (3) it decreases with the amount of extreme probability values. We also observed that, at high dependence strength levels, local maxima are far apart from global ones in terms of likelihoods. Those imply that local maxima can be reliably avoided by running EM from a few starting points and hence are not a serious issue. This is confirmed by our second set of experiments.
AB - It is common knowledge that the EM algorithm can be trapped at local maxima and consequently fails to reach global maxima. We empirically investigate the severity of this problem in the context of hierarchical latent class (HLC) models. Our experiments were run on HLC models where dependency between neighboring variables is strong. (The reason for focusing on this class of models will be made clear in the main text.) We first ran EM from randomly generated single starting points, and observed that (1) the probability of hitting global maxima is generally high, (2) it increases with the strength of dependency and sample sizes, and (3) it decreases with the amount of extreme probability values. We also observed that, at high dependence strength levels, local maxima are far apart from global ones in terms of likelihoods. Those imply that local maxima can be reliably avoided by running EM from a few starting points and hence are not a serious issue. This is confirmed by our second set of experiments.
UR - https://www.scopus.com/pages/publications/52249110529
M3 - Conference Paper published in a book
AN - SCOPUS:52249110529
SN - 8086742148
SN - 9788086742144
T3 - Proceedings of the 3rd European Workshop on Probabilistic Graphical Models, PGM 2006
SP - 301
EP - 308
BT - Proceedings of the 3rd European Workshop on Probabilistic Graphical Models, PGM 2006
Y2 - 12 September 2006 through 15 September 2006
ER -