Subvector-quantized high-density discrete hidden Markov model and its re-estimation

We investigated two methods to improve the performance of high-density discrete hidden Markov model (HDDHMM). HDDHMM employs discrete densities with a very large codebook consisting of thousands to tens of thousands of vector quantization (VQ) codewords which are constructed as the product of per-dimension scalar quantization (SQ) codewords. Although the subsequent HDDHMM is fast in decoding, it is not accurate enough. In this paper, making use of the fact that, for a fixed number of bits, VQ is more efficient than SQ, subvector quantization (SVQ) was investigated to improve the quantization efficiency while keeping the (time and space) complexity of the quantizer sufficiently low. Model parameters of the resulting SVQ-HDDHMM were further re-estimated. For the Wall Street Journal 5K-vocabulary task, it is found that the proposed SVQ-HDDHMM could be a better model (both in terms of recognition time and error rate) than conventional continuous-density HMM for practical deployment.