the hidden markov model homework solution

the hidden markov model homework solution

Hidden Markov Model: Theory and Applications

1. Introduction to Hidden Markov Models

Markov properties often occur naturally when analyzing processes that evolve in time. In particular, in the case of discrete processes, the Markov property states that given the value of the process at some time, the remaining behavior is conditionally independent of its past given this present value. Discrete time processes of this type are known as discrete time Markov chain processes, and are among the best understood and most important classes of stochastic processes. One reason for their popularity is that standard mathematical results are available for such processes, permitting in-depth study of their behavior as well as their use in many applications. The subject of phase type distribution and models, for example, shows that all families of absolutely continuous phase type distributions are in one-to-one correspondence with continuous time Markov jump processes.

When modeling behavior that evolves over time, such as speech and language, gene sequences, tourism activity, river flow, and the movement of a stock index, the modeler needs not only to express uncertainty about its present behavior, but has to do this in such a way that she can use this process to express her uncertainty about the behavior to come. Discrete time Markov chain models often seem too restrictive in that the modeled process does not contain enough historical information to satisfy an a priori expectation of a good model, but are also quite large and difficult to parameterize when they are made to contain a large amount of historical information to satisfy this expectation. In many applications, the use of Markov chains is an appropriate choice as their extensive history of development and use demonstrates. The Hidden Markov Model technique introduced by L.E. Baum and his co-workers is a highly useful extension to the Markov chain when the chain-based model is too simplistic for practical problems, but the simple strategy of assuming that the distribution of the data in each state of the chain is known is also too simple.

2. Mathematical Foundations of Hidden Markov Models

A Hidden Markov Model (HMM) is a statistical model in which the system being modeled is assumed to be a Markov process with unobservable (i.e. hidden) states. While the Hidden Markov Model’s mathematics represent the problems of state inference and parameter estimation in terms of probability distributions, almost all real-world work with HMMs is done using the forward algorithm and the Viterbi algorithm, both examples of dynamic programming. A Hidden Markov Model can also be described by a sequence of symbols, generated while in the sequence of the state. Each symbol also belongs to a finite training set of symbols. The observed state is then randomly selected from the obtained sequence. This concept model has wide application in such areas as computer vision, speech recognition, natural language processing, computational biology, and reinforcement learning.

In this paper, we have developed a theory of HMM specific for Markov systems. Also, a wide class of Markov processes were investigated for which the developed theory can be directly used. Two fundamental problems in the statistical theory are estimation of Markov process model from observation (latent Markov model problem) and estimation of process properties using model (Markov switching regression).

3. Applications of Hidden Markov Models

Recent biotechnological advances in the techniques for observing microbiological systems have led to a tremendous growth in both the size and complexity of the data collected from such systems. At the same time, simple Markov models have been useful in inferring from observations to underlying systems. A Hidden Markov Model (HMM) is a generalization of a simple Markov model in which the state of the system is the random variable while the output of the system may depend on an unknowable random variable linked to the value of the system state. HMMs have not been widely used in microbiology, but we believe their ability to represent complex real-world interrelationships makes HMMs a valuable tool in this field. Our purpose is to introduce biologists to HMMs in hopes that some will find them a useful tool.

The usefulness of HMMs lies in the ability to define a system based on a few relatively few easily obtained values coupled with the ability to model complex interrelationships. This is an incredibly powerful combination. Although measurement of the initial probabilities of system states, as well as all transition and output probabilities would be infeasible in any system of size, in many cases these values are largely fixed by the experimental design. Our work presents a comprehensive introduction to HMMs in which we start with the basic theory of HMMs as well as known statistical (Backward-Forward, Baum-Welch) methods for estimating the parameters of HMMs from examined cases, and existing algorithms for computing the probabilities of various occurrences under a model. To anchor our general introduction in working examples, we provide real-world model construction as well as model-based delegation based on solved HMMs. We also model a small bacterial regulatory network using HMMs, then compare the performance of the resulting HMMs to an alternative model of the system. Our work is accessible and our methods are general.

4. Advanced Topics in Hidden Markov Models

We suppose that the readers have mastered the basic ideas and the relatively simple computation algorithms for a Hidden Markov model (HMM). Thus, in this chapter, we focus on some relatively advanced topics on HMM. A Hidden Markov model has three major problem areas: model, learning, and the specific inference problem. In this chapter, we discuss advanced issues in these areas. Although the idea of Hidden Markov models is simple, nevertheless, there is a large volume of literature on HMM. Crucial to understanding HMM is the ability to calculate the probability of a given sequence. Presented here are some of the well-known algorithms related to HMM that enable the calculation of the probabilities of a given sequence with respect to the drawn model.

Fitting the model in a numerical sense to maximize the probability of the data, and finally, advanced topics related to using Hidden Markov models are discussed here. What we presented here is only a small subset of the numerous algorithms that are available; quite often, other practical algorithms were developed with the emphasized purpose of being computationally efficient and can be readily applied. Nevertheless, we believe that the computation techniques described here cover a broad set of known algorithms to get a strong insight for HMM.