Appendix: Calculation Basics of Markov Models

6.8 Appendix: Calculation Basics of Markov Models

There are two commonly used methods for evaluating a Markov model: expected value calculation (called “Markov cohort” analysis), and Monte Carlo simulation (first-order trials or microsimulation). It is important to understand the difference between the two analysis methods, and to recognize the terms associated with them.

In an expected value analysis, the percentage of a hypothetical cohort in a state during a cycle is multiplied by the cost or utility associated with that state. These products are summed over all states and all cycles. In TreeAge Pro, expected value calculations are the basis of most analyses, including n-way sensitivity analysis and baseline cost-effectiveness analysis.

On the other hand, in a microsimulation (a.k.a., patient level simulation), a single trial’s value is simply the sum of the rewards/tolls/payoffs for the path traversed by an “individual” taking a random walk through the model’s chance nodes (using a Monte Carlo pseudo-random number series). An expected value is estimated by averaging as many trials as possible.

In TreeAge Pro, the same Markov model can be evaluated by either expected value or microsimulation methods. Generally deterministic, expected value analysis is preferred because it is more computationally efficient; it returns a mean value much more quickly than simulation, which often requires thousands of trials to return a mean value within an acceptable error.

However, some models will require simulation; refer to the section about Patient Level Simulation for more details

Additional background discussion can be found in:

Decision Making in Health and Medicine, Hunink, and Glasziou (2001), Cambridge University.

You are urged to consult a variety of publications dealing with the concepts which underlie Markov modeling and simulation.

Non-standard Markov Models

In TreeAge Pro, the basic Markov modeling rules outlined above can be overruled in a variety of ways, for example:

Time-dependent Markov models are easily handled using tables, tunnels, and/or tracker variables (where trackers can only be used in patient level simulation models).
Discrete Event Simulation (DES) models can combine samplings from event time distributions and simulation features like tracker variables, parallel trials, and dynamic populations.
A Markov model can be analyzed using the Node() function in such a way that sensitivity analysis and other cohort-type analyses can be used, while the Markov model is actually evaluated using microsimulation trials.
EV/Cohort analysis of a Markov model can make use of a Dynamic Cohort Models with a specific starting size and composition that may change over time. Useful for models where the state of the cohort can impact the transitions between health states.