34.10 Counting "time in state" with tunnel states

A tunnel state can be used when you need to keep track of the number of cycles an individual has remained in a particular state. In a cancer state, for example, transition probabilities to other states often depend on how long the individual has been in the cancer state.

Tunnels versus simulation/trackers:

Instead of using tunnel states, some Markov models are built using tracker variables and microsimulation to count time-in-state; each approach has its advantages. Tunnel states work during both cohort analysis (i.e., rollback, n-way sensitivity analysis) and microsimulation. Trackers are more restrictive, requiring microsimulation (a.k.a. individual-level simulation) which is generally a more time-consuming analysis. Trackers, however, are extremely flexible, and can be used to keep track of unlimited continuous and discrete states, transient events, etc. Models built using tracker variables can be made structurally simpler, with fewer states/branches. Refer to the Individual-Level Simulation and Markov Models Chapter for details on tracker variables.

If a model is already using trackers/microsimulation, then it will be more efficient/parsimonious to track time-in-state, rather than “tunnel” it.

Temporary states and the _tunnel counter

In the Markov modeling literature, a temporary state is a state which an individual must exit after one cycle and a tunnel is a series of temporary states. Normally, an individual entering the tunnel state — either from another state, or at the start of the Markov process — enters temporary state #1. If an individual remains in the tunnel state for another cycle, they move in order through temporary states #2, #3, and so on.

One way to model a tunnel is to use a separate state for each temporary state, and manually set up ordered transitions between the states. However, TreeAge Pro allows you to represent a tunnel more efficiently, using a single branch from the Markov node; this makes it easy to create tunnels of any length (even with thousands of temporary states).

When you create a tunnel state in TreeAge Pro, all temporary states will use the same transition subtree. In order to specify different transitions for particular temporary states, you can refer to TreeAge Pro’s temporary state counter, a Markov keyword called _tunnel (similar to _stage). TreeAge Pro starts the _tunnel counter at 1 for someone entering a _tunnel state, and increments the counter by 1 each cycle they remain in the tunnel state. Using _tunnel to count “time in state,” your transition probabilities can look up appropriate values from tables. Logic nodes and statements can also use the _tunnel counter.

Using a tunnel state – an example

Consider the three-state cancer model shown below. Note that since there is no transition from Cancer back to itself, the patient spends one cycle in the Cancer state before exiting to another state, and therefore Cancer is a temporary state (although not a tunnel yet).

This model may not accurately represent the basic process of the disease. Cancer should probably unfold in a series of temporary states, with different probabilities of changing state (i.e., remission or death) in each successive cycle/year. And these probabilities should depend on how many cycles someone has spent in the Cancer state (which is not given by the _stage counter, since everyone starts in Pre-cancerous, and a transition to Cancer may occur at any cycle).

An “exploded” version of the cancer Markov model is shown below. An explicit chain of temporary states is used to describe each year of the cancer. While this more detailed model is still relatively small, as the required number of temporary states increases, explicitly representing all of them becomes more problematic. In TreeAge Pro, the chain of temporary states can be represented more efficiently using a single tunnel state.

The same Markov process (i.e., calculating the same results) can be built in TreeAge Pro with Cancer set as a tunnel state, as illustrated below. See the Health Care tutorial example model Cancer Tunnel.tex.

To change a state to a tunnel state:

• Select the appropriate Markov state.

• Choose Views > Markov from the toolbar.

• In the Markov View, set the Tunnel Max value equal to the maximum required number of temporary states.

The number specified determines how high the _tunnel counter will increment (corresponding to the number of copies of the state which TreeAge Pro keeps track of internally during calculations). Individuals that reach the last temporary state and transition into the state again will simply remain in the last temporary state.

Set the number of temporary states to the minimum number that will account for all differences among the temporary states. For example, if the only difference among the temporary states is a probability that changes for the first four cycles then stabilizes, you will need four temporary states.

Using the _tunnel counter

Note the use of the _tunnel counter/keyword in probabilities in the Cancer transition subtree shown in the prior section. The Die node probability is T_MORT[_tunnel]. As a portion of the cohort remains in the state, _tunnel increases, and a different probability is pulled from the table.

Merging results for temporary states

When you perform a Markov cohort analysis at a Markov node that includes a tunnel state, you are given the option to merge each tunnel’s temporary states into a single text report column or graph line. In fact, that is the default presentation of data.

This is particularly useful if there are many temporary states in a tunnel. Prior to running the analysis, TreeAge Pro will present the prompt shown above, asking whether or not to merge temporary states. If you choose not to merge temporary states, the text report will append extra columns for the second and subsequent temporary states.

One positive result of merging a tunnel’s temporary states is a simpler, more coherent line graph, as shown below.

State Probabilities Graph with Tunnels expanded:

State Probabilities Graph with Tunnels collapsed:

Populating temporary states at _stage = 0 (cycle 0)

Normally, the initial probability expression assigned to a tunnel state is used to populate only the first temporary state. It is possible, however, to distribute members of the cohort among the different temporary states at cycle 0. The Tunnel Info section of the Markov State Information dialog includes an advanced setting that will cause TreeAge Pro to evaluate the initial probability expression in a tunnel state for every temporary state.

In the above example, the "Calculate temp state initial probs" is set to true, and 25% of the cohort should start the analysis in each of the 4 temporary states associated with this tunnel state.

This could be used, for example, to create a model that uses the _tunnel counter to track the age of the members of the cohort with a realistic age distribution. Every state could be made a tunnel state, with their initial probabilities referencing tables of probabilities using the _tunnel counter (creating the age distribution).

Special binding names and tunnels

Markov bindings were first discussed within the context of clones Cloning Markov Models and using State Bindings.

TreeAge Pro supports three special Markov binding names:

• Tunnel Crossover

• Tunnel Variable

• Tunnel Table

The Tunnel Crossover binding is used at a transition node to allow the _tunnel counter to increment uninterrupted when transitioning from one state to a different state, provided both are tunnel states. Normally, the _tunnel counter would reset to 1 when moving to a different state. If the tunnel crossover value exceeds the maximum number of tunnels for the state, the maximum tunnel value is used.

The Tunnel Variable binding is used to dynamically point any transition node to a specific temporary state in the destination tunnel state. The variable must have the name "tunnel_" appended to the front of the starting health state (replace spaces with underscores). See example model Markov Tunnel Bindings for details.

The Tunnel Table binding is used in a similar way to Tunnel Variable, except that transitions to particular temporary states are determined by numbers pulled from a column in a TreeAge Pro table. The tunnel table binding will have a name like Tunnel Table C1 where C1 refers to the first value column (not index column) from a table with the name "tunnel_" appended to the front of the starting health state (replace spaces with underscores). When the cohort reaches the tunnel table binding, it looks for the table, finds the row with an index equal to the current _tunnel value, then goes to the appropriate column (C1, C2, etc.) to determine the _tunnel value for the next cycle, regardless of which jump state is selected for the binding. See example model Markov Tunnel Bindings Table for details.

To create a special binding, click "Add Special" on the State Bindings View toolbar. See below.

The Health Care tutorial example model "Markov Tunnel Bindings" illustrates the use of the Tunnel Crossover and Tunnel Variable special bindings.

The Tunnel Variable binding definition made at the Drug A node points to the Drug A state. The use of this binding at the max doses, restart node within the Drug A transition subtree means that this transition will return to the Drug A state for the next cycle. However since the binding is a Tunnel Variable, TreeAge Pro searches for variable tunnel_Drug_A ("tunnel_" + <state name> replacing spaces with underscores) to determine the temporary state for the next cycle. In this case the variable tunnel_Drug_A is defined as 1, so the temporary state will be 1 for the next cycle (restart the doses).

The Tunnel Crossover binding definition at the Drug A node points to the Drug B state and vice versa. This allows the transition for the No Response node to transition to the other state. However, since it is a Tunnel Crossover binding, the temporary state is maintained. The flow would pass from temporary state 1 in the Drug A state to temporary state 2 in the Drug B state and vice versa.