43.5 Accumulating value - time and event rewards

In DES models, values/rewards are accumulated in a similar, but slightly different way than in Markov models.

In DES models, rewards are accumulated as follows.

  • Time rewards - accumulated at time nodes based on the amount of time spent waiting for an event to occur (equivalent to a Markov state reward).

  • Event rewards - accumulated as a fixed value at any node (equivalent to a Markov transition reward).

  • Startup rewards - accumulated once as a fixed value at time nodes at _time 0.

DES Time Rewards

DES time rewards are exclusively accumulated at time nodes. The value/expression entered is appropriate for one time unit (similar to a Markov cycle); however, that value/expression is then multiplied by the time spent at that time node until an event occurs.

This is the equivalent of state rewards in a Markov model, which are accumulated based on membership in a specific health state for a fixed period of time. However, in DES models, there are no fixed cycles. Instead of accumulating a single value for that fixed cycle, the value is accumulated over the entire time spent at that node.

Let's examine this within the context of the DES.Model.With.Health.States. The figure below contains the DES View with the Mild node selected.

Note that the active payoffs (1 & 2) are for cost and effectiveness, respectively.

(If you need to change the active payoffs, use the link to Tree Preferences next to the pencil above the Name column.)

The value cMild is the value for cost and the value uMild is the value for effectiveness. When the model runs, the time node will select the next time-to-event based on the shortest time-to-event, resulting in the _event_time.

Let's assume that the following time-to-event values are calculated from formulas/distributions...

  • Time_MildToSevere = 3.80497

  • Time_MildToDead = 11.21743

The shortest time is the for the progression from Mild to Severe, so that event occurs next. At the Mild node, it does not matter which event occurs for value accumulation, just the time until that event occurs.

The cost is calculated by evaluating the cost payoff and multiplying it by the time-to-event. Consider we are in Treatment 1:

  • cMild = cMild1 = 1000

  • time to the progression to severe = 3.80497

  • cost accumulated before the first event = 1000 * 3.80497 = 3,804.97

Therefore, 3,804.87 (units of currency) are accumulated at the time node before the transition to the severe state occurs.

DES Event Rewards

DES event rewards are usually accumulated at event nodes somewhere to the right of time nodes. The value accumulated is equal to the expression you enter with no adjustment for the time-to-event.

This is the equivalent of transition rewards in a Markov model, which are accumulated based on an individual passing through a specific event in a Markov transition subtree.

The figure below contains the DES View with the Mild to Severe node selected.

The cost of the progression from Mild to Severe (parameter variable cProgress) is placed in the event reward. Any individual who passes through this event will incur the cost of the progression.

DES Startup Rewards

DES startup rewards are accumulated only once in the model at the beginning of the analysis (_time = 0). They function like event rewards in that they are a fixed value rather than being multiplied by the time-to-event. However, they should be placed at the time nodes where an individual can enter the model (initial probability > 0).

DES startup rewards are typically used for one-time-only costs that are incurred at the beginning of an analysis. For example, if you have a model that accounts for a single surgical procedure and then follows the patients into the future, the cost of surgery should be placed in the startup cost.

In many models, all startup rewards will be zero.

You can also accumulate value prior to the DES nodes by entering Upstream Payoffs. This technique is described in detail in the section Entering Upstream Payoffs Along Patient Pathways.

In the DES context, upstream payoffs are automatically added to the Startup reward for all health states.