That is the title of a helpful article by Karnon and Haji Ali Afzali (Pharmacoeconomics 2014) with a subtitle A Review and Critique of the Costs and Benefits of DES.
The paper starts by highlighting the 3 most common types of models: decision trees, Markovian cohort models, and individual level models. Decisions trees are useful for simple, short-term models, but not useful for more complex modeling tasks or longer time horizons. Cohort models are the most popular, but “…[b]ecause such models follow a cohort rather than individuals, they are subject to the Markovian assumption, which states that the subsequent (or future) pathway of an object depends only upon the present health state, not on the sequence of states that preceded it. Individual models include
discrete time and continuous time state transition models, and discrete event simulation (DES). “All of these models overcome the Markovian assumption by assigning attributes to individuals that can influence their progression through the model, but which can also be updated as individuals’ experience events within the model.” What is the difference between the discrete/continuous time transition models and DES? The former are focused around health states; DES is organized around events.
What are the pros and cons of DES? I summarize some of these below.
Pros. The primary benefits of DES is that they can more easily incorporate the following 4 scenarios:
Baseline heterogeneity. “Most cohort-based models define a single, homogeneous population (e.g., women aged 60 years with node-positive early breast cancer [10]), for whom a single set of input parameter values is specified. By facilitating the representation of heterogeneity in the eligible population, DES may improve model accuracy if the baseline characteristics are not jointly and separately normally distributed, and the baseline characteristics are not linear in effect (e.g., for each unit increase in age, the probability of an event increases by 1%). The value of representing baseline heterogeneity is a function of the magnitude of the potential improvement in model validity, and the quality of the data available to populate the model.”Continuous disease markers. In the real world, diseases often change gradually over time. DES allow for this type of modelling. For instance, if a clinical trial only has surrogate markers, being able to link these surrogate measures to outcomes is very helpful. How is it done? “Firstly, a model(s) is generated to predict changes in the disease marker variable(s) over time. Secondly, models to predict the likelihood of relevant clinical events are applied at each time point, given current disease marker levels (and other patient characteristics). Such models require robust data describing disease marker progression, and the relationship between disease markers and the incidence of relevant clinical events (e.g., diabetic retinopathy, cardiovascular events, and renal disease).” Time varying event rates. Time-varying event rates can be implemented into cohort models but it is much more complex than in DES. “the representation of time-varying probabilities in [cohort models requires] states that are entered beyond the start time of the model requires the use of tunnel states: a health state is disaggregated into multiple sub-states, each of which represents the health state at some set time beyond commencement in that state. The implementation of time-varying event rates is more straightforward in a DES model – the model simply samples a time to event from a specified survival function.” When does this make a difference? Largely when there are ‘knock-on effects’ of a given event. “Improvements in model validity from representing time-varying event rates to non-absorbing health states are likely to be larger. In these cases, variation in the distribution of the time to next event has a knock-on effect. In the stroke prevention model, new times to subsequent secondary events were sampled following the experience of each secondary event.”Influence of prior events on subsequent event rates. Again, the impact of prior events on subsequent transition states can be modelled in cohort models but the complexity multiplies quickly. “The models that best illustrate this factor are the diabetes models, which mostly use the UKPDS Outcomes Model risk equations to describe the incidence of seven diabetes-related complications. To represent all potential combinations of the seven complications would require 127 separate health states…Other examples include models in which the same event can be experienced multiple times and the number of episodes is an important predictor, for example, a model of schizophrenia recorded the number of relapse episodes as a patient attribute.”
Cons: The main con is complexity and need for additional validation.
Complexity. “The costs of DES relate to the time and expertise required to implement and review complex models, when perhaps a simpler model would suffice. The costs are not borne solely by the analyst, but also by reviewers.” Validation. All models need validation. Because DES are more complex and may appear less transparent to reviewers (e.g., academic reviewers, HTA bodies, other researchers), conducting validation exercises is critical.
So which model should you use? Decision tress, cohort models, or individual-level models like DES? The authors turn to Albert Einstein to provide some guidance:
‘Everything should be made as simple as possible, but not simpler.’
The article also conducts a literature review of all DES models that were used as part of an economic evaluation of health technologies. You can read the full article here.
