Right heart catheterization (RHC) is a well-established diagnostic tool for patients with congenital and acquired right heart disease and to actively monitor critically ill patients in the intensive care unit. Notably, previous randomized controlled trials and observational studies provide limited support for the clinical utilization of RHC in critically ill patients. However, traditional statistical methods incorporating propensity scores rely heavily on potentially invalid parametric assumptions, leading to biased results. We studied the data from the Study to Understand Prognoses and Preferences for Outcomes and Risks and Treatments (SUPPORT) and investigated differences in survival among patients with and without RHC using an innovative, semiparametric, double-robust substitution estimator called targeted maximum likelihood estimation (TMLE). We confirmed that critically ill patients who received an RHC had a significantly decreased 30-day and 60-day survival compared to patients who did not receive one after adjusting for a variety of potential confounder selection strategies. This paper will advance the understanding of TMLE for analysis of observational studies, and promote the application of TMLE in the critical care studies.
Confounder, Critical care, Right heart catheterization, Targeted maximum likelihood estimation
Despite its long-standing clinical use since 1970 [1], right heart catheterization (RHC) for critically ill patients remains controversial. Initial observational series [2] and subsequent randomized controlled trials [3] failed to demonstrate benefit. While RHC remains a fundamental diagnostic tool in the outpatient setting for a variety of cardiopulmonary diseases (notably pulmonary hypertension, severe heart failure or prior heart transplantation, and shunt quantification), some observers have written "obituaries" for the pulmonary artery catheter used to perform RHC in critical care settings [4]. Although RHC for cardiogenic shock remains a distinct entity with possible advantage [5], its broader role in critically ill patients has been appropriately limited due to the results of clinical trials.
One of the first studies to question the value of RHC in critically ill patients used an observational design [2]. Not only did it show no benefit, but it suggested increased mortality. Because this signal of harm was not seen in a subsequent meta-analysis of over 5,000 patients from 13 randomized trials [3], a reasonable hypothesis suggests that unadjusted bias existed within the observational cohort where RHC was selected for clinical indications [2].
In medical research, the marginal treatment effect at the population level is often of particular interest. It is the average causal effect of the treatment on the outcome by comparing the population if everyone was treated to the population if everyone were untreated. In practice, potential confounding covariates will invalidate such causal effects. Propensity score methods (PSM) [6,7] are widely used to address measured confounding, and Connors, et al. [2] used PSM to analyze their observational series. However, PSM suffers from problems that can make the study results misleading, biased, or challenging to interpret. First, the effect and validity of PSM for removing confounding bias heavily depends on the correct specification of the parameter propensity score model. But restricted parametric statistical models with a finite number of unknown parameters used for calculating the propensity score may not capture the true, complicated, underlying probability distribution of the real world. Second, maximum likelihood estimation (MLE) relies on an overall fit of the entire probability distribution, but the parameter of interest (such as the marginal difference of the treatment effects) is usually related to just a relevant subset. As a result, the estimation is not efficient. Third, as an estimating-equation-based methods, PSM is not robust to sparsity. Specifically, extreme propensity scores can result in unstable estimates with high variance. The estimates may be even outside the constraints of the statistical model.
TMLE [8] has been proposed to overcome these problems by offering the following advantages. First, TMLE is an instrumental tool used in semiparametric or nonparametric models avoiding the assumption of a misspecified parametric statistical model. Moreover, TMLE has excellent flexibility in incorporating a variety of machine learning methods to estimate the outcome and exposure mechanisms, which can minimize bias in comparison with misspecified regressions. Using machine learning algorithms can be particularly advantageous in avoiding model misspecification when there is a large number of covariates and potentially complex relationships among them. With semiparametric and machine learning techniques, TMLE can capture the true probability distribution. Second, TMLE targets learning with a specific focus on the parameter of interest instead of the whole distribution. As a result, it is more efficient than traditional MLE. Third, TMLE is doubly robust, which means that TMLE leads to unbiased estimates even under misspecification of one of the outcome models or the treatment model. Fourth, TMLE employs a substitution estimator that is more robust to outliers and sparsity than the estimating-equation-based estimators. Fifth, results from TMLE can be interpreted as causal effects under standard causal assumptions. However, due to its novelty and theoretical difficulty, TMLE has seldom been used in critical care studies.
Therefore, we re-analyzed the original observational study of RHC in critically ill patients but using TMLE instead of their original PSM as a novel application to study its ability to match the results of randomized controlled trials.
The Study to Understand Prognoses and Preferences for Outcomes and Risks and Treatments (SUPPORT) [2] was a 5-center study of decision making and outcomes in severely ill, hospitalized, adult patients. The five centers were: Beth Israel Hospital, Boston, Mass; Duke University Medical Center, Durham, NC; Metro-Health Medical Center, Cleveland, Ohio; St Joseph's Hospital, Marshfield, WI; and University of California Medical Center, Los Angeles. Study coordination was performed via George Washington University, Washington, DC, and the statistical center was at Duke University. The SUPPORT study design contained two phases. Phase 1 was a prospective observational study, with 4301 patients enrolled from June 1989 to June 1991. Phase 2 was a cluster randomized control trial study, with 4804 patients enrolled from January 1992 to January 1994. Eligibility criteria included patients admitted or transferred to the ICU in the first 24 hours, producing a study population of 5735 patients.
TMLE is a two-step semiparametric method. Denote A as the treatment assignment such that A = 1 means the patient received RHC and A = 0 means the patient did not receive RHC. Let W denote patient baseline covariates, and denote as the survival probability of a patient at time t given treatment assignment A and baseline covariates W. Define the target parameter as
, where Ψ maps a distribution P to a real number in R, P0 is the true data-generating distribution, Q0 is a portion of P0 relevant to Ψ.
In the first step, obtain an initial estimation of P0 or Q0 by using machine learning (super learner). Denote the initial estimator as . Given an initial estimator, we can build a fluctuation working model and define a proper loss function of which the derivative with respect to ε equals the influence curve of the target parameter. In the second step, update the initial estimate in a step targeted toward making an optimal bias-variance tradeoff for the parameter of interest , instead of the whole distribution P0. We can update the initial estimator by estimating through an interated or a one-step process (usually a logistic regression) and denote the updated version as and . The TMLE is the substitution estimator obtained by plugging or into the target parameter mapping , and TMLE solves the efficient influence curve equation , where denotes the observed influence curve under distribution . Under regular conditions, we have
Where is our true target parameter, can be estimated by . Then hypothesis testing and interval estimation can be performed.
Differences in the 30-day and 60 day survival were estimated with TMLE. Three different ranges of potential confounders were used to perform TMLE, as shown in Table 1. In the first simple set (W1), we combined literature knowledge and the findings from the original analysis [2] to select covariates that were likely to influence the use of RHC such as age, sex, race, type of insurance, primary disease category, and APACHE score. The moderate set (W2) included variables from the simple set (W1) and variables associated with RHC use from the original analysis such as cancer status, comorbidities, support model estimate of surviving 2 months, mean blood pressure, white blood cell count, heart rate, respiratory rate, temperature, albumin, sodium, pH levels, do-not-resuscitate DNR status on day 1. The full set (W3) included all potential confounders.
Table 1: Potential confounders used in implementing TMLE and PSM. View Table 1
When TMLE makes use of the simple covariate set (W1), the difference in the 30-day and 60-day survival between patients with RHC and patients without RHC were -0.03 (95% CI, (-0.056, -0.005)) and -0.03 (95% CI, (-0.057, -0.003)), respectively. When TMLE makes use of the moderate covariate set (W2), the difference in the 30-day and 60-day survival between patients with RHC and patients without RHC were -0.05 (95% CI, (-0.077, -0.023)) and -0.05 (95% CI, (-0.077, -0.023)), respectively. When TMLE makes use of the full covariate set (W3), the difference in the 30-day and 60-day survival between patients with RHC and patients without RHC were -0.06 (95% CI, (-0.087, -0.033)) and -0.06 (95% CI, (-0.089, -0.031)), respectively. All the results (Table 2) showed that patients who did not undergo RHC had a significantly higher survival probability than patients who underwent RHC.
Table 2: Differences in the 30-day and 60-day survival estimated by TMLE. View Table 2
Connors, et al. [2] used PSM to analyze the dataset in the SUPPORT study. However, there are many problems with PSM that make the study results misleading, biased, and challenging to interpret. This paper used an innovative alternative to PSM, TMLE, to confirm that patients with RHC had a significantly decreased 30-day and 60-day survival in comparison to patients without RHC during initial care. This paper will advance the understanding of TMLE for analysis of observational studies, and promote the application of TMLE in the critical care studies.