Illness severity and comorbidity rating scales have been found to predict long-term outcomes for older hospitalized adults. This study was designed to determine if these rating systems could also predict short-term hospital outcomes.
Prospective observational study of hospitalized adults 65 and older. Medical students calculated comorbidity and illness severity using the CCI and the CIRS based on information available in electronic medical records during the first 24 hours after admission.
The ICU and wards of two metropolitan hospitals.
Senior adults admitted over a three-year period. There were 597 enrolled patients with a mean age of 76.1 years. Participants were 35.5% male, 50.9% Caucasian, and 48.1% African American.
Logistic regression models found the CIRS (OR = 1.13, 95% CI 1.02, 1.26), but not the CCI (OR = 1.01, CI 0.82, 1.25), was significantly associated with in-hospital death and the receipt of discharge services (OR = 1.086, CI 1.049, 1.124) compared to (OR = 1.002, CI 0.993, 1.131). In further analysis, it was found that a CIRS score of 29 or greater was associated with significant increase in a patient's chance of death during that hospitalization (OR = 14.05, CI 1.60, 123.17) compared to the reference group (OR = 2.57 CI 1.42, 4.66) for requiring additional services post discharge.
The admission calculated CIRS is useful for predicting important hospital outcomes. Such information may allow clinicians to target intensive care to appropriate patients at high risk of death or long stays and to better anticipate end of life issues. Furthermore, this information may be helpful in risk adjustment for researchers and policy makers.
Comorbidity, Patient discharge, Hospital mortality, Length of stay
When older adults are hospitalized they have an increased risk of adverse outcomes and high healthcare utilization. In 2010 adults aged 65 and older made up 73% of hospital deaths compared to 27% for those less than 65 years of age [1]. In 2007 the average Length of Stay (LOS) for patients 65 and older was 5.6 days compared to 5.1 days for those 45 to 64 [2]. Validated measures to predict which older adults are at the highest risk of adverse outcomes would be useful for clinicians in targeting aggressive care to the highest risk patients, appropriately addressing advance care planning, and conducting discharge planning in a timely manner.
Prior studies have demonstrated that established measures of comorbidity and illness severity could be used to predict longer-term outcomes such as readmission and long-term mortality. For example, The Cumulative Illness Rating Scale (CIRS) has been validated as a predictor of readmission for hospitalized older adults [3] and hospitalization within one year for older adults in a residential setting [4]. It has been validated as a predictor of long-term mortality when assessed in inpatient [3,5] and residential [4] settings. In a more recent study, however, it was found that non-medically trained raters and oncologists would have varying interpretations of comorbidity when assessing the same patient's medical records [6]. The Charlson Comorbidity Index (CCI) has been extensively studied and validated as a predictor of re-hospitalization [7], functional decline [8,9], and mortality in a number of populations [7-13].
Although the CCI and CIRS have been validated in a number of settings, there have been few efforts to use these measures, or any others, to predict outcomes of current hospitalizations based on admission data for adults 65 and older. In addition, many of the prior studies were conducted more than a decade ago. Since then, the United States health care system has undergone significant structural changes that could potentially have a serious impact on the results of those studies if they were done today. One study found that the CIRS is associated with length of stay but not mortality in elderly inpatients when controlling for covariates [12]. Another study found associations with mortality and length of stay for inpatients 90 and older [14]. The CCI is associated with increased inpatient mortality for subpopulations of hospitalized Medicare (Medicare is government health insurance afforded to all United States citizens 65 and older and others who are determined to be disabled by the federal government whereas Medicaid is government funded health insurance given based on socioeconomic status) beneficiaries with carotid endarterectomy [15] and nosocomial infections [16]. It has also been associated with mortality and LOS when calculated for patients 16 and older using an administrative database [17].
This study was designed to determine if either of the two validated rating systems, the CCI [18] and the CIRS [19] could be used to predict hospital outcomes in adults 65 and older admitted to medical services, based on data obtained early in hospital admission. The purpose was to determine if the CIRS or the CCI could predict important out comes such as mortality or the necessity of post-admission services thereby allowing for the coordination of services and targeting of aggressive care.
Originally derived from a study of medical inpatients and validated in a group of ambulatory breast cancer patients, the CCI contains nineteen diseases weighted to predict one-year mortality [18]. The CIRS includes measures of severity of individual diagnoses in its scoring system [8].
This study was a secondary analysis of data from a prospective observational study of patients from hospital medical wards and the Intensive Care Units (ICU) of two metropolitan hospitals in a Midwestern city affiliated with a university and staffed by university faculty, residents, and students [20]. One hospital was a tertiary referral center and the other was a county funded safety net hospital. The study population consisted of patients 65 years of age and older admitted to the medical wards or ICU of the target hospitals over a 38-month recruitment period (November 2008-December 2011). To be eligible for the study, patients had to be hospitalized for at least 48 hours and (based on the study's original goal) had to have required at least one major medical decision regarding life sustaining care, procedures or surgeries requiring informed consent, or discharge services. Informed consent was obtained from each patient or his or her legal surrogate. Patients were ineligible if they were incarcerated, did not speak English, or died or were discharged before 48 hours of admission.
Medical student research assistants calculated comorbidity and illness severity, using the CCI and CIRS, based on all information available in Electronic Medical Records (EMR) during the first 24 hours after hospital admission. This included admission notes, procedure notes, medication reconciliations, nursing notes, emergency department notes, imaging studies, physical and occupational therapy assessments, treatment and care plans, and other available consults conducted within the first 24 hours.
Each student received ten hours of training on the CCI and CIRS rating scales. This included training on each of the EMR systems used, where to find relevant data, and how to apply the data to each rating scale. Once the training was complete they completed two validation sets. To evaluate inter-rater reliability, each student completed two validation data sets consisting of thirty patients and kappa scores were derived. After each set, students and the study primary investigator reviewed kappa scores, discussed areas with low agreeability, and reviewed the scoring criteria in an effort to improve reliability.
The study outcomes of in-hospital mortality, discharge services, and LOS were determined for each patient from the Regenstrief Medical Record System, an electronic record of clinical data from Eskenazi and IU Health Hospitals [21]. Discharge services included home health services, discharge to a nursing home or rehabilitation facility, and hospice.
The data used for this study were from a cohort of an investigation done by Torke, et al. [21]. The initial investigation identified 1083 patients and enrolled 597. Enrolled patients and not enrolled patients were similar in age (mean 76.1 to 77.0 years; P = 0.08), sex (female, 64.5% to 60.8%; P = 0.22), race (African American and other, 48.1% and 1% to 52.3% and 1.3%; P = 0.33), and socioeconomic status (Medicaid 34.0% to 31.4%; P = 0.46) [20].
CIRS and CCI scores were first analyzed to look for associations with the three outcome variables - in-hospital mortality, hospital LOS, and need for post-discharge services among survivors. Logistic regression models were used for the outcomes of death and need for services due to their dichotomous nature. LOS is a highly skewed continuous measure so log-linear regression models were used to analyze the association between CIRS and CCI with the log-transformation of LOS. Cohen's kappa coefficients were calculated on each individual item, for both CIRS and CCI, in order to measure inter-rater agreement for both rating scales. Bootstrapping methods were used to determine if either CIRS or CCI scores modeled the data with better fit. A random sampling of the original population was sampled with replacement 1000 times with each of the 1000 data sets having the original sample size of 597. For each of these samples, analyses were ran using both the CIRS and the CCI scores as the independent variable on the outcomes death and need for services. The log likelihood value was kept as a measure of model fit for each model. A paired t-test was then performed on the set of 1000 paired (CIRS and CCI) log likelihood outcomes. For both independent variables, CIRS gave statistically better fit than the CCI (p < 0.0001).
Based on these preliminary findings, we performed further analysis with the CIRS, categorizing the variable with logically determined cut-points based on the distribution of patient deaths within CIRS scores. These cut-points were determined after examining several grouping sizes. A three-group set-up (low 0-20, medium 21-28, and high 29+) had the most homogenous distribution of deaths, and so was used. We examined whether these groups differed in terms of death, need for services, and LOS in order to determine if there was a difference between patients with low CIRS scores compared to mid-range and high CIRS values.
Generalized linear models were performed to determine if there were differences between these CIRS score groups, looking at the odds ratios between the low score group (the referent group) and the medium and high score groups, in regards to in-hospital mortality, discharge services, and LOS. All analyses were performed using SAS v9.3 (SAS Institute, Cary, NC). All analytic assumptions were verified to have been met and goodness of fit criteria used to ensure the proper models were being used.
There were 597 enrolled patients with an average age of 76.1 years. Participants were 35.5% male, 50.9% Caucasian, and 48.1% African American (Table 1). Mean scores on the CCI were 4.83 (SD 2.75) and for the CIRS were 22.19 (SD 5.42) (Table 1).
Table 1: Patient characteristics and outcomes. View Table 1
In logistic regression models controlling for age, race, gender, and socio-economic status, we found CIRS (OR = 1.13, 95% Confidence Interval (CI) 1.02, 1.26), but not CCI scores (OR = 1.01, 95% CI 0.82, 1.25), to be significantly associated with in-hospital death and receipt of discharge services (OR = 1.086, 95% CI 1.049, 1.124 compared to OR = 1.002, 95% CI 0.993, 1.131). In log linear regression, both CIRS (p < 0.001) and CCI (p < 0.001) were significantly associated with LOS. Bootstrapping determined that, in cases of death and services, the CIRS was statistically better than the CCI (p < 0.001).
Because of the CIRS was determined to be a better fit when compared to the CCI, it was divided into three groups based on visual examination of the relationship between CIRS scores and in-hospital deaths. In unadjusted analysis, CIRS groupings were significantly associated with death, services, and LOS (Table 2). For hospital deaths in CIRS groups with scores 0-20, 21-28 and 29 and above, rates of in-hospital death were 1 (0.5%), 7 (2.7%) and 6 (7.8%) respectively (p = 0.005; Table 2). Percent of post-discharge service use was 91 (44.8%), 151 (67.6%) and 48 (67.6), p < 0.001) and median LOS was 6, 6, and 7 days (p = 0.033). Logistic regression revealed that a score of 29 or greater was associated with an OR of death of 14.05 (95% CI 1.60, 123.17) compared to the reference group and an OR of 2.57 (95% CI 1.42, 4.66) for requiring additional services post discharge (Table 2). A score of 21-28 elicited a five times greater chance of in-hospital mortality and a 1.85 times greater chance of needing post-discharge services. CIRS groupings were also significantly associated with LOS (Table 2).
Table 2: Relationship between cumulative illness rating scale (cirs) grouped by score, key hospital outcomes, and length of stay. View Table 2
Pairwise kappa statistics estimated on each validation set revealed that for the CCI, kappa scores were above 0.41, signifying moderate agreement [22] in 96% of the nineteen items in the 1st validation set and 100% in the second. Pairwise kappa agreement was greater than 0.41 in 93% of each of the fourteen CIRS items in the first validation set but only 79% in the 2nd set. The first validation set was conducted between four raters whereas the second validation set was conducted between only two of the raters because two of the raters had graduated. The median time for completion of a single CIRS record by the students was thirty minutes.
This investigation determined that the CIRS, but not the CCI, is predictive of in-hospital mortality and the need for additional services post discharge. Both the CIRS and CCI were predictive of LOS. Furthermore, it found that patients with a high CIRS score had a 14 times greater chance of in-hospital mortality than a patient with a low score. High CIRS scores predicted modest increases in LOS.
The CIRS shows promise as a useful instrument in hospital practice due to its association with several important hospital outcomes. Determination of high mortality risk within 24 hours of admission may allow clinicians to begin early, aggressive care for patients at high risk of death. This information would also identify patients for whom early discussions about goals of care are especially important. Furthermore, these findings would allow clinicians and discharge planners to begin preparation for post discharge services earlier in a patient's hospital stay limiting delays in discharge due to a lack of placement availability.
The ability to predict outcomes would not just be valuable information for clinicians but for hospital administrators because the continuous changes in hospital reimbursement methods lead to an even greater need for efficiency and cost saving measures. By implementing the use of the CIRS, for older adults, at the time of admission, policy makers would be utilizing another mechanism to streamline care for hospitalized patients. Identifying patients at risk for death would allow for the timely provision of palliative care and could prompt important discussions regarding goals of care, end-of-life care, and code status thus ensuring that care is consistent with patient and family preferences.
Because medical students were successfully trained in completing these assessments, the study demonstrates that this exercise could be a valuable learning tool in academic medical centers. Even though it must be acknowledged that the students trained to perform the chart reviews were employed as research assistants for purposes of the study, the 8-10 hours required to train an upper level medical student could be done during the clinical years of medical school because the basic knowledge required to understand and complete the assessment is completed in a student's didactic years. Furthermore, as medical schools continue to advance their curricula, the CIRS could become part of that evolution. The CIRS rating scale would allow students to put their training to practical use while assisting physicians in developing plans of care for their patients.
The minimal training required to train a medical student, resident physician, or registered nurse to use the CIRS rating scale makes it an easily implemented tool to help guide the care of older adults. In less than 30 minutes, a trained practitioner will have additional knowledge that will help physicians better prepare their patients and the medical team for what to expect for the duration of the hospitalization and allow discharge planning to be more efficient and expedited.
There were several limitations to this study. Since it was conducted in two hospitals in the same city, the results may not reflect the overall population of older adults. Furthermore, the patient population was strictly selected from medicine wards and ICUs, which may garner different results from that of surgery or other services. Between-reviewer Kappa scores were only in the moderate range. This may indicate a challenge of the CIRS is obtaining inter-rater agreement. Due to the low number of in-hospital deaths, the odds ratios for mortality have a large confidence interval. These results should be extended to larger populations of patients.
The admission calculated CIRS, which can be reliably assessed by medical students, is useful for predicting important hospital outcomes, including death, receipt of discharge services and LOS. Such information may allow clinicians to target medical care earlier in the hospital course, has the potential to reduce cost, and improve quality by identifying patient needs early in the hospital course. Furthermore, this information may be helpful in risk adjustment for researchers and policy makers when developing innovative ways to streamline care and become more efficient in today's health care market.
Dr. Torke was supported by a Geriatrics Health Outcomes Scholars Award from the John A. Hartford Foundation/American Geriatrics Society and a career development award from the National Institute on Aging (K23AG031323). The funder had no role in the design, methods, subject recruitment, data collections, analysis, or preparation of paper.