Background: To evaluate the utility of the Impact Stratification Score (ISS) in characterizing health-related disease burden for older adult patients with multiple chronic conditions (MCC).
Methods: The sample of 1226 older adult MCC patients (average age of 80, 51% female, and 89% White) completed the PROMIS-29 v2.1 profile that contains the 9 ISS items. The ISS was examined using factor analysis (i.e., correlated factors and bifactor models). We evaluated the relative validity of ISS compared with other PROMIS-29 scores using ratio of F-statistics from multivariate regressions predicting each PROMIS-29 score from patient chronic conditions and utilization patterns.
Results: Bifactor model results indicated essential unidimensionality, primarily reflecting one general construct (i.e., impact) and that, after accounting for impact, very little reliable variance remained in the two group factors. General impact scores were reliable (omega H = 0.73). ISS scores were significantly higher according to older age, female gender, and Hispanic ethnicity, increased with increasing number of chronic conditions, and were strongly related to presence of most chronic conditions and healthcare utilization rates. Relative efficiency coefficients revealed that ISS scores were more strongly related to most chronic conditions relative to PROMIS pain intensity, physical health, and pain interference scores and outperformed the PROMIS-29 physical health summary score for several conditions including arthritis, diabetes, and high blood pressure.
Conclusion: This study presents evidence that the ISS is a sufficiently unidimensional and reliable measure that may be useful in characterizing health-related disease burden among older adult ambulatory patients with two or more chronic conditions.
Older adults, Multiplechronic conditions, Impact stratification, PROMIS®, Reliability, Bifactor, Patient-reported outcomes
CFI: Comparative Fit Index; CLBP: Chronic Low Back Pain; ISS: Impact Stratification Score; KPCO: Kaiser Permanents Colorado; MCC: Multiple Chronic Conditions; Pain1: Single Item Pain Intensity Score; Pain4: Four Item Pain Interference Score; PF: Four Item Physical Function Score; PSumm: Physical Health Summary Score; PROMIS-29: Patient Reported Outcome Measurement Information System 29-item Profile Measure; RMSEA: Root Mean Square Error of Approximation; SRMR: Standardized Root Mean Residual
In 2014, the impact stratification score (ISS) was proposed by the National Institute of Health Pain Consortium's research task force on research standards for chronic low back pain (CLBP) to stratify patients by the impact CLBP has on their lives [1,2]. The ISS was constructed using a subset of items contained in the Patient-Reported Outcomes Measurement Information System (PROMIS®) 29-item profile measure (PROMIS-29). The PROMIS-29 assesses seven health domains with 4 items each (Physical Function, Pain Interference, Fatigue, Sleep Disturbance, Depression, Anxiety, and Ability to Participate in Social Roles), and includes a single item to assess pain intensity. The ISS is the sum of the items from the PROMIS-29 physical function (4 items, each scored 1-5), pain interference (4 items, each scored 1-5), and pain intensity (1 item scored 0-10) measures, resulting in a possible score range from 8 (least impact) to 50 (greatest impact).
Evidence for the reliability, validity, and clinical utility of the ISS for patients with CLBP is beginning to accumulate [3] and a recent study established its unidimensionality and support for its use as a single summed score among CLBP patients [4]. Given the applicability of the seven domains assessed by the PROMIS-29 across a wide range of patient samples including older adults with multiple chronic conditions (MCC) [5], the ISS could be useful among patient samples other than those with chronic back pain. The disease burden of older adults with MCC is substantial [6], often resulting in functional impairment [7], and it can be challenging to characterize improvements in this hard-to-treat patient population [8]. In this study, we examine the dimensionality and psychometric properties of the ISS among older adult ambulatory patients with two or more chronic conditions, examine the association of ISS scores with demographic, clinical, and healthcare utilization variables, and evaluate the ISS relative to other PROMIS-29 scale scores in identifying specific conditions for these patients.
Data and eligibility for these analyses have been described in full elsewhere [9]. Participants were recruited from Kaiser Permanente Colorado (KPCO), a not-for-profit integrated delivery system that directly provides both ambulatory and hospital-based care. KPCO members were eligible to participate if they were age 65 or older, were assigned to a primary care provider at a KPCO ambulatory clinic, had been seen for clinical care at least once in the past 12 months, had a valid email address, and had at least 2 of 13 specific chronic conditions (the conditions are shown in Table 1).
Table 1: Respondent demographic and clinical characteristics. Percentages given except where otherwise noted (n = 1,226). View Table 1
We use the following scale scores based on the PROMIS-29 v. 2.1 instrument: Physical function (PF; four items assessing ability to perform physical activities including chores around the house, climbing stairs, walking, and instrumental activities of daily living, such as running errands), with item responses from 5 (without any difficulty) to 1 (unable to do)reverse coded so that higher scores indicate poorer functioning [10]; pain interference (Pain4; four items assessing the extent to which pain hinders engagement with day to day activities, social activities, chores, and work around the home), with item responses ranging from 1 (not at all) to 5 (very much) and higher scores indicating more pain interference [11]; a single item reflecting the pain intensity a person experienced (Pain1), on average over the past 7 days on a scale from 0 (no pain) to 10 (worst pain imaginable) with higher scores indicating greater pain intensity; the 9-item ISS which includes the PF, Pain4 and Pain1 items [2] and the physical health summary score (PSumm), which is generated based on a factor model using all domains measured by the PROMIS-29 [12].
Demographics and clinical and utilization characteristics were extracted from the KPCO data warehouse: Age at baseline, sex, race/ethnicity, chronic health conditions, number of home health visits, primary and specialty care visits, emergency department visits and hospitalizations. Because a majority of our sample was White and non-Hispanic, and several other categories were relatively uncommon, we created four merged categories to support analyses: White non-Hispanic, non-White and non-Hispanic, Hispanic (any race), and missing race non-Hispanic.
The goals of this study were to: 1) Determine whether the unidimensional structure of the ISS holds in an older adult MCC patient sample; and 2) Determine the utility of the ISS in characterizing disease burden in a non-CLBP sample relative to other PROMIS-29 scores. Following [4], we first examine correlations among all items, item-rest correlations (correcting for item overlap with the total score), and Cronbach's alpha, then evaluate the Rodriguez, et al. structure in this sample with a confirmatory factor model using traditional fit indices such as the Root Mean Square Error of Approximation (RMSEA ≤ 0.08) [13], Comparative Fit Index (CFI ≥ 0.95) [14], and Standardized Root Mean Residual (SRMR ≤ 0.08) [14].
After replication of the Rodriguez, et al. structure, we conducted multivariate regression analyses to look at the relationship of the ISS score and other PROMIS-29 scores with chronic conditions. Following Hays and Revetto [15], we calculated relative efficiency of ISS compared with other PROMIS-29 scale scores using the ratio of F-statistics. The relative efficiency of the ratio of F-statistics is compared to a value of 1, with larger values indicating greater efficiency. We constructed four relative efficiency ratios for each chronic condition using the PROMIS score with the smallest F-ratio as the reference (e.g., if pain1 had the smallest F-ratio in the multivariate model, relative efficiency was calculated and compared for pain4, PF, ISS, and PSumm). Factor analyses were conducted in R [16] using the lavaan package [17]. ANOVA, multivariate regressions and calculations of relative were using SAS v9.0 [18].
Out of 3,749 people who received a survey, 1,359 participants responded (36%). Further details can be found in [19]. Of the 1,359 respondents, 1,226 (90%) completed all ISS items and were included in analyses. Respondent characteristics are shown in Table 1. The mean age was 80.5 years, and 65% of participants were age 80 or older. A majority of participants (89%) were non-Hispanic White. Per the study inclusion criteria, all had at least 2 of 13 chronic conditions; 35% had exactly 2, 31% had 3, 18% had 4, and 16% had 5 or more. Prevalence of many chronic conditions was relatively high compared to the U.S. population; for example, 37% had chronic lung disease compared with 7% nationally [20], and 31% had diabetes compared with 13% nationally [21]. More than half had at least one home health encounter in the past 12 months. The mean ISS score was 21 with a standard deviation of 10.
Cronbach's alpha for the ISS was excellent (α = 0.93) with item-rest correlations ranging from 0.67 to 0.84. Item-rest correlations were higher for Pain4 items than for Pain1 and PF items. All items were significantly (p < 0.0001) and positively correlated with one another with product-moment correlations ranging from 0.39 to 0.90 (results not shown). A confirmatory factor model representing the bifactor structure fit the data well: RMSEA = 0.049 (CI: 0.038-0.061); SRMR=0.012; CFI = 0.995, indicating that the structure reported in Rodriguez, et al. for a sample of CLBP patients applies to this sample of older adult ambulatory care patients with two or more chronic conditions.
Table 2 shows bivariate analyses of ISS scores according to patient-level characteristics. We used one-way ANOVAs to generate F-statistics and Duncan's multiple range test for pairwise comparisons. Older age was associated with higher impact (p < 0.0001); ISS was lowest for those aged 70-74 and increased with higher age groupings with scores for respondents age 85 and over nearly 3 points higher than those age 65-69 and significantly different from all other age groups. Females and Hispanics had significantly higher ISS scores than their comparison groups. For example, females scored 4.01 points higher than males (p < 0.0001), whereas Hispanics scored 5.55 points higher than White non-Hispanics (p < 0.002).
Table 2: Mean (SD) ISS scores by patient characteristics. View Table 2
ISS was significantly higher with increases in number of comorbid conditions (e.g., difference of 8.23 between 2 and 5 or more chronic conditions). ISS scores differed as expected for those with select chronic conditions. The largest score difference was 4.41 for those with depression scoring higher than those without. Those with arthritis, congestive heart failure, osteoporosis, sciatica, chronic lung disease, stroke, and diabetes also scored significantly higher than those without the condition. Having cancer, hypertension, inflammatory bowel disease, ischemic heart disease, and other heart problems did not result in a significantly different ISS score than those without the condition. All health care utilization variables were associated with higher ISS scores, with the largest differences observed according to home health encounters (5.09 higher for those with an encounter relative to those without), and number of primary care visits (7.12 higher for those with 10 or more visits in the past 12 months relative to those with 0-3 visits).
Table 3 displays associations of the five PROMIS scores with each chronic condition based on multivariate analyses with all chronic conditions as predictors of each PROMIS score. The ISS has the highest relative efficiency value for four of the 13 conditions (hypertension, depression, diabetes, and arthritis), and PSumm had the highest relative efficiency for three of the conditions (congestive heart failure, chronic lung disease, and osteoporosis). Pain1 and Pain4 scores were most efficient for sciatica, and ischemic heart disease, respectively, whereas PF was most efficient for stroke. Relative efficiency was not calculated for cancer, other heart problems, and inflammatory bowel disease due to non-significant associations in the multivariate models.
Table 3: Association of select PROMIS-29 scores with chronic conditions: F-Ratios and relative efficiency (N = 1226). View Table 3
Although the ISS was proposed as a measure of chronic pain impact for individuals with CLBP, its discriminating properties could be useful in characterizing disease burden among other patient groups. The goal of this study was to evaluate the sensitivity of the ISS to reflect disease burden among older adult patients with multiple chronic conditions. We explored this possibility by first determining the suitability of the unidimensional factor structure observed in a sample of CLBP patients [4], and then after examining ISS scores in this patient population to establish whether they differed meaningfully according to patient demographic and clinical characteristics, evaluating the efficiency of the ISS in characterizing presence of specific conditions in this patient group relative to other common measures from the PROMIS-29 including the component parts of the ISS and the commonly-used physical health summary score.
Older adult patients with multiple chronic conditions represent a growing and heterogenous group that is challenging to treat and whose disease burden is difficult to characterize with a common metric. In this study, the ISS was shown to have similar dimensional structure and psychometric properties as that reported by Rodriguez, et al. in a sample of CLBP patients. Further, the ISS scores differed meaningfully according to patient demographic and clinical characteristics and demonstrated higher relative efficiency than other PROMIS scores for four chronic conditions.
These results should be interpreted with the following limitations in mind. First, the data in this study include patients from a single large homogeneous health system (KPCO) and results may not generalize to other patient samples. Further, generalizability of the results is limited by the 36% response rate and the limited racial/ethnic diversity of the sample. We also considered only 13 chronic conditions, leaving many unmeasured. Finally, the ISS was evaluated relative to other scores in the PROMIS-29. Further studies should compare the performance of the ISS to non-PROMIS measures. Despite these limitations, this study provides compelling support for the utility of the ISS in a unique outpatient sample of older adults with chronic conditions.
Given these promising preliminary findings, it would be worthwhile to further evaluate the ISS in complex older adult patient samples and for conditions other than CLBP. If ISS scores can be shown to change meaningfully over time, especially in response to treatment or other clinical intervention, the ISS may be a good candidate to form the basis of a quality measure for this highly heterogeneous patient group.
The title page lists all individuals who have contributed significantly to this manuscript.
We have no conflicts of interest to disclose.
Edelen led study design, acquisition of data, analysis and interpretation of data, and preparation of manuscript; Rodriguez assisted with analysis and interpretation of data; Qureshi assisted with preparation of manuscript; Herman assisted with interpretation of data; Hays assisted with interpretation of data.
The funders had no role in the design, methods, subject recruitment, data collections, analysis and preparation of paper.
Data collection funded by the National Institute on Aging (contract#HHSN271201500064C NIH NIA, PI: Edelen).
Analyses, interpretation and manuscript preparation funded by the National Center for Complementary and Integrative Health (NCCIH). Grant No. 1R01AT010402-01A1, MPI: Herman, Hays.