The clinical outcome data for patients who had a particular intervention for the lumbar degenerative disease is based on changes in symptomatology. This functional data had been acquired in a prospective manner using standard tools e.g., ODI, RMQ, SF-36 etc. Such data may be lacking in many facilities of developing countries due to various reasons. We are proposing a retrospective comparative questionnaire to obtain similar outcome data after intervention for patients with lumbar degenerative spinal disease. The process of content validation and reliability of the questionnaires known as the UM (University of Malaya) general well-being index post-intervention for spinal disorders and UM post-intervention lumbar specific spinal disability index is presented. This questionnaire is simple but encompasses key functional outcomes. There is a summative score that would indicate the overall outcome. It can be conducted efficiently in person or via the telephone. It would require further validation in clinical use for patients with lumbar degenerative disease.
Lumbar degenerative spinal disease, Outcome, Questionnaire, Content validation, Reliability, Qualify of life (QOL)
ADL: Activities of Daily Living; CVI: Content Validity Index; EFA: Exploratory Factor Analysis (EFA); ODI: Oswestry Disability Index; PA: Parallel Analysis; RMQ: Roland-Morris Questionnaire; QOL: Quality of Life; SF-36: Shoft Form-36 Questionnaire.
Research concerning degenerative spinal disorders relies on the patient's perspective on their quality-of-life (QOL) to assess the effect of the treatment [1]. There were multiple questionnaires available in the field which evaluates their spine-related disability index, including the Oswestry Disability Index [2], Roland-Morris Classification [3] and Quebec Classification Score [4]. While other questionnaires were used to assess the patient's quality of life, namely Short Form-36 (SF-36) [5] and EQ5D questionnaires [6]. These are comprehensive questionnaires that serve as a good tool to gauge patients' specific and general functional state at one point in time taking into account the symptoms and their effects on daily activities and quality of life. If this assessment is done at the various timelines, it can be used to assess the response to a specific therapeutic modality.
There are various outcome instruments available and the selection of the most appropriate instrument wound be determined by the researcher [7]. The commonly used questionnaire Oswestry Disability Index [2], Roland-Morris Classification [3], Short Form-36 (SF-36) [5], need data recorded in a prospective manner at various points in time to assess the progress of the condition or response to treatment. This would require infrastructure and manpower to conduct such research as it is time-consuming. In many parts of the developing world, this sort of facility and support is lacking; even if it were available sometimes in a busy clinical practice such evaluation could not be conducted on every patient.
Lumbar degenerative disease is a common disorder. There is a cohort of patients who would benefit from surgical intervention. The surgical options for degenerative spinal disorders are varied; ranging from simple neural decompression using microscope or endoscope; to more intricate approaches involving fusion surgery. The surgical options that are advocated by the surgeon are based on their training, experience, and locality. In the current times, the literature pertaining to this subject is predominantly from developed countries. The surgical strategies employed in the developed world to manage lumbar degenerative disease may differ from the developing or less developed countries. There is a paucity of literature on the techniques used and the outcomes regarding degenerative spinal disease from the developing world. Because of the limited resources, these countries tend to practice a more basic neural decompression surgery rather than fusion surgery; however, this outcome data is not widely available because of limitation of obtaining prospective data in these countries due to various reasons including lack of funding, heavy clinic load, logistics and manpower. Therefore, these sets of data do not get into mainstream literature but are possibly presented only in local meetings in a form of the surgeons' experience. In an effort to capture this basic outcome data from this cohort of patients, we embarked on devising a truncated questionnaire that permits for a broader retrospective comparative evaluation of general well-being and spine specific disabilities for lumbar degenerative spinal disorder patients who have had an intervention. The outcome measures would permit the researcher to appraise the benefit of a particular intervention in terms of functional domains.
The questionnaire has two large domains of general well-being and lumbar specific spinal disabilities index. Each domain has five questions, and the responses are objective with either one of the three to four responses. The choice of questions in each domain is general (pain intensity, personal care, sleeping, social life engagement/recreation and work) and specific (walking, sitting, change in position, bladder dysfunction, lower limb numbness). These parameters were chosen in an attempt to reflect a more holistic appraisal of the functional state despite it being a short questionnaire. This assessment can be done retrospectively either in person or through the telephone which brings out the advantages of greater response rate, relatively inexpensive but quick data collection and the ability for interviewers to probe for incomplete or ambiguous answers [8]. This method is also relevant in the current pandemic. We are presenting our assessment tool and the validation process.
Content validation certifies whether an instrument measures what is intended and verifies whether the items properly reflect the content domain of interest and whether the scale dimensions are consistent with each proposed item and provide clarity to respondents [9,10]. Content validity aspects include the relevance of the item and its clarity, which are determined by an evaluation of the instrument items by a group of experts with previous experience or currently recognized in the field of study, namely practicing surgeons, physicians or experts [11]. The present study aimed to validate the content of a developed instrument to assess the general well-being and quality of life of individuals with the lumbar degenerative disease who underwent lumbar spine surgeries (intervention).
The stages for the development and validation of an instrument encompass four distinct phases: Planning, construction, quantitative analysis, and validation, and lastly pilot study among patients [11]. The instrument planning and construction phases consisted of an extensive literature review and expert opinions. Content validation and the pilot study are described in the present study. This project was approved according to protocol no. 2020819-8994 of the Medical Research Ethics Committee, University Malaya Medical Centre, Kuala Lumpur, Malaysia.
The content validation (phase I) was done in two rounds. In the first round, the instrument was initially subjected to content validation by a committee of twenty medical experts with an interest in the field of degenerative spinal disease. The content validation questionnaire was developed in English, it was evaluated for the relevance and clarity of each item developed. The second round of content validation was repeated among fifteen experts after the initial responses and comments were considered and relevant changes made to fine-tune the instruments. Once the content validations by the experts were concluded a pilot study was conducted to evaluate the construct validity. Figure 1 demonstrates a flow chart in the development of a new questionnaire.
Figure 1: Flow chart of development of new questionnaire.
View Figure 1
Phase I (Pre-test): Based on the protocol described by Polit, et al. [12], we choose twenty medical practitioners including neurosurgeons, orthopaedic spine surgeons, rheumatologists, neuropsychologist, and rehabilitation physician with experience in the care of patients with the lumbar degenerative disease to participate. These included both local and foreign medical practitioners. The length of experience of the experts ranged from 8 to 25 years. In the first round of content validation, twenty experts with seventeen local Malaysian experts and three international experts responded within a specified period while fifteen experts from a similar cohort participated in the second round.
Phase II (Pilot study): A pilot study was conducted from a pool of patients who underwent lumbar spine surgeries for degenerative spine diseases in a tertiary hospital in Kuala Lumpur from the period of January 2008 to December 2019. As such, a total of 104 patients were successfully contacted and interviewed via telephone within a period of 2 weeks. The minimum sample size for EFA, according to Gorsuch [13], should be at least 100, therefore the collected data (n = 104) was adequate to calculate factor loadings for each question.
Phase I (Pre-test): The content validity of the developed instrument was statistically analyzed by a Content Validity Index (CVI). To calculate the CVI, each item was ranked on a four-point scale (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant). For each item, the CVI was calculated as the number of experts who provided a rank of 3 or 4, divided by the total number of experts. The Kappa Modified Coefficient was used to determine the degree of relevance agreement of the CVI and was calculated according to protocols set by Polit, et al. Twenty experts participated in the first phase of content validation, the acceptable CVI value for each item ranged from 1.00 to 0.70, and the Kappa Modified value from 1.00 to 0.68. In the second phase, the items considered as acceptable were those that had a CVI ranging from 1.00 to 0.80 and a Modified Kappa value ranging from 1.00 to 0.80 because fifteen experts participated in these phases. Because the CVI considered the number of experts consulted for each phase, the difference in the number of experts in the different content validation phases of the present study did not influence the statistical results.
Phase II (Pilot study): To do the construct validity of the questionnaire, the pilot study data were analyzed first through Parallel Analysis. Parallel Analysis is a Monte Carlo simulation method that helps scholars in defining the number of factors to retain in Principal Component and Exploratory Factor Analysis. This technique delivers a superior difference to other techniques that are generally used for the same purpose, such as the Scree test or the Kaiser's eigenvalue-greater-than-one rule. In this study the parallel analysis was done based on this method using online software for PA (www.statatoddo.com). Parallel analysis (PA) is a method used to define the number of factors in factor analysis. There are several factors that affect the results of a PA: the choice of the eigenvalue percentile, the strength of the factor loadings, the number of variables, and the sample size of the study. Followed by parallel analysis, Exploratory Factor Analysis (EFA) was applied to evaluate the construct validity and to determine the contribution of each item (loading factor) to related components (factor). The EFA was done using Varimax rotation to determine the factor structure among 10 items related to the scale. All statistical test was done using SPSS Ver 25.
The first round of content validation: In the first round of content validation, the domains were considered satisfactory (CVI between 0.70 and 1.00) for evaluating the general disability and the region-specific domains. All items in the General Characteristics Domain obtained satisfactory scores (Table 1) and hence all the items in this domain were maintained in the second version of the instrument. Item 3 which assessed sleeping although achieved acceptable scores in relevancy (CVI = 0.75, Kappa = 0.746) and clarity (CVI = 0.70, Kappa = 0.746) was modified to ascertain the sleeping problems was due to the spinal problems instead of other issues as pointed out by one of the experts. The items that assessed pain intensity and social life engagement & recreation also had their wordings changed for improved understanding, for example in the pain intensity section, the options of "improved, maintained and worsened" were changed to "reduced, persistent and worsened". Items 6-10 presents the common scores of the lumbar-specific items. All items achieved a CVI of more than 0.85 except for relevancies in item 8 which assesses the change in positions of patients (CVI = 0.75) and clarity in item 9 accessing patient's bladder dysfunction (CVI = 0.75). To keep the questions more focused and maintain clarity to obtain a consistent understanding of the question either by reading or listening to the question by the respondent, we added the phrase "After the spinal intervention, …" to each of the questions. Wordings were also improved while some grammatical errors were pointed out by the experts to keep the questions all in the present tense.
Table 1: The first phase of content validation of the general disability domain. View Table 1
The second round of content validation: Table 2 represents the content validation of the general disability domain and lumbar-specific domain in the second round of the content validation process. The second version of the instrument was resubmitted to a similar expert committee for reevaluation of all items. 15 experts responded and participated in the second phase of content validation. All items managed to achieve a CVI of > 0.85 for both relevancy and clarity. Hence, all items were successfully validated after appropriate corrections were made prior to the second phase of validation.
Table 2: The second phase of content validation of the general disability domain. View Table 2
In this section, the results of the parallel analysis are presented for the adapted instrument including 10 items. The parallel analysis (Figure 2) indicated that only the eigenvalue of the first extracted factor was more than the eigenvalue of simulated data (λ = 1.646) which suggest that a single factor can be extracted from the data and therefore in the next step for exploratory factor analysis the number of extracted components is considered as 1 factor.
Figure 2: Scree plot and parallel analysis for items related to knowledge.
View Figure 2
Exploratory Factor Analysis was using Varimax rotation was applied to determine the factor structure among 10 items related to the general well-being scale. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.863, above the suggested value of 0.6, and Bartlett's test of sphericity was significant (χ2 (45) = 420.581, p < 0.05).
In the current study, all initial communalities were above the threshold. The results of EFA on all 10 items were forced into one component based on the results of the parallel analysis. The results showed that this component explained 46.6 % of the variance. According to these results, as shown in Table 3, all items showed a loading factor above 0.5 except item 9 and item 10 had a loading less than 0.5.
Table 3: Factor loadings based principal component analysis with varimax rotation for 10 items related to general well-being scale. View Table 3
In order to investigate the effect of time on the outcomes of construct validity, the sample was divided into four categories and EFA was performed on each subset of data (Table 4) because the current study is based on a retrospective study that took place from 2008 to 2020. The results revealed that, with the exception of Q9 and Q10, which had low loading factors in the pooled sample, the remainder of the items had acceptable loading factors across different subgroups, indicating that the passage of time had no effect on the results of the EFA.
Table 4: Exploratory factor analysis (EFA) based on time (Loading Factors). View Table 4
Lumbar degenerative spinal disease is a "wear and tear" condition that affects the more mobile parts of the spine mainly the cervical and lumbar region [14]. Most patients' manifest symptoms of pain or numbness as their primary problem. The threshold to offer surgical intervention also hinges on the severity of the symptoms leading to functional disability in executing the activities of daily living as well as the quality of life in terms of vocation and recreation. Therefore, the success of a particular intervention rendered to a patient with the degenerative spinal disease can be gauged by assessing the resolution or improvement of the presenting symptoms [15], where the ideal outcome of treatment is a return to the normal or usual quality of life for a given age and medical condition [16]. This gives a good indicator of the functional benefit that the patient had derived from a particular intervention/treatment within his own environment. Therefore, in cases of degenerative lumbar spinal disease, the symptomatology together with functional disability play a key part in making the management decisions as well as assessment of outcomes of these patients before and after the intervention.
Wilson and Cleary proposed a taxonomy for different measures of health outcome, which divides these outcomes into five levels: Biological and physiological factors, symptoms, functioning, general health perceptions, and overall quality of life [17]. Consequently, the lumbar spine specific and general well-being questionnaires in various forms have been used to this end [2-4]. However, the key features of these questionnaires are they are conducted in a prospective manner indicating the symptoms and functional disability at that one point in time. Some of these questionnaires are lengthy having multiple stems for one particular symptom that is being assessed eg. Oswestry Disability Index [2], by doing so it is able to tease the finer details of a symptom or activity. Though it is a more detailed assessment tool acquiring these data accurately from the subjects needs patients cooperation and compliance. These questionnaires have undergone vigorous validation and are a standard means of furnishing outcome data for degenerative spinal disorders in many works of literature.
Given the busy service and lack of research assistants, prospective functional data are not routinely collected for most of the cases of degenerative spinal disease that sought treatment in our facility. Based on the clinical follow-up, it is apparent that most of the patients who have undergone surgical intervention seem to have improved in functional ability. We wanted to study our surgical outcomes in a more scientific manner, but we realized that we lacked the prospective data. This initiated us to devise an assessment tool to perform a retrospective comparative assessment in a more standardized manner to allow a more systematic assessment of the outcome. The retrospective assessment tool will allow a comparison of the perceived degree of improvement in symptoms before and after the intervention. This comparison can be done at the various timeline from the intervention. However, the further the timeline is from the index surgery there is the possibility that the patient may not be able to make a reliable comparison. To minimize this sort of bias, we keep the questions more functional in terms of their overall experience in a particular item of assessment rather than asking the patient to score the specific symptoms in a retrospective manner. Scoring symptoms retrospectively may be fraught with errors. However, evaluating the patient's experience of change in various functional items after an intervention in a broader sense rather than specific scoring would minimize the errors in terms of recall biasness. Without a retrospective instrument, we cannot measure the result more objectively in a standardized manner due to the lack of prospective functional data.
The stark limitation of these existing questionnaires eg. ODI, RMQ, SF-36, etc. is the fact that the data must be collected in a prospective fashion at two different time points eg. before and after the intervention to make the comparison of the outcome after a particular intervention. It would be a more sensitive and specific way of presenting the outcome data, but to obtain such data would need resources and patient commitment to answer these detailed questionnaires. In order to achieve, a similar endpoint of outcome assessment after a particular intervention in a patient with degenerative spinal disease; we have validated a single assessment tool using a comparative questionnaire to gauge the difference that the patient has experienced after the intervention. The questionnaire looks at a few common functional parameters to be a surrogate of the overall functional ability of the patient. The questions do not interrogate details of the symptom or activities like the Oswestry Disability Index (ODI) because the answers expected are more of a reflection of the difference that the patient experiences after the intervention. Therefore, the options within each parameter are kept broader (Appendix 1) but distinct, nevertheless. Despite losing on the finer symptomatology and functional analysis, this assessment tool is more usable and able to derive reasonable outcome assessments to indicate the usefulness of certain interventions in patients with lumbar degenerative disease particularly in resource-limited countries.
The questionnaire underwent content validation and reliability test. Factor loadings are correlation coefficients between observed variables and latent common factors [18]. Factor loadings are part of the outcome from factor analysis, which serves as a data reduction method designed to explain the correlations between observed variables using a smaller number of factors. A factor loading that falls outside of the interval bounded by (± cutoff value) is large and is thus retained. On the other hand, a factor loading that does not meet the criterion indicates that the corresponding observed variable should not load on the corresponding common factor.
The general well-being domains that we focused on were pain intensity, personal care, sleeping, social life engagement/recreation and work. The first question was pertaining to pain intensity. We kept this question broad to encompass overall pain and not specific to the low back pain and lower limb pain. We are assessing the overall pain syndrome related to the degenerative spinal disorder and hope that by tackling the primary pain source, the other secondary pain related issues will resolve or diminish. However, there were limitations to this question, as addressed by the experts, where patients would sometimes confuse non-spine related pain at the same time and resulting in differing expected results. However, this question has a factor loading of 0.776, which makes it a strong indicator that this question is highly relevant and important to access the patient's general well-being outcome. The second question included in the questionnaire was about personal care and activity of daily living (ADL). Options with concepts of being fully ADL independent, partially dependent and total dependent were being implemented in this section. Like the first question, we appreciate there may be other factors affecting the independence to daily living apart from the spine issue. This question has a factor loading of 0.700, which also makes up a relatively strong contributor to a patient's quality of life.
The lumbar spine specific domains that we focused on were walking, sitting, changing position, bladder dysfunction and lower limb numbness as these functions had specific relation to the lumbar spine. For example, in the first question which focuses on walking, we emphasized any presence of back or leg pain and if it causes any restriction in walking as a result of the back pain. This question has an acceptable factor loading of 0.796. The only 2 items which had a factor loading of < 0.400 were domains of bladder dysfunction (0.396) and lower limb numbness (0.390). This could be explained as the incidence of bladder dysfunction because of degenerative spine disease was relatively uncommon even before surgery in our pool of patients (14.4%), hence making this question irrelevant in most of the patients in the pool. Meanwhile, in the domain of lower limb numbness, patients may perceive numbness in different ways; while some patients felt the residual numbness was acceptable and negligible while some patients still felt the numbness in the lower limbs affecting their daily activities. Most papers considered loading factors greater than 0.5 and any lower than that should be excluded and removed from the questionnaire. However, according to [18-21], the cutoff value is arbitrarily selected depending on the field of study, and (± 0.4) seems to be preferred by many researchers. Hence, as the values of questions 9 & 10 were close to the cutoff point of 0.4, it is not being excluded in the current study. However, further evaluation should be made in future studies to determine its value and to decide on whether these 2 items should be excluded in the future.
The date of the intervention and the date of the assessment is recorded in the questionnaire to determine the timeline. The same instrument can be used to record outcome measures at various timelines. Hence, analyzing the benefit of that index intervention at various points in time. Bearing in mind, the confounding factors that would affect the outcome apart from the intervention would have a similar effect on the subject's assessment of symptomatology in both the prospective and retrospective questionnaire.
The date of the intervention and the date of the assessment is recorded in the questionnaire to determine the timeline. The same instrument can be used to record outcome measures at various timelines. Hence, analyzing the benefit of that index intervention at various points in time. Bearing in mind, the confounding factors that would affect the outcome apart from the intervention would have a similar effect on the subject's assessment of symptomatology in both the prospective and retrospective questionnaire.
There will be certain limitations of a retrospective questionnaire to assess functional outcomes. One of the key elements is the ability to remember the severity of the symptomatology at presentation. The risk of this biasness has been alleviated by focusing on the experiential change as perceived by the subject of the symptoms in functional terms rather than grading the severity of the symptom at presentation in a retrospective manner. The analysis on the effect of time on the outcomes of construct validity indicated that the passage of time had no effect on the results of the EFA (Table 4). This is a novel instrument to permit a more standardized means of measuring outcomes for the cohort of lumbar degenerative disease patients who do not have prospective clinical data. By having such an instrument, more developing countries would be able to describe the outcomes in reference to the time of intervention of various sorts of treatments that are performed for patients with lumbar degenerative spinal disease.
The content validation of the UM general well-being index post-intervention for spinal disorders and UM post-intervention lumbar specific spinal disability index shows it is a valid and reliable instrument to give an impression of the outcome after intervention in a patient with degenerative lumbar spinal disease. The functional parameters that were gauged were relevant to the disease condition. This tool will afford a standardized means of performing the retrospective comparative assessment of the effectiveness of a particular intervention. This is a valuable tool to describe outcome measures in a cohort of patients who do not have prospective pre-intervention data.