Interrater and Intrarater Reliability Using Prechtl's Method of Qualitative Assessment of General Movements in Infants

Joanne S. Katz

Join Us \| Latest Articles \| Contact

Journal Home

Editorial Board

International Journal of Pediatric Research

DOI: 10.23937/2469-5769/1510014

Interrater and Intrarater Reliability Using Prechtl's Method of Qualitative Assessment of General Movements in Infants

Joanne S. Katz^1* and Agnes Perenyi²

¹Physical Therapy Program, Downstate Medical Center, State University of New York, Brooklyn, NY, USA
²Department of Pediatrics, Division of Neonatology, Downstate Medical Center, State University of New York, Brooklyn, NY, USA

^*Corresponding author: Joanne S. Katz, Physical Therapy Program, Downstate Medical Center, State University of New York, 450 Clarkson Avenue, Box 16, Brooklyn, NY 11203, USA. Email: joanne.katz@downstate.edu
Int J Pediatr Res, IJPR-2-014, (Volume 2, Issue 1), Original Research; ISSN: 2469-5769
Received: November 10, 2015 | Accepted: January 13, 2016 | Published: January 15, 2016
Citation: Katz JS, Perenyi A (2016) Interrater and Intrarater Reliability Using Prechtl's Method of Qualitative Assessment of General Movements in Infants. Int J Pediatr Res 2:014. 10.23937/2469-5769/1510014
Copyright: © 2016 Katz JS, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Abstract

Purpose: To establish interrater and intrarater reliability of two novice raters (the two authors) with different educational background in assessing general movements (GM) of infants using Prechtl's method.

Methods: Forty-three infants under 20 weeks of post-term age were recruited from our Level III neonatal intensive care unit (NICU) and NICU follow-up clinics of our medical center. The infants were observed using the GM assessment either during the writhing movement or the fidgety movement age periods.

Results: There was no significant difference (p > 0.05) between the two observers on interrater reliability and between Trials 1 and 2 for interrater reliability.

Conclusion: Novice raters need to establish their interrater and intrarater reliabilities in order to correctly identify GM patterns. The ability to correctly identify GM patterns in infants may be influenced by the raters' varying educational background.

Introduction and Purpose

Due to the recent advances in obstetrical and neonatal intensive care, an increasing number of preterm infants survive [1,2]. The surviving infants have high risk for often multiple morbidities, repeated hospitalizations after discharge and adverse neurodevelopmental (ND) outcomes. Several studies suggest improvement in early neurodevelopment in some of the subgroups of preterm infants [3-5]. Other authors report no improvement or unchanged ND outcomes, especially in very preterm infants [6,7]. Therefore, prediction of adverse outcome has paramount importance because of the significance of early initiation of appropriate therapeutic interventions.

Prechtl described the spontaneous motor activity of human fetuses and term and preterm infants as well as the quality and timing of the appearance of such movements [8-10]. He also indicated that the quality and presence or absence of general movements (GMs) reflect the condition and function of the central nervous system [11]. These spontaneous gross motor movement patterns originate by a central pattern generator intraspinally and from the medulla similar to the central automatisms for breathing, sucking and for locomotion such as swimming, crawling and walking.

The infant's GMs as gross motor movements are called writhing movements (WMs) which are preceded by variable preterm GMs before 36weeks. WMs can be observed after birth in both term and preterm infants. These GMs involve the whole body in variable sequence of the neck, trunk and extremities. They are described by Prechtl as "complex, elliptical, fluent, and are of moderate to large amplitude, with an intensity, force and speed that increases and decreases over time" [12]. The same GMs observed in preterm infants are frequently observed as WMs with faster speed and larger amplitude. If the GMs appear as monotonous and less complex, they are referred to as poor repertoire (PR) movements [12].

The writhing GM period lasts about six to nine weeks following term birth. At that time, they gradually disappear and a new pattern called fidgety movements (FMs) emerges. FMs exist until about 5 months (20 weeks) of age. The FMs are described as gross motor movements of small amplitude, moderate speed and acceleration in all directions involving the neck, trunk and extremities. FMs can be observed in the alert infant except while fussing, crying, and being fed or cared for (handled) in any way [12].

There are four abnormal GM patterns described in Prechtl's assessment: 1) chaotic (CH) GMs which are large-amplitude movements which are disordered in appearance with consistently abrupt movements, 2) absence of FMs, 3) abnormal FMs, which demonstrate exaggerated amplitude, speed and jerkiness; and 4) cramped synchronized (CS) movements, which are observed from preterm age onwards and described as rigid movements with no fluency and smoothness. All trunk and extremity muscles contract and relax almost simultaneously. Both the CS movements and absent FMs have a high predictive value for the development of cerebral palsy (CP) [11-14]. GMs have also been shown to be predictive for motor, cognitive, language, and behavioral impairments [15,16].

The purpose of this study was to examine the interrater and intrarater reliability of Prechtl's method of GM assessment in young infants in order to establish the observation skills of the two researchers with different educational background and experience. Both examiners were trained in the basic and advanced Prechtl's GM assessment courses just prior to data collection. However, they would still be considered to be novice raters, as they were only beginning to introduce Prechtl's Method into their clinical practice. Their goal was to utilize this assessment as a standard evaluation tool and part of the infant ND evaluation in the neonatal intensive care unit (NICU) as well as in the ND follow-up clinic.

Methods

Study design

This was a single-center, observational study with prospectively collected clinical data. We used a repeated-measures design across two trials with a physical therapist and neonatologist to determine intrarater and interrater reliability of Prechtl's GM assessment.

Subjects

Infants (n = 43) were recruited either from the NICU or in the ND follow-up clinic with gestational age (GA) between 24-41 weeks at birth, with chronological age between term age and 20 weeks of postnatal age at the time of the assessments. Corrected age was used in preterm infants. Exclusion criteria included infants with major congenital anomalies and genetic syndromes. The study was approved by the Institutional Review Board. Written and signed informed consent was obtained from the parent of each infant.

Instrument

Prechtl's Method of qualitative assessment of general movements was used in this study. This has been shown to be a reliable assessment method by authors who are experienced in its use in clinical practice [17-19]. In the present study, the reliability of novice (i.e., recently completed training courses and passing exams of the method prior to starting the study) observers using Prechtl's Method was assessed.

Procedures

Observation of GMs in the study participants was done according to the method described by Prechtl which involves videotaping the awake infant without disturbing him/her and the environment (i.e. feeding or holding the infant or offering pacifier) [12]. A digital camera was placed on a tripod above the infant's isolette or crib (in the NICU) or examination table (in the ND follow-up clinic) in order to videotape the infant lying supine, with diaper and no clothing. A three to five minute video clip from each recording was transferred in each case to a computer file for scoring by the researchers. If care giving had been necessary, it always had priority, so the videotaping was interrupted and repeated several times if necessary. The Individual Trajectory Form [12] was used to document the infant's movements. Infants from six to eight weeks of postnatal age were observed for the presence and quality of their (writhing) GMs and were scored with either normal, PR or CH GMs. Infants after that age until 20 weeks of postnatal age were observed for the presence/ absence or quality (normal versus abnormal) of FMs. Infants with CS GMs were also recorded. Each infant was assessed on at least one occasion by both observers for interrater reliability. Depending upon the length of stay in the NICU and subsequent outpatient follow-up, some infants were assessed on multiple occasions.

Interrater reliability compared the two researcher's assessments of each videotaped infant's GMs. Intrarater reliability was established by comparison of both researchers' individual assessments two weeks following initial observations.

Data analysis

Interrater and intrarater reliability were assessed using Cohen's Kappa statistic [20].

Results

There were 76 ratings performed on 43 infants, with 34 ratings assessing GMs during the writhing movement period (8 of them prior to term age) and 42 during the fidgety movement phase of development. Table 1 shows the ratings for all assessments by the two investigators on two occasions. Out of 34 ratings of WMs, there were 14 (41%) discordant ratings involving disagreement of PR versus normal WMs. Assessments during the fidgety period included four abnormal and 13 absent fidgety ratings. There were eight (18%) discordancies out of 42 ratings, with three of them due to normal versus abnormal FMs, and four due to normal versus absent FMs. CS movements were observed in 18 ratings, with three discordancies. Only one CH movement was found.

Table 1: Ratings for the Two Raters for all Assessments. View Table 1

Simple Kappa and 95% confidence limits for the interrater and intrarater assessment for WMs and FMs are found in Table 2. The second author demonstrated higher intrarater reliability than the first author, although the difference between the two raters was not significant (p > 0.05). Interrater reliability was lower than that of the intrarater reliability for both researchers. There were no significant differences (p > 0.05) between the two researchers on interrater reliability or between trials one and two for intrarater reliability.

Table 2: Intrarater and Interrater Results for Assessment of Writhing and Fidgety Movements. View Table 2

Discussion

Prechtl's method of qualitative assessment of GMs has been shown to be a reliable and valid evaluation to assess young infants' gross motor performance regarding their GM patterns which reflects their brain maturation or brain pathology [12,17-19,21,22]. The neurologic basis of the presence of GMs is not entirely clear. The mechanism possibly involves maturation changes of the motor neurons, changes in muscle innervation, increasing Renshaw inhibition, or decreasing excitability of motor neurons due to supraspinal and intraspinal organization [23].

The technique of the GM assessment is based on the so-called Gestalt perception by which changes in movement quality is perceived. The global Gestalt perception results in the evaluation of the GMs sensing and noting the fluidity, complexity and variability of these movements by the examiner [24]. This visual Gestalt perception, which involves pattern recognition, is used when dynamic and static images are globally seen and perceived. As fatigue interferes with the observer's Gestalt perception, observers are counseled to never assess GMs for more than 45 minutes. Additionally, when observing multiple abnormal GM recordings, it is necessary to watch normal GM recordings to recalibrate the observer's own Gestalt perception [12]. The two authors of this study followed these suggestions during the study period.

The GM evaluations can be carried out by videotaped assessment as well as by direct observation. The advantage of the videotaped assessment lies in the fact that the videotapes can be replayed with normal and high speed, with the latter being helpful with assessing the complexity and variability of GMs. The stage of alertness of the infant is important during assessment. Interacting (i.e. care giving, toys, bright colors in the environment) with the infant may stop GMs altogether by diverting his/her attention. GMs exhibited by a crying infant may be abrupt, jerky or tremorous. Offering a pacifier attenuates the GM response; small amplitude movements are exhibited by the infant with arms and hips in flexion and the knees in extension [25-27].

The validity of GM assessments as a predictor of gross motor development, thus the ultimate ND outcome, varies with the age when the assessments are done. The best prediction involves serial assessments of each infant if possible. The value of single assessments to predict abnormal gross motor development (i.e. CP) improves with advanced postnatal age. The accuracy of these predictive values when the GM assessment is done during the WM age is 75-80%, while during the FM age the accuracy of the prediction of CP reaches 85-98% [13,22,28].

Previous studies [14,17-19] showed a range of 44-99% interrater reliability with average Kappa value of 88%. The Kappa value > 75 % is considered excellent agreement. The reported intrarater reliability was found between 85-100% [29]. In this study, intrarater reliability was generally considered to be high (0.62-0.88), however interrater reliability was poor (0.35-0.39). The disagreement in the interrater reliability between the two observers in this study may be explained with the differences of the 'trained eyes'. Similar to our conclusion, Adde et al. [30] indicate that professional training and background knowledge play a role in GM evaluations. The second author is a neonatologist who works with infants in the age groups represented in this study. The first author is a pediatric physical therapist that has experience with pediatric patients across a wider age range and thus may have missed some of the subtleties of movements that young infants will exhibit. Due to the discordancies in interrater assessments, Bernhardt et al. [17] recommend subsequent discussions among clinicians when rating infants with Prechtl's method.

The difficulty of evaluation of infants in the fidgety stage may lie in the fact that six to eight week old infants (postnatal age) may be at transition between the writhing and fidgety stage, with FMs being possibly very minimal, thus more difficult to assess correctly.

Since the purpose of the present study was not to establish the predictive value of the Prechtl method of GM assessment, we did not include the ND outcomes of the infants who had been assessed. In fact, by the time of the completion of the study, some of the diagnostic study results and short-term ND outcomes were still unknown.

Conclusions

Although Prechtl's method of assessment of GMs in infants has been shown to be reliable, novice raters using this method should be cautious, as their inter- and intrarater reliability may not be as high as what is described in the literature. The ability to correctly identify GM patterns in infants may be influenced by the rater's varying educational background. By completing this study, we established our own reliabilities. Since the participants of the study had been the first group of infants we had assessed, we plan to re-assesses our inter- and intrarater reliability, as well as incorporate the method in the ND assessment of our patients, both in the NICU and in the ND follow-up clinic.

Conflict of interest

The authors declare no conflict of interest.

Grant support

There are no sources of funding associated with the study in this manuscript.

References