Partial-mouth Periodontal Examination Protocols for Population-based Surveillance of Periodontitis
Duong T Tran1*, Isabel C Gay2 and Muhammad F Walji1
1School of Dentistry, The University of Texas Health Science Center at Houston, USA
2School of Dental Medicine, East Carolina University, USA
*Corresponding author: Duong T. Tran, DDS, MPH, PhD, School of Dentistry, The University of Texas Health Science Center at Houston, 7500 Cambridge St. Houston, TX 77054, USA, Tel: 713-486-4031, E-mail: Duong.T.Tran@uth.tmc.edu
Int J Oral Dent Health, IJODH-2-025, (Volume 2, Issue 1), Research Article; ISSN: 2469-5734
Received: January 13, 2016 | Accepted: February 13, 2016 | Published: February 16, 2016
Citation: Tran DT, Gay IC, Walji MF (2016) Partial-mouth Periodontal Examination Protocols for Population-based Surveillance of Periodontitis. Int J Oral Dent Health 2:025. 10.23937/2469-5734/1510025
Copyright: © 2016 Tran DT, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Objective: This study aimed at evaluating bias in the estimates of prevalence of moderate and severe periodontitis for each of four previously identified partial-mouth periodontal examination protocols (PMPE) compared to the gold standard full-mouth periodontal examination (FMPE).
Material and Methods: Full-mouth examination data of 3,248 adults ≥ 30 years old from the National Health and Nutrition Examination Survey (NHANES) dataset 2011-2012 was computed to provide periodontitis prevalence according to the case definition by the Centers of Disease Control and Prevention and American Academy of Periodontology (CDC/AAP). Periodontitis prevalence was also computed for the PMPE protocols according to the CDC/AAP definition. As in the partial-mouth exams, ≤ 50% of sites are examined, the criteria of the definition was adjusted accordingly (reduced by half) when computing periodontitis prevalence for PMPE protocols. Absolute bias, relative bias, sensitivity and inflation factor were calculated for the PMPE protocols.
Results: Using the CDC/AAP definition, the ranges of relative bias of moderate and severe periodontitis prevalence were 6.8% to 18.1% and 28.4% to 41% in absolute value for the PMPE protocols, respectively. With half-reduced CDC/AAP definition, half-mouth four-site protocol provided small absolute bias (4.2%) and relative bias (11.4%) for the estimates of moderate periodontitis prevalence; corresponding biases for severe periodontitis were -1.1% and -8.2%.
Conclusion: The prevalence of moderate and severe periodontitis could be estimated with limited bias when a half-mouth four-site protocol and a half-reduced CDC/AAP case definition were used in combination not only for the NHANES 2009-2010 but also for the NHANES 2011-2012.
Periodontitis, Surveillance, Partial-mouth periodontal examination protocols, CDC/AAP case definition of periodontitis
Periodontitis is a significant public health problem with potential systemic health impact [1,2]. Monitoring the disease at population level is an important step for planning, implementing and evaluating programs that can promote oral health. Different partial-mouth periodontal examination protocols (PMPE) have been used in population-based surveillance of periodontitis [3-5]; however, researchers have shown that some of the most used protocols for the diagnosis of periodontitis have inherent flaws either grossly underestimating the prevalence of the disease or providing inaccurate estimates of severity [6-9]. After 30 years of intense debate, the Joint EU/USA Periodontal Epidemiology Working Group recommended full-mouth periodontal examination protocol (FMPE) for the surveillance, or PMPE protocols if resources are limited .
Currently, the “gold standard” clinical exam for periodontal diseases includes the probing of six sites per tooth of all present teeth . However, FMPE protocol has been used in only few population-based surveillance systems worldwide [10,12,13] because the full-mouth evaluation of 168 sites (six sites per tooth) is labor, time and resource demanding . While less than ten minutes per subject are usually allotted for a full-mouth periodontal exam in the U.S. National Health and Nutrition Examination Survey (NHANES) , a FMPE required on average 28.8 minutes according to Owens et al.  or even up to 45 minutes according to Aimetti et al. . The use of FMPE has restricted the capacity to monitor the disease in the population, particularly at state and local levels due to limited resources even though public health activities are designed to target state and local populations .
Researchers have proposed simplified periodontal exams for many decades. Among them were partial-mouth periodontal exam protocols (PMPE) which are clinical assessments of representative sets of teeth or probing sites . Using NHANES dataset 2009-2010, a recent study showed that half-mouth four-site protocol provided limited bias in estimating the prevalence of periodontitis according to the case definition by the Centers of Disease Control and Prevention and American Academy of Periodontology (CDC/AAP) . The half-mouth four-site protocol examines mesiobuccal (MB), distobuccal (DB), mesiolingual (ML), and distolingual (DL) sites of teeth in maxillary and mandibular quadrants randomly selected. The CDC/AAP periodontitis case definition  was introduced for population-based surveillance in 2007 and has been implemented in several surveillance systems [20-22].
There have been only three studies which have assessed bias associated with the half-mouth four-site protocol in estimating periodontitis prevalence using the CDC/AAP definition [7,18,20]. Eke et al. showed that the sensitivity of the half-mouth four-site protocol was not acceptable for surveillance [7,20]; however, these results were biased due to the use of the same CDC/AAP definition for both, full-mouth and partial-mouth protocols. As in the partial-mouth exams, ≤ 50% of sites are examined, the criteria of the definition should be adjusted accordingly (reduced by half) when computing periodontitis prevalence for PMPE protocols ; Tran et al. provided evidence of bias caused by application of the same CDC/AAP definition to both FMPE and PMPE protocols.
As the performance of a PMPE protocol was affected by the level of periodontitis in population , the half-mouth four sites protocol needs to be tested in different populations worldwide. For NHANES 2011-2012, sample design was changed to oversample Non-Hispanic Asians in addition to the ongoing oversample of Hispanics, non-Hispanic Blacks, older adults, and low income whites/others . We hypothesized that the half-mouth four sites protocol also perform well with the NHANES 2011-2012 dataset. Our specific aim was to evaluate bias associated with the half-mouth four sites protocol together with other previously identified low-bias PMPE protocols in estimating moderate and severe periodontitis prevalence according to the CDC/AAP case definition on the latest NHANES dataset 2011-2012 available. We didn’t include mild periodontitis in this study because data reliability for mild periodontitis was poor (kappa = 0.36) .
Materials and Methods
Partial-mouth periodontal examination protocols and data origin
PMPE protocol performance was re-evaluated using existing data from the NHANES 2011-2012 dataset . Four PMPE protocols identified as having good performance in previous studies were re-assessed in this study [10,18]. As measurements on mid-buccal and mid-lingual sites are not used in the CDC/AAP case definition, these sites were excluded from the four PMPE protocols as well. The protocols were: 1) mesiobuccal (MB), distobuccal (DB), mesiolingual (ML), and distolingual (DL) measurements on ten teeth of the Community Periodontal Index (CPI protocol) - maxillary and mandibular first and second molars, maxillary right and mandibular left central incisors; 2) MB-DB measurements on all teeth; 3) MB-DL measurements on all teeth; 4) MB-DB-ML-DL measurements on teeth from half-mouth; one random upper quadrant and one random lower quadrant (half-mouth 4 sites). The randomness of the selected quadrant for each subject was created using Stata’s random-number function.
NHANES 2011-2012 is a representative sample of the U.S. population including 4,365 adults aged ≥ 30 years for periodontal examination. From the original dataset, we applied participant exclusion criteria as shown in figure 1. Subjects with less than six teeth were excluded as several PMPE protocols had missing values for these subjects. A total of 3,248 subjects met the inclusion criteria of the PMPE protocols.
Figure 1: Selection of NHANES 2011-2012 subjects. View Figure 1
NHANES participants were examined by dentists using Hu-Friedy PCP-2 [2-12] periodontal probes (Hu-Friedy, Chicago, IL, USA) at mobile examination centers. Each measurement was rounded to the lowest whole millimeter. Periodontal measurements were performed at six sites per tooth. Calibration of examiners for NHANES 2011-2012 subjects was described  but the inter-rater agreement has not been published. For the NHANES 2009-2010, kappa scores was 0.7 for moderate and severe periodontitis according to the CDC/AAP case definition; Inter-rater agreement was poor for mild periodontitis (Kappa = 0.36) .
We used the same method as described in our previous study , comparing periodontitis prevalence determined by FMPE with the prevalence determined by PMPE protocols using the NHANES 2011-2012 dataset. CDC/AAP  defined periodontitis as follows: Mild: ≥ 2 interproximal sites with ≥ 3 mm CAL, and ≥ 2 interproximal sites with ≥ 4 mm PD (on different teeth) or 1 site with ≥ 5 mm PD; Moderate: ≥ 2 interproximal sites with ≥ 4 mm CAL (on different teeth), or ≥ 2 interproximal sites with ≥ 5 mm PD (on different teeth); Severe: ≥ 2 interproximal sites with ≥ 6 mm CAL (on different teeth) and ≥ 1 interproximal site with ≥ 5 mm PD (on the same site with CAL ≥ 6 mm or on different sites).
In order to correctly estimate the true prevalence provided by the gold standard full mouth periodontal exam using the CDC/AAP case definition, a PMPE protocol should be used with a case definition adjusted accordingly to the number of sites examined. As an example: if two sites of 3mm attachment loss need to be found in a full-mouth examination for a diagnosis of periodontitis, then there should be only one 3mm site required to be present for the partial-mouth exam. This adjustment would limit the bias in estimating the prevalence for PMPE protocols as compared to FMPE protocol . Because the PMPE protocols examined less than 50% of total sites, we suggested a half-reduced CDC/AAP case definition for the PMPE protocols to be used as follows: Mild: ≥1 interproximal sites with ≥ 3 mm CAL, and ≥ 1 interproximal sites with ≥ 4 mm PD (on the same site with CAL ≥ 3 mm or on different sites) or 1 site with ≥ 5 mm PD; Moderate: ≥ 1 interproximal sites with ≥ 4 mm CAL, or ≥ 1 interproximal sites with ≥ 5 mm PD; Severe: ≥ 1 interproximal sites with ≥ 6 mm CAL and ≥ 1 interproximal site with ≥ 5 mm PD (on the same site with CAL ≥ 6 mm or on different sites). In this research, we chose to evaluate only moderate and severe periodontitis categories to eliminate a potential bias risk for PMPE protocols in mild periodontitis prevalence due to poor inter-rater agreement .
Comparisons were conducted by: 1) applying the half-reduced CDC/AAP definition to the PMPE protocols and 2) applying the CDC/AAP definition to the PMPE protocols. The CDC/AAP definition was always applied to the FMPE protocol.
The comparison was based on absolute bias, relative bias, sensitivity, and inflation factor :
In case two definitions were applied to FMPE and PMPE protocols, the PMPE protocol may overestimate the periodontitis prevalence, resulting in a positive relative bias and an inflation factor < 100%. For the NHANES sample, PMPE protocol associated bias for prevalence estimate was stratified by age groups at 10-year intervals, race/ethnicity, gender, education level, and median number of teeth. These variables affect periodontitis prevalence  which may impact bias for the estimates provided by PMPE protocols [6,28]. Linear regression analysis was conducted to examine the relation between periodontitis status classified by FMPE and PMPE protocols. Patients with missing data of any variable were not included in the subgroup analysis for that variable. Data analysis was conducted using STATA 13 .
For the NHANES sample, age ranged between 30 to 80 years. Approximately half the subjects were male, having some college education or higher. About one third of the subjects were white. Mean number of teeth was 23.4, median was 25 teeth, and the range was 6 to 28 teeth. 11.4% of sites had PD ≥ 3 mm and 22.1% of sites had CAL ≥ 3 mm (Table 1).
Table 1: Characteristics of the NHANES 2011-2012 sample. View Table 1
Accuracy of the PMPE protocols in the NHANES sample
CDC/AAP case definition applied to the PMPE protocols: According to the CDC/AAP case definition, the prevalence of moderate and severe periodontitis was 37% and 13.4% respectively from full-mouth examination. The prevalence estimated by the PMPE protocols varied from 30.3% to 34.5% for moderate periodontitis and from 7.9% to 9.6% for severe periodontitis. The PMPE protocols underestimated the prevalence of severe periodontitis more than the prevalence of moderate periodontitis. Among the four PMPE protocols, the full-mouth MB-DL protocol provided the smallest bias; it underestimated the prevalence of moderate and severe periodontitis by 6.8% and 28.4%, respectively (Table 2).
Table 2: Accuracy of PMPE protocols in estimating periodontitis prevalence according to CDC/AAP case definition in NHANES sample. View Table 2
Half-reduced CDC/AAP case definition applied to the PMPE protocols: Biases for all PMPE protocols decreased, particularly remarkably for severe periodontitis, when the half-reduced CDC/AAP case definition was applied to the PMPE protocols. Among four protocols, the relative bias provided by MB-DB protocol was the smallest (6.8%) for moderate periodontitis but largest for severe periodontitis (-17.2%). CPI and MB-DL protocols provided very small relative bias for severe periodontitis (-0.7% and -1.5%, respectively); however, they provided relative bias of around 15% for moderate periodontitis (Table 3). Table 4 showed that full-mouth MB-DL, followed by half-mouth four-site protocol, had a good fit to the periodontitis status determined by FMPE.
Table 3: Accuracy of PMPE protocols in estimating periodontitis prevalence according to half-reduced CDC/AAP case definition in NHANES sample. View Table 3
Table 4: Simple linear regression models of the relation between periodontitis status classified by FMPE and PMPE protocols. View Table 4
Bias for PMPE protocols stratified by age, gender, education and number of teeth: Table 5, Figure 2 and Appendix 1 present the prevalence estimates and associated bias for each PMPE protocol, stratified by age, gender, education and number of teeth. The half-mouth four-site protocol provided absolute bias < 8% and relative bias < 1% in absolute value compared to FMPE protocol for moderate periodontitis across all strata. Regarding severe periodontitis, absolute bias for half-mouth four-site protocol were < 4%; and associated relative bias was < 13% in absolute value, except for age group 30-39 years old (23.4%) and 50-59 years old (-19.6%). The full-mouth MB-DL protocol performed well with relative bias < 20% in absolute value for most of strata except for moderate periodontitis of age group 30-39 years old (23.3%), 50-59 years old (27.5%), and group having some college or higher (23.6%). The full-mouth MB-DB and CPI protocols followed in the list but they provided some relative biases > 20% across all strata.
Figure 2: Prevalence of moderate and severe periodontitis for FMPE and PMPE protocols stratified by age and sex. CDC/AAP case definition was used with FMPE and half-reduced CDC/AAP case definition was used with PMPE protocols. HM4s = half-mouth four site protocol. View Figure 2
Table 5: Accuracy of four best performed PMPE protocols in estimating periodontitis prevalence according to half-reduced CDC/AAP case definition in NHANES sample by age and gender. View Table 5
This paper presents a proposal for approaching an epidemiologic-based periodontal assessment in a manner that may adjust for misclassification of disease when a full-mouth assessment cannot be obtained. We used the most recent NHANES dataset 2011-2012 to re-evaluate bias associated with four PMPE protocols in two situations: 1) using the CDC/AAP definition and 2) using the half-reduced CDC/AAP definition for the PMPE protocols. In the first situation, although the full-mouth MB-DL performed better than other protocols, it did not closely estimate severe periodontitis prevalence in the NHANES sample (absolute bias, -3.8%; relative bias, -28.4%; sensitivity, 71.6%). In the second situation, among the four PMPE protocols evaluated, the half-mouth four-site protocol consistently provided small absolute bias and relative bias across all strata of age, gender, education level, and number of teeth. Therefore, the half-mouth four-site protocol should be considered as an alternative for periodontal assessment in surveillance when full-mouth periodontal examination is not feasible. When using the half-mouth four-site protocol, the half-reduced CDC/AAP definition should be used in computing the periodontal prevalence in order to approximate the prevalence as determined by full-mouth periodontal examination and CDC/AAP definition.
Our results corroborate the findings from a previous study on the NHANES 2009-2010 dataset . Results on the two NHANES consecutive datasets consistently showed that the half-mouth four-site protocol provided small absolute bias (< 5%) and relative bias (< 12%) in absolute value for moderate periodontitis; and for severe periodontitis (absolute bias < 2%, relative bias < 11% in absolute value). Although the half-mouth four-site protocol provided a relative bias of 23.4% for severe periodontitis in age group 30-39 years, the corresponding absolute biases for this age group were small (1.1%). In addition to small absolute bias, we can infer from a study that a PMPE protocol with a relative bias ≤ 20% is considered to have an acceptable performance  as relative bias equal to sensitivity minus one .
We also observed the reduction of bias in estimating periodontitis prevalence according to the CDC/AAP definition when computing the prevalence for the PMPE protocols in combination with the suggested half-reduced CDC/AAP definition. This does not mean that we suggest replacing CDC/AAP case definition by half-reduced CDC/AAP definition in defining moderate or severe periodontitis because the half-reduced CDC/AAP case definition does not meet the standards case definitions for moderate or severe periodontitis. We suggest that by using the half-mouth four-site protocol and half-reduced CDC/AAP definition for computation, we can provide limited bias in estimating gold standard prevalence by FMPE protocol and CDC/AAP definition. Applying the CDC/AAP definition to the PMPE protocols contribute partly to the underestimation of prevalence .
The other three PMPE protocols performed poor as compared to the half-mouth four-site protocol across all strata. We observed a consistent substantial overestimation of periodontitis prevalence in the age group 30-39 years and in subjects with ≥ 25 teeth for CPI protocol in the NHANES 2011-2012 as well as in the NHANES 2009-2010 . Although the full-mouth MB-DB, MB-DL protocols both examine 56 sites, the former might be easier and faster to examine only buccal sites. However, they both provided relative biases > 20% in several subgroups. A previous study on the NHANES 2009-2010 also showed that they performed poorer than the half-mouth four-site protocol . Regarding PMPE protocols, the Joint EU/USA Periodontal Epidemiology Working Group recommended full-mouth MB-B-DL, followed by half-mouth six sites, and half-mouth MB-B-DL protocols for periodontitis surveillance . However, there was no direct evidence showing full-mouth MB-B-DL and half-mouth MB-B-DL protocols performed well with the CDC/AAP case definition. The half-mouth six-site protocol would be equal to the half-mouth four-site protocol if using the CDC/AAP case definition because measurements on mid-buccal and mid-lingual sites are not used in the CDC/AAP case definition. Full-mouth MB-DL protocol had a slightly better fit to the periodontitis status determined by FMPE than half-mouth four-site protocol but performed poor in subgroups. Therefore, half-mouth four-site protocol should be considered as an alternative for FMPE.
We leveraged assessment the PMPE protocol performance on two consecutive NHANES datasets 2009-2010 and 2011-2012 which represent the U.S. adult population. Since full-mouth examination protocol has been implemented in the NHANES since 2009, we were able to compare partial-mouth periodontal data with the full-mouth gold standard data set. The excellent performance of the half-mouth four-site protocol on the two NHANES data cycles provided strong support for its use in periodontitis surveillance. We hypothesized that our result could be generalized to other studies that examine the general population. The limitation of our study was the inability to assess the inter-rater agreement between examiners for periodontal measurements in the NHANES 2011-2012 as it has not been published. Therefore, we could not assess bias for mild periodontitis as the inter-rater agreement was poor for the mild periodontitis in NHANES 2009-2010.
Within the limitation of this study, the gold standard prevalence of moderate and severe periodontitis could be estimated with limited bias when a half-mouth four-site protocol and a half-reduced CDC/AAP case definition were used in combination not only for the NHANES 2009-2010 but also for the NHANES 2011-2012. The full mouth periodontal examination should be used whenever possible to improve validity of prevalence estimates; the half-mouth four-site protocol could be an alternative in case when the full mouth periodontal examination is not feasible.
Petersen PE, Ogawa H (2012) The global burden of periodontal disease: towards integration with chronic disease prevention and control. Periodontol 2000 60: 15-39.
Linden GJ, Lyons A, Scannapieco FA (2013) Periodontal systemic associations: review of the evidence. J Periodontol 84: S8-8S19.
Dye BA, Thornton-Evans G (2007) A brief history of national surveillance efforts for periodontal disease in the United States. J Periodontol 78: 1373-1379.
White DA, Tsakos G, Pitts NB, Fuller E, Douglas GV, et al. (2012) Adult Dental Health Survey 2009: common oral health conditions and their impact on the population. Br Dent J 213: 567-572.
Gökalp SG, Doğan BG, Tekçiçek MT, Berberoğlu A, Unlüer S (2010) National survey of oral health status of children and adults in Turkey. Community Dent Health 27: 12-17.
Susin C, Kingman A, Albandar JM (2005) Effect of partial recording protocols on estimates of prevalence of periodontal disease. J Periodontol 76: 262-267.
Eke PI, Thornton-Evans GO, Wei L, Borgnakke WS, Dye BA (2010) Accuracy of NHANES periodontal examination protocols. J Dent Res 89: 1208-1213.
Leroy R, Eaton KA, Savage A (2010) Methodological issues in epidemiological studies of periodontitis--how can it be improved? BMC Oral Health 10: 8.
Kingman A, Susin C, Albandar JM (2008) Effect of partial recording protocols on severity estimates of periodontal disease. J Clin Periodontol 35: 659-667.
Susin C, Dalla Vecchia CF, Oppermann RV, Haugejorden O, Albandar JM (2004) Periodontal attachment loss in an urban population of Brazilian adults: effect of demographic, behavioral, and environmental risk indicators. J Periodontol 75: 1033-1041.
Holtfreter B, Albandar JM, Dietrich T, Dye BA, Eaton KA, et al. (2015) Standards for reporting chronic periodontitis prevalence and severity in epidemiologic studies: Proposed standards from the Joint EU/USA Periodontal Epidemiology Working Group. J Clin Periodontol 42: 407-412.
Centers for Disease Control and Prevention. National Health and Nutrition Examination Survey. Atlanta: Centers for Disease Control and Prevention 2013.
Aimetti M, Perotto S, Castiglione A, Mariani GM, Ferrarotti F, et al. (2015) Prevalence of periodontitis in an adult population from an urban area in North Italy: findings from a cross-sectional population-based epidemiological survey. J Clin Periodontol 42: 622-631.
Centers for Disease Control and Prevention. Examination Components, page 67. Atlanta: Centers for Disease Control and Prevention 2014.
Owens JD, Dowsett SA, Eckert GJ, Zero DT, Kowolik MJ (2003) Partial-mouth assessment of periodontal disease in an adult population of the United States. J Periodontol 74: 1206-1213.
Eke PI, Genco RJ (2007) CDC Periodontal Disease Surveillance Project: background, objectives, and progress report. J Periodontol 78: 1366-1371.
Kingman A, Albandar JM (2002) Methodological aspects of epidemiological studies of periodontal diseases. Periodontol 2000 29: 11-30.
Holtfreter B, Kocher T, Hoffmann T, Desvarieux M, Micheelis W (2010) Prevalence of periodontal disease and treatment demands based on a German dental survey (DMS IV). J Clin Periodontol 37: 211-219.
Page RC, Eke PI (2007) Case definitions for use in population-based surveillance of periodontitis. J Periodontol 78: 1387-1399.
Eke PI, Dye BA, Wei L, Thornton-Evans GO, Genco RJ; CDC Periodontal Disease Surveillance workgroup: James Beck (University of North Carolina, et al. (2012) Prevalence of periodontitis in adults in the United States: 2009 and 2010. J Dent Res 91: 914-920.
Holtfreter B, Schwahn C, Biffar R, Kocher T (2009) Epidemiology of periodontal diseases in the Study of Health in Pomerania. J Clin Periodontol 36: 114-123.
Figueiredo A, Soares S, Lopes H, dos Santos JN, Ramalho LM, et al. (2013) Destructive Periodontal Disease in Adult Indians from Northeast Brazil: Cross-Sectional Study of Prevalence and Risk Indicators. J Clin Periodontol 40:1001-1006.
Centers for Disease Control and Prevention. National Health and Nutrition Examination Survey. Atlanta: Centers for Disease Control and Prevention 2014.
Dye BA, Li X, Lewis BG, Iafolla T, Beltran-Aguilar ED, et al. (2014) Overview and quality assurance for the oral health component of the National Health and Nutrition Examination Survey (NHANES), 2009-2010. J Public Health Dent 74: 248-256.
Centers for Disease Control and Prevention. Oral Health - Periodontal. Atlanta: Centers for Disease Control and Prevention 2014.
Eke PI, Page RC, Wei L, Thornton-Evans G, Genco RJ (2012) Update of the case definitions for population-based surveillance of periodontitis. J Periodontol 83: 1449-1454.
Burt B; Research, Science and Therapy Committee of the American Academy of Periodontology (2005) Position paper: epidemiology of periodontal diseases. J Periodontol 76: 1406-1419.
Kingman A, Morrison E, Löe H, Smith J (1988) Systematic errors in estimating prevalence and severity of periodontal disease. J Periodontol 59: 707-713.
StataCorp (2013) Stata Statistical Software: Release 13. College Station, TX: StataCorp LP.
Nelson DE, Holtzman D, Bolen J, Stanwyck CA, Mack KA (2001) Reliability and validity of measures from the Behavioral Risk Factor Surveillance System (BRFSS). Soz Praventivmed 46 Suppl 1: S3-42.