Keywords

Meta-analysis, Diagnostic test, DerSimonian and Laird, Odds ratio

Introduction

In order to acquire trustworthy findings from a scientific research, it is essential to design a comprehensive study plan, to appropriately collect data, to select adequate statistical methods for evaluation and to interpret the results accurately. Therefore, both the insecurity arising from impractical data, and the inconsistent and contradictory results can be eliminated by combining the previous studies on the same topic. This approach is defined as "meta-analysis" which provides a joint and accurate decision-making opportunity [1]. Meta-analysis aims to predict the related parameters more accurately by increasing the sample size via statistical analysis of the results obtained from the published or unpublished individual studies which are related to a special topic [2,3]. Meta-analysis has been used in 1980s mostly to assess the clinical efficacy of individual medical interventions and since then, it has been a required and advocated statistical analysis in various disciplines [4].

Today, a large number of diseases can be diagnosed and treated. Diagnostic tests which confirm the presence or absence of a disease, give information about the prognosis of the disease and in certain situations, determine the response to treatment have an essential role in medical field [5].

The estimation methods used in meta-analysis have been investigated by numerous studies. Viechtbauer, et al. showed that the Paule and Mandel (PM) estimator is the same as the so-called empirical Bayes estimator [6]. The PM estimation method retains many of the advantages of the method of moments, because it is semiparametric and requires no convergence diagnostics [7]. The moment-based method proposed by DerSimonian and Laird (DL) is most commonly used to estimate the heterogeneity variance. DL method is the default approach in many software routines. Simulation studies have found that the method can be biased and thus, other methods have been introduced [3]. The maximum likelihood (ML) method is asymptotically efficient, but requires an iterative solution. The Sidikve Jonkman (SJ) estimator has methodological similarities with the PM estimator. Although the Hedges ve Olkin (HO) estimator is simple to compute and does not require an iterative numerical solution, it is not widely used [8,9]. On the other hand, restricted maximum likelihood (REML) estimation is a generally well-known estimation technique in the statistical literature [10]. Bowden, et al. conducted an empirical study by comparing DL and PM estimation methods and stated that as the variance between studies increased, τ² value of PM was greater than τ² obtained from DL [11].

Study Method

Diagnostic test

Diagnostic tests are utilized to identify the presence or absence of a condition in order to develop an appropriate treatment plan [12]. Many performance measures are used to evaluate a diagnostic test. These measures include sensitivity, specificity, false positive rate, false negative rate, Positive Predictive Value (PPV), Negative Predictive Value (NPV), positive likelihood ratio (LR+), negative likelihood ratio (LR-), accuracy, Youden Index (YI) and Diagnostic Odds Ratio (DOR) (Table 1 and Table 2) [13-15].

Meta-analysis

Meta-analysis is a statistical method that aims to provide more reliable and accurate findings via combining and summarizing the results from previous individual studies [3,16,17]. In 1954, Cochran developed a method for parameter estimation by bringing together researches made in different places, times and areas in an appropriate form [18]. Meta-analysis has been used in 1980s mostly to assess the clinical efficacy of individual medical interventions and since then, it has been a required and advocated statistical analysis in various disciplines [4].

In meta-analysis, different estimations are provided depending upon the contents of the study and these estimations are essential for determining the combined effect and assigning study weights. One of the models utilized in the meta-analysis is the fixed-effects model and the other is the random-effects model [9,16,17].

The fixed-effects model is based on the ground of the assumption that all studies included in the analysis predict the same effect size. In other words, it is assumed that if a trial has an effect, this effect does not interact with the study criteria and it remains constant. In the fixed-effects model, it is assumed that the differences between the effect sizes are the results of the sampling error. In this model, relatively narrower confidence intervals are obtained, accurate information about the homogeneity of the studies cannot be estimated since the between-study variance is not taken into account, and studies with small sample size may not be as sensitive as the ones with large samples [1,9,17,19,20].

The random-effects model makes calculations taking into consideration both the variances between the studies and within each study. The random-effects model assumes that the heterogeneity of all effect sizes arises both from the sampling error and the variations within the study population. Since the between-study variances are taken into account with this model, the homogeneity of the studies can be assessed, and it is more sensitive in small sample sized studies [1,9,17,19,20].

Methods to estimate between-study variance

In the meta-analysis, there are various methods to estimate the between-study variance. Some of those are DerSimonian-Laird (DL), Restricted Maximum Likelihood (REML), Sidik and Jonkman (SJ), Hedges and Olkin (HO), Maximum Likelihood (ML), and Paule and Mandel (PM) estimation methods (Table 3) [3,9,10,21].

DerSimonian and Laird (DL) method: DL estimator is a non-iterative method that is frequently used as the default approach in many softwares [3,9]. τ² which is the between-study variance for random effect size model, and w_i which is the reverse of fixed effect variance for each study are used to calculate the new weights as

$$w_i^{*}=\frac{1}{[v_i+{\widehat{\tau }}^2]}$$

From here ${\widehat{\tau }}^2,$

$${\widehat{\tau }}^2=\left\{\begin{array}{ccc}\frac{Q-(k-1)}{{\sum }_{i=1}^k{w}_i-\frac{{\sum }_{i=1}^k{w}_i{}^2}{{\sum }_{i=1}^k{w}_i}}& ,& Q\ge df\\ 0& ,& Q<df\end{array}\right.$$

When ${\widehat{\tau }}^2$ equals to zero, it transforms from the random-effects model to the fixed-effects model.

$$Q={\displaystyle {\sum }_{i=1}^k{w}_i}{(\ln O{R}_i)}^2-\frac{{({\sum }_{i=1}^k{w}_i\ln O{R}_i)}^2}{{\sum }_{i=1}^k{w}_i}$$

The above-mentioned Q value is calculated as

$$\ln {\overline{T}}_{DL}=\frac{{\sum }_{i=1}^k{w}_i^{*}\ln O{R}_i}{{\sum }_{i=1}^k{w}_i^{*}}$$

The combined effect size is calculated as

$$\ln {\overline{T}}_{DL}=\frac{{\sum }_{i=1}^k{w}_i^{*}\ln O{R}_i}{{\sum }_{i=1}^k{w}_i^{*}}$$

The variance of the combined estimation is calculated as

$$Var({\overline{T}}_{DL})=\frac{1}{{\sum }_{i=1}^k{w}_i^{*}}$$

and % (1 – α) the confidence interval is calculated as stated below

$\exp \left[\ln ({\overline{T}}_{DL})-z_{\frac{\alpha }{2}}\sqrt{Var({\overline{T}}_{DL})}\right]\le \theta \le \exp \left[\ln ({\overline{T}}_{DL})+z_{\frac{\alpha }{2}}\sqrt{Var({\overline{T}}_{DL})}\right]$ [17,22,23].

Restricted Maximum Likelihood (REML) method: REML estimation method is a well-known technique in the statistical literature and in this estimation method, the between-study variance (τ²) is calculated via double-iterative process. The first iteration includes the estimation of maximum likelihood estimator of ${\widehat{\tau }}_{ML}^2$ [3,8,10]. The estimate of ${\widehat{\tau }}_{REML}^2$ is obtained by the derivative of the restricted log-likelihood function.

$$\ln L({\tau }^2)=-\frac{k}{2}\ln (2\pi )-\frac{1}{2}{\displaystyle \sum _{i=1}^k\ln (v_i+{\tau }^2)}-\frac{1}{2}{\displaystyle \sum _{i=1}^k\frac{({(y_i-{\widehat{\mu }}_{RE}({\widehat{\tau }}_{ML}^2))}^2}{v_i+{\tau }^2}-\frac{1}{2}}\ln \left({\displaystyle \sum _{i=1}^k\frac{1}{v_i+{\tau }^2}}\right)$$

with respect to τ² equals to zero and the resulting solution of the equation for τ² is,

$${\widehat{\tau }}_{REML}^2=\max \left\{0,\frac{{\sum }_{i=1}^k{(w_{i,RE})}^2({(y_i-{\widehat{\mu }}_{RE}({\widehat{\tau }}_{ML}^2))}^2-v_i}{{\sum }_{i=1}^k{(w_{i,RE})}^2}+\frac{1}{{\sum }_{i=1}^k{w}_{i,RE}}\right\}$$

From here, it is provided via

$$w_{i,RE}=\frac{1}{v_i+{\widehat{\tau }}_{REML}^2}$$ [9]

Sidik and Jonkman (SJ) method: This estimation method is proposed by Sidik and Jonkman and it is a non-iterative technique based on weighted least squares method [24]. To obtain the SJ estimator ${\widehat{r}}_i=\frac{v_i}{{\widehat{\tau }}_0^2}$ (assuming ${\widehat{\tau }}_0^2$ ≠ 0) and with this equation ${\widehat{q}}_i={\widehat{r}}_i+1$ values are calculated.

Here, ${\widehat{\tau }}_0^2=\frac{{\sum }_{i=1}^k{(y_i-\overline{y})}^2}{k}$ is the initial estimate of the between-study variance. Then, the SJ estimator is obtained by setting the quantity $\sum {\widehat{q}}_i{}^{-1}{(y_i-{\widehat{\mu }}_{\widehat{q},RE})}^2$ equal to its expected value ${\widehat{\tau }}_{SJ}^2=\frac{1}{k-1}{\sum }_{i=1}^k{\widehat{q}}_i{}^{-1}{(y_i-{\widehat{\mu }}_{\widehat{q},RE})}^2$ [9].

Hedges and Olkin (HO) method: Hedges and Olkin estimation method was first defined by Cochran [18]. Hedges (1983) discussed the estimation method for the between-study variance component in the meta-analytic context. The estimator is obtained by setting the sample variance

$$S_y^2=\frac{1}{k-1}{\displaystyle \sum _{i=1}^k{(y_i-\overline{y})}^2}$$

equal to its expected value and solving τ², which yields

$${\widehat{\tau }}_{HO}^2=\max \left\{0,\frac{1}{k-1}{\displaystyle \sum _{i=1}^k{(y_i-\overline{y})}^2-\frac{1}{k}{\displaystyle \sum _{i=1}^k{v}_i}}\right\}\text{ (9)}$$

Maximum Likelihood (ML) method: ML estimator is asymptotically efficient but it requires an iterative solution. According to the marginal distribution of y_i ~ N(μ,v_i + τ²), the estimation of ${\widehat{\tau }}_{ML}^2$ is obtained by maximizing the log-likelihood function.

$$\ln L(\mu ,{\tau }^2)=-\frac{k}{2}\ln (2\pi )-\frac{1}{2}{\displaystyle \sum _{i=1}^k\ln (v_i+{\tau }^2)}-\frac{1}{2}{\displaystyle \sum _{i=1}^k\frac{{(y_i-\mu )}^2}{(v_i+{\tau }^2)}}$$

Setting partial derivatives with respect to µ and τ² which are equal to zero, and solving the likelihood equations for the two parameters to be estimated, the ML estimators for µ and τ² can be obtained as follows

$${\widehat{\mu }}_{RE}({\widehat{\tau }}_{ML}^2)=\frac{{\sum }_{i=1}^k{w}_{i,RE}{y}_i}{{\sum }_{i=1}^k{w}_{i,RE}}$$

$${\widehat{\tau }}_{ML}^2=\max \left\{0,\frac{{\sum }_{i=1}^k{(w_{i,RE})}^2({(y_i-{\widehat{\mu }}_{RE}({\widehat{\tau }}_{ML}^2))}^2-v_i)}{{\sum }_{i=1}^k{(w_{i,RE})}^2}\right\}$$

From here, it is provided via

$w_{i,RE}=\frac{1}{v_i+{\widehat{\tau }}_{ML}^2}$ [9]

Paule and Mandel (PM) method: The Paule and Mandel estimation method has most of the advantages of the method of moments due to its' semiparametric characteristics and the lack of requirement of convergence diagnostics [7]. This method is essentially equivalent to the Emprical Bayes estimator discussed by Morris [9,25]. Using the random effect weights, this method is equivalent to empirical Bayes method. Paule and Mandel, proposed a special form of Q with a_i equation [26].

$${\widehat{\tau }}_{PM}^2=\frac{Q_{gen}-\left({\sum }_{i=1}^k{a}_i{v}_i-\frac{{\sum }_{i=1}^k{a}_i^2{v}_i}{{\sum }_{i=1}^k{a}_i}\right)}{{\sum }_{i=1}^k{a}_i-\frac{{\sum }_{i=1}^k{a}_i^2}{{\sum }_{i=1}^k{a}_i}}$$

$$a_i=w_{i,RE}=\frac{1}{v_i+{\tau }^2}.$$

The generalized Q statistic is

$Q_{gen}={{\displaystyle \sum w_{i,RE}(y_i-{\widehat{\mu }}_{RE}({\tau }^2))}}^2\sim {\chi }_{k-1}^2$ [9]

Implementation

In the application section of the study, we conducted a meta-analysis in order to evaluate the effectiveness of Clinical Oral Examination (COE) for predicting the diagnosis of oral dysplasia or Oral Squamous Cell Carcinoma (OSCC) of mucosal lesions that were submitted for biopsy and were diagnosed histologically. A Clinical Oral Examination (COE) is the principal strategy used to detect abnormal oral mucosal changes including OSCC and oral dysplasia, which is the initial stage of cellular transformation to malignancy [27]. Automated literature searches of articles published from Jan 1, 1966, through Jan 20, 2010, was completed by using the PubMed, Web of Knowledge and the Cochrane Library databases via using the search terms "oral mucosal lesion screening" and "oral lesions". A total of 1,252 articles have met the inclusion criteria (1,195 studies in PubMed, 38 in the Cochrane Library and 19 in Web of Knowledge). Additional articles which included clinically detected lesions that were identified by means of visual examination and other visual techniques were also entered as subsets of data. In all enrolled studies, the main inclusion criterion was the presence of histological diagnoses which were obtained after tissue biopsy of clinically detected oral mucosal lesions.

In conclusion, twenty-four observational studies which included 7,079 patients and 1,956 biopsies met the inclusion criteria [27]. The analyses for diagnostic test and meta-analysis of the data were performed by using Open Meta-Analyst, R Packages, Meta Essential 1.4, STATA 13.0 statistical software.

Results and Discussion

First of all, the sensitivity, specificity, odds ratio, accuracy, Positive Likelihood Ratio (PLR), Negative Likelihood Ratio (NLR), Positive Predictive Value (PPV), Negative Predictive Value (NPV), and Youden Index (YI) were calculated (Table 4). When the Odds Ratio values (OR) of the studies were considered, both very high (OR = 4815) and very low OR values (OR = 0.068) were observed. The accuracy value, which is expected to be high in a favorable diagnostic test, has varied between 0.065 and 0.995 among the studies.

The Q test was utilized to evaluate the heterogeneity between studies. As a result, it was assessed that the studies were heterogeneous (Q = 68.94, p = 0.00 < 0.05). Thus, random effect model was used for meta-analysis.

Using Open Meta Analyst statistical software, the meta-analysis of the random effect model that was performed according to the DerSimonian and Laird (DL) estimation method revealed that the pooled sensitivity value of COE was high [0.953 (95% CI: 0.895-0.979)] and the pooled specificity was low [0.25 (95% CI: 0.124-0.44)] (Table 5).

When the PLR and NLR were considered, the pooled PLR value was 1.053 (95% CI: 1.00-1.11) and the pooled NLR was 0.469 (95% CI: 0.341-0.645). In general, a PLR value above 10.0 indicates that the test makes a significant contribution to the diagnostic process and a NLR below 0.2 indicates that the test is good at ruling out diseases [15,28]. Additionally, PLR and NLR values of 1 demonstrate that the test provides no information about the likelihood of the disease. In our study, the pooled odds ratio (OR) was 6.031 (95% CI: 2.208-16.471), revealing the ineffectiveness of the COE in prediction of oral dysplasia or OSCC.

The results of the analyses obtained with DL, REML, ML, PM, HO, and SJ estimation methods using R, Open Meta Analyst, Meta Essential softwares are presented in Table 6. The DL estimation method was present in all the software programs used in the study. The Q statistic value that was calculated for evaluation of the homogeneity by using the DL method in R, Open Meta Analyst and Meta Essential softwares yielded to 68.943 (p < 0.0001), and the lowest I² and τ² values were obtained. Based on these results, it can be concluded that a moderate level of heterogeneity was present. In R and Open Meta Analyst softwares, Restricted Maximum Likelihood (REML), Maximum Likelihood (ML), Paule and Mandel (PM), Hedges and Olkin (HO) and Sidik and Jonkman (SJ) estimation methods were utilized and similar results were obtained with both softwares. According to the results, the highest I² and τ² values were obtained using the non-iterative SJ estimation method. The analysis with the PM estimation method which is simple and does not require distributional assumption, the lowest I² value was obtained following REML and ML estimation methods (I² = 72.80%).

The publication bias was investigated by using the Egger weighted regression method and a funnel plot chart was prepared (Table 7 and Figure 1). Egger regression method and the funnel plot chart showed that, with 95% confidence intervals, publication bias was not present (p = 0.087 > 0.05).

Conclusion

The results of our study indicate that Clinical Oral Examination (COE) is not a sufficient technique for the diagnosis. Except Der Simonian and Laird (DL) estimation methods, analyses could be performed with other estimation methods in ready-made softwares. Furthermore, it can be concluded that the appropriate software program for meta-analysis varies depending on the user's needs and preferences.

Conflict of Interest

The authors declare that they have no conflict of interest.

References

1. Kurt S (2009) Meta Analizin Klinik Çalismalarda Kullanimi Üzerine Bir Uygulama. Yüksek Lisans Tezi, Marmara Üniversitesi Ekonometri Anabilim Dali, Istanbul, 77.
2. Karahan AE (2014) Hayvancilikta Meta Analiz Uygulamalari. Yüksek Lisans Tezi, Ege Üniversitesi Zootekni Anabilim Dali, Izmir, 53s.
3. Langan D, Higgins JPT, Jackson D, Bowden J, Veroniki AA, et al. (2018) A comparison of heterogeneity variance estimators in simulated random-effects meta-analyses. Res Synth Methods 10: 83-98.
4. Crombie IK, Davies HT (2009) What is meta-analysis? Hayward Medical Communications.
5. Çubuk M (2011) Tani Dogrulugunu Ölçülerini Etkileyen Faktörlerin Belirlenmesi. Hacettepe Üniversitesi Biyoistatistik Anabilim Dali, Ankara, 107.
6. Viechtbauer W, Lopez-Lopez JA, Sanchez-Meca J, Marin-Martinez F (2014) A comparison of procedures to test for moderators in mixed-effects meta-regression models. Psychol Methods 20: 360-374.
7. Jackson D, Veroniki AA, Law M, Tricco AC, Baker R (2017) Paule-Mandel estimators for network meta-analysis with random inconsistency effects. Res Synth Methods 8: 416-434.
8. Langan D, Higgins JPT, Simmonds M (2017) Comparative performance of heterogeneity variance estimators in meta-analysis: A review of simulation studies. Res Synth Methods 8: 181-198.
9. Veroniki AA, Jackson D, Viechtbauer W, Bender R, Bowden J, et al. (2016) Methods to estimate the between-study variance and its uncertainty in meta-analysis. Res Synth Methods 7: 55-79.
10. Thorlund K, Wetterslev J, Awad T, Thabane L, Gluud C (2011) Comparison of statistical inferences from the DerSimonian-Laird and alternative random-effects model meta-analyses - an empirical assessment of 920 Cochrane primary outcome meta-analyses. Res Synth Methods 2: 238-253.
11. Bowden J, Tierney JF, Copas AJ, Burdett S (2011) Quantifying, displaying and accounting for heterogeneity in the meta-analysis of RCTs using standard and generalised Q statistics. BMC Med Res Methodol 11: 41.
12. White S, Schultz T, Enuameh YAK (2011) Synthesizing evidence of diagnostic accuracy. Lippincott Williams &Wilkins, JBI Book Collection, Philadelphia, 116.
13. Aslan D, Sandberg S (2007) Simple statistics in diagnostic tests. JMB 26: 309-313.
14. Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PMM (2003) The diagnostic odds ratio: A single indicator of test performance. J Clin Epidemiol 56: 1129-1135.
15. Karaagoglu E, Karakaya J, ve Kiliçkap M (2016) Tani Testlerinin Degerlendirilmesinde Istatistiksel Yöntemler. Ankara, 261.
16. Borenstein M, Hedges LV, Higgins J, Rothstein H (2009) Introduction to meta-analysis. John Wiley & Sons, Ltd, Chichester, UK.
17. Yeniay H (2013) Yasam Verilerinin Meta Analizi. Yüksek Lisans Tezi, Hacettepe Üniversitesi, Istatistik Anabilim Dali, Ankara, 72.
18. Cochran WG (1954) The combination of estimates from different experiments. Biometrics 10: 101.
19. Kinay E (2012) Üniversite Giris Sinavi Yordama Yeterliligi Çalismalarinin Meta Analizi. Yüksek Lisans Tezi, Ankara Üniversitesi Egitim Bilimleri Enstitüsü, Ankara, 110.
20. Küçükönder H (2007) Meta Analiz ve Tarimsal Uygulamalar, Yüksek Lisans Tezi, Kahramanmaras Sütçü Imama Üniversitesi Zootekni Anabilim Dali, Kahramanmaras, 100.
21. Panityakul T, Bumrungsup C, Knapp G (2013) On estimating residual heterogeneity in random-effects meta-regression: A comparative study. J Stat Theory Appl 12: 253-265.
22. DerSimonian R, Laird N (1986) Meta-analysis in clinical trials. Control Clin Trials 7: 177-188.
23. Petitti DB (2000) Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis. Oxford University Press, New York, 306.
24. Sidik K, Jonkman JN (2005) Simple heterogeneity variance estimation for meta-analysis. Journal of the Royal Statistical Society: Series C (Applied Statistics) 54: 367-384.
25. Morris CN (1983) Parametric empirical Bayes inference: Theory and applications. Journal of the American Statistical Association 78: 47.
26. Paule RC, Mandel J (1982) Consensus values and weighting factors. Journal of Research of the National Bureau of Standards 87: 377-385.
27. Epstein JB, Güneri P, Boyacioglu H, Abt E (2012) The limitations of the clinical oral examination in detecting dysplastic oral lesions and oral squamous cell carcinoma. J Am Dent Assoc 143: 1332-1342.
28. Deeks JJ (2001) Systematic reviews in healthcare: Systematic reviews of evaluations of diagnostic and screening tests. BMJ 323: 157-162.
29. Onofre MA, Sposto MR, Navarro CM, Motta ME, Turatti E, et al. (1997) Potentially malignant epithelial oral lesions: Discrepancies between clinical and histological diagnosis. Oral Dis 3: 148-152.
30. Zheng W, Soo KC, Sivanandan R, Olivo M (2002) Detection of squamous cell carcinomas and pre-cancerous lesions in the oral cavity by quantification of 5-aminolevulinic acid induced fiuorescence endoscopic images. Lasers Surg Med 31: 151-157.
31. Epstein JB, Zhang L, Poh C, Nakamura H, Berean, K, et al. (2003) Increased allelic loss in toluidine blue-positive oral premalignant lesions. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 95: 45-50.
32. Maraki D, Becker J, Boecking A (2004) Cytologic and DNA-cytometric very early diagnosis of oral cancer. J Oral Pathol Med 33: 398-404.
33. Ram S, Siar CH (2005) Chemiluminescence as a diagnostic aid in the detection of oral cancer and potentially malignant epithelial lesions. Int J Oral Maxillofac Surg 34: 521-527.
34. Du GF, Li CZ, Chen HZ, Chen XM, Xiao Q, et al. (2007) Rose bengal staining in detection of oral precancerous and malignant lesions with colorimetric evaluation: A pilot study. Int J Cancer 120: 1958-1963.
35. Epstein JB, Silverman S Jr, Epstein JD, Lonky SA, Bride MA (2008) Analysis of oral lesion biopsies identified and evaluated by visual examination, chemiluminescence and toluidine blue. Oral Oncol 44: 538-544.
36. Allegra E, Lombardo N, Puzzo L, Garozzo A (2009) The usefulness of toluidine staining as a diagnostic tool for precancerous and cancerous oropharyngeal and oral cavity lesions. Acta Otorhinolaryngol Ital 29: 187-190.
37. Wilder-Smith P, Lee K, Guo S, Zhang J, Osann K, et al. (2009) In vivo diagnosis of oral dysplasia and malignancy using optical coherence tomography: Preliminary studies in 50 patients. Lasers Surg Med 41: 353-357.
38. Arduino PG, Surace A, Carbone M, Elia A, Massolini G, et al. (2009) Outcome of oral dysplasia: A retrospective hospital-based study of 207 patientswith a long follow-up. J Oral Pathol Med 38: 540-544.
39. Nagaraju K, Prasad S, Ashok L (2010) Diagnostic efficiency of toluidine blue with Lugol's iodine in oral premalignant and malignant lesions. Indian J Dent Res 21: 218-223.
40. Koch FP, Kunkel M, Biesterfeld S, Wagner W (2011) Diagnostic efficiency of differentiating small cancerous and precaneerous lesions using mucosal brush smears of the oral cavity - A prospective and blinded study. Clin Oral Investig 15: 763-769.
41. Jerjes W, Upile T, Conn B, Hamdoon Z, Betz CS, et al. (2010) In vitro examination of suspicious oral lesions using optical coherence tomography. Br J Oral Maxillofac Surg 48: 18-25.
42. Prout MN, Sidari JN, Witzburg RA, Grillone GA, Vaughan CW (1997) Head and neck cancer screening among 4611 tobacco users older than forty years. Otolaryngol Head Neck Surg 116: 201-208.
43. Epstein JB, Feldman R, Dolor RJ, Porter SR (2003) The utility of tolonium chloride rinse in the diagnosis of recurrent or second primary cancers in patients with prior upper aerodigestive tract cancer. Head Neck 25: 911-921.
44. Remmerbach TW, Weidenbach H, Müller C, Hemprich A, Pomjanski N, et al. (2003) Diagnostic value of nucleolar organizer regions (AgNORs) in brush biopsies of suspicious lesions of the oral cavity. Anal Cell Pathol 25: 139-146.
45. Chen Y-W, Lin J-S, Fong JH-J, Wang I-K, Chou S-J, et al. (2007) Use of methylene blue as a diagnostic aid in early detection of oral cancer and precancerous lesions. Br J Oral Maxillofac Surg 45: 590-591.
46. Bhalang K, Suesuwan A, Dhanuthai K, Sannikorn P, Luangjarmekorn L, et al. (2008) The application of acetic acid in the detection of oral squamous cell carcinoma. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 106: 371-376.
47. Farah CS, McCullough MJ (2007) A pilot case control study on the efficacy of acetic acid wash and chemiluminescent illumination (ViziLite) in the visualisation of oral mucosal white lesions. Oral Oncol 43: 820-824.
48. Mehrotra R, Singh MK, Pandya S, Singh M (2008) The use of an oral brush biopsy without computer-assisted analysis in the evaluation of oral lesions: A study of 94 patients. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 106: 246-253.
49. Mclntosh L, McCullough MJ, Farah CS (2009) The assessment of diffused light illumination and acetic acid rinse (Microlux/DL) in the visualisation of oral mucosal lesions. Oral Oncol 45: e227-e231.
50. Mehrotra R, Singh M, Thomas S, Nair P, Pandya S, et al. (2010) A cross-sectional study evaluating chemiluminescence and autofluorescence in the detection of clinically innocuous precancerous and cancerous oral lesions. J Am Dent Assoc 141: 151-156.
51. Koch FP, Kaemmerer PW, Biesterfeld S, Kunkel M, Wagner W (2011) Effectiveness of autofluorescence to identify suspicious oral lesions - A prospective, blinded clinical trial. Clin Oral Investig 15: 975-982.
52. Güneri P, Epstein JB, Kaya A, Veral A, Kazandi A, et al. (2011) The utility of toluidine blue staining and brush cytology as adjuncts in clinieal examination of suspicious oral mucosal lesions. Int J Oral Maxillofac Surg 40: 155-161.

---