Background: Cervical cancer is one of the world’s deadliest forms of cancer and Pap smear is the most popular screening test for diagnosis in early stages. The aim of our study was to assess potential screening rule of ThinPrep Pap test.
Methods: In this cross sectional, study Cervical samples were collected from all women who referred to a pathology center for Papanicolaou test and all samples were prepared with ThinPrep process. To assess validity, biopsy was selected as gold standard. 131 women who had ThinPrep Pap test and biopsy were considered for the analysis. The participants were selected purposeful. Three thresholds were used to define test positivity: 1) Atypical squamous cells of undetermined significance (Asc-us) 2) Low grade squamous intraepithelial lesion (LSIL) 3) High grade squamous intraepithelial lesion (HSIL) and worse. Inter and intra observer reliability were evaluated using kappa (simple and weighted) as well as Fleiss kappa and validity were assessed by the well-known validity estimates for qualitative variables.
Results: Intra observer reliability was moderate for pathologists with low and moderate experience (kappa was 0.44 and 0.46 respectively) and was good for experienced pathologist [kappa (WK) = 0.64]; however, inter observer reliability was poor (Fliess kappa=0.12). For diagnosis of ASCUS and worse, the sensitivity was 96.3% and for diagnosis of HSIL and worse the sensitivity and specificity were 86.6% and 95.1%, respectively.
Conclusion: ThinPrep pap is an acceptable screening test for diagnosis of cervical cancer in early stages. However, experience and specialty have effect on reliability’s results.
Cancer is one of the main causes of death in the world . Cervical cancer is one of the world’s deadliest – but most easily preventable – forms of cancer for women, responsible for more than 270 000 deaths annually, 85% of which occur in developing countries . In Iran, the incidence rate of cervical cancer is less than other countries in the world but the risk of mortality of this cancer is high in this region. Also, because cervical cancer is preventable and it can be diagnosed in primary stages, the existence of a screening test is essential . There are several available and affordable tests that can effectively detect pre-cancerr, as well as several affordable treatment options . The Pap smear (cytology) is the only test that has been used in large populations and that has been shown to reduce cervical cancer incidence and mortality . It is generally accepted that pap programs have had a significant impact on cervical cancer rates in high resource setting over the past several decades .
Liquid-based cytology preparations (LBCP) are replacing conventional Papanicolaou (CP) shown to improve detection rates of cervical intraepithelial neoplasia when compared with conventional preparations. Although the Validity and reliability of a test are one of the most important methodological issues in all research sciences, researchers didn't use the correct statistical methods in many articles . More than 170 scientific studies involving the Thin-Prep Pap test have demonstrated its benefits, including increased disease detection, reduction of equivocal diagnoses, improved specimen adequacy, adjunctive molecular testing, and morphology assessment. However, due to the lack of studies about reliability and validity of Thin-Prep Pap smear test, doing a study about this important topic seems essential. Our study aimed to assess potential screening rule of Thin-Prep Pap test to manage patients correctly and diagnose cervical cancer in early stages.
Materials and Methods
In this cross-sectional study, cervical samples were collected from all women who referred to a pathology center for Papanicolaou test and all samples were prepared with Thin-Prep process (CytoFast Company). The women who had a history of cervical neoplasia and radiation history were excluded from the study. The participants were enrolled between October 2015 and March 2017.
This study investigated the reliability and validity of the ThinPrep Papanicolaou test. For assessment of inter-observer reliability, 97 ThinPrep slides in the pathology center were selected and given to 3 pathologists with different experiences (less experienced, with moderate experience and experienced). We considered pathologist with less than five-year experience as less experienced, five to ten years as moderate and finally more than ten years' experience having fellowship in gynecology pathology as an experienced pathologist. Cytology results were reported using the Bethesda system (TBS). Based on this system, pathologists reported the results in 9 categories: 1) Atypical squamous cells of undetermined significance (Asc-us), 2) Atypical squamous cells – cannot exclude HSIL (ASC-H), 3) Low grade squamous intraepithelial lesion (LSIL), 4) High grade squamous intraepithelial lesion (HSIL), 5) (CIN1), 6) CIN2,3, 7) Squamous cell carcinoma (SCC), 8) Atypical Glandular Cells not otherwise specified (AGC-NOS), 9) Adenocarcinoma. The pathologists reviewed the slides independently and in 3 different centers with the same way. All specimens were taken by two educated and expert health workers so that sampling errors would be low.
For the investigation of intraobserver reliability, 20 slides from 97 were selected randomly and gave to pathologists after about 4 to 6 weeks so that investigated them once again. In the second time observing, the slides number were changed for the blinding process.
For the investigation of validity, the biopsy was selected as a gold standard. So, all women who had normal and abnormal ThinPrep Pap test and also biopsy were interned to our study. Finally, 131 women who had ThinPrep Pap test and biopsy attended in this study. The biopsy was taken by gynecologists and examination of pathology specimens were carried out by an expert and experienced pathologist. The biopsy results were reported in categories as the same as cytology (pap smear test) results. Therefore the results of histopathology and Pap smear were compared with each other and true and false positive and negative were computed. Three thresholds were used to define test positivity: 1) Atypical squamous cells of undetermined significance (Asc-us) and worse 2) Low grade squamous intraepithelial lesion (LSIL) and worse 3) High grade squamous intraepithelial lesion (HSIL) and worse.
Inter and intraobserver reliability was assessed using Fleiss and weighted kappa. For evaluation of validity, the Open Epi website was used. Then validity statistics, sensitivity, specificity, positive predictive value, negative predictive value, positive and negative likelihood ratio, diagnostic accuracy and odds ratio, were calculated for each threshold. We used SPSS version 16.0 for data analysis.
In this section, 3 pathologists observed 20 Pap smear slides twice with 4 to 6 weeks interval. The pathologists’ reports were considered binary, positive and negative, simple and weighted kappa for every pathologist was calculated. Kappa was 0.46 for the pathologist (with moderate experience), 0.44 for the little experienced pathologist and 0.64 for the experienced pathologist.
Fleiss kappa was 0.11 showing poor inter-observer reliability.
As for histopathology and thin perp pap smear test's results and using the Open epi web site, sensitivity, specificity, positive and negative predictive value, positive and negative likelihood ratio, diagnostic accuracy and odds ratio were calculated for every three thresholds. To diagnosis of ASCUS and worse, the sensitivity and specificity were 96.36% and 34.21%, respectively, for diagnosis of LSIL and worse sensitivity was 86.54% and specificity was 75.9% and for diagnosis of HSIL and worse the sensitivity and specificity were 86.67% and 95.05%, respectively. The results of validity were summarized in Table 1a, b.
|Thin-Prep Pap Smear||ASCUS||27||1||0||4||2|
*Atypical squamous cells of undetermined significance (ASCUS) ,**Low grade squamous intraepithelial lesion (LSIL) ,***High grade squamous intraepithelial lesion (HSIL)
|ASCUS and worse||LSIL and worse||HSIL and worse|
|Positive predective value||51.40%||70.30%||83.80%|
|Negative predective value||92.80%||89.50%||96%|
|Positive likelihood ratio||1.4||3.6||17.5|
|Negative likelihood ratio||0.1||0.17||0.14|
The Pap smear is generally accepted as the most successful screening test for cancer detection . We showed that ThinPrep Pap smear is also an acceptable screening test for early detection of cervical cancer. Experienced pathologists in the pathology centers improve the reliability of the ThinPrep in the diagnosis of cervical cancers in the early stage. Our study will help to gynecologists, midwives and executive managers in the public health system to select a correct diagnostic method and manage the patient and costs properly.
In our study, intra observer reliability in pathologists with low and moderate experience was moderate and in experienced pathologist was good. This result indicated that experience influenced on intra observer reliability . While in previously published studies, intra observer reliability was good . Inter observer reliability of the Thin-Prep in our study was poor in contrast to the published papers with moderate reliability .
The reasons that inter and intra observer reliability are poor in our study compared with other studies can be described as below:
2. Lack of postgraduate training and specialist periodicals for pathologists.
3. The main reason that inters and intraobserver reliability is poor, is the lack of specialist pathologists specializing in the field of gynecology. In this study, the pathologists that reviewed slides were general and did not have a specialty in gynecology.
For the investigation of validity, 131 women with positive and negative biopsy were interned in this study. Sensitivity for diagnosis of ASCUS and worse was higher than other studies done in the past . This is because, in this study, all patients' Pap smear and pathology slides were examined by a pathologist who had fellowship in the pathology of gynecology and had a high level of experience and skill in this field and all slides were examined with high accuracy. In our study, by increasing the threshold, the specificity of the test increased and this was in keeping with other studies . These results indicated that Thin-Prep Pap test is very sensitive to the diagnosis of early stages of the disease and much more specific in a high degree of disease. Also, we showed that the Thin-Prep Pap smear has high sensitivity and has no problems such as sampling errors and low screening quality in comparison with conventional Pap smear . The sensitivity and other statistics in our study indicated that Thin-Prep Pap test is more valid than the conventional Pap smear.
It is good to know that in many pathology centers screeners examine the pap smear slides to help the pathologists, but there is no supervise and quality control on their performance. This can have a negative effect on the correct diagnosis of disease. So, it is recommended that monitor the performance of screeners and pathologists and quality control should be taken into account.
Because the patients’ characteristics and clinical data were not available, we could not estimate the added diagnostic value of the Thin-Prep Pap test. We consider this as one of the main limitations of our study.
In conclusion, in summary, based on our results Thin-Prep Pap test can be considered as a screening test for early detection of cervical cancer. However, poor inter-observer reliability and moderate intra observer reliability should be taken into account. Experience and specialty affect reliability's results. Therefore using of experienced and specialist pathologists in the pathology centers and quality control on their performance can improve the reliability of the test and help to use this test as a test in the diagnosis of cervix cancer in early stages.
This study is a part of the MSc thesis supported by Shahid Beheshti University of Medical Sciences.
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Conflict of interest
The authors declare no conflict of interest.
- Cervical Cancer 2017. Available from: https://www.cdc.gov/cancer/cervical/.
- New WHO guide to prevent and control cervical cancer 2014. Available from: http://www.who.int/mediacentre/news/releases/2014/preventing-cervical-cancer/en/.
- Khodakarami N, Farzaneh F, Yavari P, Khayamzadeh M, Taheripanah R, Akbari M. New Cervical Cancer Screening program for Low-risk Iranian Women. The Iranian Journal of Obstetrics, Gynecology And Infertility. 2014; 17(95):10.
- WHO. Comprehensive Cervical Cancer Control, guide to essential practice. second:[Available from: www.who.int/reproductivehealth.
- Screening for cervical cancer 2017. Available from: http://www.who.int/cancer/detection/cervical_cancer_screening/en/.
- New evidence on the impact of cervical cancer screening and treatment using HPV DNA tests, visual inspection, or cytology 2009.
- Sabour Siamak. Validity and reliability of the 13C-methionine breath test for the detection of moderate hyperhomocysteinemia in Mexican adults; statistical issues in validity and reliability analysis. Clinical Chemistry and Laboratory Medicine (CCLM). 2014; 0(0)DOI
- Sabour S. Validity and reliability of the new Canadian Nutrition Screening Tool in the 'real-world' hospital setting Methodological issues. European Journal of Clinical Nutrition. 2015; 69(7)DOI
- Sabour Siamak. Adherence to guidelines strongly improves reproducibility of brachial artery flow-mediated dilation. jatherosclerosis. 2016; 251
- Sabour Siamak. Reliability of a new modified tear breakup time method: methodological and statistical issues. Graefe's Archive for Clinical and Experimental Ophthalmology. 2015; 254(3)DOI
- Sabour S, Abbasnezhad O, Mozaffarian S, Nazari H. Accuracy and Precision in Medical Researches; Common Mistakes and Misinterpretations. World Journal of Research and Review (WJRR). 2017; 4(4):58-60.
- Settakorn J, Rangdaeng S, Preechapornkul N, Nateewatana S, Pongsiralai K, Srisomboon J, et al. Interobserver Reproducibility with LiquiPrepTM Liquid-based Cervical Cytology Screening in a Developing Country. Asian Pacific Journal of Cancer Prevention. 2008; 9:92-96.
- van Grinsven S., Hagenmaier F., van Loon C.J.M., van Gorp M.J., van Kints M.J., van Kampen A.. Does the experience level of the radiologist, assessment in consensus, or the addition of the abduction and external rotation view improve the diagnostic reproducibility and accuracy of MRA of the shoulder?. Clinical Radiology. 2014; 69(11)DOI
- Sabour S.. Re: does the experience level of the radiologist, assessment in consensus, or the addition of the abduction and external rotation view improve the diagnostic reproducibility and accuracy of MRA of the shoulder?. Clinical Radiology. 2015; 70(3)DOI
- Klinkhamer P, Vooijs G, Haan A. Intraobserver and interobserver variability in the diagnosis of epithelial abnormalities in cervical smears. Acta Cytol. 1988; 32(6):794-800.
- Gupta Dilip K., Komaromy-Hiller Gabor, Raab Stephen S., Nath Manju E.. Interobserver and Intraobserver Variability in the Cytologic Diagnosis of Normal and Abnormal Metaplastic Squamous Cells in Pap Smears. Acta Cytologica. 2001; 45(5)DOI
- Confortini M, Biggeri A, Cariaggi M P, Carozz iF M, Minuti P A, Russo A, et al. Intralaboratory reproducibility in cervical cytology. Results of the application of a 100-slide set. Acta Cytol. 1992; 37(1):49-54.
- Sama Donatella, Cotignoli Tullia, Guerrini Laura, Maioli Patrizia, Sintoni Catia, Bucchi Lauro. Intralaboratory Reproducibility of Cervical Cytology Diagnoses in the External Quality Assurance Scheme of the Emilia-Romagna Region of Italy. Gynecologic Oncology. 1996; 60(3)DOI
- Stoler H, Schiffman M. Interobserver Reproducibility of Cervical Cytologic and Histologic Interpretations. JAMA. 2001; 285(11)
- Educational curriculum of Pathology (in persian). Ministry of Health and Medical Education: 2013.
- General Pathology. Available from: http://pathology.ubc.ca/educational-programs/residency-training/general-pathology/.
- Arbyn Marc, Sankaranarayanan Rengaswamy, Muwonge Richard, Keita Namory, Dolo Amadou, Mbalawa Charles Gombe, Nouhou Hassan, Sakande Boblewende, Wesley Ramani, Somanathan Thara, Sharma Anjali, Shastri Surendra, Basu Parthasarathy. Pooled analysis of the accuracy of five cervical cancer screening tests assessed in eleven studies in Africa and India. International Journal of Cancer. 2008; 123(1)DOI
- Chen Changxian, Yang Zhijun, Li Zhuang, Li Li. Accuracy of Several Cervical Screening Strategies for Early Detection of Cervical Cancer: A Meta-Analysis. International Journal of Gynecologic Cancer. 2012; 22(6)DOI
- Hussein T., Desai M., Tomlinson A., Kitchener H.C.. The comparative diagnostic accuracy of conventional and liquid-based cytology in a colposcopic setting. BJOG: An International Journal of Obstetrics and Gynaecology. 2005; 112(11)DOI
- Farzaneh F, Tamimi M, Amiri Z, Alizadeh K. The value of Pap smear in detecting cervical neoplasia compared with histopathologic findings in patients referred to Taleghani Hospital, Tehran 2007-2009. Pajoohandeh Journal. 2011; 16(2):92-97.