Saim Yologlu, Ahmet Kadir Arslan, Seyma Yasar, Fatma Hilal Yagin, Cemil Colak

Interactive web-based software for evaluating diagnostic tests and roc curve analyses in health sciences

Aim: This study aims to develop web-based and user-friendly DTROC software in which clinicians and researchers can perform twoand three-dimensional receiver operating characteristic (ROC) curve analyses and calculate diagnostic test performance metrics. Materials and Methods: An illustrative example was presented to implement two- and three-dimensional ROC analyses in medicine. As a case study, three-dimensional ROC analysis in DTROC was applied to the ktemp variable (measurements on the neuropsychometric test for “temporal factor”) in the open-access data set named "AL” (Alzheimer's disease neuropsychometric marker dataset). DTROC web-based software was developed using “plotly”, “pROC”, “dplyr”, “shiny”, “shinydashboard”, “rhandsontable”, “shinyBS”, “DT”, “epiR”, “DiagTest3Grp”, and "the “shiny" libraries. Results: The developed web-based DTROC software allows clinicians and researchers to analyze two and three-dimensional ROC analyzes, ROC curve comparisons, optimum cut-off calculations, required sample size estimation for diagnosis, and calculate metrics for two/more than two-class diagnostic tests without any programming language knowledge or theoretical background about the analyzes mentioned above. When the results for the ktemp variable in the AL data set used in practice were examined, the volume under the surface was found to be 0.77. According to the application results in DTROC Three-dimensional ROC analysis, it can be said that the ktemp variable has a distinguishing feature in detecting AL. Conclusion: All the comprehensive features of DTROC can be accessed free of charge through a graphical user interface that makes the analysis process very easy for users and automates the analysis process. With all these features, DTROC provides much more comprehensive features and various applications than commercial and free software to achieve two/three-dimensional ROC analyses and calculate diagnostic test metrics for two / more classes.

PDF

___

1. Obuchowski NA. Receiver operating characteristic curves and their use in radiology. Radiology 2003;229:3-8.
2. Pepe MS. The statistical evaluation of medical tests for classification and prediction: Medicine; 2003.
3. Obuchowski NA. Estimating and comparing diagnostic tests’ accuracy when the gold standard is not binary1. Acad Radiol 2005;12:1198-204.
4. Zhou X, Obuchowski N, McClish D. Comparing the accuracy of two diagnostic tests. Statistical Methods in Diagnostic Medicine New York: John Wiley & Sons 2002:165-94.
5. Nie NH, Bent DH, Hull CH. SPSS: Statistical package for the social sciences: McGraw-Hill New York; 1975.
6. StataCorp L. Stata data analysis and statistical Software. Special Edition Release 2007;10:733.
7. Schoonjans F, Zalata A, Depuydt C, et al. MedCalc: a new computer program for medical statistics. Comput Methods Programs Biomed 1995;48:257-62.
8. Team R. R Studio: integrated development for R. RStudio, Inc, Boston, MA URL http://www rstudio com. 2015;42:14.
9. Goksuluk D, Korkmaz S, Zararsiz G, et al. easyROC: an interactive web-tool for ROC curve analysis using R language environment. R J 2016;8:213.
10. Storandt M, Hill RD. Very mild senile dementia of the Alzheimer type: II. Psychometric test performance. Arch Neurol 1989;46:383-6.
11. Søreide K. Receiver-operating characteristic curve analysis in diagnostic, prognostic and predictive biomarker research. J Clin Pathol 2009;62:1-5.
12. Goehring C, Perrier A, Morabia A. Spectrum bias: a quantitative and graphical analysis of the variability of medical diagnostic test performance. Stat Med 2004;23:125-35.
13. Obuchowski NA. ROC analysis. AJR Am J Roentgenol 2005;184:364-72.
14. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36.
15. Xiong C, van Belle G, Miller JP, et al. Measuring and estimating diagnostic accuracy when there are three ordinal diagnostic groups. Stat Med 2006;25:1251-73.
16. Mossman D. Three-way rocs. Medical Decision Making 1999;19:78-89.
17. López Ratón M, Rodríguez Álvarez MX, Cadarso Suárez CM, et al., editors. OptimalCutpoints: an R package for selecting optimal cutpoints in diagnostic tests2014: American Statistical Association.
18. Sievert C, Parmer C, Hocking T, et al. plotly: Create Interactive Web Graphics via ‘plotly. js’. R package version 2017;4:110.
19. Robin X, Turck N, Hainard A, et al. Package ‘pROC’. 2012-09-10 09: 34, 2020.
20. Wickham H, Francois R, Henry L, et al. dplyr: A grammar of data manipulation. R package version 04. 2015;3.
21. Chang W, Cheng J, Allaire J, et al. Package ‘shiny’. See http://citeseerx ist psu edu/viewdoc/download. 2015.
22. Chang W. shinydashboard: create dashboards with ‘Shiny’. R package version 0.5. 3. 2016.
23. Owen J. rhandsontable: Interface to the ‘Handsontable. js’ library. R package version 03. 2018;7.
24. Bailey E. shinyBS: twitter bootstrap components for Shiny. R package version 061. 2015.
25. Xie Y, Cheng J, Tan X. DT: A Wrapper of the JavaScript Library ‘DataTables’. R package version 0.2. 2016.
26. Luo J, Xiong C, Luo MJ. Package ‘DiagTest3Grp’. 2012.
27. Stevenson M, Stevenson MM, BiasedUrn I. Package ‘epiR’. 2020.
28. Frey F. SPSS (Software). The International Encyclopedia of Communication Research Methods 2017:1-2.
29. Chambers J. Software for data analysis: programming with R: Springer Science & Business Media; 2008.
30. Sing T, Sander O, Beerenwinkel N, et al. ROCR: visualizing classifier performance in R. Bioinformatics 2005;21:3940-1.
31. Robin X, Turck N, Hainard A, et al. pROC: an opensource package for R and S+ to analyze and compare ROC curves. BMC bioinformatics 2011;12:1-8