Time-Dependent Receiver Operating Characteristic Analysis and Applications in the Field of Medicine

When there is a time-dependency between the biomarker and the event of interest (death, disease, relapse etc.), classical receiver operating characteristic (ROC) analysis may not be able to estimate the true performance of the biomarker. For such cases, time-dependent ROC, an extended version of the standard ROC, is developed. In this study, the aim is to demonstrate applications of this modified ROC method on medical datasets and find out if it should be preferred over classical ROC for time-dependent situations. Comparison between classical ROC and Kaplan-Meier (KM) estimator, which is one of the two time-dependent ROC estimators, has been made using datasets in this study. Nearest Neighbor Estimator (NNE), the alternative of KM estimator, is also applied on all datasets. Then the findings of these two approaches are compared. It is concluded that time-dependent ROC method is superior to the standard ROC analysis. In general, the closer to the event time, the higher performance is observed. Especially, biomarkers measured at last 12 or 6 months before the event are determined to be better at classification than the earlier measurements. Though in all applications KM and NNE yielded very similar results, the latter is considered to be more appropriate to evaluate the performance of a biomarker when a time dependent data is also censored. Results of this study show that time-dependent ROC analysis performs more accurately when there is a time dependency between the biomarker and the event of interest; therefore, it is recommended.

Keywords:

Time-dependent ROC, Diagnostic tests, Biomarker Kaplan-Meier, Nearest neighbor estimator,

PDF

___

Akritas M. 1994. Nearest neighbor estimation of a bivariate distribution under random censoring. Ann Stat, 22: 1299-1327.
Antolini L, Boracchi P, Biganzoli E. 2005. A time-dependent discrimination index for survival data. Stat Medic, 24: 3927-3944.
Balasubramanian R, Lagakos S. 2001. Estimation of the timing of perinatal transmission of HIV. Biometrics, 57: 1048-1058.
Blanche P, Dartigues JF, Jacqmin-Gadda H. 2013. Review and comparison of ROC curve estimators for a time-dependent outcome with marker-dependent censoring. Biometrical J, 5: 687-704.
Blanche P, Dartigues JF, Jacqmin-Gadda, H. 2013. Estimating and comparing time-dependent areas under receiver operating characteristic curves for censored event times with competing risks. Stat Medic, 32(30): 5381-5397.
Chambless L, Diao G. 2006. Estimation of time-dependent area under the ROC curve for long-term risk prediction. Stat Medic, 25: 3474-3486.
Diefenbach CS, Li H, Hong F, Gordon LI, Fisher RI, Bartlett NL. 2015. Evaluation of the international prognostic score (IPS-7) and a simpler prognostic score (IPS-3) for advanced Hodgkin lymphoma in the modern era. Br J Haematol, 171(4): 530-538.
Etzioni R, Pepe MS, Longton G, Chengcheng H, Goodman G. 1999. Incorporating the time dimension in receiver operating characteristic curves: a case study of prostate cancer. Medic Decision Making, 19: 242-251.
Heagerty PJ, Saha-Chaudhuri P. 2013. survivalROC: Time-dependent ROC curve estimation from censored survival data. R package version 1.0.3. URL: http://CRAN.R-project.org/package=survivalROC. (February 10, 2022).
Heagerty PJ, Lumley T, Pepe MS. 2000. Time-dependent ROC curves for censored survival data and a diagnostic marker. Biometrics, 56: 337-344.
Heagerty PJ, Zheng Y. 2005. Survival model predictive accuracy and ROC curves. Biometrics, 61: 92-105.
IBM Corp. 2011. IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY: IBM Corp.
Kaplan E, Meier P. 1958. Nonparametric estimation from incomplete observations. J Amer Stat Assoc, 53: 457-481.
Leisenring W, Pepe MS, Longton G. 1997. A marginal regression modelling framework for evaluating medical diagnostic tests. Stat Medic, 16: 1263-1281.
Martinez-Camblor P, Bayon GF, Perez-Fernandez S. 2016. Cumulative/dynamic ROC curve estimation. J Stat Comput Simul, 86(17): 3582-3594.
Paydaş S, Lacin S, Doğan M, Barışta İ, Yıldız B, Seydaoğlu G. 2021. IPS-3 validation in 1012 cases with classical hodgkin lymphoma. Leukemia Res, 102: 106519.
Pepe MS. 1997. A regression modeling framework for ROC curves in medical diagnostic testing. Biometrika, 84: 595-608.
R Development Core Team. 2015. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL: http://www.R-project.org. (February 10, 2022).
Ünal İ. 2010. Tanı Standards and analytic techniques used in evaluation of diagnostic tests. PhD thesis, Çukurova University, Health Science Institute, Adana, Türkiye, pp: 161.
Ünal İ. 2018. Is the transformation useful to estimate the area under the ROC curve with skewed data? Cukurova Med J, 43: 141-147.
Zheng Y, Cai T, Feng Z. 2006. Application of the time-dependent ROC curves for prognostic accuracy with multiple biomarkers. Biometrics, 62: 279-287.
Zheng Y, Heagerty PJ. 2004. Semiparametric estimation of time-dependent ROC curves for longitudinal marker data. Biostatistics, 5(4): 615-632.
Zheng Y, Heagerty PJ. 2007. Prospective accuracy for longitudinal markers. Biometrics, 63: 332-341.