Çağatay ÇATAL, Cengiz TOGAY, Emrah DÖNMEZ, Alper ÖZCAN1, Bedir TEKİNERDOĞAN

Assessment of environmental factors affecting software reliability: a survey study

Currently, many systems depend on software, and software reliability as such has become one of the key challenges. Several studies have been carried out that focus on the impact of external environmental factors that impact software reliability. These studies, however, were all carried out in the same geographical context. Given the rapid developments in software engineering, this study aims to identify and reinvestigate the environmental factors that impact software reliability by also considering a different context. The environmental factors that have an impact on software reliability as reported in earlier studies have been analyzed and synthesized. Subsequently, a survey study is conducted to analyze the impact of 32 environmental factors from the perspective of multiple stakeholders. Several statistical analysis methods were applied for the analysis. Data were collected from 24 organizations and 70 software professionals. Most factors shown in top 10 lists of previous studies remain in the top 10 in our study, but their order is different. Testing coverage is now the most significant factor and testing effort is considered as the second most significant factor. The environmental factors defined previously retain their impact. The ordering of the importance of the environmental factors has changed though.

PDF

___

1] Friedman MA, Voas JM. Software Assessment: Reliability, Safety, Testability. New York, NY, USA: John Wiley & Sons, 1995.
[2] Febrero F, Calero C, Moraga M. A systematic mapping study of software reliability modeling. Information and Software Technology 2014; 56 (8): 839-849.
[3] Zhu M, Pham H. A two-phase software reliability modeling involving with software fault dependency and imperfect fault removal. Computer Languages, Systems & Structure 2018; 53: 27-42. [4] Utkin V, Coolen FPA. A robust weighted SVR-based software reliability growth model. Reliability Engineering & System Safety 2018; 176: 93-101.
[5] Yazdanbakhsh O, Dick S, Reay I, Mace E. On deterministic chaos in software reliability growth models. Applied Soft Computing 2016; 49: 1256-1269.
[6] Wang H, Fei H, Yu Q, Zhao W, Yan J et al. A motifs-based maximum entropy Markov model for realtime reliability prediction in system of systems. Journal of Systems and Software 2019; 151: 180-193.
[7] Organizaci O. ISO-IEC 25010: 2011 Systems and Software Engineering-Systems and Software Quality Requirements and Evaluation (SQuaRE)-System and Software Quality Models. 2011. [8] Zhu M, Zhang X, Pham H. A comparison analysis of environmental factors affecting software reliability. Journal of Systems and Software 2015; 109: 150-160.
[9] Zhu M, Pham H. Environmental factors analysis and comparison affecting software reliability in development of multi-release software. Journal of Systems and Software 2017; 132: 72-84.
[10] Zhang X, Pham H. An analysis of factors affecting software reliability. Journal of Systems and Software 2000; 50 (1): 43-56.
[11] Zhang X, Shin MY, Pham H. Exploratory analysis of environmental factors for enhancing the software reliability assessment. Journal of Systems and Software 2001; 57 (1): 73-78.
[12] Mishra A, Yazici A, Cetin S. Software evolution in Turkey. Tehnicki Vjesnik 2016; 23 (3): 929-935.
[13] Loganathan A, Muthuraj RJ. A new methodology for data reduction in software reliability studies. Communications in Statistics: Case Studies, Data Analysis and Applications 2016; 2 (3-4): 101-105.
[14] Pfleeger SL, Kitchenham BA. Principles of survey research: part 1: turning lemons into lemonade. ACM SIGSOFT Software Engineering Notes 2001; 26 (6): 16-18.
[15] Kitchenham BA, Pfleeger SL. Principles of survey research part 2: designing a survey. ACM SIGSOFT Software Engineering Notes 2002; 27 (1): 18-20.
16] Kitchenham BA, Pfleeger SL. Principles of survey research part 6: data analysis. ACM SIGSOFT Software Engineering Notes 2003; 28 (2): 24-27.
[17] Kelley K, Clark B, Brown V, Sitzia J. Good practice in the conduct and reporting of survey research. International Journal for Quality in Health Care 2003; 15 (3): 261-266.
[18] Aday LA, Cornelius LJ. Designing and conducting health surveys: a comprehensive guide.San Francisco, CA, USA: Jossey-Bass, 2006.
[19] Bowling A. Research Methods in Health: Investigating Health and Health Services. Maidenhead, UK: McGraw-Hill Education; Open University Press, 2014.
[20] Johnson JW. A heuristic method for estimating the relative weight of predictor variables in multiple regression. Multivariate Behavioral Research 2000; 35 (1): 1-19.
[21] Wohlin C, Runeson P, Host M, Ohlsson MC, Regnell B, Wesslen A. Experimentation in Software Engineering: an Introduction. Norwell, MA, USA: Kluwer, 2000.
[22] Easterbrook S, Singer J, Storey MA, Damian D. Selecting empirical methods for software engineering research. In: Shull F, Singer J, Sjøberg DIK (editors). Guide to Advanced Empirical Software Engineering. London, UK: Springer, 2008, pp. 285-311.
[23] Zhou X, Jin Y, Zhang H, Li S, Huang X. A map of threats to validity of systematic literature reviews in software engineering. In: 2016 23rd Asia-Pacific Software Engineering Conference (APSEC); New York, NY, USA; 2016. pp. 153-160.