Real-time anomaly detection and mitigation using streaming telemetry in SDN

Measurement and monitoring are crucial for various network tasks such as traffic engineering, anomaly detection, and intrusion prevention. The success of critical capabilities such as anomaly detection and prevention depends on whether the utilized network measurement method is able to provide granular, near real-time, low-overhead measurement data or not. In addition to the measurement method, the anomaly detection and mitigation algorithm is also essential for recognizing normal and abnormal traffic patterns in such a huge amount of measured data with high accuracy and low latency. Software-defined networking is an emerging concept to enable programmable and efficient measurement functions for these kinds of challenging requirements. In this paper, we present a new, real-time, model- driven anomaly detection and mitigation platform. Model-driven streaming telemetry and exponential smoothing are the underlying approaches of the platform. A customized collector is proposed to gather streaming telemetry metrics, and Holt’s prediction algorithm is improved to handle real-time streaming data and decrease false positives. The developed system is tested on a campus network and the success rate of the system is calculated as 92%.

PDF

___

[1] Bawany NZ, Shamsi JA, Salah K. DDoS attack detection and mitigation using SDN: methods, practices, and solutions. Arabian Journal for Science and Engineering 2017; 42 (2): 1-17.
[2] Yu M, Jose L, Miao R. Software Defined Traffic Measurement with OpenSketch. Berkeley, CA, USA: Networked Systems Design and Implementation; 2013. pp. 29-42.
[3] Giotis K, Androulidakis G, Maglaris V. Leveraging SDN for efficient anomaly detection and mitigation on legacy networks. In: 3rd European Workshop on Software Defined Networks; Budapest, Hungary; 2014. pp. 85-90. doi: 10.1109/EWSDN.2014.24
[4] Wang P, Chao KM, Lin HC, Lin WH, Lo CC. An efficient flow control approach for SDN-based network threat detection and migration using support vector machine. In: IEEE 13th International Conference on e-Business Engineering; Macau; 2016. pp. 56-63. doi: 10.1109/ICEBE.2016.020
[5] Giotis K, Argyropoulos C, Androulidakis G, Kalogeras D, Maglaris V. Combining OpenFlow and sFlow for an effective and scalable anomaly detection and mitigation mechanism on SDN environments. Computer Networks 2014; 62 (1): 122-136. doi: 10.1016/j.bjp.2013.10.014
[6] Rehman SU, Song WC, Kang M. Network-wide traffic visibility in OF@TEIN SDN testbed using sFlow. In: 16th Asia-Pacific Network Operations and Management Symposium; Hsinchu, Taiwan; 2014. pp. 1-6. doi: 10.1109/APNOMS.2014.6996541
[7] Buragohain C, Medhi N. FlowTrApp: An SDN based architecture for DDoS attack detection and mitigation in data centers. In: 3rd International Conference on Signal Processing and Integrated Networks; Noida, India; 2016. pp. 519-524. doi: 10.1109/SPIN.2016.7566750
[8] Jose L, Yu M, Rexford J. Online measurement of large traffic aggregates on commodity switches. In: Proceedings of the USENIX HotICE; Boston, MA, USA; 2011. pp. 13-14.
[9] Chowdhury S, Bari M, Ahmed R, Boutaba R. PayLess: A low cost network monitoring framework for Software Defined Networks. In: Proceedings of the 14th IEEE/IFIP NOMS; Krakow, Poland; 2014. pp. 1–9. doi: 10.1109/NOMS.2014.6838227
[10] Yu C, Lumezanu C, Zhang Y, Singh V, Jiang G et al. FlowSense: Monitoring network utilization with zero measurement cost. In: Proceedings of the 14th IPAM; Toronto, Canada; 2013. pp. 31–41. doi: 10.1007/978-3-642-36516-4_4
[11] Wang MH, Wu SY, Yen LH, Tseng CC. PathMon: Path-specific traffic monitoring in OpenFlow-enabled networks. In: 8th International Conference on Ubiquitous and Future Networks; Vienna, Austria; 2016. pp. 775-780. doi: 10.1109/ICUFN.2016.7537143
[12] Ballard JR, Rae I, Akella A. Extensible and scalable network monitoring using OpenSAFE. In: Proceedings of the 2010 Internet Network Management Conference on Research on Enterprise Networking; Berkeley, CA, USA; 2010. p. 8.
[13] Khan F, Hosein N, Ghiasi S, Chuah CN, Sharma P. Streaming solutions for fine-grained network traffic measurements and analysis. IEEE/ACM Transactions on Networking 2014; 22 (2): 377-390. doi: 10.1109/TNET.2013.2263228
[14] Zhang Y. An adaptive flow counting method for anomaly detection in SDN. In: Proceedings of the 9th ACM CoNEXT; California, USA; 2013. pp. 25–30. doi: 10.1145/2535372.2535411
[15] Shirali S, Ganjali Y. Efficient implementation of security applications in OpenFlow controller with FleXam. In: IEEE 21st Annual Symposium on High-Performance Interconnects; San Jose, CA, USA; 2013. pp. 49-54. doi: 10.1109/HOTI.2013.17
[16] Braga R, Mota R, Passito A. Lightweight DDoS flooding attack detection using NOX/OpenFlow. In: Proceedings of the 35th Conference on Local Computers; Denver, CO, USA; 2010. pp. 408-415. doi: 10.1109/LCN.2010.5735752
[17] Mehdi SA, Khalid J, Khayam SA. Revisiting traffic anomaly detection using software defined networking. In: Proceedings of the 14th International Conference on Recent Advances in Intrusion Detection; Menlo Park, CA, USA; 2011. pp. 161-180.
[18] He D, Chan S, Ni X, Guizani M. Software-defined-networking-enabled traffic anomaly detection and mitigation. IEEE Internet of Things Journal 2017; 4 (6): 1890-1898. doi: 10.1109/JIOT.2017.2694702
[19] Vasilomanolakis E, Karuppayah S, Mühlhauser M, Fischer M. Taxonomy and survey of collaborative intrusion detection. ACM Computing Surveys 2015; 47 (4): 1-33. doi: 10.1145/2716260
[20] Khairi MH, Sharifah HS, Abdul NM, Abdullah AS, Hassan MK. A review of anomaly detection techniques and distributed denial of service (DDoS) on software defined network (SDN). Engineering, Technology & Applied Science Research 2018; 8 (2): 2724-2730.
[21] Miller Z, Deitrick W, Hu W. Anomalous network packet detection using data stream mining. Journal of Information Security 2017; 2 (4): 158–168.
[22] Cheng J, Xu R, Tang X, Sheng VS, Cai C. An abnormal network flow feature sequence prediction approach for DDoS attacks detection in big data environment. Computers, Materials and Continua 2018; 55 (1): 95-119.
[23] Nataf E, Festor O. End-to-end YANG-based configuration management. In: IEEE Network Operations and Management Symposium; Osaka, Japan; 2010. pp. 674-684. doi: 10.1109/NOMS.2010.5488381
[24] Putina A, Rossi D, Bifet A, Barth S, Pletcher D et al. Telemetry-based stream-learning of BGP anomalies. In: Proceedings of the 2018 Workshop on Big Data Analytics and Machine Learning for Data Communication Networks; New York, NY, USA; 2018. pp. 15-20. doi: 10.1145/3229607.3229611
[25] Manso P, Moura J, Serrão C. SDN-based intrusion detection system for early detection and mitigation of DDoS attacks. Information 2019; 10 (3): 106. doi: 10.3390/info10030106
[26] McKeown N, Anderson T, Balakrishnan H, Parulkar G, Petrson L et al. OpenFlow: Enabling innovation in campus networks. ACM SIGCOMM Computer Communication 2008; 38 (2): 69-74. doi: 10.1145/1355734.1355746
[27] Holt CC. Forecasting seasonals and trends by exponentially weighted moving averages. International Journal of Forecasting 2004; 20 (1): 5-10. doi: 10.1016/j.ijforecast.2003.09.015
[28] Eliseev D, Farkhadov M. Modern methods to collect, store, and process big data in large-scale systems. In: 5th Interna- tional Conference on Control, Instrumentation, and Automation; Shiraz, Iran; 2017. pp. 179-182. doi: 10.1109/ICCIAu- tom.2017.8258674
[29] Miller Z, Dickinson B, Deitrick W, Hu W, Wang AH. Twitter spammer detection using data stream clustering. Information Sciences 2014; 260: 64–73. doi: 10.1016/j.ins.2013.11.016
[30] Dobesova Z. Programming language Python for data processing. In: International Conference on Electrical and Control Engineering; Yichang, China; 2011. pp. 4866-4869. doi: 10.1109/ICECENG.2011.6057428
[31] Garima S, Rani S. Review on time series databases and recent research trends in time series mining. In: 5th International Conference - Confluence, The Next Generation Information Technology Summit; Noida, India; 2014. pp. 109-115. doi: 10.1109/CONFLUENCE.2014.6949290
[32] MacQueen J. Some methods for classification and analysis of multivariate observations. In: Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability; Berkeley, CA, USA; 1967. pp. 281-297.
[33] Ahmed M, Mahmood N, Hu J. A survey of network anomaly detection techniques. Journal of Network and Computer Applications 2016; 60: 19–31. doi: 10.1016/j.jnca.2015.11.016