Sangeetha SELVARAJ, Sudha Sadasivam GANGADHARAN

Privacy preserving hybrid recommender system based on deep learning

Deep learning models are widely being used to provide relevant recommendations in hybrid recommender systems. These hybrid systems combine the advantages of both content based and collaborative filtering approaches. However, these learning systems hamper the user privacy and disclose sensitive information. This paper proposes a privacy preserving deep learning based hybrid recommender system. In hybrid deep neural network, user’s side information such as age, location, occupation, zip code along with user rating is embedded and provided as input. These embedding’s pose a severe threat to individual privacy. In order to eliminate this breach of privacy, we have proposed a private embedding scheme that protects user privacy while ensuring that the nonlinear latent factors are also learnt. In this paper, we address the privacy in hybrid system using differential privacy, a rigorous mathematical privacy mechanism in statistical and machine learning systems. In the reduced feature set, the proposed adaptive perturbation mechanism is used to achieve higher accuracy. The performance is evaluated using three datasets with root mean square error (RMSE), mean absolute error (MAE), mean squared error (MSE), R squared, precision and recall. These evaluation metrics are compared with varying values of privacy parameter ϵ . The experimental results show that the proposed solution provides high user privacy with reasonable accuracy than the existing system. As the engine is generic, it can be used on any recommendation framework.

PDF

___

[1] Tianqing Z, Gang L, Wanlei Z, Philip S Y. Differential Privacy and Applications. NY, USA: Springer International Publishing, 2017.
[2] Sangeetha S, Sudhasadasivam G. Privacy of big data: a review. In: Dehghantanha A, Choo KK (editors). Handbook of Big Data and IoT Security, NY, USA: Springer Cham, 2019, pp. 5-23.
[3] Zhang D, Chen X, Wang D, Shi J. A survey on collaborative deep learning and privacy- preserving. In: IEEE Third International Conference on Data Science in Cyberspace, Guangzhou, China; 2018. pp. 652-658.
[4] Hamm J, Cao P, Belkin M. Learning privately from multiparty data. In: Proceedings of 33rd International Conference on Machine Learning, NY, USA; 2016. pp. 555-563.
[5] Arun R, Shivani A. A differentially private stochastic gradient descent algorithm for multiparty classification. In: Proceedings of Machine Learning Research, La Palma, Canary Islands, Spain; 2012, pp. 933-941.
[6] Pathak MA, Rane S, Raj B. Multiparty differential privacy via aggregation of locally trained classifiers. In: Proceedings of the 23rd International Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada; 2010, pp. 1876-1884.
[7] Narayanan A, Shmatikov V. Robust de-anonymization of large sparse datasets. In: IEEE Symposium on Security and Privacy (sp 2008), Oakland, CA; 2008, pp. 111-125.
[8] Mironov I, McSherry F. Differentially private recommender systems: Building privacy into the Netflix prize contenders. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France; 2009, pp. 627-636.
[9] Friedman A, Berkovsky S, Kaafar MA. A differential privacy framework for matrix factorization recommender systems. User Modeling and User-Adapted Interaction 2016; 26(5): 425-458. doi: 10.1007/s11257-016-9177-7.
[10] Shokri R, Stronati M, Song C, Shmatikov V. Membership inference attacks against machine learning models. In: IEEE Symposium on Security and Privacy (SP), San Jose, CA; 2017, pp. 3-18.
[11] Fredrikson M, Jha S, Ristenpart T. Model inversion attacks that exploit confidence information and basic countermeasures. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, Denver, Colorado, USA; 2015, pp. 1322-1333.
[12] Abadi M, Chu A, Goodfellow I, McMahan HB, Mironov I et al. Deep Learning with Differential Privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria; 2016, pp. 308-318.
[13] Dowlin N, Gilad-Bachrach R, Laine K, Lauter K, Naehrig M et al. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning, NY, USA; 2016, pp. 201–210.
[14] Ma Z, Liu Y, Liu X, Ma J, Ren K. Lightweight privacy-preserving ensemble classification for face recognition. IEEE Internet of Things Journal 2019; 6 (3): 5778-5790. doi: 10.1109/JIOT.2019.2905555.
[15] Ma Z, Liu Y, Liu X, Ma J, Li F. Privacy-preserving outsourced speech recognition for smart iot devices. IEEE Internet of Things Journal 2019; 6 (5): 8406-8420.
[16] McMahan HB, Ramage D, Talwar K, Zhang Li. Learning differentially private recurrent language models. In: 6th International Conference on Learning Representations; Vancouver, BC, Canada ; 2018. pp. 1–14.
[17] McSherry F, Talwar K. Mechanism design via differential privacy. In: Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science, IEEE Computer Society, Washington, DC, USA; 2007. pp. 94- 103.
[18] Phan N, Wang Y, Wu X, Dou D. Differential privacy preservation for deep auto-encoders: an application of human behavior prediction. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, 2016; pp.1309-1316.
[19] Phan N, Wu X, Hu H, Dou D. Adaptive Laplace mechanism : differential privacy preservation in deep learning. In: Proceedings - IEEE International Conference on Data Mining ICDM; New Orleans, USA; 2017. pp. 385-394.
[20] Kiran R, Pradeep K, Bharat B. DNNRec: A novel deep learning based hybrid recommender system. Expert Systems with Applications 2020; 144. doi: 10.1016/j.eswa.2019.113054.
[21] Sahoo A, Pradhan C, Barik R, Dubey H. DeepReco : deep learning based health recommender system using collaborative filtering. Computation 2019; 7: 25. doi: 10.3390/computation7020025.
[22] Covington P, Adams J, Sargin E. Deep neural networks for YouTube recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems, Boston, Massachusetts, USA, 2016; pp. 191-198.
[23] Wei J, He J, C K, Zhou Y, Tang Z . Collaborative filtering and deep learning based recommendation system for cold start items, Expert Systems With Applications 2016, 69: 29-39, doi: 10.1016/j.eswa.2016.09.040.
[24] Dwork C. Differential privacy. In: Bugliesi M., Preneel B., Sassone V., Wegener I. (eds). Automata, Languages and Programming, ICALP 2006. Lecture Notes in Computer Science. Heidelberg, Berlin: Springer, 2006, pp. 1-12.
[25] Dwork C, Roth A. The algorithmic foundations of differential privacy. Hanover, MA, USA: Now Publishers Incorporation, 2014.
[26] Fanti G, Pihur V, Erlingsson Ú. Building a RAPPOR with the unknown: privacy-preserving learning of associations and data dictionaries. Proceedings on Privacy Enhancing Technologies 2016; (3): 41-61.
[27] Erlingsson Ú, Pihur V, Korolova A. RAPPOR: Randomized aggregatable privacy-preserving ordinal response. In: Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security - CCS ’14, Scottsdale Arizona, USA; 2014, pp. 1054-1067.
[28] Shin H, Kim S, Shin J, Xiao X. Privacy enhanced matrix factorization for reccommendation with local differential privacy. IEEE Transactions on Knowledge and Data Engineering 2018; 30 (9): 1770-1782, doi:10.1109/TKDE.2018.2805356.
[29] Berlioz A, Friedman A, Kaafar MA, Boreli R, Berkovsky S. Applying differential privacy to matrix factorization. In: Proceedings of the 9th ACM Conference on Recommender Systems; Vienna, Austria; 2015. pp. 107–114.
[30] Goodfellow I, Bengio Y, Courville A. Deep Learning. USA: The MIT Press, 2016.
[31] Sweeney L. k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowlege-Based Systems 2002; 10 (5): 557 – 570, doi: 10.1142/S0218488502001648.
[32] Shun Zhang, Laixiang Liu, Zhili Chen, Hong Zhong. Probabilistic matrix factorization with personalized differential privacy.Knowledge-Based Systems 2019; 183: 104864, doi: https://doi.org/10.1016/j.knosys.2019.07.035.