Concurrency control algorithms for deduplicated cloud storage

Deduplication of data is essential to effectively use cloud storage. As the metadata in deduplicated cloudstorage are shared across multiple users, concurrent updates may result in inconsistencies. A coarse-grained lockingstrategy that has been proposed earlier to overcome this difficulty is not suited for inline deduplication owing to poorperformance. In the present work, a fine-grained locking strategy that overcomes this shortcoming is proposed. Ametadata structure along with a set of concurrent control mechanisms to accomplish this is presented. This strategy isshown to improve the throughput by as much as 60% with only marginal lock overhead.

PDF

___

[1] Zeng W, Zhao Y, Ou K, Song W. Research on cloud storage architecture and key technologies. In: 2nd IEEE 2009 Interaction Sciences Information Technology, Culture and Human International Conference; 24–26 November 2009; Seoul, Republic of Korea: IEEE. pp. 1044-1048.
[2] Policroniades C, Pratt I. Alternatives for detecting redundancy in storage systems data. In: USENIX 2004 Annual Technical Conference; 27 June–2 July 2004; Boston, MA, USA: Usenix Association. pp. 73-86.
[3] Clements AT, Ahmad I, Vilayannur M, Li J. Decentralized deduplication in SAN cluster file systems. In: 11th USENIX 2009 Annual Technical Conference; 14–19 June 2009; Santa Clara, CA, USA: Usenix Association. pp. 101-114.
[4] Fu Y, Jiang H, Xiao N, Tian L, Liu F. Aa-dedupe: An application-aware source deduplication approach for cloud backup services in the personal computing environment. In: IEEE 2011 Cluster Computing International Conference; 26–30 September 2011; Austin, TX, USA: IEEE. pp. 112-120.
[5] Thwel T, Thein NL. An efficient indexing mechanism for data deduplication. In: IEEE 2009 Current Trends in Information Technology Conference; 15–16 December 2009; Dubai, United Arab Emirates: IEEE. pp. 1-5.
[6] Xu L, Hu J, Mkandawire S, Jiang H. SHHC: A scalable hybrid hash cluster for cloud backup services in data centers. In: 31st IEEE 2011 Distributed Computing Systems Workshops International Conference; 21–24 June 2011; Minneapolis, MN, USA: IEEE. pp. 61-65.
[7] Guo F, Efstathopoulos P. Building a high performance deduplication system. In: USENIX 2011 Annual Technical Conference; 15–17 June 2011; Portland, OR, USA: Usenix Association. pp. 1-14.
[8] Efstathopoulos P, Guo F. Rethinking deduplication scalability. In: 2nd USENIX 2010 Hot Topics in Storage and File Systems Conference; 22–25 June 2010; Berkeley, CA, USA: Usenix Association. pp. 1-5.
[9] Debnath B, Sengupta S, Li J. ChunkStash: Speeding up inline storage deduplication using flash memory. In: USENIX 2010 Annual Technical Conference; 22–25 June 2010; Boston, MA, USA: Usenix Association. pp. 1-15.
[10] Lillibridge M, Eshghi K, Bhagwat D, Deolalikar V, Trezis G, Camble P. Sparse indexing: Large scale inline deduplication using sampling and locality. In: 7th USENIX 2009 File and Storage Technologies Conference; 24–27 February 2009; San Francisco, CA, USA: Usenix Association. pp. 111-123.
[11] Zhu B, Li K, Patterson H. Avoiding the disk bottleneck in the data domain deduplication file system. In: 6th USENIX 2008 File and Storage Technologies Conference; 28–29 February 2008; San Jose, CA, USA: Usenix Association. pp. 1-14.
[12] Strzelczak P, Adamczyk E, Herman-Izycka U, Sakowicz J, Slusarczyk L, Wrona J, Dubnicki C. Concurrent deletion in a distributed content-addressable storage system with global deduplication. In: 11th USENIX 2013 File and Storage Technologies Conference; 13–15 February 2013; San Jose, CA, USA: Usenix Association. pp. 161-174.
[13] Sundarrajan R, Neelamegam K, Prabagaran VT. Improve file sharing and file locking in a cloud. In: IBM 2010 White Paper; IBM. pp. 1-22.
[14] Nurmi D, Wolski R, Grzegorczyk C, Obertelli G, Soman S, Youseff L, Zagorodnov D. The eucalyptus open-source cloud-computing system. In: 9th IEEE/ACM 2009 Cluster Computing and the Grid International Symposium; May 18 2009; TBD Shanghai, China: IEEE. pp. 124-131.