学术报告:SP-Cache: Load-balanced, Redundancy-free Cluster Caching with Selective Partition

发布时间:2018-06-15  阅读次数:1181

活动时间:2018-6-19  10:00am
活动地点:济事楼 417

 

SP-Cache: Load-balanced, Redundancy-free Cluster Caching with Selective Partition

 

王威 
  •   香港科技大学助理教授
 
邀请人:史清江教授

Abstract:
Big data clusters increasingly employ in-memory solutions to improve I/O performance. However, the routinely observed file popularity skew and load imbalance create hot spots in a cluster, which significantly degrade the benefits of in-memory caching. Common approaches to tame load imbalance include copying multiple replicas of hot files and creating parity chunks using storage codes. Yet, these techniques either suffer from high memory overhead due to cache redundancy or incur non-trivial encoding/decoding complexity. In this talk, I will present an effective approach to achieve load balancing without cache redundancy or encoding/decoding overhead. Our solution, which we call SP-Cache, selectively partitions files based on their popularity and evenly caches those partitions across the cluster. We develop an efficient algorithm to determine the optimal number of partitions for a hot file---too few partitions are incapable of mitigating hot spots, while too many are susceptible to stragglers. We have implemented SP-Cache in Alluxio, a popular in-memory distributed storage for data-intensive clusters. Real cloud deployment and trace-driven simulations show that, compared to the state-of-the-art solution, SP-Cache reduces the file access latency by up to 40% in both the mean and the tail, using 40% less memory.
 
Bio:
Wei Wang is currently an Assistant Professor in the Department of Computer Science and Engineering at the Hong Kong University of Science and Technology (HKUST). He is also affiliated with HKUST Big Data Institute. Wei received Ph.D. from the University of Toronto in 2015, and M.Eng. and B.Eng from Shanghai Jiao Tong University, all in the Department of Electrical and Computer Engineering. His research interests cover the broad area of distributed systems, with special emphasis on big data and machine learning, cloud computing, and computer networks in general. His works have been published in top venues such as TPDS, TMC, SC, INFOCOM, ICDCS, ICNP, CoNEXT and SIGMETRICS. He is a recipient of the 2015 Chinese Government Award for Outstanding Students Abroad and the Best Paper Finalist Award at the USENIX ICAC 2013. He was recently named as the Distinguished TPC member of IEEE INFOCOM 2018.

联系我们

地址:中国 上海曹安公路4800号同济大学软件学院

邮编:201804

联系电话:86-21-69589585,69589332(FAX)

 Copyright© 2017 同济大学软件学院

技术支持: 上海维程