Distributed System

The growing disparity in data storage and retrieval needs of modern applications is driving the proliferation of a wide variety of distributed key-value (KV) stores. However, the complexities in implementing these distributed KV stores and adapting ever-changing storage requirements present unique opportunities and engineering challenges.

Voltron: A Versatile Distributed Key-Value Storage Framework

Voltron is a modular and compositional development platform that eases DOBS programming. Voltron is based on the insight that DOBS shares common distributed management functionalities (e.g., replication, consistency, and topology), and thus their development can be modularized and reused for building other/new stores. Voltron takes a single-server data store implementation (called datalet), and seamlessly enable DOBS services atop the datalets by leveraging pre-built control modules (called controlet). The resulting DOBS can be easily extended for new types of services. We demonstrate how Voltron can enable a wide variety of DOBS services with minimal engineering efforts.

Github
image

MOS: Micro Object Architecture

The current practice of distributed object store is to either use a single set of configuration parameters or rely on statically configured storage policies for a cloud object store deployment, even when the store is used to support different types of applications with evolving requirements. This crucial mismatch between the different applications requirements and capabilities of the object store is problematic and should be addressed to achieve high efficiency and performance. MOS is a Micro Object Storage architecture, which supports independently configured microstores each tuned dynamically to the needs of a particular type of workload. We also design an enhancement, MOS++, that extends MOS's capabilities through fine-grained resource management to effectively meet the tenants’ SLAs while maximizing resource efficiency. We have implemented a prototype of MOS++ in OpenStack Swift using Docker containers.

image

MBal: A Load Balanced Distributed Memory Caching System

MBal, a high-performance in-memory object caching framework with adaptive Multi-phase load Balancing, which supports not only horizontal (scale-out) but vertical (scale-up) scalability as well. MBal is able to make efficient use of available resources in the cloud through its fine-grained, partitioned, lockless design. This design also lends itself naturally to provide adaptive load balancing both within a server and across the cache cluster through an event-driven, multi-phased load balancer. While individual load balancing approaches are being leveraged in in-memory caches, MBal goes beyond the extant systems and offers a holistic solution wherein the load balancing model tracks hotspots and applies different strategies based on imbalance severity---key replication, server-local, or cross-server coordinated data migration.

image

Related Publications    Full list

MOS: Workload-aware Elasticity for Cloud Object Stores

Ali Anwar, Yue Cheng, Aayush Gupta, Ali R. Butt

ACM HPDC '16    

Taming the Cloud Object Storage with MOS

Ali Anwar, Yue Cheng, Aayush Gupta, Ali R. Butt

ACM PDSW '15