Storage Developer Conference

Consistent Hashing provides a mechanism through which independent actors in a distributed system can reach an agreement about where a resource is, who is responsible for its access or storage, and even derive deterministically a prioritized list of fall-backs should the primary location be down. Moreover, consistent hashing allows aspects of the system to change dynamically while minimizing disruptions. We've recently developed a new consistent hashing algorithm, which we call the Weighted Rendezvous Hash. Its primary advantage is that it obtains provably minimum disruption during changes to a data storage system. This presentation will introduce this algorithm for the first time, and consider several of its applications.

Learning Objectives

What is Consistent Hashing
Traditional applications of consistent hashing
The implementation of the new algorithm: Weighted Rendezvous Hash
Why Weighted Rendezvous Hash is more efficient than previous algorithms
Applications of Weighted Rendezvous Hash in data storage systems

Three emerging trends must be considered when assessing how storage should operate at extreme scale. First, continuing expansion in the volume of data to be stored is accompanied by increasing complexity in the metadata to be stored with it and queries to be executed on it. Second, ever increasing core and node counts require corresponding scaling of application concurrency while simultaneously increasing the frequency of hardware failure. Third, new NVRAM technologies allow storage, accessible at extremely fine grain and low latency, to be distributed across the entire cluster fabric to exploit full cross-sectional bandwidth. This talk describes Distributed Application Object Storage (DAOS) – a new storage architecture that Intel is developing to address the functionality, scalability and resilience issues and exploit the performance opportunities presented by these emerging trends.

Learning Objectives

Exascale / Big Data
Scalable Distributed Storage Systems
Object Storage
Persistent Memory

Successive generations of storage solutions have increased decentralization. Early NAS systems made all the decisions on a single server, down to sector assignment. Federated NAS enabled dynamic distribution of the namespace across multiple storage serves. The first Object Clusters delegated CRUD-based management of both object metadata and data to OSDs.

Current generation of Object Clusters uses Consistent Hashing to eliminate the need for central metadata. However, Consistent Hashing and its derivatives, combined with the prevalent use of TCP/IP in storage clusters results in performance hot spots and bottlenecks, diminished scale-out capability and dis-balances in resource utilization.

These shortcomings will be demonstrated with a simulation of a large storage cluster. An alternative next generation strategy that simultaneously optimizes available IOPS "budget" of the back-end storage, storage capacity, and network utilization will be explained. Practically unlimited load-balanced scale-out capability using Layer 5 (Replicast) protocol for Multicast Replication within the cluster will be presented.

Learning Objectives

Why neither Consistent Hashing nor TCP/IP scale
How CCOW (Cloud Copy-on-Write) and Replicast provide for infinite scale-out
Impact on IOPS, Storage Capacity and Network utilization
Simulation: queuing and congestion in a 4000 node cluster Actual
Results: measured with a more affordable cluster

Memory is the key to fast big data processing. This has been realized by many, and frameworks such as Spark and Shark already leverage memory performance. As data sets continue to grow, storage is increasingly becoming a critical bottleneck in many workloads.

To address this need, we have developed Tachyon, a memory-centric fault-tolerant distributed storage system, which enables reliable file sharing at memory-speed across cluster frameworks such as Apache Spark, MapReduce, and Apache Flink. The result of over three years of research and development, Tachyon achieves both memory-speed and fault tolerance.

Tachyon is Hadoop compatible. Existing Spark, MapReduce, Flink programs can run on top of it without any code changes. Tachyon is the default off-heap option in Spark. The project is open source and is already deployed at many companies in production. In addition, Tachyon has more than 100 contributors from over 30 institutions, including Yahoo, Tachyon Nexus, Redhat, Baidu, Intel, and IBM. The project is the storage layer of the Berkeley Data Analytics Stack (BDAS) and also part of the Fedora distribution.

In this talk, we give an overview of Tachyon, as well as several use cases we have seen in the real world.

Scalability is critically important to distributed storage systems. Exabyte-scale storage is already on the horizon and such systems involve tens of thousands of nodes. But today’s Internet protocols were never designed to handle such cases. In systems this big it's impossible to maintain connections and sessions with every storage device. However, multiple round trip connection setups, TLS handshakes, and authentication mechanisms, compounded with the unyielding speed of light and geo-dispersed topologies create a perfect storm for high latency and bad performance. In this presentation we explore the available options to achieve security, performance, and low latency in a system where persistent sessions are an unaffordable luxury.

Learning Objectives

Limitations of today’s Internet protocols in large distributed systems
How to implement a secure, connectionless, single-round-trip network protocol
What the network topology will look like in an exabyte-scale globally dispersed storage system