Storage Developer Conference

The Lightning Memory-Mapped Database (LMDB) was introduced at LDAPCon 2011 and has been enjoying tremendous success in the intervening time. LMDB was written for the OpenLDAP Project and has proved to be the world's smallest, fastest, and most reliable transactional embedded data store. It has cemented OpenLDAP's position as world's fastest directory server, and its adoption outside the OpenLDAP Project continues to grow, with a wide range of applications including big data services, crypto-currencies, machine learning, and many others.

The talk will cover highlights of the LMDB design as well as the impact of LMDB on other projects.

Learning Objectives

Highlight problems with traditional DB storage designs
Explain benefits of single-level-store
Explain corruption-proof design and implementation
Compare and contrast leading data structures: B+tree, LSM, Fractal Trees

The Bw-Tree is an ordered key-value store, built by layering a B-tree form access method over a cache/storage sub-system (LLAMA) that is lock-free and organizes storage in a log-structured manner. It is designed to optimize performance on modern hardware, specifically (i) multi-core processors with multi-level memory/cache hierarchy, and (ii) flash memory based SSDs with fast random reads (but inefficient random write performance). The Bw-Tree is shipping in three of Microsoft’s server/cloud products – as the key sequential index in SQL Server Hekaton (main memory database), as the indexing engine inside Azure DocumentDB (distributed document-oriented store), and as an ordered key-value store in Bing ObjectStore (distributed back-end supporting many properties in Bing).

Learning Objectives

Bw-Tree data structure
Lock-free design for high concurrency
Log-structured storage design for flash based SSDs
Page-oriented store (LLAMA) for building access methods on top
Bw-Tree Applications in Production at Microsoft

In-Memory Database appliances are rapidly evolving, becoming in effect the main operating stored image for both analytic and cognitive computing applications in the next generation of data center and cloud in-rack storage.

Co-opting of DRAM with proximal NAND-Flash mass storage being combined with Near Data Processing re-imagines the entire computing paradigm by effectively turning an entire database image into a content-addressable look alike. Candidates for Storage Class Memory are nearing market introduction and with Near Data Processing abilities will radically change Database Management Systems.

Learning Objectives

An understanding of performance metrics for IMDB systems
The importance of silicon photonic interconnects
The evolution to a low latency high bandwidth DB environment
Competing elements for market supremacy.
Cognitive computing - the next step