Performance Modeling and Simulation for Accumulo Applications

Back to Schedule

Abstract

Apache Accumulo is known for being a high performance sorted key/value database, but achieving high performance in your application still requires good development practices. Often, developers will extrapolate from small-scale tests to argue that the application will perform well at higher scales. Unfortunately, design and implementation flaws that aren't visible at small scale inevitably show up in production at a much higher cost to fix.

Sqrrl is an application built on Accumulo that leverages log storage, indexing, graphs, and statistics modeling while supporting high throughput ingest and distributed analytic processing. At Sqrrl, we ensure reliable performance using a variety of modeling and simulation techniques. This talk will show examples of insights and performance improvements gained from micro-benchmarking, analog simulation, and predictive model validation.

Speakers

Adam Fuchs

CTO, Sqrrl

Adam Fuchs is one of the original developers and architects of Apache Accumulo. He is now the chief technology officer for Sqrrl, leveraging the techniques and design patterns learned from a long career in data processing at the National Security Agency to build a cybersecurity threat hunting platform. Adam got his undergraduate degree in computer science at the University of Washington and attended graduate school at the University of Maryland, College Park. He now lives in Seattle with his wife and two kids, where he enjoys running up mountains in his spare time.