Effective Management of High Volume Numeric Data with Histograms
Fred Moyer | Circonus
ABOUT THE TALK
How do you capture and organize billions of measurements per second so that:
You can answer a rich set of queries effectively (percentiles, counts below X, aggregations across streams)
You don’t blow through your AWS budget in minutes
In this talk, we'll take a look at both log-linear and cumulative histograms and how they provide advantages over storing data as quantiles, averages, and other histogram implementations such as linear and fixed-bucket. An open source histogram software library will be explained, and sample statistical operations using it will be shown. You'll come away with an understanding of use histograms to make your data engineering life easier.
Download Slides
Fred moyer
Engineer | Circonus
Fred Moyer is an Engineer at Circonus. He is a recovering Perl and C programmer, and these days likes to hack in Go and is learning Lua. Fred has been active in open source development for almost 20 years. He is a 2013 White Camel award winner, obtained a graduate degree in Mechanical Engineering, and his undergrad double in Mechanical Engineering and Materials Science.
Experience more insightful data science & engineering talks at DataEngConf NYC '17
