Four Minute Paper: Facebook’s time series database, Gorilla

Premise for the paper

Whats unique about Gorilla

Use cases for Gorilla

  • Observability for hundreds of different systems, focusing on reliable real-time monitoring of production systems.
  • Identify issues with new software releases.
  • Be highly available and continue to operate with network failures.
  • Be fault tolerant with multi-region architecture.

Attributes of Gorilla

  • Gorilla is a write thru cache.
  • It’s aggregate data is more important than individual data points.
  • Recent data points are of higher value (85% of queries read past 26 hrs of data).
  • ACID guarantees are not necessary, but a high percentage of writes must succeed.
  • Optimized for being available for reads and writes in the face of failures prioritized over availability of any older data.

Challenges

  • High data insertion rate.
  • Total data quantity.
  • Real-time aggregation.
  • Reliability requirements.

Solutions

  • In-memory cache (past 26 hrs) of the on-disk persistent store.
  • Compressed data to reduce size of cache 12x.
  • Multi-instance Gorilla architecture (reads go to closest instance) for fault tolerance.

Architecture

ref: https://www.vldb.org/pvldb/vol8/p1816-teller.pdf
  • Compression algorithm
  • In-memory data structure
  • Persistent on-disk data structure
  • Fault tolerance

Conclusion

--

--

--

Live simply. Program stuff.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Serverless end-to-end tracing, troubleshooting & performance monitoring with Lumigo

Implement backup with Barman

CSS spec: Cascade Layers — quick introduction and usages

Python logging string format

Bank Robbers — CodinGame C++ Implementation

My 30-Day Leetcode challenge — Week 1

Version Control System

{UPDATE} DualBrain+ Entrenamiento cerebral Hack Free Resources Generator

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Jessica G

Jessica G

Live simply. Program stuff.

More from Medium

Reconsider before using IAM Auth in Amazon MSK Connect

Schema naming strategies in Confluent Schema Registry

Overview of AQUA for Amazon Redshift

Boxfuse Setup & Deployment for Play framework in Scala: