Four Minute Paper: Facebook’s time series database, Gorilla

Premise for the paper

Whats unique about Gorilla

Use cases for Gorilla

  • Observability for hundreds of different systems, focusing on reliable real-time monitoring of production systems.
  • Identify issues with new software releases.
  • Be highly available and continue to operate with network failures.
  • Be fault tolerant with multi-region architecture.

Attributes of Gorilla

  • Gorilla is a write thru cache.
  • It’s aggregate data is more important than individual data points.
  • Recent data points are of higher value (85% of queries read past 26 hrs of data).
  • ACID guarantees are not necessary, but a high percentage of writes must succeed.
  • Optimized for being available for reads and writes in the face of failures prioritized over availability of any older data.

Challenges

  • High data insertion rate.
  • Total data quantity.
  • Real-time aggregation.
  • Reliability requirements.

Solutions

  • In-memory cache (past 26 hrs) of the on-disk persistent store.
  • Compressed data to reduce size of cache 12x.
  • Multi-instance Gorilla architecture (reads go to closest instance) for fault tolerance.

Architecture

ref: https://www.vldb.org/pvldb/vol8/p1816-teller.pdf
  • Compression algorithm
  • In-memory data structure
  • Persistent on-disk data structure
  • Fault tolerance

Conclusion

--

--

--

Live simply. Program stuff.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

A Beginner’s Guide to Debugging in Ruby

Linux on Windows…What?

ARCore Cupcakes #1 — Render a Scene without AR for phones without ARCore

Shopping Wine UI —  Flutter Web

Machine Maintenance 101

How I explain to my friends what we do at work

Does enterprise scale Dynamics require multiple instances?

Road Map to become an expert in compitative programming

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Jessica G

Jessica G

Live simply. Program stuff.

More from Medium

NVMe performance vs. power management

Pre-commit Check with Shell Script to Avoid Using Certain Words

MIT 6.824: Distributed Systems Lecture 1

IBM Sterling SFTP Server Adapter 2.0 vs. SFTP Server Adapter 1.0: Performance Comparison