Data Systems⚪New

🌳 The Storage Black Box

Choose the right underlying storage engine (LSM vs B-Tree) for a specific workload.

⬇️ Foundation

The Librarian (Filing books immediately vs. throwing them in a pile)

An append-only sequence of records. The simplest and fastest way to write data.

Sorted String Tables. Immutable on-disk files where keys are sorted, merged in the background.

The industry standard for storage. Fixed-size pages on disk updated in place using a wide tree structure.

Storing data by column rather than by row to optimize for analytical (OLAP) queries.

Scale out across multiple machines using Replication (copying) and Partitioning (splitting).

Integrate disparate systems by deriving new datasets from a source of truth via Batch and Stream processing.

✔

🤔 Reflection:

"You are building a logging system with 99% writes. Would you choose a B-Tree or an LSM-Tree? Why?"