10 min read

MongoDB

A recipe app stores a recipe, its ingredients, its steps and its photos. In a relational database that is five tables and four joins every time someone opens a recipe. In MongoDB it is one document, shaped exactly like the recipe screen, read in one go. That is the core idea: store data the way your application reads it.

MongoDB is a document database. Instead of rows and columns, it stores JSON like documents in collections, and each document can hold nested objects and arrays. This chapter covers how to model data for it, and the two diagrams that explain how it stays up and how it scales: replica sets and sharding.

Your answer

A blog post has comments. Would you store comments inside the post document, or in their own collection? What would make you change your answer?

Documents and collections#

A document is stored as BSON, a binary form of JSON that adds types like dates and exact decimals. Documents live in collections, the rough equivalent of tables, and every document has a unique _id. There is no fixed schema by default: two documents in the same collection can have different fields, though in practice you should add schema validation once the shape settles.

json

{
  "_id": "rcp_8841",
  "title": "Kerala fish curry",
  "author": { "id": "u_17", "name": "Meera" },
  "tags": ["seafood", "spicy", "dinner"],
  "ingredients": [
    { "item": "kingfish", "qty": "500 g" },
    { "item": "kokum", "qty": "4 pieces" }
  ],
  "steps": ["Soak the kokum...", "Grind the masala..."],
  "rating": { "avg": 4.6, "count": 213 }
}

One read returns the whole recipe, with no joins. Queries can reach inside it ({ "tags": "spicy" } matches any document whose tags array contains "spicy"), and indexes work on nested fields and arrays too.

Embed or reference#

The single biggest modelling decision. Embed related data inside the document, or reference it by id in another collection, the way you would with a foreign key.

the rule of thumbdecide per relationship

Embed	when	The data is read together, belongs to one parent, and stays bounded in size: a recipe's ingredients, an order's line items, an address on a user.
Reference	when	The data is shared by many parents, is updated on its own, or can grow without limit: authors shared by many recipes, comments on a viral post.
Hybrid	when	Embed a small copy of what you display and reference the rest: the author's name embedded for display, their full profile referenced.

Sample row

post: { title, author: { id, name }, latest_comments: [ ...last 5 ], comment_count }

Every document has a 16 MB size limit. That limit is rarely the real problem; the problem is reading and rewriting a huge document on every small change. Unbounded arrays, such as all comments ever on a post, are the classic mistake.

So for the question above: a handful of comments on a personal blog, embed them. Thousands of comments on a popular post, or comments that need their own queries ("all comments by this user"), give them their own collection with a post_id, and embed only the latest few for fast display.

Embedding is denormalization

The embedded author name is a copy. When the author changes their name, every recipe holding the old copy is now stale. That is often acceptable for a display name, but decide it on purpose, and know which code updates the copies. The trade is the one in the normalization chapter, seen from the other side.

Staying up: replica sets#

A production MongoDB deployment is a replica set: usually three members holding copies of the same data. One is the primary, which takes all writes. The others are secondaries, which copy the primary's operation log (the oplog) and replay it.

Figure 1. All writes go to the primary and flow to the secondaries through the oplog. If the primary disappears, the remaining members elect a new one, usually within about ten seconds.

Two settings decide how safe a write is. Write concern says how many members must have the write before MongoDB reports success. w: 1 means just the primary: fast, and a write can be lost if the primary dies before a secondary copied it. w: "majority" means most members have it, so it survives a failover. Use majority for anything you care about; recent MongoDB versions use it as the default. Read preference says where reads go. Reading from secondaries spreads load, but they can lag slightly behind, so you may read something older than what you just wrote.

Scaling out: sharding#

When one replica set cannot hold the data or the write traffic, MongoDB splits a collection across several replica sets, called shards, using a shard key you choose.

Figure 2. The app talks to a router, which looks up which shard owns which range of the shard key and sends each query only where it needs to go. Each shard is itself a replica set.

Data is divided into ranges of the shard key, called chunks, and a background balancer moves chunks between shards to keep them even. A query that includes the shard key goes to one shard. A query without it has to ask every shard and merge the answers, a scatter gather, which gets slower as you add shards.

Everything depends on the shard key, and it is hard to change later. Compare four choices for an orders collection.

Shard key

Shard 2 carries 17.4% (even would be 16.7%)

Writes spread evenly because hashing scatters customers randomly. Queries for one customer's orders hit one shard. Range queries across customers, such as all orders today, must visit every shard.

Good shard key or not?

An ObjectId _id, which starts with a timestamp

tenant_id for a SaaS app with one giant customer

user_id for a social app, queried mostly by user

status (pending, shipped, delivered)

Transactions#

Single document writes have always been atomic, which is one more reason to keep data that changes together in one document. Since version 4.0 MongoDB also supports multi document ACID transactions, extended to sharded clusters in 4.2. They work, and they cost more than in a relational database, so a design that needs them constantly is a hint the data might belong in Postgres.

Choice	What you gain	What you pay	Pick it when
Embed	One read returns everything, and single document writes are atomic.	Duplicated data goes stale, and growing arrays make documents heavy.	Data read together, owned by one parent, bounded in size.
Reference	One copy of each fact, and children can be queried on their own.	Extra queries or $lookup joins, which are slower than relational joins.	Shared data, independently updated data, unbounded lists.
Shard on a hashed key	Even writes and even storage across shards.	Range queries become scatter gather.	High write volume where most queries are by that key.

Checkpoint#

Checkpoint

1. When should comments be referenced rather than embedded in a post?

2. Why is created_at a poor shard key for an orders collection?

3. What does write concern majority protect against?

Worth memorising

Model for how you read. Embed what is read together and bounded, reference the rest.
16 MB per document, and unbounded arrays are the classic mistake.
Replica set: one primary, secondaries replay the oplog. Use write concern majority.
The shard key decides everything. High cardinality, not monotonic, matches your main query.

Say this in 60 seconds

MongoDB stores JSON like documents in collections, so I model data the way the application reads it. I embed data that is read together, owned by one parent and bounded in size, and reference data that is shared, updated independently or unbounded, sometimes embedding a small display copy. For availability it runs as a replica set, a primary taking writes and secondaries replaying the oplog, with automatic elections, and I use write concern majority so acknowledged writes survive failover. To scale, it shards collections by a shard key through mongos routers, and the key must be high cardinality, not monotonically increasing, and match the main query, or writes pile onto one shard and queries scatter to all of them.

Next: DynamoDB Related: Sharding