Library
7 min read

Concurrency vs parallelism

Published

A single cook making four dishes puts the dal on to simmer, chops onions while it cooks, starts the rice, checks the dal, rolls rotis while the rice steams. Four dishes are in progress at once, but at any instant the cook is doing exactly one thing. That is concurrency: structuring work so many tasks make progress by interleaving, especially while some of them wait.

Four cooks each making one dish is parallelism: four things literally happening at the same instant, which needs four pairs of hands.

Rob Pike's line from his 2012 talk is the one to remember: "Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once."

Your answer

Node.js runs your JavaScript on a single thread, yet one Node process happily serves 10,000 open connections. Meanwhile, resizing 10,000 photos in that same process is painfully slow. Why does one thread handle the first job well and the second badly?

See the difference#

Four requests, three ways to run them
core 1R1R2R3R4R1R2R3R4
waiting on DB
All done at 160 msCore busy 100%Striped: a core held by a request that is only waiting
Concurrency on one core. While request 1 waits for the database, the event loop runs request 2, then 3, then 4. The waits overlap, so all four finish at 160 ms instead of 400 ms, with no extra cores. This is how Node.js and Python asyncio serve thousands of connections.
Each block is 10 ms. Context switching and scheduling overhead are left out, which flatters threads slightly.

Switch between the two workloads and the whole story falls out:

  • I/O bound work (most web backends) spends most of its time waiting: on the database, on another API, on the disk. Concurrency overlaps the waits. One core with an event loop gets close to the speed of four cores, because the cores in the parallel version were mostly idle.
  • CPU bound work (image processing, compression, encryption, ML inference) never waits. Interleaving it changes nothing. Only more cores, parallelism, makes it faster.

The tools, mapped#

  • Event loop (Node.js, Python asyncio, Nginx, Redis). One thread, non blocking I/O, callbacks or async/await. Excellent concurrency for I/O, no parallelism. One slow CPU task blocks everyone, which is why a 2 second JSON parse in a Node request handler freezes the whole server.
  • Threads. Can give both. On one core they interleave (concurrency); on many cores they run truly at the same time (parallelism). Python's GIL long prevented threads from running Python code in parallel; Python 3.13 added an experimental free threaded build, and multiprocessing sidesteps it with processes.
  • Goroutines (Go) and virtual threads (Java 21). Cheap user space tasks multiplexed onto a pool of OS threads, one per core. You write simple blocking code, and the runtime gives you concurrency across thousands of tasks and parallelism across the cores.
  • Processes and machines. Parallelism by running more copies. That is how you scale Node (one process per core, or more containers) and how batch systems like Spark spread work.

See processes vs threads vs async for how these differ in memory, isolation and failure.

Concurrency bugs need no parallelism#

A race condition is two tasks interleaving in an unlucky order. That can happen on a single core: an async function reads a balance, awaits a network call, and another request changes the balance in between. Concurrency brings the bugs; parallelism only makes them more frequent. The fixes are the same: atomic operations, locks, or not sharing mutable state.

ts
// Two concurrent requests, one thread, still a race.
async function withdraw(userId: string, amount: number) {
  const balance = await db.getBalance(userId)       // both read 500
  if (balance < amount) throw new Error('insufficient')
  await db.setBalance(userId, balance - amount)     // both write 500 - amount
}
// Fix: one atomic statement.
// UPDATE wallets SET balance = balance - $1 WHERE user_id = $2 AND balance >= $1

How much can more cores help?#

Parallelism has a limit set by the part of the work that cannot be split. That is Amdahl's law: if 10% of a job must run serially, no number of cores makes it more than 10 times faster.

Amdahl's law
10 %
16
Speed up with these cores1 / (0.1 + 0.90 / 16) = 6.40x
Speed up with infinite cores10.0x
Efficiency per core40%
6.4x
faster on 16 cores

The serial part dominates quickly. Past a point, extra cores are mostly idle. Shrinking the serial part (a lock everyone waits on, a single coordinator) pays more than adding hardware.

ChoiceWhat you gainWhat you payPick it when
Concurrency (event loop, async)Thousands of in flight I/O tasks on one thread, tiny memory per task, no locks in single threaded runtimes.No speed up for CPU work, one blocking call stalls everything, async code is harder to read.Web servers, API gateways, proxies, chat servers: anything waiting on networks and disks.
Parallelism (cores, processes, machines)Real speed up for CPU heavy work.Shared state needs synchronisation, limited by the serial fraction, more memory.Image and video processing, compression, ML inference, analytics, builds and tests.

In the interview#

Interviewer
Node.js is single threaded. How can it handle so many connections?
Classic. The answer is concurrency through non blocking I/O.
You
Most of the time a request is waiting on the database or another service. Node never blocks a thread on that wait: it hands the I/O to the operating system (epoll on Linux) and the event loop runs other requests' JavaScript in the meantime, coming back when the result is ready. So one thread is enough for many concurrent connections as long as each callback does a little CPU work. The weakness is CPU heavy work, which blocks the loop for everyone, so I would move that to worker threads, a separate service, or a queue.
Names the mechanism, the condition it depends on, and its failure mode.
Interviewer
And to use all 16 cores?
You
Run 16 processes, via the cluster module or, more often, 16 containers behind the load balancer. That is parallelism across processes, each still concurrent inside.
Concurrency or parallelism is what helps?
An API that calls three other APIs and waits for all three
Encoding a 4K video
A WebSocket server with 50,000 idle connections
Running 2,000 unit tests

Checkpoint#

Checkpoint

1. Why does an event loop handle many connections well but CPU work badly?

2. Can a race condition happen on a single core?

3. 5% of a job is serial. What is the most speed up any number of cores can give?

Worth memorising
  • Concurrency: dealing with many things at once. Parallelism: doing many things at once.
  • I/O bound: concurrency helps. CPU bound: only parallelism helps.
  • Never block the event loop with CPU work. Worker threads, processes or a queue.
  • Races come from interleaving, not from cores.
  • Amdahl: max speed up = 1 / serial fraction.
Say this in 60 seconds

Concurrency is structuring a program so many tasks are in progress at once by interleaving them, which pays off when tasks spend time waiting; parallelism is running tasks at the same instant on multiple cores, which is the only thing that speeds up pure computation. An I/O bound web backend gets most of its throughput from concurrency, which is why a single threaded event loop like Node.js serves thousands of connections: it hands waits to the operating system and runs other callbacks meanwhile. CPU bound work like image processing needs parallelism across cores, processes or machines, and blocks an event loop if run inside it. Go's goroutines and Java's virtual threads give both. Race conditions come from interleaving, so they happen even on one core, and the fix is atomic operations or locks. Amdahl's law limits parallel speed up to one over the serial fraction.