concurrency-scalingAnswer last reviewed July 2026

Say your Node service has maxed out the box it runs on. What has to be true before you can just throw more machines at it?

The good answers name every place per-process state hides, then do the shared-resource math before anyone gets to add instances.

What an AI-prepared candidate might say

It basically has to be stateless. Anything that needs to survive past a request goes to shared infrastructure, so sessions go in Redis or a signed cookie, uploads go to object storage, background jobs go on a queue, and that way any instance can serve any request. Then a load balancer spreads traffic across the instances, with health checks to pull bad ones out of rotation, and sticky sessions only if something really needs affinity. Shared state lives in databases and caches that every instance can see. Once that's all true, scaling is just adding instances behind the balancer, and an autoscaler can do it off CPU or request metrics. The usual blockers are in-memory sessions, writing to local disk, or code that just assumes there's one instance. You fix all of those by moving the state out of the process.

Senior
Locked

The full list of places state hides (sessions, caches, limiters, singleton jobs, sockets, local disk) and what each one does to you at N instances.

Unlock the depth
Staff
Locked

The DB-connection budget, cache stampedes at fleet scale, the metric a Node service should actually autoscale on, and keeping rollouts safe under version skew.

Unlock the depth
Follow-up chain
Say your Node service has maxed out the box it runs on. What has to be true before you can just throw more machines at it? | NodeBook