Node.js interview questions: Concurrency and scaling

They help when the work is CPU-bound. So heavy parsing, compression, image manipulation, cryptography, that kind of thing, because those block the event loop and a worker gets them off the main thread. They don't help with I/O-bound work, Node already handles that asynchronously. And they can actually hurt when the tasks are small, because spawning threads and passing messages has overhead, which is why everyone uses a pool that reuses workers. The rule of thumb I've seen is that the computation should be big enough to outweigh the cost of sending the data over and getting the result back. Then SharedArrayBuffer lets threads share memory instead of copying it, and Atomics gives you safe reads and writes and waiting on that shared memory. That's how you avoid races when multiple threads are touching the same buffer.

Cluster forks a bunch of Node processes that cooperate so one port serves all of them. With the default scheduling policy, which is round-robin, the primary process accepts the incoming connections and hands them out to workers in turn, so the load stays roughly even. On Windows the default is different, the OS decides which process gets each connection there. Every worker runs your full server code. For long-lived stateful protocols like WebSockets, or Socket.IO holding session state, you need sticky sessions, meaning the same client always lands on the same worker, and round-robin doesn't guarantee that, so you add affinity at a proxy or hash by client address or something like that. There's also SO_REUSEPORT as an alternative to the whole arrangement, where each worker opens its own listening socket and the kernel spreads the connections between them.

So nothing travels by reference between processes or workers, it all gets copied. process.send to a child process serializes the message, JSON-style by default, or there's an advanced mode based on structured clone, and writes it over the IPC pipe. postMessage between worker threads structured-clones the value into the receiving thread. Plain data is fine either way, and structured clone also handles Buffers, Maps, Sets, circular references, that kind of thing. The cost grows with the message size, so big payloads get slow and memory-hungry. There are two escape hatches I know of. Transferables move ownership without copying, so an ArrayBuffer or a MessagePort listed in the transfer list detaches on the sender's side and reappears on the receiver's. And SharedArrayBuffer goes further, both sides read and write the same memory and nothing gets copied at all.

It basically has to be stateless. Anything that needs to survive past a request goes to shared infrastructure, so sessions go in Redis or a signed cookie, uploads go to object storage, background jobs go on a queue, and that way any instance can serve any request. Then a load balancer spreads traffic across the instances, with health checks to pull bad ones out of rotation, and sticky sessions only if something really needs affinity. Shared state lives in databases and caches that every instance can see. Once that's all true, scaling is just adding instances behind the balancer, and an autoscaler can do it off CPU or request metrics. The usual blockers are in-memory sessions, writing to local disk, or code that just assumes there's one instance. You fix all of those by moving the state out of the process.

You size the pool around the number of CPU cores, and you reuse workers instead of spawning one per task, since that amortizes the startup cost. Then you put a queue in front so tasks can wait when every worker is busy. Each task goes out to a free worker with postMessage and the result comes back the same way. When it saturates you either queue with a limit, reject new tasks, or push backpressure up to the caller. You want per-task timeouts too, so a stuck task doesn't hold a worker forever, and you restart workers that crash. Honestly, libraries like piscina package most of this up already, the sizing, the queueing, the timeouts, so hand-rolling it is rarely necessary. And you watch queue depth and task latency to tell when the pool is undersized.

You run multiple instances and replace them one at a time. Bring up a new instance, wait for its health check to pass, shift traffic over to it, then send SIGTERM to an old one, and that kicks off graceful shutdown, so it stops accepting new connections, finishes the in-flight requests, and exits before the platform's grace period runs out. Kubernetes rolling updates, blue-green, canary, they're all basically the same swap with a different order and blast radius. The app really has two jobs in all of this. Handle SIGTERM properly, and report readiness accurately so the balancer only routes to instances that can actually serve. If you do it right, capacity stays above demand the whole time and clients never notice anything. The usual failures are instances getting killed mid-request, or traffic getting sent to instances that aren't ready yet.

An in-process cache, so a Map or an LRU living inside the instance, is the fastest thing you can do. No network hop, no serialization, it's just a heap read. The downsides are duplication, since every instance caches its own copy, inconsistency between instances, and you lose it all on every restart. A shared cache like Redis gives all the instances one view, survives deploys, and each key gets cached once. But you pay a network round trip and serialization on every hit, plus it's a new dependency you have to operate. The pattern I've seen is layering them, a small short-TTL in-process cache in front of Redis in front of the origin, so hot keys are nearly free and the shared layer keeps the fleet roughly consistent. Which way you lean depends on how hot the data is, how much staleness you can tolerate, and how expensive the origin fetches are.

They all run another program, the differences are mostly ergonomics. spawn is the base one. It starts the process and gives you streams for stdin, stdout, and stderr, so it fits long-running processes or big output. exec runs your command through a shell and buffers the entire output, then hands it to a callback. That works for short commands with small output, because I'm pretty sure the buffer has a size cap on it. execFile is basically exec without the shell, it runs the binary directly with an argument array, so it's faster and safer with untrusted input. And fork is spawn specialized for Node scripts. It launches a new Node process and sets up an IPC channel, so parent and child can exchange messages with send and the 'message' event. That's the primitive sitting under cluster and process pools.

You listen for the worker's exit and error events, log whatever happened, and start a replacement. To avoid a tight crash loop you add exponential backoff between restarts and cap the number of attempts, and past the cap you alert instead of respawning. You read the exit code to tell a clean exit from a crash. And the restart logic stays outside the worker itself, so a broken worker can't break its own recovery. Health checks or heartbeat messages catch the workers that hang without actually dying. Then at the outer layer, a process manager like systemd or PM2 or the orchestrator supervises the parent the same way. The goal is basically automatic recovery from transient failures, containing the repeated ones, and enough logging to figure out what killed the worker.