So Node runs all your JavaScript on one thread. Then how does one process hold tens of thousands of open connections at the same time without falling over?

The waiting happens in the kernel, and that's the piece most people miss. When a connection comes in, libuv registers the socket's file descriptor with whatever readiness mechanism the platform has - epoll on Linux, kqueue on macOS, IOCP on Windows. Then the loop parks itself in its poll phase, sleeping inside a single syscall, till the kernel reports that some descriptor has activity. An idle connection costs you one file descriptor, some kernel socket-buffer memory, and a little bit of JS heap state. No thread, no stack. So ten thousand idle sockets are basically free, because epoll_wait scales with how many sockets have events, and idle sockets have none.

There's a second, smaller mechanism for operations which have no readiness model. Regular files always report ready, but the actual read can still block on the disk, so that work goes to libuv's thread pool - four threads by default. The fs module runs there, dns.lookup also runs there, and the async crypto functions pbkdf2, scrypt and randomBytes too. But dns.resolve* doesn't - that one goes through c-ares on non-blocking sockets. And createHash and createCipheriv simply run synchronously on whatever thread called them.

So the full picture is one V8 thread running every callback, the kernel watching your sockets, and four pool threads soaking up the file and DNS work. Once you keep this split in your head, you can start predicting things. Ten thousand concurrent HTTP requests hitting a handler that queries Postgres is fine, because that is all socket work. But ten thousand concurrent fs.readFile calls - four will run and the rest will queue. The async API looks identical in both cases, and the scaling is completely different.

Once this is running in production, what I actually care about is what stalls that one shared thread, and what tells me about it first. Any synchronous work in a handler will do it. A multi-megabyte JSON.parse, a backtracking regex, a tight loop over some big array. Every other connection's callbacks wait behind it. The symptom is p99 climbing on every endpoint at once while host CPU looks moderate, because one core is pegged and the rest are sitting idle. The tool I reach for is perf_hooks.monitorEventLoopDelay, a native histogram sampled at 10ms resolution by default. Export the p99 and the max, never the mean. And pair it with performance.eventLoopUtilization, because ELU near 1.0 with low lag means a steady stream of small callbacks is saturating the thread without any single visible stall.

The other thing that saturates is the pool. I've seen this exact incident - an auth service doing pbkdf2 on every login fills all four threads, then dns.lookup, which shares the same pool, starts queueing behind it, so outbound HTTP from the same process becomes slow and nobody connects it back to logins. Bumping UV_THREADPOOL_SIZE buys some headroom. The durable fixes are moving password hashing into worker threads using the *Sync variants (because pbkdf2Sync blocks only the worker's own thread, while async crypto called from a worker lands right back on the same process-wide pool), and getting dns.lookup out of hot paths with keep-alive agents or dns.resolve.

For capacity, it's one process per core with cluster or container replicas, connection count bounded by fd limits and per-socket memory, and throughput bounded by per-request CPU on the loop thread. Do this math before you pick the platform. 5ms of CPU per request caps a process near 200 req/s, no matter how asynchronous the I/O is. Fan-out I/O workloads fit this architecture. CPU-per-request workloads don't.