HTTP Parsing with llhttp

A TCP read can stop halfway through an HTTP header. That is why Node needs an HTTP parser at all.

The socket hands Node raw bytes, and nothing in those bytes tells the socket where one HTTP message ends and the next begins. HTTP's rules live one layer above the socket. So Node puts a parser in between. The parser remembers the exact spot where it stopped reading. When more bytes show up, it continues from there. It calls into JavaScript only once it has read enough of the message to do something with it.

Here is the smallest version of the problem.

socket.write('GET /slow HTTP/1.1\r\nHos');
socket.write('t: example.test\r\n\r\n');

The first write ends right after Hos. At that point Node has seen the start of the Host header name and nothing more. The second write finishes the word Host, adds its value, and then sends the blank line that closes the header section.

A single socket read can hold all sorts of partial or complete HTTP data.

half of a request line
one complete header section
headers plus part of the body
two complete requests plus part of a third

The parser has to handle all of these.

TCP carries bytes and nothing else. It has no concept of where an HTTP request begins or ends. The bytes can arrive a few at a time, in one large block, or as several HTTP messages bunched together. HTTP/1.1 is what gives those bytes structure, and the parser reads that structure to work out what each byte means.

So Node never assumes that one socket read equals one HTTP message. A read might stop before the request line is even finished. The next one might carry a whole GET request with no body. A read during a large upload might hold only the first few kilobytes of it. And on a busy connection, one read can contain a finished request plus the start of the next pipelined one.

Node feeds bytes into llhttp as they arrive. llhttp tracks where it is in the current HTTP message. If a buffer ends before the message is done, llhttp consumes the valid part it got and returns. The next time Node hands it a buffer, it carries on from where it left off.

When you debug this code, the parser's position is the thing to follow, not the socket chunks. Where a chunk happens to end tells you almost nothing, because chunks and HTTP messages do not line up.

You run into this the moment you start debugging. A packet capture can show one header smeared across several TCP packets. Meanwhile a single Node stream chunk can hold several header lines together. And a parser error will sometimes point into the middle of the buffer, since the bytes before the bad one were already accepted. What a byte means depends entirely on the state the parser was in when it got there.

An HTTP parser reads HTTP/1.1 bytes and reports the pieces it finds. The method, the URL text, the version, the headers, the body bytes, any trailers, and the point where the message ends. In Node, the parser doing that job is called llhttp.

llhttp is the native HTTP/1.1 parser library that node:http uses. Its definition is written in TypeScript in the llhttp project and then generated into C, which is what Node compiles and embeds. Node wraps that C parser in its own native and JavaScript code, attaches it to each socket, and turns the parser's events into http.IncomingMessage and http.ServerResponse behavior.

Bytes move through these layers before your code runs.

connected socket bytes
  -> Node HTTP parser wrapper
  -> llhttp execution
  -> parser callbacks
  -> IncomingMessage and ServerResponse
  -> request listener or client response callback

A parser callback is a function that Node registers with the parser, so the parser can report progress as it works through the bytes. llhttp fires callbacks for message start, URL spans, header field spans, header value spans, the end of the header section, body chunks, chunk positions, trailers, and message completion. Node's wrapper catches each of those and updates its own HTTP objects.

The timing of those callbacks explains a lot of how Node HTTP feels to use. Your request listener does not run until the parser has accepted the request line and the whole header section. If parsing fails early, your handler never gets a req object at all. Once the handler is running, body bytes can still be arriving behind it. Trailers show up only after the body has finished. And an upgrade request leaves the normal HTTP path entirely, handing the raw socket to whatever code takes over the next protocol.

Bytes Before Objects

http.createServer() builds the server object and holds on to your request listener. .listen() starts accepting sockets. When a connected socket reaches the HTTP server, Node attaches a parser to it and begins feeding the readable bytes in.

Three separate things are at work here, and each one has a distinct job.

net.Socket
  owns connection state and readable bytes

HTTP parser
  owns partial HTTP/1.1 parse state

IncomingMessage
  exposes parsed headers and body as JavaScript state

The socket is the transport layer. It deals with connection state and the raw bytes moving across it. The parser sits above that and applies the rules of HTTP/1.1 grammar. IncomingMessage is the JavaScript object your handler actually receives, and it represents one parsed HTTP message.

Splitting the work this way is what lets a single connection carry more than one request. A keep-alive socket, the kind that stays open to serve more than one request, can handle one request, then another later on. Between messages, Node resets the parser state but keeps the socket open. Every request still gets a fresh IncomingMessage of its own.

Parsing starts before JavaScript sees the request event.

Node reads bytes off the socket into buffers, and the parser consumes those buffers. llhttp walks the HTTP grammar and reports each piece it finds through callbacks. Node gathers the header pieces, works out when the headers are complete, builds or fills in the IncomingMessage, pairs it with a ServerResponse, and only then emits the server's request event.

The parser is attached at the HTTP layer, so normal server code receives req data instead of raw socket bytes. The socket still exists at req.socket, but the HTTP server has parser machinery connected to the readable side of that socket. If your application reads straight from the socket, it competes with the parser for those bytes and breaks the HTTP server's handling of them. In HTTP server code, the request stream is how you read the body.

The ordering here is easy to get wrong. Node can accept a TCP connection and emit the server's lower-level connection event before a single HTTP request has been parsed. The socket exists and the parser can be attached, but the client might not have sent enough HTTP bytes yet. The request event holds off until the parser has finished the request line and the headers. A slow client can keep that socket open for a while, with the parser still waiting on the blank line that ends the header section. The length of that wait is governed by server.headersTimeout.

This handler runs only after the headers have been parsed.

import http from 'node:http';

http.createServer((req, res) => {
  console.log(req.method, req.url);
  res.end('ok\n');
}).listen(3000);

By the time that callback runs, req.method, req.url, req.httpVersion, req.headers, and req.rawHeaders are all populated. The body stream can still be incomplete, though. On a large upload, the headers usually land first and the body keeps coming in afterward.

The client side works the same way, just in the other direction. http.request() writes an outgoing request. The socket then starts receiving the response bytes. The parser reads the status line, the headers, and the framing of the response body. Node emits the client's response event with an IncomingMessage that represents that response.

Errors show up at whatever point the parser had reached. A bad method token in an incoming request can fire clientError before the server ever emits request. On the client side, a malformed response from an upstream server shows up as an error on the ClientRequest. Either way, the parser rejected the bytes while Node was still partway through building the message object.

When something breaks, the useful question is where. Transport problems surface as socket errors. The HTTP grammar and the parser limits produce parser errors. And anything that goes wrong after Node has built req and res is a handler error.

A plain net.Server and an http.Server behave very differently, even though both begin with a connected socket. A net.Socket just hands you chunks whenever something reads from the stream. An HTTP server slots a parser in between the socket and your handler. That parser eats the HTTP framing, strips it out of the body stream, and surfaces the message metadata as properties.

An inbound request runs in this order.

accept connected socket
  -> attach HTTP parser
  -> parse request line and headers
  -> create request and response objects
  -> emit request
  -> stream body bytes

The client response side runs through the reverse sequence.

assign socket to ClientRequest
  -> write request bytes
  -> attach response parser
  -> parse status line and headers
  -> emit response
  -> stream response body bytes

The two paths line up step for step, but the error event is different on each side. A parse failure on the server side goes to clientError. The peer there is the client talking to your server. The same failure on the client side surfaces as an error on the request object, because the peer is now the upstream server whose response Node refused.

The State Machine

llhttp works as a state machine. At any moment it sits in one state, reads the next bytes, and moves to another state according to the HTTP grammar. Whatever state it is in decides which bytes count as valid and which callback fires next.

The complete state graph is large. The trace below covers enough of it to follow what happens.

start
  -> request line
  -> header field
  -> header value
  -> headers complete
  -> body or next message
  -> trailers
  -> message complete

For chunked bodies, the body section has more steps.

chunk size
  -> chunk data
  -> chunk complete
  -> next chunk size
  -> trailers
  -> message complete

These states survive from one socket read to the next. Feed llhttp a buffer that ends in the middle of a header name, and the parser records exactly where it stopped. The next execute call picks up from that spot with the new bytes. Saving the state like this is what lets Node cope with TCP reads that cut off partway through HTTP syntax.

The parser is incremental. A single llhttp_execute() call might parse a whole request, only part of one, several pipelined requests at once, or one request plus the start of the next. It all comes down to which bytes Node passes in. The call consumes as much valid input as it can and hands back a status code. HPE_OK means the input was consumed. A pause code means parsing stopped deliberately. Anything else in the HPE_* family means the parser rejected the bytes.

Inside the request line, the parser is hunting for a method token, a space, the request target bytes, another space, and the HTTP version. It does not need the whole message in memory first. It reports spans as it comes across them, and Node collects the parts it cares about and stores them on the parser or the message.

Header parsing repeats the same steps for every field. llhttp reads a field name, hits the : byte, reads the value, and finishes the pair at the line ending. Node keeps the original name and value for rawHeaders and also builds up the normalized view that becomes message.headers. It carries on until it reaches the blank line that closes the header section.

The headers-complete point is where Node finally has enough to build the main request object. By then it has the method, the URL text, the version, the full header section, and the body framing rules that the headers spell out. With that in hand, Node creates the IncomingMessage and decides how the rest of the body should flow.

What the parser tracks depends on how the body is framed. With a Content-Length body, it counts down how many body bytes are left in the current message. With chunked transfer coding, it follows the chunk size lines, the chunk data, the chunk endings, and any trailer fields. And for a body that ends only when the connection closes, completion comes from EOF through llhttp_finish() rather than a length counter.

Callbacks come in two kinds. Span callbacks point at a run of bytes and say which field those bytes belong to, the URL or a header value, for example. Completion callbacks mark a finishing point, like the end of the headers, the end of a chunk, or the end of the whole message. Node maps both kinds onto its own HTTP message lifecycle.

Node's HTTP wrapper sits between the raw parser callbacks and your code. It is responsible for policy, the maximum header size, how duplicate headers are handled, building the IncomingMessage, pushing into the body stream, and firing error events. llhttp handles the byte grammar and the state changes, nothing more.

A span callback can fire more than once for a single field. A TCP read can stop in the middle of one. A long field can straddle two buffers. So a URL callback might get /user on one call and s/42 on the next, or a header value callback might get part of a long cookie value now and the rest a little later. Node keeps appending those spans until the matching completion callback says the field is finished.

That split is why the parser exposes data callbacks and completion callbacks separately. A data callback reports more bytes for the current field. A completion callback reports that the field is now done. Node needs both before it can build a stable JavaScript property out of the field.

There is an extra step in header parsing. To the parser, field names and values are just byte ranges. JavaScript works in strings. Node then has to decode those bytes and store them following HTTP's rules for header fields, while also keeping enough of the original around for rawHeaders. All of that runs before req.headers becomes the plain object you read in a handler.

The state machine never leaves the parser. What reaches JavaScript is a coarser set of lifecycle events. In your code you get request, data, end, clientError, upgrade, and close. Down in llhttp the positions are much finer, things like method, URL, version, header field, header value, body, chunk size, trailer field, trailer value, and complete.

Pipelining is another thing that comes straight out of incremental parsing. Pipelining means a client sends several HTTP/1.1 requests on one persistent connection without waiting for each response first. The parser can finish a message, drop back to the start state, and keep consuming bytes that are already sitting in the same buffer. Node still has to keep the responses in the right order at the HTTP layer. Subchapter 5 gets into the keep-alive side of that. The parser part of it rests on one plain point. A single socket buffer can hold more than one HTTP message.

Partial bodies work the same way. Say Content-Length: 10 arrives and the first body callback reports 4 bytes. The parser still has 6 bytes outstanding for this message. If the socket closes after only those 4 bytes, completion fails, because the body promised 10 bytes and delivered 4. That is a failure to complete the HTTP message, which is a different problem from how the JavaScript stream is formatted.

A parser pause is a deliberate stop. llhttp has pause states so that Node can stop feeding it bytes and pick up again later. Node leans on this around accepted upgrades. Backpressure, the mechanism that slows a producer when a consumer falls behind, sits one level up from the pause. When the readable side of the IncomingMessage has buffered enough body data, Node stops reading from the socket, so fewer buffers reach llhttp until the consumer asks for more. After an accepted upgrade, the parser has already read the HTTP request that asked for the upgrade, and then parsing stops so the bytes after it can belong to the new protocol.

A parser reset puts a parser back in its start state but keeps its configuration intact. Once a message finishes, Node can reset the parser and reuse it for the next HTTP/1.1 message on the same connection, or hand it back to its pool of idle parsers.

Resetting is safe because the per-message JavaScript state lives somewhere else, on the IncomingMessage, the ServerResponse, and the objects around them. The parser holds only its own parsing state. So once a message finishes and Node has detached the message objects, the parser is free to go back to its start state for the next one.

Parse errors come out of that same state machine. The same byte can be fine in one state and illegal in another. A question mark is perfectly valid inside a request target, but not inside a method token. A carriage return is expected at the end of a header line, yet illegal in the middle of a header value. Each byte gets its meaning from the state the parser is in right then.

So most parse errors come down to this. The parser was enforcing one specific HTTP rule, and the next byte broke it. Node wraps the HPE_* code it gets back in an error object and sends it out through the HTTP error path.

From Callback to Message

The callback layer is where raw HTTP progress turns into Node object state.

Take the request line first. llhttp reads the method bytes, the request target bytes, and the version bytes. Node stores the method token as req.method, the target text as req.url, and the version as req.httpVersion, with the major and minor version numbers available separately. By the time any of those are set, the parser has already checked enough grammar to be sure they are valid HTTP/1.1.

Next come the headers, in name and value pairs. Node gathers each name and value as the span callbacks report them, and the completion callbacks mark where one pair ends and the next starts. Node holds on to that raw list of pairs for rawHeaders, and it can derive the normalized header views from the list later.

Once the headers are complete, Node has enough to build the main request object. It picks the incoming message class, attaches the socket, sets the request properties, starts up the readable stream side, and on a server allocates the matching outgoing response object. Your request listener runs at exactly that point, once all of that is done.

The body stream hangs off that same IncomingMessage, even though the actual body bytes can arrive later. As body callbacks fire, they push bytes into the readable side. When the message completes, that completion state gets recorded. If the connection closes or errors, both the stream and the socket state are updated.

The JavaScript stays small.

http.createServer((req, res) => {
  console.log(req.method);
  console.log(req.headers);
  req.pipe(res);
});

That tiny handler is already pulling on three results of parsing. req.method came out of the request line. req.headers came from the collected header pairs, run through Node's normalization. And req itself, as a readable stream, delivers the body bytes once the parser has worked out which bytes are body.

The parser fills in the response side for clients too. A status line becomes res.statusCode, res.statusMessage, and res.httpVersion. The response headers turn into the same header views you get on a request. The response body bytes flow out through the response IncomingMessage.

Building the message means keeping the per-message state apart from the state that gets reused. Both the parser and the socket can be reused across messages. An IncomingMessage, on the other hand, belongs to exactly one HTTP message, so a second request on the same socket gets its own second IncomingMessage. That separation is what lets parser reset and keep-alive work without ever mixing up the JavaScript request objects.

A few Node options change how these objects get built without touching the HTTP grammar at all. The IncomingMessage and ServerResponse options let advanced users plug in their own subclasses. highWaterMark adjusts the stream buffer thresholds. optimizeEmptyRequests changes how the empty-body stream is set up for requests that carry neither Content-Length nor Transfer-Encoding. All of those act on what the parser produces. llhttp itself is still parsing the same grammar underneath.

This callback path also explains why some failures give you so little to go on. When the parser rejects bytes before the headers are complete, all you reliably have is the socket and the parser error. When it rejects bytes mid-body, Node might already have handed you a request object, but the message can still finish incomplete. The error came from the parser in both cases.

Header Materialization

Headers begin as bytes in the socket buffer. Node then exposes a few different JavaScript views of them, because different jobs need the headers in different forms.

message.rawHeaders holds the received names and values in one flat array. The casing is exactly as it came in, duplicate fields are still duplicated, and the original order is preserved.

[
  'Host', 'example.test',
  'X-Trace', 'a',
  'x-trace', 'b'
]

That form is the one you want when you are debugging parser input, checking what a proxy forwarded, or chasing duplicate-header behavior. For everyday lookup it is clumsy, since you have to walk the array in pairs yourself.

message.headers gives your application a lower-cased object instead. The names become lower-case keys, and the values come through as strings or arrays depending on the header and the duplicate rules. Current Node builds this view lazily, so req.headers is only constructed the first time your code reads it.

Computing it lazily saves work on request paths that never need the normalized object. One handler might read nothing but req.url and pipe the body straight on. A forwarding handler might only ever look at rawHeaders. In both cases Node can hold on to the raw parser output and build the normalized object only if something asks.

The normalized object uses lower-case keys because HTTP field names are case-insensitive in the first place. Lower-casing them up front means your lookups behave the same no matter how the client capitalized things.

const type = req.headers['content-type'];
const host = req.headers.host;

Both of those read from the normalized view. The original wire casing is still there in rawHeaders if you need it for diagnostics or forwarding.

Duplicate header names need more care.

In the normalized message.headers object, some header names are treated as single-value fields. For names like host, content-length, content-type, and authorization, Node keeps the first value and drops later duplicates by default. set-cookie is always an array. Duplicate cookie values are joined with ; . Any other duplicates are joined with , .

joinDuplicateHeaders changes that drop behavior for the single-value set. Turn it on and Node joins the duplicate values with , instead of throwing the later ones away. Either way, rawHeaders still holds the original pairs as received.

const server = http.createServer({
  joinDuplicateHeaders: true
}, (req, res) => {
  console.log(req.headers);
  console.log(req.headersDistinct);
  res.end('ok\n');
});

That option only affects the normalized headers object. headersDistinct is a separate view again, where every value is an array and duplicates are kept as their own elements. It sidesteps the join-and-discard policy in the result while still lower-casing the names.

The three views break down like this.

rawHeaders
  received names, received casing, pair order

headers
  lower-case keys, Node duplicate policy

headersDistinct
  lower-case keys, array values, duplicate values retained

Which one you reach for depends on what you are debugging. For ordinary routing, headers is almost always what you want. Forwarding headers onward usually calls for rawHeaders, or for rebuilding them deliberately. And when duplicates are the actual problem, headersDistinct and rawHeaders together tend to give the clearest picture.

None of them is free. headersDistinct preserves every duplicate as an array, though it still lower-cases the names. rawHeaders is the most faithful to the wire, casing and order intact, but you do the lookup by hand. message.headers is the easiest to read from, at the cost of Node's duplicate rules being baked in. All three are just JavaScript views over the parser output. None of them is the parser.

Duplicate Content-Length behaves differently. The body framing depends on a single length, so two conflicting Content-Length declarations can make the parser reject the message before it ever reaches routing. The duplicate-header views only come into play for messages the parser accepts far enough to build the message object at all.

Outgoing headers go through a different path. http.validateHeaderName() and http.validateHeaderValue() run the same low-level checks Node uses when you set headers on an outgoing request or response. Reach for them when your code takes header names or values from users and you want the failure to land before you start building the outgoing message.

import { validateHeaderName, validateHeaderValue } from 'node:http';

validateHeaderName(name);
validateHeaderValue(name, value);

Node also validates on calls like response.setHeader() or http.request({ headers }). The standalone functions just let you choose where the failure happens instead. An invalid name throws a TypeError with ERR_INVALID_HTTP_TOKEN. An invalid value can throw ERR_HTTP_INVALID_HEADER_VALUE or ERR_INVALID_CHAR.

The number of headers and the total size of the headers are two separate limits. server.maxHeadersCount defaults to 2000 and caps how many received pairs Node carries into the normalized and distinct views, and setting it to 0 removes that cap. maxHeaderSize caps the total number of header bytes instead. A request with hundreds of tiny fields pushes against the count limit, while a request with one enormous field pushes against the byte limit.

When you debug, keep these four apart. A count limit governs how many fields get materialized. A byte limit governs how much header input the parser accepts at all. The validation functions are about outbound syntax. Duplicate joining is purely a change to the JavaScript view.

The full path runs like this.

parse header bytes
  -> enforce parser grammar and size limits
  -> collect raw header pairs
  -> create normalized and distinct views on demand
  -> validate outgoing headers when sending

If a request fails before your handler ever runs, start with the parser grammar and the limits. If a header value looks different inside the handler than you expected, the materialization rules are the place to look. And if an outgoing response throws, the problem is on the validation path.

Body Bytes

The parser deals only with HTTP framing. JSON, form data, multipart uploads, and every other payload format are somebody else's problem, handled by later code. Once the parser has picked out the body bytes, Node hands them to you through the IncomingMessage readable stream.

For a fixed-length body, llhttp keeps a running count of the bytes still owed against Content-Length. Each body callback reports a run of body bytes, and Node pushes them into the request stream. When the count hits zero, the parser can finish the message and get ready for the next one on the same connection.

With chunked transfer coding, llhttp reads the chunk size and then takes exactly that many following bytes as chunk data. Only the data bytes reach your request stream. The chunk size lines and the delimiters never leave the parser layer, so your code reads body bytes and never the chunk syntax around them.

http.createServer((req, res) => {
  req.on('data', chunk => console.log(chunk.length));
  req.on('end', () => res.end('done\n'));
}).listen(3000);

Each chunk here is a body chunk from the readable stream. Its size is decided by buffering and by how the parser delivered the bytes, and it tells you nothing about the application payload format. With chunked transfer coding, these stream chunks do not even have to line up with the HTTP chunk sizes.

Backpressure reaches the parser through the stream state and the socket reads. When the readable side of the IncomingMessage has buffered enough, Node can stop pulling more bytes off the socket until the consumer catches up. Fewer socket reads mean fewer parser executions, and the parser just stays parked at its current spot in the body.

This is where a frequent server bug comes from. A handler that ignores the request body leaves those body bytes unread, and they still belong to the current HTTP message. Until Node can either finish or drain that body under its lifecycle rules, the connection is not a good one to reuse. Subchapter 5 deals with the keep-alive and pooling fallout. At the parser level, the point is just that the message is still incomplete.

message.complete tells you whether Node received and parsed the whole HTTP message. It is most useful when a connection closes mid-body. A destroyed socket can end the request stream through a connection-failure path even though the HTTP message never actually finished.

This is also why req exists before the full body does. The headers alone are enough to build the request object and route it. The body can still be streaming in while your handler checks the metadata, opens a file, or sets up a call to an upstream service.

The parser's body callbacks report the spans of bytes that belong to the message body. A 10 MiB upload can come in across many socket reads, and Node can push out many body chunks for it. A single parser execution can also hold enough bytes to finish a fixed-length body and run straight on into the next pipelined message. The parser is keeping track of positions in the HTTP message. The stream layer is keeping track of what JavaScript has consumed. Those are two different counters.

A fixed-length body runs these steps.

headers complete
  -> content length stored
  -> body bytes pushed to IncomingMessage
  -> remaining length decremented
  -> message complete when remaining length reaches zero

A chunked body adds a few more.

headers complete
  -> chunk size parsed
  -> chunk bytes pushed to IncomingMessage
  -> zero-size chunk
  -> trailers parsed
  -> message complete

Both paths come out through the same req.on('data') API. What changes underneath is the parser work, because the HTTP framing on the wire is different.

An unread body has a few telltale signs. Your handler sends its response early, yet the socket stays busy because the client is still uploading. The connection closes instead of going back into the reuse pool. A later keep-alive request ends up waiting behind the leftover body cleanup. All of those trace back to the same situation, a message whose headers were accepted but whose body is still in flight.

If your server code means to ignore the body, req.resume() is the straightforward way to drop it. It reads and discards the body bytes so the message can still reach completion. If you instead need limits, the body readers with byte caps come in Subchapter 4. The parser stops at framed bytes. Everything about body policy happens after that point.

Client response bodies put the same pressure on you. The parser reads the response headers, emits response, and then the body bytes flow through the response IncomingMessage. Leave that response body unread and the socket stays bound to it, which hurts agent pooling and reuse down the line.

Parse Errors and Limits

Malformed input shows up as a parser return code well before it could ever reach your route code.

llhttp error codes all carry the HPE_* prefix. HPE_INVALID_METHOD says the method token was rejected. HPE_INVALID_HEADER_TOKEN says a byte in a header failed the header grammar. Codes like HPE_INVALID_CONSTANT and HPE_INVALID_VERSION each point at the particular parser state that turned the input down.

The code is really just the parser pointing at where it was. It names the HTTP rule it was enforcing at the moment a byte broke it.

const server = http.createServer((req, res) => {
  res.end('ok\n');
});

server.on('clientError', (err, socket) => {
  console.error(err.code);
  socket.end('HTTP/1.1 400 Bad Request\r\n\r\n');
});

Once you have a clientError listener, the socket outcome is entirely up to your code. A parser failure here happened before any req or res was built. If you want to reply with an HTTP response, you write it to the socket by hand. And your listener has to close or destroy the socket before it returns.

Node attaches two fields that help when you are diagnosing parser errors. err.bytesParsed is roughly how many bytes were parsed before the error. err.rawPacket is the buffer that was in play when it failed. Log both carefully, because a raw packet can contain credentials or body data.

The absence of req and res is itself the clue. Your usual response helpers, header setters, and framework middleware only run after parsing has succeeded. A parser error arrives before any of that exists. The socket is the one output path you have left.

So a clientError handler should stay small. It checks whether the socket is still writable, writes a minimal HTTP response if that makes sense, and then ends or destroys the socket. For logging, err.code, err.bytesParsed, the peer address, and a redacted sample of the packet are plenty. A full packet log can leak secrets.

A broken method token shows it plainly.

GE T / HTTP/1.1
Host: example.test

The space sitting inside the method token breaks the request-line grammar. llhttp returns an HPE_* error, and Node surfaces it through clientError. The request listener never runs, because the parser threw out the start line before Node could build a valid request object.

Header overflow is a different failure. It means the header section came in larger than the configured byte limit. In Node v24 the default ceiling is 16 KiB. The process-wide value is http.maxHeaderSize, and --max-http-header-size sets it. A server can override it per instance with the maxHeaderSize option, and a client request can set its own maxHeaderSize for the response headers it reads.

const server = http.createServer({
  maxHeaderSize: 1024
}, (req, res) => {
  res.end('ok\n');
});

That server takes at most 1024 bytes of request headers per request. Go over the limit and the parser reports HPE_HEADER_OVERFLOW. By default Node answers that code with 431 Request Header Fields Too Large. A custom clientError listener can send something else, but it still has to close out the socket cleanly.

--max-http-header-size changes the default limit for the process.

node --max-http-header-size=32768 server.js

That flag moves the default byte ceiling for the whole process. The per-server maxHeaderSize is the local override on top of it. The unit is bytes throughout. A header with non-ASCII characters on the wire still costs bytes, and the parser limit is counting bytes.

server.maxHeadersCount is the count limit for incoming header pairs, defaulting to 2000. Node applies it as it adds header lines into the request object's normalized views. Lower it and fewer pairs show up through headers and headersDistinct. Set it to 0 and the count limit is gone.

These two do not fail the same way. HPE_HEADER_OVERFLOW is about bytes, so a large cookie, a long bearer token, or a stack of forwarding metadata can trip it. server.maxHeadersCount is about the number of fields and how many get materialized onto the request object. A flood of tiny X-Thing-N headers can build up count pressure while staying well under the byte ceiling.

Timeouts sit alongside these limits as a third axis. server.headersTimeout is about time, how long Node will wait for a complete header section to arrive. maxHeaderSize is about size, how big that section may get. server.maxHeadersCount is about quantity, how many fields Node will carry into its views. Between them they catch a different bad client each, whether it is too slow, too large, or simply sending too many fields.

The timeout and the byte size can both kill the request before your handler sees it. Count pressure is gentler, you still get a request object, just with the normalized and distinct views capped. When the exact list of received pairs is what you care about, read rawHeaders during diagnostics.

requireHostHeader enforces one HTTP/1.1 rule. In Node v24, http.createServer() sets it to true by default. An HTTP/1.1 request with no Host header gets a 400 Bad Request straight from the server path. Set it to false and that check relaxes, for talking to peers that send non-conforming requests.

const server = http.createServer({
  requireHostHeader: true
}, handler);

That option lives at the parser and server layer. It decides whether Node will accept a structurally complete HTTP/1.1 request that is missing the required host metadata. By the time routing code runs, the request has already cleared that check.

Clients hit a similar problem from the other side. An http.request() can fail at DNS, at TCP, at TLS for HTTPS, or in the HTTP parse. On plain node:http, an invalid response header from an upstream server comes back as an error on the ClientRequest. The parser rejected bytes from that response before Node could give you a valid IncomingMessage.

That ordering should change how you retry. A parse error means the peer sent response bytes outside the grammar or limits Node accepts. Retrying blindly tends to hit the exact same parser failure again. Until the logs say otherwise, treat a parser error as a protocol-compatibility or upstream-correctness problem. Socket reuse can complicate the picture, but the first read on it stays the same. The parser rejected the response grammar or the response limits.

For production logs, capture these.

error code
bytes parsed
server or client path
local and remote socket address
header size and count settings
leniency setting

That is enough to tell a peer that closed the TCP connection apart from a parser that rejected a header, and both apart from header bytes that ran past the configured ceiling.

It helps to group these failures by the first object that is missing.

When there is no req at all, either the parser failed before it could build the request, or the socket failed before any message existed. Look at clientError, socket errors, err.code, bytesParsed, and the header-size settings.

When you do have a req but req.complete is false, the message started and then failed to finish. Look at the body framing, the timing of the socket close, the declared length, and whether the client simply stopped sending.

And when req is fully there and the route itself throws, the parser already did its job. That failure belongs to your code or to application-level body parsing.

Grouping them this way stops a parser error from turning into a long framework debugging session. A method token rejected before the request was even built is a problem to fix above your routes. A header section that blew past maxHeaderSize sits above your body parsers. And an upstream server that returns invalid response syntax every single time is above your retry loop, not inside it.

Limits can also disagree from one deployment hop to the next. A reverse proxy might allow bigger headers than Node does, and Node might allow bigger headers than the service behind it. So a client can send a request that clears the first hop and fails at the second. When you are chasing a 400 or a 431, write down the configured header byte limit at every hop that parses the message.

requireHostHeader has the same trap. A proxy in front might add a Host, rewrite it, or reject it outright. Node's own default rejects any HTTP/1.1 request that arrives without one. When the request came through an intermediary, a missing header can be the proxy's doing rather than what the original client sent. Chapter 10's proxy subchapter gets into forwarding policy. Here the scope is just Node enforcing the server option before routing.

Lenient Parsing

Strict parsing is the default. Out of the box, Node's parser rejects malformed HTTP syntax rather than guessing at what the peer meant.

insecureHTTPParser switches on llhttp's leniency flags for a single server or client request. The process flag --insecure-http-parser does the same thing globally. Leniency here means the parser will accept inputs that strict mode turns down, things like invalid header values, invalid HTTP versions, certain framing combinations, bare line feeds, and some deviations in chunk formatting.

const server = http.createServer({
  insecureHTTPParser: true
}, (req, res) => {
  res.end('accepted\n');
});

That option exists for compatibility, nothing else. It widens the grammar Node will accept. It also raises the odds that your Node process and some other HTTP hop will read the same bytes differently. Request smuggling and proxy normalization come up in the later security and proxy chapters. At the parser level the change is small. Lenient parsing just counts more byte sequences as valid HTTP.

Reserve it for a known peer, a compatibility problem you have actually measured, and a route or service you control. It does not belong in a default server setup.

Leniency works on the client side too. When an upstream server sends response syntax that Node's strict parser would reject, http.request({ insecureHTTPParser: true }) can accept it anyway. The cost is identical to the server case. You have changed which protocol grammar the client will accept.

Leniency happens before any application validation. A lenient parser still gives you req.headers, req.url, and a body stream, and your application is still free to reject the request afterward. The parser's decision just comes first, back when Node is deciding whether the bytes count as HTTP at all.

Leniency is a separate knob from duplicate joining, too. joinDuplicateHeaders only changes the JavaScript view of headers that were already accepted. insecureHTTPParser changes which byte sequences get accepted in the first place, before any view exists. One acts on the object you read. The other acts on the grammar itself.

Think of grammar leniency as a deployment decision rather than a local one. When a reverse proxy, a Node server, and an upstream service disagree about what syntax is allowed, the same bytes can be read as different messages at different hops. Chapter 25 covers the attack classes that come from that. The parser-level point is short. A lenient parser changes which messages can get into your application.

On the client side, a narrow wrapper beats the process flag. When a single upstream sends a bad response version or header value, set insecureHTTPParser on that one request path and leave the rest of the process strict. Doing it that way keeps the exception sitting right there in the code where a reviewer will see it.

There is one last trap with leniency, around testing. It can make a test pass purely by accepting bad input, which only ever proves that the parser you configured accepted that input. So keep your test fixtures explicit about strict versus lenient mode, so a later reader knows which behavior the test is actually exercising.

Trailers

Trailers are header fields that arrive after the body instead of before it. In HTTP/1.1 you almost only ever see them with chunked transfer coding. The parser surfaces them once it has read the end of the body and then read the trailer section that follows.

The timing is the part to get right. message.trailers, message.rawTrailers, and message.trailersDistinct are filled in at the 'end' event. Before the body has finished, those fields are empty or only partly there.

http.createServer((req, res) => {
  req.resume();
  req.on('end', () => {
    console.log(req.rawTrailers);
    console.log(req.trailersDistinct);
    res.end('ok\n');
  });
});

rawTrailers is built just like rawHeaders. It is one array of received trailer names and values, with the casing and the pair order kept as they came. trailers is the normalized object view of them. trailersDistinct gives lower-cased keys with array values, keeping the separate received values.

The parser has to actually reach message completion before any of those views can be trusted. Abandon the body partway through and the trailer state stays incomplete.

Some protocol designs put trailers to good use, but most application request handling should not lean on them for routing or authorization. By the time a trailer is readable, the whole body has already come in.

The parser reaches them like this.

body bytes
  -> zero-size chunk
  -> trailer field and value pairs
  -> blank line
  -> message complete

Only at message completion can Node fill in the trailer views. Before that point a handler has the body bytes and the ordinary headers to work with, while anything that comes after the body is still pending.

Sending trailers goes through a separate API on the outgoing side. response.addTrailers() queues trailers onto a chunked response. On the way in, the parser turns received trailer fields into message.trailers, message.rawTrailers, and message.trailersDistinct. The two paths only meet at the wire format, then split back into Node's outgoing and incoming APIs.

The timing settles how you should use them. Anything you need for routing, authentication, content limits, or rejecting a request early has to be in the headers, because trailers land too late for all of it. Genuine post-body metadata, like an integrity check or some backend protocol convention, is the right fit, as long as you read it after 'end' and handle the case where the trailer is missing.

Idle Parsers

An idle HTTP parser is one that Node holds on to after it has finished parsing a message, ready to be used again. Allocating a parser is not free. Holding parsers and reusing them cuts down the repeated allocation work in a busy HTTP process.

http.setMaxIdleHTTPParsers() controls how many idle parser objects Node keeps -

import http from 'node:http';

http.setMaxIdleHTTPParsers(500);

The default in Node v24 is 1000. Turn it down and an application that sees bursty HTTP traffic holds less parser memory between bursts. Turn it up and an application that constantly creates and releases parsers does less allocation work.

Reusing parsers is a different thing from pooling HTTP connections. A connection pool is about sockets. An idle parser pool is about parser objects. A keep-alive socket carries several HTTP messages over its life, and an idle parser can later be attached to work on a different socket entirely. They are two resources, with two separate settings to tune them.

Reuse only works because of parser reset. After a parser finishes a message and Node detaches the message state, the parser can drop back to its start state while keeping its configuration, the request or response mode, the callbacks, and the leniency flags. From there Node either keeps it idle or attaches it to another parsing job.

Most applications never touch this setting, and that is fine. The time to change it is when a memory profile shows HTTP parser objects being retained, or an allocation profile shows parser churn. It tunes a runtime resource. It does not change how any request behaves.

Set it too low and you pay for more parser allocation during bursts. Set it too high and you hold more memory after the bursts pass. The right number comes from your traffic pattern and your memory profile, and has nothing to do with request semantics.

Parser retention also sits apart from agent tuning, and it is easy to reach for the wrong knob. http.Agent governs client-side socket reuse. The server keep-alive options govern how long incoming sockets live. http.setMaxIdleHTTPParsers() governs only how many parser objects are kept around. So a service drowning in open sockets will not be helped by changing parser retention, and a service still holding parser memory long after traffic dropped will not be helped by changing the socket pool.

The parser pool exists because setting up a parser carries state, the callbacks, the mode, the configuration, and some native memory. Reusing a parser keeps that setup ready to go. Resetting it clears out the per-message state. And the idle retention number decides how many of those reset parsers Node bothers to keep.

Upgrade Handoff

An Upgrade header is a request to switch protocols once the HTTP/1.1 request is done. The parser still has to read that HTTP request first. Node can only decide whether to hand off the socket after the request line and header section have both come through valid.

A typical upgrade request opens like this.

GET /chat HTTP/1.1
Host: example.test
Connection: Upgrade
Upgrade: websocket

The parser reads this as an ordinary HTTP request that happens to carry upgrade metadata. If the server accepts the upgrade, Node emits the upgrade event with three values, req, socket, and head.

const server = http.createServer({
  shouldUpgradeCallback: req => req.url === '/chat'
});

server.on('upgrade', (req, socket, head) => {
  socket.destroy();
});

shouldUpgradeCallback is a Node v24 server option for deciding which upgrade attempts to accept. It gets the incoming request and returns a boolean. An accepted attempt fires upgrade. A rejected one carries on down the ordinary request path.

Without that option, the default decision depends on whether the server has an upgrade listener at all. In Node v24.9 and newer, if an upgrade is accepted but no upgrade listener exists, Node destroys the socket, rather than leave an upgraded socket that nothing is handling.

Once upgrade fires, the HTTP parser is finished with that socket. Node takes its normal HTTP data handling off the socket. From here the code for the upgraded protocol reads the raw bytes itself. head holds the bytes that were already read past the HTTP headers and actually belong to that upgraded protocol.

WebSocket picks up right after that handoff. The HTTP parser recognizes the request, checks the HTTP grammar, and passes the socket along. The rest of it, frame parsing, ping and pong, close frames, masking, and reconnects, is covered in the realtime chapter.

The division of labor is clean. Before upgrade, Node enforces the HTTP/1.1 grammar and builds the HTTP objects. After upgrade, your handler or a protocol library takes over the socket and every byte that comes after.

A lot of upgrade handlers mishandle the head buffer. The same socket read that completed the upgrade request can also have delivered bytes past the HTTP headers. The parser consumes the HTTP portion and leaves those extra bytes alone, for the upgraded protocol. Node passes them to you as head. A handler that ignores head quietly drops the first bytes of the upgraded stream.

A rejected upgrade stays in ordinary HTTP request-and-response territory, so your normal handler can return a status code and a body. An accepted upgrade leaves that path for good. The socket becomes a raw stream that the upgrade handler controls. Response helpers, HTTP timeouts, body parsing, and keep-alive reuse stop applying to anything that comes after.

Underneath that handoff is the parser pause from earlier. llhttp reaches the upgrade point and stops treating the bytes after it as HTTP/1.1. Node then hands control to the upgrade event. The parser has done the HTTP part of the exchange, and the next protocol begins with the socket and the head bytes.

Related Reading

• Previous - Node.js http.Server Request/Response Lifecycle
• Next - Node.js HTTP Routing and Middleware Without a Framework
• Node.js net Module: TCP Servers, Sockets & IPC
• Node.js Buffer Operations: Views, Copies, and Memory Ownership
• WebSocket Protocol and Node Servers