One HTTP response, byte by byte

HTTP is the protocol that web clients and servers use for requests and responses. The client starts a request, the server reads it and sends back a response. HTTP/1.1 writes the control part of each message as plain text and carries the message over TCP.

The Internet Engineering Task Force publishes HTTP standards as Request for Comments documents, RFCs in short. RFC 9110 defines HTTP semantics i.e the meanings of methods, status codes and fields. RFC 9112 defines the HTTP/1.1 wire syntax i.e the required order and encoding of bytes. The response you're about to serve is exactly 52 bytes.

Figure 1.2 - A status line, one field line, the blank-line terminator, and the body together make the complete 52-byte response.

There are four pieces, and I'm using RFC 9112's own vocabulary for them. Please stick to these terms - they are precise, and the rest of the course assumes you know them.

1. The status line - HTTP-version SP status-code SP reason-phrase, for example HTTP/1.1 200 OK. SP means one space byte. The three-digit status code tells the client the result category. 200 means the request succeeded. The reason phrase is the text after the code, and RFC 9112 says a client SHOULD ignore that phrase. SHOULD is a standards keyword for a requirement you can skip only when you have a valid reason.
2. Field lines, also called headers. Each line has a field name and a value. You'll send content-length: 13. Field names are case-insensitive, so Content-Length and content-length mean the same thing. This course stores and emits lowercase names. Converting different spellings into one stored spelling is called normalization.
3. The header block terminator - a bare CRLF on a line by itself, so the wire shows \r\n\r\n. This is the only signal that headers have ended and the body is starting. "Blank line" is too loose a description here. It is exactly CR LF CR LF.
4. The body - raw bytes. HTTP doesn't assume a body contains text. It can be text, an image, or any other byte sequence.

[!INFO] The response head contains the status line, field lines and the blank-line terminator. The response body starts immediately after that terminator.

Every line in the head ends with \r\n - a carriage return (byte 0x0d) followed by a line feed (byte 0x0a). A bare \n won't cut it.

RFC 9112 allows a recipient (the program reading a message) to accept a lone LF as a line terminator. curl accepts such a response, but other clients reject it. A sender still has to generate the proper CRLF sequence. Lesson 3 compares curl with Node's fetch client.

See, TCP hands the receiver a stream with no message markers in it, so HTTP itself has to say where a body stops. This is called message framing. An HTTP/1.1 response can use one of these three framing methods.

1. content-length: N            read exactly N more bytes, then you are done
  2. transfer-encoding: chunked   chunks carry their own lengths (built in Chapter 10)
  3. close-delimited              no length is sent; the body ends when the server
                                  closes the connection (HTTP/1.0's old default)

Chunked transfer coding divides a body into pieces. Each piece starts with its own byte length, so the recipient can find the end without any content-length field. Chapter 10 teaches this and builds its decoder. A close-delimited body ends when the TCP connection closes, which means that connection can't carry another response after it.

If you send none of these and keep the connection open, the client has to keep on reading, because no received byte marks the end. curl -v prints no chunk, no close, no size. Assume close to signal end. Means it selected close-delimited framing and kept waiting for a close that never came.

The numeric content-length value must match the body exactly.

• If it's too large, the client waits for bytes that never arrive. Same hang as before.
• If it's too small, the client stops reading early. And on a reused connection, the tail of your body gets read as the start of the next response. Silent corruption.

RFC 9110 defines the content-length value as the number of octets in the body. An octet is an eight-bit byte. But JavaScript's str.length counts UTF-16 code units instead. A code unit is one numeric part of JavaScript's internal string encoding. The string "Hëllo" has str.length === 5 but it takes 6 bytes after UTF-8 encoding, because ë uses two UTF-8 bytes. UTF-8 is the byte encoding normally used for web text. Buffer.byteLength(str) gives you the number of encoded UTF-8 bytes.

Buffer.byteLength("Hello, World!"); // 13 - this is the number that goes on the wire

ASCII is an older character encoding whose letters, digits and basic punctuation each take one byte in UTF-8. For those strings, str.length and Buffer.byteLength agree. Still, use Buffer.byteLength, because the values differ the moment a string contains a character that encodes to more than one byte.

Today's response skips content-type. That field identifies the media type of a body, like plain text or JSON. The exercise doesn't need it, and a later chapter adds media-type selection.