Working with Buffers

I’m pretty sure you’re here because either you wish to learn how do I apply all the knowledge learnt in the previous chapters, or something in your Node.js service is consuming excessive amounts of memory. Or maybe because your high-throughput binary protocol parser is executing at extremely slow speeds. The culprit is almost always a fundamental misunderstanding of how Node.js Buffers handle memory. This chapter is your guide out of that hell. We’re going to dismantle the most dangerous assumption in Node.js development: that Buffer.slice() behaves like Array.prototype.slice(). It doesn’t. Not even close. One creates a new, independent copy of data; the other creates a “view” - a mere window into the exact same underlying memory as the original. This is the heart of zero-copy operations.

When used correctly, these views are incredibly powerful, letting you process massive amounts of data with almost no memory overhead. When misunderstood, they create the most insidious, hard-to-debug memory leaks you’ll ever encounter - leaks where a tiny 10-byte slice holds a 1GB buffer hostage in memory, preventing the garbage collector from reclaiming it. We’ll walk through the war stories, the late-night debugging sessions, and the production outages that forged this knowledge. You’ll learn the critical difference between a view (slice, subarray) and a true copy (Buffer.copy()), and when to use each. We’ll explore the intimate relationship between Buffers and TypedArrays, how they share the same memory foundation (ArrayBuffer), and how that can be both a superpower and a source of subtle data corruption. By the end of this, you won’t just know the API; you’ll have developed a deep, almost instinctual respect for memory semantics. You’ll understand why your service is using 10GB of memory for 1GB of data and, more importantly, how to fix it for good.

The Anatomy of a Gigabyte-Scale Memory Leak

Ever seen a service that should use a tidy 500MB of RAM suddenly decide it needs 10GB to live? It’s an amazing feature of misunderstanding memory. Let’s walk through how you could, with the best intentions and perfectly clean-looking code, build this exact disaster. It’s the best way to learn how to prevent it.

Imagine a common scenario: a service that ingests large batches of data. Maybe they’re logs, maybe they’re multipart file uploads. The task is simple: for each incoming chunk (which could be several megabytes), you need to parse a small, fixed-size header to extract an identifier, like a session ID. You’d write a function that looks something like this. Be honest, you’ve probably written this code a dozen times.

// This function gets called for thousands of incoming multi-megabyte chunks.
function getSessionId(logBuffer) {
  // The session ID is always the first 16 bytes.
  const headerSlice = logBuffer.slice(0, 16);

Stop right here. This exact line - logBuffer.slice(0, 16) - is where your production system begins its death spiral. Here’s what’s actually happening in the Node.js internals. When you call slice(), Node.js doesn’t allocate new memory. Instead, it creates a tiny JavaScript object (about 72 bytes on V8) that contains three critical pieces of information: a pointer to the parent buffer’s ArrayBuffer, an offset (0 in this case), and a length (16). This object lives on the V8 heap, but it maintains a strong reference to the external memory where logBuffer stores its actual data.

The V8 garbage collector sees this reference and marks the entire parent buffer as “reachable.” Even though you only care about 16 bytes, the GC must keep the entire multi-megabyte buffer alive. In V8’s generational garbage collection system, this parent buffer gets promoted from the young generation to the old generation after surviving two scavenges, making it even harder to collect. I’ve seen this pattern keep 100MB buffers alive for hours in production, all for the sake of storing a handful of 16-byte session IDs.

  // We'll store this slice in a map or cache to batch requests later.
  return headerSlice;
}

This code will inevitably cause production failures. It looks innocent, but logBuffer.slice(0, 16) is the line that will cause your production environment to fail completely. Here’s what happens. You’re processing, say, 100MB of logs per minute. Your service’s memory usage (the Resident Set Size, or RSS) should stay relatively flat. Instead, you watch it climb, gigabyte by gigabyte. You’re holding onto 10GB of memory to manage what should be, at most, a few megabytes of session IDs.

So you take a heap snapshot, and what you see makes no sense. The profiler shows you thousands of tiny 16-byte Buffer objects, but it claims they are collectively responsible for retaining gigabytes of memory. Your first thought is, “The profiler is broken.” It isn’t. It’s showing you a fundamental truth: the slice is not a copy. It’s a view. That 16-byte headerSlice object is just a lightweight JavaScript wrapper, but it holds an internal reference to the original, multi-megabyte logBuffer. As long as that tiny slice is alive - say, sitting in your cache - the garbage collector cannot reclaim the entire large buffer.

You weren’t leaking a few bytes. You were leaking the massive parent buffer for every single request. Multiply that by thousands of requests, and you have a recipe for the exact 10GB memory leak we’re dissecting. This is the consequence of misinterpreting one of the most common methods in the Buffer API. Let’s dig into why.

Understanding Buffer Memory Architecture

Before you can truly understand the slice disaster, you have to understand what a Buffer actually is. Most developers, especially those coming from Python or Ruby, have a vague mental model of buffers as “just a bunch of bytes.” That’s not wrong, but it’s dangerously incomplete. The most important thing to burn into your brain is this: Node.js Buffer memory does not live on the V8 heap.

This is the key. When you create a string or a JavaScript object, its memory is allocated and managed by the V8 garbage collector on the V8 heap. This is the memory you see in a typical Chrome DevTools heap snapshot. Buffers are different. To achieve high performance, especially for I/O operations, Node.js allocates Buffer memory outside of the V8 heap in what’s often called “external” memory. This memory is allocated in C++ land, often directly from the operating system.

Why does this matter? Because V8 is highly optimized for managing lots of small, short-lived objects. It’s not designed for handling large, contiguous blocks of memory like a 50MB file read from disk. Delegating this to C++ allows Node.js to work more directly with the OS’s own I/O mechanisms, avoiding costly memory copies between V8’s world and the system’s world.

Node.js is smart about this. To avoid the overhead of constantly asking the operating system for memory for every little buffer, it maintains an internal buffer pool. As of Node.js v23, this pool has a fixed size of Buffer.poolSize (8192 bytes or 8KB). When you create a new Buffer smaller than Buffer.poolSize >>> 1 (4096 bytes), Node.js will slice a piece off this pre-allocated pool instead of asking the OS for new memory. This is a significant performance optimization for small buffers.

The JavaScript Buffer object you interact with is essentially a lightweight wrapper, a sort of “handle” or “pointer” that lives on the V8 heap. This small JavaScript object holds a reference to the chunk of raw, external memory. The V8 garbage collector knows about this link. While the raw memory isn’t on the heap, V8 is aware that the JavaScript Buffer object is keeping that external memory “alive.” When the JavaScript Buffer object is garbage collected, Node.js knows it can then release the associated external memory.

Let’s visualize the relationship. Imagine you have a large block of memory allocated by Node’s C++ core. We’ll call this the ArrayBuffer, which is the standard JavaScript object for representing a fixed-length raw binary data buffer.

// Internally, Node.js does something like this (simplified):
// 1. Allocate a large block of raw memory (the backing store).
const backingStore = allocate_raw_memory(1024); // A 1KB C++ allocation

This allocation happens in Node’s C++ layer through a call to malloc() or, for aligned memory, posix_memalign(). The operating system finds a contiguous block of physical RAM and maps it into your process’s virtual address space. This memory is completely outside V8’s control - V8 doesn’t even know it exists yet. The allocation is tracked by Node’s internal memory allocator, which maintains metadata about the size and alignment of each block. If this allocation fails (rare but possible under memory pressure), Node.js will throw a RangeError with the message “Array buffer allocation failed” - a sign your system is genuinely out of memory.

// 2. Create an ArrayBuffer that points to it.
const arrayBuffer = new ArrayBuffer(1024); // Represents the raw memory

The ArrayBuffer is JavaScript’s bridge to this raw memory. When you create an ArrayBuffer, V8 creates a small JavaScript object on the heap (about 40 bytes) and associates it with the external memory through a mechanism called an “external backing store.” V8’s garbage collector is informed about this external memory through AdjustAmountOfExternalAllocatedMemory(), which helps it make intelligent decisions about when to trigger garbage collection. If you allocate too many ArrayBuffers too quickly, V8 will aggressively trigger major GC cycles to try to free up external memory, which can tank your application’s performance.

// 3. Create a Buffer object that is a "view" over that ArrayBuffer.
//    A Buffer is essentially a Node.js-specific version of Uint8Array.
const bigBuffer = Buffer.from(arrayBuffer);

The bigBuffer object itself is small - it just contains some metadata like its offset and length. The actual 1024 bytes of data live outside the V8 heap. This separation is the source of both immense power and immense confusion. It’s what allows for “zero-copy” operations, but it’s also what enables a tiny JavaScript object on the heap to prevent a massive chunk of external memory from being freed. In our log parser disaster, the thousands of 16-byte headerSlice objects were all tiny JavaScript handles on the heap, but each one carried with it an invisible string, keeping the original multi-megabyte logBuffer’s memory alive. We weren’t leaking JavaScript objects; we were leaking the massive external memory blocks they pointed to. After enough 3am debugging sessions, you learn to respect this distinction.

Views and References: slice, subarray, and Buffer.from

Now that we have a clearer picture of where buffer memory lives, let’s talk about the tools you use to manipulate it. This is the point where theory becomes practice, and where most developers make an incorrect decision. The three main functions we need to understand are Buffer.slice(), Buffer.subarray(), and Buffer.from() (when used with another buffer or ArrayBuffer).

Let’s start with the one that caused my team so much pain: slice(). If you come from a JavaScript background, your habitual programming patterns suggest that Array.prototype.slice() creates a shallow copy. You slice an array, you get a new array, and you can modify one without affecting the other. This is a lie when it comes to buffers.

Buffer.prototype.slice() does not create a copy. It creates a view.

Let me say that again, because it’s the most critical sentence in this entire chapter: Buffer.slice() creates a view that shares memory with the original buffer. It carves out a new Buffer object, but this new object points to the exact same bytes in the same underlying ArrayBuffer as the original.

Let me show you the innocent-looking code that nearly cost me my sanity.

// Imagine this is a 50MB buffer read from a network stream.
const massiveBuffer = Buffer.alloc(50 * 1024 * 1024);
massiveBuffer.write("USER_ID:12345|REST_OF_DATA...");

That Buffer.alloc() call just triggered a cascade of events in Node’s internals. First, Node.js checks if the requested size (52,428,800 bytes) is larger than Buffer.poolSize >>> 1 (4096 bytes). It is, so Node bypasses the buffer pool entirely. It makes a direct call to the C++ layer to allocate 50MB of memory. On Linux, this typically results in an mmap() system call for large allocations, which maps anonymous pages into your process’s address space. The kernel doesn’t actually allocate physical RAM yet - it uses a technique called “demand paging” where physical pages are only allocated when you first write to them. This is why Buffer.alloc() zeroes the memory - it forces the kernel to allocate real physical pages immediately.

The write() operation then copies your string data into this buffer using optimized SIMD instructions when available. V8’s string encoding machinery converts the UTF-8 JavaScript string into raw bytes. For ASCII characters, this is a straight copy. For multi-byte UTF-8 characters, the encoder has to carefully track byte boundaries to avoid splitting characters. This encoding happens in a tight C++ loop that’s been optimized to process multiple bytes per CPU cycle using vector instructions.

// This creates a VIEW into the same memory. No copy!
const userIdSlice = massiveBuffer.slice(9, 14); // Extracts "12345"
console.log(userIdSlice.toString()); // Output: 12345

That operation is incredibly fast because it doesn’t need to allocate 5 bytes and copy data into them. It just creates a tiny new JavaScript object with a different offset and length that points back to the original 50MB ArrayBuffer. Now, what happens if we modify the view?

// Let's modify the slice.
userIdSlice.write("99999");

This single write operation just corrupted your original 50MB buffer. When you call write() on the slice, Node.js calculates the absolute position in the parent ArrayBuffer: slice’s base offset (9) plus the write position (0) equals byte position 9 in the parent’s memory. The string “99999” gets encoded to UTF-8 bytes [0x39, 0x39, 0x39, 0x39, 0x39] and written directly into the parent buffer’s memory at positions 9-13. There’s no copy-on-write mechanism, no protection, no warning. The write happens through a direct memory pointer operation in C++, bypassing all of JavaScript’s safety mechanisms. In production, I’ve seen this pattern corrupt binary protocol headers, overwrite critical metadata, and even expose sensitive data from one request to another.

// Now let's look at the original buffer again.
console.log(massiveBuffer.toString("utf-8", 0, 20));
// Output: USER_ID:99999|REST_O

Did you see that? We changed the userIdSlice, and it mutated the massiveBuffer. They are two Buffer objects that reference the exact same memory location. When you modify the data through one Buffer reference, the change is immediately visible through the other Buffer reference because they both point to the same underlying ArrayBuffer. The V8 documentation on TypedArray views, which Buffers are based on, confirms this shared memory behavior is intentional and fundamental to their design.

So what about subarray()? In current Node.js versions, Buffer.prototype.subarray() is effectively the same as Buffer.prototype.slice(). Both create a view into the same memory, not a copy. The Node.js documentation recommends subarray() for clarity when you want to signal that you’re working within the TypedArray specification, as subarray is the standard TypedArray method for creating views.

const mainBuffer = Buffer.from([1, 2, 3, 4, 5]);
const sub = mainBuffer.subarray(1, 3); // A view of bytes [2, 3]

That subarray() call creates a new Buffer object with just three properties that matter: a reference to mainBuffer’s ArrayBuffer, an offset of 1, and a length of 2. The total cost is about 72 bytes on the V8 heap for the JavaScript object itself. No memory is allocated for the actual data. The view’s internal [[ViewedArrayBuffer]] slot points directly to the parent’s backing store. When you access sub[0], V8 performs pointer arithmetic: it takes the base address of the parent’s memory, adds the view’s offset (1 byte), and reads from that location. This happens entirely in compiled machine code without any JavaScript overhead.

sub[0] = 99; // Modify the view
console.log(mainBuffer); // Output: <Buffer 01 63 03 04 05> (Note the 99 is 0x63)

The third character is Buffer.from(). This one is tricky because its behavior changes completely depending on the input type you provide.

• Buffer.from(string): Allocates new memory and copies the string data into it.
• Buffer.from(array): Allocates new memory and copies the byte values.
• Buffer.from(arrayBuffer): Creates a VIEW that shares memory with the provided ArrayBuffer. This is a zero-copy operation.
• Buffer.from(buffer): Allocates new memory and copies the data from the source buffer. This is a full copy!

The distinction between Buffer.from(arrayBuffer) and Buffer.from(buffer) is a common source of bugs. The former is a zero-copy view, while the latter is a full-copy operation. The TypedArray view that silently corrupted our binary protocol taught me to never trust without measuring. We had a function that was sometimes passed an ArrayBuffer and sometimes a Buffer, and the subtle difference in Buffer.from() semantics was causing unexpected copies in our hot path, tanking performance.

Zero-Copy Operations

The term “zero-copy” is misleadingly appealing. It sounds like achieving performance gains without any costs. You’re not. There’s a trade-off, and you need to understand it. A zero-copy operation means you are not copying the data payload. You are, however, still creating a new JavaScript object - the view itself. This object has a small memory footprint on the V8 heap, but its creation is orders of magnitude faster than allocating a new memory block and then iterating over the original data to copy it byte by byte.

Let’s quantify this. Let’s say we have a 10MB buffer and we want a 1KB chunk from it.

const largeBuffer = Buffer.alloc(10 * 1024 * 1024); // 10MB
const chunkSize = 1024; // 1KB

This allocation triggers a single mmap() syscall for 10,485,760 bytes. The kernel reserves virtual address space but doesn’t allocate physical pages yet - that happens on first write through demand paging. Node.js tracks this allocation in its external memory accounting, adding 10MB to process.memoryUsage().external. V8’s garbage collector is notified through Isolate::AdjustAmountOfExternalAllocatedMemory(), which influences when the next major GC cycle triggers. If external memory grows too fast, V8 will panic and force a synchronous GC, blocking your event loop for potentially hundreds of milliseconds.

// --- The Zero-Copy View ---
console.time("view creation");
const view = largeBuffer.subarray(5000, 5000 + chunkSize);
console.timeEnd("view creation");

The subarray() operation completes in nanoseconds. It allocates exactly 72 bytes on the V8 heap for the new Buffer object and sets three fields: buffer pointer, offset (5000), and length (1024). No memory barrier, no cache invalidation, no TLB flush. The CPU can keep this entire operation in L1 cache. The performance counter shows ~0.007ms because that’s mostly the overhead of console.time() itself - the actual subarray operation takes less than 100 nanoseconds on modern CPUs.

// --- The Full Copy ---
console.time("copy creation");
const copy = Buffer.alloc(chunkSize);
largeBuffer.copy(copy, 0, 5000, 5000 + chunkSize);
console.timeEnd("copy creation");

On a modern Node.js install, the results are telling:

• view creation: 0.007ms
• copy creation: 0.024ms

The cost of creating a view is effectively constant time, O(1). It doesn’t matter if you’re viewing 10 bytes or 10 megabytes; you’re just creating a small JS object with some pointers and offsets. The cost of a copy, however, is linear time, O(n). It’s directly proportional to the amount of data you’re copying. For a 1MB chunk from a 100MB buffer, the view is still nearly instantaneous while the copy takes a measurable slice of a millisecond. In a hot path, this adds up. Our own telemetry has shown that replacing unnecessary copies with views in a critical parsing loop can cut CPU usage by 30%.

But here’s the trade-off, the common mistake that developers make. You hear “zero-copy” and think “faster.” So you embark on an “optimization” pass, replacing copies with views wherever you see them. But what you’re really doing is trading CPU cycles for memory management complexity. The view is fast because it doesn’t have to manage its own memory; it simply borrows the parent’s. This creates a strong reference that the garbage collector must respect. As long as your view is alive, the entire parent buffer is pinned in memory.

This is the fundamental trade-off: you trade memory safety for speed. You are telling the runtime, “Trust me, I know what I’m doing. Keep this giant chunk of memory around because I need this tiny piece of it.” The runtime will do exactly what you ask. And if you’re not careful, it will trust you all the way to an out-of-memory exception. The correct “optimization” is not to use views everywhere, but to understand when the cost of a small, explicit copy is infinitely cheaper than the memory cost of retaining a giant parent buffer.

TypedArrays and Buffer Relationships

To truly get this, you have to understand that Buffers are part of a bigger family: TypedArrays. This isn’t just trivia; it’s the key to how Buffers interact with the entire modern JavaScript ecosystem, from web APIs to WebAssembly. Since Node.js 3.0, the Buffer class is a direct subclass of Uint8Array.

const buf = Buffer.from("hello");
console.log(buf instanceof Uint8Array); // true

This is a huge deal. It means a Buffer is a Uint8Array, and you can use it anywhere a Uint8Array is expected. It’s what allows for seamless interoperability between Node’s core APIs and web platform standards.

A Buffer is a view over raw memory, where each element is an 8-bit unsigned integer (a byte). But there are other kinds of views. You can have an Int16Array (16-bit signed integers), a Float32Array (32-bit floating-point numbers), and so on. The crucial part is that all these different TypedArray views can share the exact same underlying block of memory, which is represented by an ArrayBuffer object.

This is where things get really powerful, and by powerful, I mean you can silently corrupt all your data without a single error being thrown. Let’s walk through it. Imagine you’re parsing a binary network message. You have a single ArrayBuffer containing the data. You could create multiple views to interpret different parts of it.

// A 12-byte ArrayBuffer
const messageArrayBuffer = new ArrayBuffer(12);

This creates 12 bytes of memory initialized to zero, as specified by the ECMAScript standard. Unlike what you might expect from low-level programming, new ArrayBuffer() always zero-initializes its memory for security and consistency. The memory is allocated from V8’s ArrayBuffer allocator, which for small buffers like this uses the process heap rather than mmap. V8 maintains this memory in a special zone that’s separate from regular JavaScript objects, allowing it to be accessed directly by typed arrays without going through V8’s object property access machinery.

// View 1: A Buffer to write a string into the end
const stringView = Buffer.from(messageArrayBuffer, 4, 8);
stringView.write("CONFIRMD");

This creates a Buffer view starting at byte offset 4 with length 8. The write() operation encodes “CONFIRMD” to UTF-8 bytes and writes them at positions 4-11 of the ArrayBuffer. Each character becomes one byte since they’re all ASCII. The write happens through a direct memcpy() in Node’s C++ layer, bypassing V8’s property access overhead entirely. If you had multiple threads accessing this ArrayBuffer through SharedArrayBuffer, this write would NOT be atomic - you could get partially written data visible to other threads.

// View 2: An Int32Array to read a 32-bit (4-byte) integer at the start
const intView = new Int32Array(messageArrayBuffer, 0, 1);
console.log("Initial integer value:", intView[0]); // 0

So far, so good. The two views point to different, non-overlapping parts of the same memory. But what happens if a bug - a simple off-by-one error in an offset calculation - causes them to overlap?

// Let's introduce a bug. We accidentally create the string view at offset 0.
const buggyStringView = Buffer.from(messageArrayBuffer, 0, 8);

This view now overlaps with intView. They both reference bytes 0-3 of the ArrayBuffer. This is completely legal in JavaScript - there’s no runtime check for overlapping views. V8 doesn’t track view relationships or enforce any mutual exclusion. From the engine’s perspective, these are just two independent objects that happen to point to the same memory region. This design choice prioritizes performance over safety, putting the burden of correctness entirely on the developer.

// We write some status update.
buggyStringView.write("CANCELED");

This write operation just triggered silent data corruption. The string “CANCELED” gets encoded to bytes [0x43, 0x41, 0x4E, 0x43, 0x45, 0x4C, 0x45, 0x44]. The first four bytes - ‘C’, ‘A’, ‘N’, ‘C’ - overwrite the exact same memory that intView[0] reads from. There’s no warning, no error, no exception. The write succeeds, and your integer data is now garbage. In production, I’ve seen this pattern corrupt packet headers, destroy cryptographic nonces, and cause financial calculation errors that took weeks to trace.

// Now, let's go read our integer from the other view.
console.log("Corrupted integer value:", intView[0]); // 1128353859
// That's the 32-bit integer representation of the bytes for "CANC"

This is the error that fails without throwing exceptions. No exceptions were thrown. The code ran “successfully.” But you just overwrote a critical integer field by writing a string into what you thought was a separate part of the message. The bug isn’t in the write operation; it’s in the creation of the view that aliased memory it shouldn’t have. This scenario teaches you a vital lesson with cold, hard precision: when you share an ArrayBuffer across multiple views, you are the memory manager. You must be absolutely certain of your offsets and lengths, or you will face the nightmare of debugging silent data corruption.

When Views Share Memory (and When They Don’t)

By now, you should be healthily paranoid about shared memory. The rule of thumb is: if an operation doesn’t explicitly say it “copies” or “allocates,” assume it shares memory. Let’s build a mental map of the common Buffer and TypedArray operations and put them into “View” (shares memory) or “Copy” (allocates new memory) buckets.

Operations that Create Views (Zero-Copy):

• Buffer.prototype.slice(start, end)
• Buffer.prototype.subarray(start, end)
• new Uint8Array(arrayBuffer, byteOffset, length) (and all other TypedArray constructors that take an ArrayBuffer)
• Buffer.from(arrayBuffer, byteOffset, length)

These are your high-performance, high-risk tools. They are incredibly fast for creating sub-sections of existing data for temporary processing. The key word here is temporary. If the view you create is short-lived and goes out of scope quickly, you get all the performance benefits without the memory retention risk.

Operations that Create Copies (Allocating New Memory):

• Buffer.alloc(size)
• Buffer.from(string)
• Buffer.from(array)
• Buffer.from(buffer)
• Buffer.prototype.copy() (the method itself, which copies into an existing buffer)
• Uint8Array.prototype.slice(start, end) (Note the critical difference! TypedArray.slice() copies, whereas Buffer.slice() views.)

This last point is a landmine. I’ve seen it burn senior engineers. Because Buffer is a Uint8Array, it inherits both methods, but Buffer overrides slice() to create views instead of copies. If you were to somehow call the Uint8Array prototype’s slice method directly on a buffer (via Uint8Array.prototype.slice.call(buf, ...)), you’d get a copy instead of a view. This inconsistency between Buffer.slice() and TypedArray.slice() is a design quirk that can cost you your sanity. The Node.js team has gone to great lengths to make Buffer’s behavior internally consistent, but this fundamental difference with standard TypedArrays remains.

Let’s look at a scenario where the distinction is crucial. You’re reading a large file, say a 1GB video file, and you just need to parse the first 1KB for metadata.

import { readFileSync } from "fs";
const videoBuffer = readFileSync("large_video.mp4"); // 1GB in memory

That readFileSync call just blocked your event loop while Node.js read 1GB from disk. Under the hood, libuv opens the file with open(), gets the file size with fstat(), allocates a buffer of that size, and then reads the entire file with a single read() syscall (or multiple reads for very large files). The entire 1GB is loaded into a single contiguous ArrayBuffer. Your process’s RSS just jumped by 1GB, and the OS might have even started swapping other processes to disk to make room. This synchronous operation can freeze your server for several seconds on slow disks.

// The WRONG way for long-term storage
const metadataView = videoBuffer.slice(0, 1024);

This creates a 72-byte Buffer object that holds a reference to the entire 1GB ArrayBuffer. The view’s retained size is 1GB, but its shallow size is just 72 bytes. If you pass this to a cache, a global variable, or any long-lived data structure, you’ve just created a memory leak. The garbage collector sees the reference chain: your cache → metadataView → videoBuffer’s ArrayBuffer, and concludes the entire 1GB must be kept alive. I’ve debugged production systems where hundreds of these tiny views collectively retained tens of gigabytes of memory.

// If we pass metadataView to another part of our app that holds onto it...
// we are keeping the entire 1GB videoBuffer in memory just for that 1KB view.

The correct approach here, if you need to hold onto that metadata for any length of time, is to perform a strategic copy.

import { readFileSync } from "fs";
const videoBuffer = readFileSync("large_video.mp4"); // 1GB in memory
 
// The RIGHT way for long-term storage
const metadataCopy = Buffer.alloc(1024);
videoBuffer.copy(metadataCopy, 0, 0, 1024);

The Buffer.alloc(1024) allocates exactly 1024 bytes from Node’s buffer pool (since it’s under 4KB). This memory is zeroed for security. The copy() operation then triggers a highly optimized memcpy() in C++ that can move data at several GB/s on modern hardware. The CPU’s SIMD instructions copy 32 or 64 bytes per cycle, making this 1KB copy complete in microseconds. Most importantly, metadataCopy has its own independent ArrayBuffer with no reference to the original 1GB buffer.

// Now, videoBuffer can be garbage collected as soon as it goes out of scope.
// We've spent a few microseconds copying 1KB to save 1GB of memory.

This decision framework - “Is this data short-lived or long-lived?” - is the key to wielding views and copies effectively. For temporary, in-function processing, views are your best friend. For data that needs to be stored, cached, or passed between different parts of your application, an explicit copy is your insurance policy against massive memory leaks.

Copy Semantics and Buffer.copy()

So, we’ve established that sometimes you absolutely need a copy. The primary tool for this in Node.js is Buffer.prototype.copy(). It’s a low-level, high-performance method designed to be the memcpy of the JavaScript world. Its signature is buf.copy(targetBuffer, targetStart, sourceStart, sourceEnd).

It’s important to note that copy() writes into an existing targetBuffer. You must allocate the destination buffer yourself before you call it. This gives you fine-grained control but also adds a step to the process.

const source = Buffer.from("abcdefghijklmnopqrstuvwxyz");
const target = Buffer.alloc(10);

The Buffer.from(string) encodes the 26-character alphabet into 26 bytes of UTF-8 (all ASCII, so one byte per character). Node allocates this from its buffer pool since it’s under 4KB. The Buffer.alloc(10) creates another small buffer, also from the pool but from a different offset. These two buffers might actually be slices of the same underlying 8KB pool slab, but they’re non-overlapping regions with independent lifecycles.

// Copy the first 10 bytes from source into target.
source.copy(target, 0, 0, 10);
console.log(target.toString()); // 'abcdefghij'

This copy() operation resolves to a single memcpy(target_ptr + 0, source_ptr + 0, 10) call in C++. Modern CPUs optimize this with SIMD instructions, moving multiple bytes per cycle. The operation completes in nanoseconds for such small buffers. The data is physically duplicated - changes to source won’t affect target and vice versa.

// Copy 'klmno' from the source into the middle of the target.
source.copy(target, 3, 10, 15); // target, targetStart, sourceStart, sourceEnd
console.log(target.toString()); // 'abcklmnohij'

The performance of Buffer.copy() is heavily optimized in Node’s C++ core. For copying data between buffers, it will almost always be faster than any manual, byte-by-byte loop you could write in JavaScript. Memory profiling results show that the time taken is directly proportional to the number of bytes copied, and the constant factor is very low.

However, there’s a more convenient way to create a copy that many people reach for: Buffer.from(buffer). As we touched on earlier, this specific overload of Buffer.from() is explicitly a copy operation.

const original = Buffer.from("This is the original buffer");
// Create a new buffer with a copy of the original's data.
const clone = Buffer.from(original);

The Buffer.from(buffer) constructor is deceptive in its simplicity. Internally, it allocates a new ArrayBuffer of the exact same size as the original (28 bytes here), then performs a memcpy() of the entire contents. This happens in Node’s C++ layer through the node::Buffer::Copy() function. The new buffer is completely independent - it has its own backing store with no references to the original. This is crucial for memory isolation and preventing the retention issues we’ve been discussing.

clone.write("That"); // Modify the clone
console.log(original.toString()); // 'This is the original buffer'
console.log(clone.toString()); // 'That is the original buffer'

Internally, Buffer.from(buffer) is essentially doing an alloc and a copy for you. It’s syntactic sugar for the two-step process. In most cases, the performance difference is negligible, and the convenience of a one-liner often wins. However, if you are in an extremely hot path where you need to reuse an existing destination buffer to avoid allocation overhead (a technique called buffer pooling), then using Buffer.copy() directly is the way to go.

Knowing when to copy is the art. The science is knowing how. The rule is simple: if the data needs to outlive its original, massive parent buffer, you must give it a new home. Allocate a new buffer of the exact size you need and copy the data into it. This breaks the link to the parent, allowing the garbage collector to do its job.

It’s the solution we eventually implemented for our log parser. Instead of storing the slice, we did this:

function getSessionId(logBuffer) {
  // Instead of a view, we make an explicit copy.
  const sessionId = Buffer.alloc(16);
  logBuffer.copy(sessionId, 0, 0, 16);

This pattern costs us 16 bytes of allocation plus a few nanoseconds for the memcpy(). The Buffer.alloc(16) gets memory from Node’s buffer pool (it’s under 4KB), and the memory is zeroed for security. The copy() operation then moves exactly 16 bytes from the source. The crucial difference: sessionId has its own ArrayBuffer with no reference to logBuffer. When this function returns and logBuffer goes out of scope, the entire multi-megabyte buffer can be immediately garbage collected. Your heap profiler will show 16-byte buffers with 16-byte retained sizes - exactly what you’d expect.

  // Now, storing 'sessionId' retains only 16 bytes, not the whole logBuffer.
  return sessionId.toString("utf-8");
}

This one-line change from slice to alloc+copy saved us gigabytes of RAM. It might seem less “efficient” on the surface because it’s doing more work (allocating and copying), but in the grand scheme of the system’s health, it was infinitely more efficient.

SharedArrayBuffer and Cross-Thread Views

The plot thickens when we introduce Node.js worker threads. For a long time, JavaScript was single-threaded. If you wanted to do CPU-intensive work, you’d block the main event loop, and your application’s performance would grind to a halt. Worker threads changed the game, allowing for true parallelism. But how do you share data between threads without expensive serialization and copying?

The answer is SharedArrayBuffer (SAB). A regular ArrayBuffer cannot be accessed by multiple threads. If you pass one to a worker, a copy is made. A SharedArrayBuffer, however, is a special type of ArrayBuffer whose underlying memory block can be referenced and manipulated by multiple threads simultaneously.

This is where our understanding of views becomes paramount. You can create a SharedArrayBuffer on the main thread, pass it to a worker thread, and then both threads can create TypedArray or Buffer views over that same block of memory.

// main.js
import { Worker } from "worker_threads";
 
// Create a SharedArrayBuffer of 4 bytes.
const sab = new SharedArrayBuffer(4);

This allocates 4 bytes of memory that can be simultaneously accessed by multiple JavaScript contexts. Unlike regular ArrayBuffer, this memory is mapped into multiple address spaces using platform-specific mechanisms (shared memory on POSIX, memory-mapped files on Windows). The allocation is page-aligned for atomic operations support. V8 tracks this specially - it can’t move or compact this memory during garbage collection because multiple isolates might be accessing it simultaneously.

// Create a view over it on the main thread.
const mainThreadView = new Int32Array(sab);
mainThreadView[0] = 123; // Initial value

This write is NOT atomic by default. On x86-64, a 32-bit aligned write is atomic at the hardware level, but JavaScript makes no such guarantees. Without using Atomics.store(), this write could be torn - another thread might see a partially written value. The value 123 is written directly to the shared memory without any synchronization primitives, meaning there’s no guarantee when other threads will see this update due to CPU cache coherency delays.

const worker = new Worker("./worker.js");
worker.postMessage({ sab });

The postMessage doesn’t copy the SharedArrayBuffer - it transfers a reference to the same memory. Both threads now have access to the same 4 bytes of RAM. This is fundamentally different from regular ArrayBuffer messaging, which clones the data. The worker thread gets its own Int32Array view, but it points to the exact same memory pages as the main thread’s view.

worker.on("message", () => {
  console.log("Main thread sees:", mainThreadView[0]); // Output: 456
});

// worker.js
import { parentPort } from "worker_threads";
 
parentPort.on("message", ({ sab }) => {
  const workerView = new Int32Array(sab);
  console.log("Worker sees initial value:", workerView[0]); // Output: 123

The worker immediately sees the value 123 that was written by the main thread. But this isn’t guaranteed without proper synchronization. Due to CPU cache coherency protocols, there could be a delay between when one thread writes and when another thread sees the update. On weakly-ordered memory architectures (like ARM), you might not see the update at all without memory barriers.

  // Modify the memory from the worker thread.
  workerView[0] = 456;
  parentPort.postMessage("done");
});

This is mind-bendingly powerful. We just modified memory in one thread and saw the result instantly in another, with zero copying and zero serialization overhead. This is the foundation for high-performance parallel computing in Node.js. You can have a worker thread performing complex calculations on a large dataset while the main thread reads the results as they become available.

However, this introduces a whole new class of problems: race conditions. Since two threads can read and write to the same memory at the same time, you need synchronization primitives to coordinate access. This is where Atomics come in. The Atomics object provides methods for performing atomic reads, writes, and read-modify-write operations on SharedArrayBuffer views. These operations are guaranteed to complete without being interrupted by another thread, preventing data corruption.

Using SharedArrayBuffer is an advanced technique, and it brings the challenges of concurrent programming directly into your Node.js application. But understanding that it’s all built on the same foundation of views (TypedArrays) over a shared block of memory (SharedArrayBuffer) demystifies the magic. It’s the same principle as slice and subarray, just extended across the thread boundary.

Memory Retention and Garbage Collection

We’ve talked a lot about memory retention, but let’s formalize it. This is the mechanism behind our 10GB log parser leak. In a garbage-collected language like JavaScript, an object is kept in memory as long as there is a reachable reference to it from the “root” set (e.g., the global object, the current call stack).

When you create a Buffer view with slice() or subarray(), you create two objects with a relationship.

1. The View Object: The new Buffer instance (userIdSlice). It’s a small object on the V8 heap.
2. The Parent Buffer Object: The original Buffer (massiveBuffer), which holds the reference to the large external ArrayBuffer.

The view object maintains an internal reference to its parent buffer. According to V8’s memory model, as long as the view object is reachable, its parent buffer is also considered reachable. The garbage collector sees the reference from userIdSlice to massiveBuffer and says, “Nope, can’t collect massiveBuffer yet, someone still needs it.” It has no idea you only care about 16 bytes out of the 50 megabytes. It just sees a valid reference and honors it.

This is why the heap snapshot was so confusing. The profiler correctly identified that the userIdSlice objects were small. But it also has a concept of “retained size” vs. “shallow size.”

• Shallow Size: The size of the object itself. For our slices, this was tiny, just a few dozen bytes for the JavaScript object wrapper.
• Retained Size: The size of all memory that is being kept alive solely because this object exists. For our slices, the retained size was enormous, because they were the only thing keeping the 50MB parent buffers from being garbage collected.

The heap snapshot showed 890MB retained by 10KB of slices. It looked like an accounting error, but it was the brutal truth of view semantics. Once we understood this, the fix was obvious: we had to sever the link between the small piece of data we needed and its giant parent. The only way to do that is with a copy.

// Before: A view that retains the parent
function createView(parent) {
  return parent.slice(0, 10);
}

This function returns a view that maintains a strong reference to parent’s ArrayBuffer. If parent is 10MB, your 10-byte view keeps all 10MB alive. The V8 garbage collector traces the reference chain and marks the entire parent as reachable. This pattern is responsible for the majority of Buffer-related memory leaks in production Node.js applications.

// After: A copy that lets the parent be freed
function createCopy(parent) {
  return Buffer.from(parent.slice(0, 10));
}

The Buffer.from(buffer) constructor call is the key. It takes the 10-byte view created by slice(), allocates a new 10-byte ArrayBuffer, copies the data into it, and returns a new Buffer object that points to this new, small allocation. The original parent buffer is no longer referenced by the returned object, and the temporary view created by slice() can be immediately collected. This pattern, Buffer.from(buf.slice(...)), is a common and effective way to create a “trimmed” copy of a small section of a large buffer. It’s the antidote to view-based memory retention. After enough production incidents, you learn to spot a missing copy like a hawk.

Binary Protocol Parsing with Views

Now let’s apply these concepts to a real-world scenario: parsing a custom binary protocol. This is common in high-performance systems, IoT, and gaming, where the overhead of JSON or XML is unacceptable. A binary protocol defines a strict layout of data in a sequence of bytes.

For example, a message might be structured like this:

• Bytes 0-1: Message Type (Uint16)
• Bytes 2-3: Message Length (Uint16)
• Byte 4: Flags (Uint8)
• Bytes 5-20: Session ID (16-byte UUID string)
• Bytes 21-end: Payload (raw bytes)

A naive approach to parsing this would involve a lot of slicing and copying.

// Naive, copy-heavy parsing
function parseMessageWithCopies(buffer) {
  const messageType = buffer.slice(0, 2).readUInt16BE();
  const messageLength = buffer.slice(2, 4).readUInt16BE();
  const flags = buffer.slice(4, 5).readUInt8();

Each of these lines creates a temporary view just to read a primitive value. The slice(0, 2) creates a Buffer object (72 bytes on heap), then readUInt16BE() reads two bytes and converts them from big-endian to native endianness. The view is immediately discarded but not before V8 allocates it, tracks it, and eventually garbage collects it. With thousands of messages per second, you’re creating massive GC pressure for no reason. These intermediate views serve no purpose - you could read directly from the original buffer.

  const sessionId = buffer.slice(5, 21).toString("utf-8");
  const payload = buffer.slice(21); // This slice could be huge!
  return { messageType, messageLength, flags, sessionId, payload };
}

This code works, but it’s creating five new Buffer objects for every single message. If you’re processing thousands of messages per second, those allocations add up, putting pressure on the garbage collector and slowing down your application.

A zero-copy approach, on the other hand, leverages views to read the data without creating copies of the data itself. We can use the offset-based read methods directly on the main buffer, or create TypedArray views for more complex data types.

// Efficient, zero-copy parsing
function parseMessageWithViews(buffer) {
  const messageType = buffer.readUInt16BE(0); // Read directly from offset
  const messageLength = buffer.readUInt16BE(2);
  const flags = buffer.readUInt8(4);

These direct reads are lightning fast. No intermediate objects, no allocations, no GC pressure. The readUInt16BE() method calculates the memory address (buffer base + offset), reads two bytes, and performs the endianness conversion in optimized C++ code. The entire operation stays in CPU cache. For high-frequency parsing, this difference between creating a view then reading versus reading directly can mean the difference between 10,000 and 100,000 messages per second.

  // For the session ID and payload, we create views
  const sessionIdView = buffer.subarray(5, 21);
  const payloadView = buffer.subarray(21);
  return { messageType, messageLength, flags, sessionIdView, payloadView };
}

This version is significantly more efficient. It creates no intermediate copies for the primitive number types. It creates two views for the session ID and payload, but no data is duplicated. The sessionIdView and payloadView are lightweight pointers back into the original message buffer.

This is the pattern that finally saved us 8GB of RAM in our TCP service. We use a view because the processing is temporary. If we needed to store the sessionIdView or payloadView long-term (e.g., in a cache or a request map), we would be right back in the memory retention trap. The contract of a function like this should be clear: it returns views that are only valid for the immediate scope of processing. If a caller needs to persist that data, it is the caller’s responsibility to perform the copy.

This is a critical design pattern for high-performance libraries. A parsing function should perform zero-copy operations and return views. The consumer of the function then decides whether the data is short-lived (use the view directly) or long-lived (create a copy). This separates concerns and puts the memory management decision in the hands of the code that has the most context.

Platform Endianness and TypedArray Views

When you’re working with binary data that comes from the network or a file, you can’t escape the concept of endianness. It refers to the order in which a multi-byte number (like a 16-bit or 32-bit integer) is stored in memory.

• Big-Endian (BE): The most significant byte comes first. This is common in network protocols (often called “network byte order”). The number 0x12345678 would be stored as 12 34 56 78.
• Little-Endian (LE): The least significant byte comes first. This is the native format for most modern CPUs, including Intel and AMD x86-64. The same number would be stored as 78 56 34 12.

Forgetting about endianness will lead to completely garbled data when reading binary protocols. Node.js Buffers provide explicit methods for this: readUInt16BE, readUInt16LE, writeInt32BE, etc. These are your safest bet when you know the exact endianness of the data you’re parsing.

But what if you’re using TypedArray views directly on an underlying ArrayBuffer? This is where it gets tricky. TypedArrays (like Int16Array, Float64Array) read and write data using the host system’s native endianness. On my x86 laptop, that’s little-endian. If I create an Int16Array view over a buffer that contains big-endian network data, I will read garbage.

// A 16-bit integer, 258, in Big-Endian format is [0x01, 0x02]
const networkBuffer = Buffer.from([0x01, 0x02]);
 
// Using the Buffer method correctly:
console.log(networkBuffer.readUInt16BE(0)); // 258, Correct!

The readUInt16BE() method explicitly handles endianness conversion. It reads bytes at positions 0 and 1, then combines them as (buffer[0] << 8) | buffer[1], which correctly interprets big-endian data regardless of platform endianness. This happens in Node’s C++ layer with optimized byte-swapping instructions like bswap on x86 or rev on ARM when needed.

// Using a TypedArray view on a little-endian machine:
const int16View = new Int16Array(networkBuffer.buffer, networkBuffer.byteOffset, 1);
console.log(int16View[0]); // 513, Incorrect! (It read 0x0201)

This is a disaster waiting to happen. How do we control endianness when using generic TypedArray views? The answer is the DataView object. A DataView is a low-level interface for reading and writing data to an ArrayBuffer that lets you explicitly specify the endianness for each operation. It’s more verbose than using a TypedArray, but it gives you absolute control.

const arrayBuffer = new ArrayBuffer(4);
const dataView = new DataView(arrayBuffer);
 
// Write a 32-bit integer in Big-Endian format
dataView.setInt32(0, 123456789, false); // false for big-endian

The setInt32() with false writes the bytes as [0x07, 0x5B, 0xCD, 0x15] - most significant byte first. DataView internally handles the byte ordering regardless of platform endianness. On a little-endian system, it reverses the bytes before writing. On a big-endian system, it writes them directly. This abstraction layer costs a few CPU cycles but guarantees correct behavior across all platforms.

// Read it back in Little-Endian format (will be wrong)
console.log(dataView.getInt32(0, true)); // Some garbage number
 
// Read it back correctly in Big-Endian format
console.log(dataView.getInt32(0, false)); // 123456789

This DataView cast seemed fine until it corrupted everything. In one of our services, a developer had used a Float32Array to quickly parse a list of floating-point numbers from a network stream, assuming the host endianness matched the network endianness. It worked fine on their development machine. But when deployed to a different cloud architecture with a different endianness (a rarity these days, but it happens), the service started reading completely nonsensical data. The fix was to replace the direct Float32Array view with a loop that used a DataView to read each float with the correct, explicitly-stated endianness. It was a painful reminder that hidden assumptions about the execution environment are a recipe for production failures. When in doubt, be explicit. Use Buffer’s BE/LE methods or use a DataView.

Production Patterns for Zero-Copy

After experiencing these production issues, my team developed a set of thoroughly validated patterns for working with buffers. These aren’t just theoretical best practices; they are essential patterns for production systems.

Pattern 1: The Temporary View for Synchronous Processing

This is the most common and safest use of zero-copy. When you need to process a chunk of a larger buffer within a single function scope, a view is perfect.

// Strategic view for temporary processing
function processChunk(largeBuffer, offset, length) {
  const view = largeBuffer.subarray(offset, offset + length);
  const result = performComplexCalculation(view);

This pattern is safe because the view’s lifetime is scoped to the function execution. The view is created, used, and becomes unreachable when the function returns. V8’s escape analysis can often optimize this further - if the view doesn’t escape the function, it might not even allocate the Buffer object on the heap, keeping everything in registers. The key insight: synchronous, function-scoped views are nearly always safe from retention issues.

  // Once the function returns, 'view' is eligible for GC.
  return result;
}

We use a view here because the processing is temporary and synchronous. The view doesn’t escape the function’s scope. If we needed to store this chunk long-term or use it in an asynchronous callback, we’d copy instead. Here’s why that decision matters…

Pattern 2: The Defensive Copy for Asynchronous Operations and Storage

Any time buffer data needs to cross an asynchronous boundary or be stored in a collection, you must assume it needs to be copied. The original buffer might be reused or garbage collected by the time your callback executes.

const longLivedCache = new Map();
 
function processAndCache(dataBuffer) {
  const key = dataBuffer.subarray(0, 16); // Temporary view for the key
  const value = dataBuffer.subarray(16); // Temporary view for the value

These views are created for immediate processing. They’re lightweight - just 72 bytes each on the heap - but they hold references to dataBuffer’s entire ArrayBuffer. If we stored these views directly in our cache, we’d create a memory leak. The entire dataBuffer would be retained for as long as the cache entry exists, which could be hours or days in a production system.

  // Before storing, we make a defensive copy.
  const storedValue = Buffer.from(value);
  // The key is converted to a string, which is implicitly a copy.
  longLivedCache.set(key.toString("hex"), storedValue);
}

Here, we create views to initially parse the buffer. But the moment we decide to put the value into our longLivedCache, we immediately create a copy. This ensures our cache entry is self-contained and doesn’t unexpectedly hold a reference to a much larger dataBuffer.

Pattern 3: The Parser Protocol (Views out, Copies in)

This is the library author’s pattern. Write parsing functions that are purely zero-copy and return views. Document clearly that the returned values are views and may be invalidated if the original buffer changes.

/**
 * Parses a message header from a buffer.
 * WARNING: Returns a view into the original buffer. Do not store
 * the returned value long-term without creating a copy.
 * @param {Buffer} buffer The source buffer.
 * @returns {{id: Buffer, body: Buffer}} Views for id and body.
 */
function parseHeader(buffer) {
  return {
    id: buffer.subarray(0, 8),
    body: buffer.subarray(8),
  };
}

This function contract is critical. The JSDoc explicitly warns that returned values are views. This shifts the memory management decision to the caller, who has more context about data lifetime. The function itself is pure and fast - no allocations beyond the two small Buffer objects for the views. This pattern scales to millions of operations per second because it does the minimum necessary work.

// Consumer of the function decides the memory strategy
const rawMessage = getMessageFromNetwork();
const { id, body } = parseHeader(rawMessage);
 
// I need to use 'id' later, so I'll copy it.
const savedId = Buffer.from(id);
// I'm just logging the body, so the temporary view is fine.
logBodyPreview(body);

This pattern provides maximum performance for consumers who can handle the data immediately and maximum safety for those who need to store it, by forcing them to be explicit about their intentions.

Debugging Memory Issues with Views

When you suspect a view-related memory leak, your primary tool is the heap snapshot. You can generate these using the Chrome DevTools for Node.js or programmatically with modules like heapdump.

The process is usually:

1. Take a heap snapshot when your application is in a stable, low-memory state.
2. Apply a load to your application that you suspect triggers the leak.
3. Take a second heap snapshot.
4. Take a third snapshot after some more time to confirm the growth trend.

In the snapshot viewer, you’ll want to use the “Comparison” view to see what objects were allocated between snapshots. When debugging our log parser, we saw a massive increase in the number of Buffer objects.

When you click on one of these Buffer objects, the profiler will show you its properties. The key is to look for the internal reference to the parent buffer. In Chrome DevTools, this is often shown under a property like [[backing_store]] or by inspecting the object’s retainers. You’ll see your tiny 16-byte Buffer slice, and in its retainer chain, you will find the massive multi-megabyte parent Buffer it’s keeping alive.

Another powerful technique is to use process.memoryUsage(). This gives you a quick overview of your application’s memory consumption.

• rss (Resident Set Size): The total memory the process is using in RAM. This is the number you see in top or Task Manager. A runaway rss is a strong indicator of a leak, often from external memory like buffers.
• heapTotal and heapUsed: The memory managed by the V8 heap.
• external: The memory used by C++ objects bound to JavaScript objects, which prominently includes Buffers.
• arrayBuffers: (Added in Node.js 13.9.0) Memory used by ArrayBuffers and SharedArrayBuffers, including all Node.js Buffers.

In our leak, heapUsed was growing slowly, but external and rss were exploding. This told us the leak wasn’t in standard JavaScript objects but in the external memory managed by Node.js - a classic signature of a Buffer retention problem.

After profiling, we discovered our views were aliasing each other in another service. We had a circular buffer implementation where we would wrap around by creating a view. A bug in our offset logic caused a new view to overlap slightly with an old view, inadvertently keeping the old view (and thus the entire buffer) alive far longer than intended. The heap snapshot was the only way to visualize that chain of references.

Best Practices for Buffer Manipulation

If I could distill all this pain and suffering down into a set of guiding principles, it would be these.

After enough production incidents, you learn to profile memory retention before deploying any new code that heavily manipulates buffers. You don’t just test for correctness; you test for memory behavior under load. You start using views for temporary, synchronous processing but reach for explicit copies for any data that is long-lived, asynchronous, or stored in a collection. You internalize the parent-child relationship between a view and its underlying buffer because you’ve debugged the alternative at 3 AM. You test with memory profilers because you’ve been burned by assumptions one too many times.

You document your function signatures relentlessly. If a function returns a view, you scream it from the rooftops in the JSDoc comments. You make it impossible for the next developer to accidentally misuse your API and create a leak. You learn to recognize the code smell of a slice() or subarray() whose result is being assigned to a variable with a wider scope, like an object property or a module-level variable. You see that and you immediately ask, “Shouldn’t that be a copy?”

And most importantly, you treat every zero-copy operation with suspicion. You don’t see it as a free performance boost; you see it as a powerful tool with significant risks. You ask yourself, “What is the lifetime of the data I’m creating? And what is the lifetime of the data I’m referencing?” If those two lifetimes are different, a copy is almost always the right answer.

Memory Profiling Data

Here is a sample of the kind of data we collected during our investigation. The test creates 100,000 small objects derived from a single 50MB buffer.

Test Scenario 1: Using slice() (creating views)

const largeBuffer = Buffer.alloc(50 * 1024 * 1024);
const views = [];
for (let i = 0; i < 100000; i++) {
  views.push(largeBuffer.slice(0, 10));
}
// At this point, take a heap snapshot.

• process.memoryUsage() Output:
  • rss: ~78 MB
  • heapUsed: ~8 MB
  • external: ~50.5 MB
• Heap Snapshot Analysis:
  • Shallow size of all Buffer objects in the views array: ~800 KB (100,000 * ~8 bytes/object)
  • Retained size: ~50 MB. The entire largeBuffer is retained by the views.

Test Scenario 2: Using a strategic copy

const largeBuffer = Buffer.alloc(50 * 1024 * 1024);
const copies = [];
for (let i = 0; i < 100000; i++) {
  // Creating a copy for each item
  copies.push(Buffer.from(largeBuffer.slice(0, 10)));
}

Each iteration creates a temporary view with slice(0, 10), then immediately copies it with Buffer.from(). The temporary view is eligible for collection as soon as Buffer.from() completes. The copy has its own 10-byte ArrayBuffer with no reference to largeBuffer. After the loop, we have 100,000 independent 10-byte buffers totaling ~1MB of memory, and largeBuffer can be garbage collected, freeing 50MB.

// At this point, 'largeBuffer' can be garbage collected.

• process.memoryUsage() Output (after GC is triggered):
  • rss: ~32 MB
  • heapUsed: ~9 MB
  • external: ~1.5 MB
• Heap Snapshot Analysis:
  • The 50MB largeBuffer is gone.
  • The copies array holds 100,000 small Buffer objects, each with its own 10-byte backing store. Total external memory is approximately 1MB (100,000 * 10 bytes) plus some overhead.

These measurements clearly quantify the trade-off. The view-based approach used less CPU upfront but retained 50MB of memory it didn’t need. The copy-based approach used slightly more CPU in the loop but resulted in a vastly smaller memory footprint.

Closing - Respecting Memory Semantics

I still remember the junior engineer who, after we’d fixed the log parser leak, asked with genuine curiosity, “So why don’t we just use copies everywhere? It seems safer.” It’s a fair question. The answer is that real engineering is about making informed trade-offs. We could copy everything, and our applications would be simpler to reason about but also slower and less efficient. We could have services that use twice the CPU and memory they need to, and in a large-scale system, that’s a cost you can’t afford.

The goal isn’t to fear zero-copy operations; it’s to respect them. It’s to understand that shared memory is a mechanism with both powerful advantages and serious risks. When you create a view, you are making a promise to the runtime - a promise that you understand the lifecycle of both the view and its parent. Mastery is about understanding the consequences of each call. It’s about looking at const view = buf.slice(0, 10) and not just seeing a line of code, but seeing the internal reference it creates back to the parent buffer and asking, “Is that a reference I’m prepared to manage?” When you can answer that question instinctively, you’ll never look at memory the same way again. You’ll have gained deep operational knowledge from production experience.