Directory inventory
Create a scanner that walks the watched tree, applies ignore policy, records relative paths, counts files and directories, and writes repeatable scan reports.
Lab 05 / Node Runtime Labs
Atomic File Watcher + Build Cache.
Build a watch-mode CLI that treats filesystem events as prompts to verify state. It scans a directory, fingerprints inputs, persists cache data, normalizes noisy saves, queues rebuilds, and writes reports that explain the session.
Watcher commandsIncremental build checks
- 01Scan
$ npm run scan -- watched --metadata --hashCaptures file identity, content hash, and cache input.
- 02Watch
$ npm run watch -- watchedStarts the native watcher path.
- 03Poll
$ npm run watch -- watched --poll --poll-interval 1000Exercises the fallback watcher with measured work.
- 04Batch
$ npm run watch -- watched --debounce 50 --concurrency 2 --rebuild-delay-ms 500Makes rebuild batching and queue limits observable.
- 05Symlinks
$ npm run scan -- watched --metadata --hash --follow-symlinksCompares link treatment with the default scan.
15Phases
8.5/10Difficulty
0 depsPackage installs
What this lab builds
The finished tool verifies filesystem state.
Metadata and hash cache
Store file type, size, timestamps, permissions, device and inode identity, symlink targets, SHA-256 hashes, and persistent cache data.
Watch event pipeline
Receive native watcher events, add polling fallback, normalize uncertainty, group atomic writes, debounce bursts, and verify state through rescans.
Incremental rebuild system
Classify added, changed, deleted, and unchanged paths, queue rebuild jobs with bounded concurrency, skip cache hits, clean deleted outputs, and report the session.
Complete phase plan
Every phase in Lab 05.
The build starts with a tiny command surface and ends with a complete watch-session report. Each phase adds one observable mechanism and one artifact to inspect.
00
Create the project
Create the watcher project shell and a CLI that accepts a directory argument.
- Create a private ESM package with an engine requirement for Node.js v24 or newer.
- Add npm scripts for scan, watch, demo-watch, and simulate-atomic.
- Create src, watched, .cache, and reports so source files, watched input, persistent state, and evidence stay separate.
- Build the CLI entry point that dispatches scan and watch commands from raw argv strings.
- Print help output with the planned metadata, hash, poll, debounce, concurrency, rebuild delay, symlink, and cache flags.
- Accept a watched directory, default to watched, and reject unknown commands, flags, and invalid directories with a failure exit code.
A project shell with package metadata, folder boundaries, help output, directory parsing, and strict input handling.
01
Scan the directory tree
Build the initial file inventory for the watched tree.
- Create src/scan.js and expose scanTree(root, options) as the shared inventory boundary.
- Walk nested directories with fs.promises.readdir and Dirent objects, then sort entries for deterministic output.
- Ignore .git, node_modules, .cache, and reports by default while recording ignored paths as evidence.
- Store portable relative POSIX-style paths instead of machine-specific absolute paths.
- Count files, directories, ignored paths, and scan errors during traversal.
- Write reports/scan.json with root, generatedAt, counts, files, ignored paths, and errors.
A scan report that describes every regular file under watched plus ignored paths and traversal errors.
02
Record file metadata
Store size, mtime, mode, inode, device, and file type for each scanned entry.
- Add the --metadata flag so extra filesystem calls are opt-in and visible.
- Use lstat for each entry so symlinks are reported as links by default.
- Record size, mtimeMs, ctimeMs, birthtimeMs, mode, permissions, dev, ino, and nlink.
- Create one file type classifier for files, directories, symlinks, FIFOs, sockets, block devices, character devices, and unknown entries.
- Read symlink targets and keep target strings separate from link metadata.
- Record metadata failures with path, code, and phase so races do not crash the scan.
A metadata-rich inventory that can separate content changes, permission changes, identity changes, and symlink state.
03
Add content hashing
Compute a content hash for every regular file.
- Add the --hash flag so content reads are explicit in scan and watch runs.
- Create src/hash.js with a helper that hashes one file and returns algorithm, digest, and byte count.
- Stream file bytes into crypto.createHash('sha256') instead of buffering whole files.
- Hash regular files only and skip directories, symlinks, and special files by policy.
- Preserve path-sorted report output even if hashing later runs with concurrency.
- Record ENOENT, EACCES, EPERM, and other hash failures as report errors.
A content fingerprint layer that gives the cache a stable input hash for ordinary files.
04
Build the persistent cache
Save the latest inventory to .cache/index.json and load it on the next run.
- Create src/cache.js with loadCache(cachePath) and saveCacheAtomic(cachePath, inventory).
- Define a cache schema with version, root, createdAt, updatedAt, options, and filesByPath.
- Load the previous cache before scanning so diffing has the prior state.
- Save through a same-directory temporary file and rename over the target cache path.
- Support --cache <path> and --no-cache to make cache location and bypass mode observable.
- Report cache.path, cache.loaded, cache.loadMs, and cache.saveMs in reports/scan.json.
An atomic cache file that survives between runs and becomes the previous inventory for later diffs.
05
Detect added, changed, and deleted files
Compare the current inventory against the cached inventory and classify paths.
- Create src/diff.js and export diffInventories(previous, current).
- Compare filesByPath keys to classify added and deleted paths.
- Compare hash, size, mtimeMs, ctimeMs, mode, identity, and type to explain changed paths.
- Count unchanged paths and keep unchanged details compact.
- Write reports/diff.json with counts, added, changed, deleted, and unchanged data.
- Preserve the order: load cache, scan, diff, write reports, then save the current cache.
A diff report that explains rebuild decisions with added, changed, deleted, and unchanged classifications.
06
Add fs.watch
Receive live filesystem events from the watched directory tree.
- Create src/watch.js so long-running watch state stays inside a session module.
- Run the existing scan path before registering live watchers to capture an initial baseline.
- Register directory watchers for the root and nested directories, using recursive watching where the platform supports it.
- Normalize every raw event into id, time, eventType, filename, watchRoot, absolutePath, relativePath, source, and uncertainty notes.
- Keep a bounded event buffer and append normalized events to reports/events.jsonl.
- Record watcher registration and delivery errors as structured events with path, operation, phase, code, and message.
A live watch session that records raw filesystem events without pretending they are final logical changes.
07
Add fs.watchFile fallback
Support polling mode with --poll.
- Add --poll and --poll-interval <ms> flags for watch mode.
- Register fs.watchFile pollers for regular files discovered by the startup baseline.
- Run periodic tree discovery so polling mode can notice newly created files.
- Convert curr and prev Stats snapshots into the same event stream used by native watching.
- Track listener functions and call fs.unwatchFile during cleanup or deleted-file handling.
- Write reports/watch-state.json with mode, interval, registered files, estimated stat checks, callbacks, discovery scans, and event counts.
A polling observation mode whose latency and filesystem cost are visible in the watch-state report.
08
Handle atomic writes
Detect editor-style temporary write and rename patterns as one logical file change.
- Create src/simulate-atomic-write.js to write a same-directory temp file and rename it over the target.
- Create src/atomic.js as the event burst grouping layer above raw watcher events.
- Recognize temp name patterns such as .name.tmp, name~, .swp, and .tmp.
- Group rename bursts by directory and a short time window such as 100ms.
- After the burst settles, rescan affected paths and compare metadata identity and hash.
- Write reports/logical-events.json with logical events, rawEventIds, tempPaths, and final path state.
Logical atomic-replace events that preserve raw event evidence while reporting one target-file change.
09
Debounce rebuilds
Group rapid events into one rebuild cycle after the event stream settles.
- Add --debounce <ms> with a default interval such as 150ms.
- Create src/debounce.js to collect events and fire after quiet time.
- Store affected paths in a Set and mark the whole watched root for rescan when filename data is missing.
- When the timer fires, scan with metadata and hashes, diff against cache, enqueue rebuild work, and save cache after scheduling.
- Append batch records to reports/batches.jsonl with batch timing, event counts, paths, and diff counts.
- Confirm reports and cache files stay outside watched input or remain ignored.
Batch records that show how raw and logical events become verified rebuild cycles.
10
Add a rebuild queue
Run rebuild jobs under a concurrency limit.
- Create src/queue.js with a concurrency-limited task queue that tracks active and pending work.
- Add --concurrency <n> with a default such as 2 and reject values below 1.
- Represent each rebuild job with id, paths, createdAt, startedAt, finishedAt, and status.
- Create src/rebuild.js with placeholder async rebuild work that writes per-path output records under reports/build-output.
- Coalesce duplicate affected paths into pending jobs while preserving active job history.
- Append queue samples to reports/queue.jsonl whenever active or pending counts change.
A rebuild queue that exposes pressure, enforces the concurrency cap, and records job state changes.
11
Add incremental build cache
Skip unchanged files during rebuild.
- Add a buildCache section to .cache/index.json separate from filesByPath.
- Store inputHash, inputMetadata, outputPath, outputHash, builtAt, and builderVersion for each built source.
- Warm the build cache during watch startup before relying on unchanged-file hits.
- Rebuild added and changed paths, then validate unchanged paths against input fingerprint, builder version, and output existence.
- Clean generated outputs and cache records for deleted source files.
- Append rebuild summaries to reports/rebuilds.jsonl with cache hits, misses, rebuilt paths, skipped paths, and deleted outputs.
A build cache that can prove when work was skipped and when one changed file caused one rebuilt output.
12
Handle symlinks and permissions
Define and implement policy for symlinks, unreadable files, and special files.
- Add --follow-symlinks to scan and watch modes while keeping the default as record links and skip targets.
- Record skipped entries for symlinks, special files, and unsupported file types with path, kind, action, and reason.
- When requested, resolve symlink targets with realpath, stat the target, and hash followed regular files.
- Track visited realpaths to report symlink cycles instead of traversing forever.
- Handle EACCES, EPERM, ENOENT, and ELOOP during scan, metadata, hash, watch, and rebuild operations.
- Write platform notes when symlink or permission behavior changes what the learner can verify locally.
An edge-case policy report that makes skipped, followed, errored, and platform-specific filesystem behavior visible.
13
Add graceful shutdown
Close watchers and save cache state on SIGINT and SIGTERM.
- Create src/shutdown.js with a registry of cleanup functions from watch mode.
- Handle SIGINT and SIGTERM as the normal control path for stopping a watch process.
- Make shutdown idempotent with states such as running, stopping, and forced.
- Stop event intake, close FSWatcher handles, unwatch polling paths, and clear timers.
- Flush pending debounce work or mark active and pending rebuild jobs with honest final state.
- Save cache atomically and write reports/shutdown.json with signal, reason, closed resources, cache result, queue state, and duration.
A shutdown report that records how the watcher stopped and what state was preserved for the next run.
14
Generate watch report
Produce a final Markdown report for a complete watch session.
- Create src/report.js to read JSON and JSONL artifacts without reaching into watcher internals.
- Implement src/demo-watch.js to reset a fixture, start watch mode, edit files, simulate an atomic write, delete a file, and stop through shutdown.
- Clear generated report artifacts at demo startup while preserving reports/reflections.md.
- Generate reports/watch-report.md with options, root, scan counts, raw events, logical groups, batches, rebuild jobs, queue high-water mark, cache hits, misses, skipped files, errors, and shutdown reason.
- Optionally generate reports/watch-report.html from the same data model.
- End the Markdown report with one concrete filesystem rule written from the observed evidence.
A final watch report that connects raw events, verified diffs, rebuild work, cache results, and shutdown state.
Watch evidence
Reports are part of the runtime tool.
Lab 05 treats reports as build output. The learner can inspect the scan baseline, cache load, diff reasons, raw events, logical groups, queue pressure, rebuild decisions, shutdown state, and the final watch report.
scan.jsonbaseline inventory, metadata, hash, cache timing, and scan errorsdiff.jsonadded, changed, deleted, unchanged, and reason fieldsevents.jsonlraw native or polling watch events in append orderlogical-events.jsonatomic write groups and verified final statebatches.jsonldebounced rebuild batches with affected pathsqueue.jsonlactive and pending rebuild queue samplesrebuilds.jsonlcache hits, misses, rebuilt paths, and deleted outputswatch-state.jsonwatch mode, poll counters, registrations, and event totalsshutdown.jsonsignal, cleanup, cache save, and queue state at exitwatch-report.mdthe final session report from scan through shutdownChoose your NodeBook package.
Switch between one-reader pricing and team licenses for up to 25 team members.
Individual pricing is for one reader and one personal purchase record.TEAM LICENSES INCLUDE UP TO 25 MEMBERS. DOWNLOADED CONTENT MAY BE
SHARED INTERNALLY WITH UP TO 30 PEOPLE TOTAL.
SHARED INTERNALLY WITH UP TO 30 PEOPLE TOTAL.
Downloadable book bundle
Digital Bundle
Volume I as EPUB, light/dark PDFs, slides, cheatsheets, and future updates.
$19.99$49.99
One-time purchase- Volume I EPUB for offline reading
- Light and dark PDF editions
- Slide decks for chapter review
- Cheatsheets for quick lookup
- Future Digital Bundle updates
- Lifetime access to the files
This is the downloadable Volume I study bundle. It does not include Node Runtime Labs.
Best value
Everything together
NodeBook Pro
All Labs plus the downloadable Volume I bundle in one purchase. Save $9.99 vs buying separately.
$49.99$99.99
One-time purchaseNode Runtime Labs+Digital Bundle
- Everything in Node Runtime Labs
- Everything in the Digital Bundle
- Seven included lab projects
- Three upcoming labs when released
- Future updates for both products
- Lifetime access to purchased files
This includes both paid products: Node Runtime Labs and the Digital Bundle.
Premium labs
Complex runtime projects
Node Runtime Labs
Seven long-form builds: recorder, binary store, stream workbench, resolver, watcher, task runtime, and protocol gateway.
$39.99$79.99
One-time purchase- Node Runtime Flight Recorder
- Binary File Store / Append-Only Log Database
- Stream Processing Workbench
- Module Resolution Inspector
- Atomic File Watcher + Incremental Build Cache
- Async Task Runtime / Local Job Orchestrator
- Custom Binary Protocol Gateway
- Three more labs upcoming
This is the paid labs bundle. It does not include EPUB, PDFs, slides, or cheatsheets.
Team downloadable files
Digital Bundle Team
Volume I files, slides, cheatsheets, and updates for a small engineering team.
$59.99
One-time team license- Everything in the Digital Bundle
- Supports up to 25 team members
- Share internally with up to 30 people
- Single purchase for the team
- Future Digital Bundle updates
- Lifetime access to purchased files
Covers up to 25 team members and internal sharing with up to 30 people.
Team value
Everything for the team
NodeBook Pro Team
Node Runtime Labs plus the downloadable Volume I bundle in one team license.
$149.99
One-time team licenseNode Runtime Labs+Digital Bundle
- Everything in NodeBook Pro
- Supports up to 25 team members
- Share internally with up to 30 people
- One receipt and license record
- Future updates for both products
- Save $29.99 vs team products separately.
Covers up to 25 team members and internal sharing with up to 30 people.
Team labs
Runtime training projects
Node Runtime Labs Team
Seven long-form builds for onboarding, study groups, and internal training.
$119.99
One-time team license- Node Runtime Flight Recorder
- Binary File Store / Append-Only Log Database
- Stream Processing Workbench
- Module Resolution Inspector
- Supports up to 25 team members
- Share internally with up to 30 people
- Three more labs upcoming
Covers up to 25 team members and internal sharing with up to 30 people.
Other labs in the bundle
Lab 05 sits inside the runtime lab track.
The bundle includes seven runtime projects covering process observation, binary storage, streams, module resolution, file watching, async orchestration, and custom protocols.