Node.js was traditionally designed as a single-threaded runtime optimized for asynchronous, non-blocking I/O (I/O-bound operations). However, modern server-side applications frequently encounter compute-intensive tasks (CPU-bound workloads): validating and transforming large JSON/Protocol Buffer payloads, generating PDFs, compressing and encrypting media streams, computing cryptographic hashes, or running lightweight machine learning inference. Executing these CPU-heavy routines directly on the main thread blocks the libuv event loop and causes severe latency spikes for all concurrent HTTP traffic.
Historically, multiprocessing was achieved via operating system process isolation using child_process (fork, spawn) or the cluster module. Starting in Node.js 10.5+ and fully stabilized in Node.js 12+, the platform provides the worker_threads module, introducing true OS-level multithreading within a single process:
+-------------------------------------------------------------------------+
| Single OS Process (PID: 14205) |
+-------------------------------------------------------------------------+
| Main Thread | Worker Thread #1 |
| - libuv Event Loop | - libuv Event Loop |
| - V8 Isolate A (Heap A, Stack A) | - V8 Isolate B (Heap B, Stack B) |
| - Call Stack | - Call Stack |
+-----------------------------------+-------------------------------------+
| |
+------------------+------------------+
|
+-------------------------------------------------+
| Shared Virtual Memory |
| - SharedArrayBuffer (Zero-copy concurrent R/W) |
| - Atomics Synchronization API (wait / notify) |
+-------------------------------------------------+The fundamental architectural difference between Node.js Worker Threads and threads in languages like Java, C++, or Go is strict state isolation:
transferList), or shared linear memory (SharedArrayBuffer) backed by thread-safe Atomics operations.Communication between the main thread and worker threads is orchestrated via message channels (MessageChannel / MessagePort), representing abstractions over asynchronous libuv task queues:
[Main Thread: V8 Isolate 1] [Worker Thread: V8 Isolate 2]
| |
| --- postMessage({ task: 1 }) [Structured Clone] -> | (Deserialization into Heap 2)
| |
| --- postMessage(arrayBuffer, [arrayBuffer]) -----> | (Zero-copy: Detach Heap 1)
| (Transfer ownership) |
| |
| <======== SharedArrayBuffer (Atomics) =========> | (Direct concurrent memory access)port.postMessage(data) serializes the object graph into an internal binary representation, passes the bytes across the libuv message queue, and deserializes them into the destination thread's V8 heap. For large payloads (e.g., 100 MB), serialization consumes substantial CPU cycles.ArrayBuffer in the transfer list:
port.postMessage({ buffer }, [buffer]);buffer.byteLength immediately becomes 0, preventing simultaneous out-of-sync mutations.Atomics synchronize concurrent access, eliminating race conditions.HTTP route handler delegates CPU-bound workload to the ThreadPool manager, receiving a deferred Promise.
If an idle worker is available, the task is dispatched immediately. If all workers are busy, it enters a FIFO queue.
Binary payloads and ArrayBuffers are dispatched across MessagePort with ownership transferred via transferList.
Thread executes computationally heavy logic on a dedicated CPU core without blocking the main event loop.
Worker posts result back to main thread, resolving the client promise and pulling the next task from the queue.
When sharing memory directly between threads, standard lock primitives must be built using CPU atomics. Using Atomics.wait() and Atomics.notify(), we construct a mutual exclusion lock (Mutex):
/**
* SharedMutex: Mutual exclusion lock backed by SharedArrayBuffer and Atomics.
* Index 0: lock status (0 = UNLOCKED, 1 = LOCKED).
*/
export class SharedMutex {
private static readonly UNLOCKED = 0;
private static readonly LOCKED = 1;
constructor(private readonly sharedBuffer: Int32Array) {}
public static createBuffer(): Int32Array {
// 4 bytes for a single Int32 cell
const sab = new SharedArrayBuffer(4);
return new Int32Array(sab);
}
public lock(): void {
// Atomically attempt to acquire the lock
while (
Atomics.compareExchange(
this.sharedBuffer,
0,
SharedMutex.UNLOCKED,
SharedMutex.LOCKED
) !== SharedMutex.UNLOCKED
) {
// If already held, put the worker thread to sleep at the OS kernel level
// NOTE: Atomics.wait is prohibited on the main thread of Node.js!
Atomics.wait(this.sharedBuffer, 0, SharedMutex.LOCKED);
}
}
public unlock(): void {
const oldValue = Atomics.exchange(this.sharedBuffer, 0, SharedMutex.UNLOCKED);
if (oldValue === SharedMutex.LOCKED) {
// Wake up one thread sleeping on this memory address
Atomics.notify(this.sharedBuffer, 0, 1);
}
}
public runSynchronized<T>(action: () => T): T {
this.lock();
try {
return action();
} finally {
this.unlock();
}
}
}Spawning a new Worker on every incoming request is an anti-pattern that exhausts memory. In production, maintain a long-lived pool sized to physical CPU cores with task queuing, timeout monitoring, and graceful teardown:
import { Worker } from 'node:worker_threads';
import os from 'node:os';
import path from 'node:path';
interface Task<T, R> {
id: string;
data: T;
resolve: (result: R) => void;
reject: (err: Error) => void;
timeoutTimer: NodeJS.Timeout;
}
interface WorkerRecord {
worker: Worker;
busy: boolean;
activeTaskId: string | null;
}
export class ProductionThreadPool<T = unknown, R = unknown> {
private readonly workers: WorkerRecord[] = [];
private readonly taskQueue: Task<T, R>[] = [];
private isTerminating = false;
constructor(
private readonly workerScriptPath: string,
private readonly poolSize: number = os.cpus().length,
private readonly taskTimeoutMs: number = 30_000
) {
this.initPool();
}
private initPool(): void {
for (let i = 0; i < this.poolSize; i++) {
this.spawnWorker();
}
}
private spawnWorker(): void {
const worker = new Worker(this.workerScriptPath);
const record: WorkerRecord = {
worker,
busy: false,
activeTaskId: null,
};
worker.on('message', (result: { taskId: string; payload: R; error?: string }) => {
this.handleWorkerResult(record, result);
});
worker.on('error', (err) => {
console.error(`[Worker Error] Thread crashed unexpectedly:`, err);
this.replaceCrashedWorker(record);
});
this.workers.push(record);
}
public execute(data: T): Promise<R> {
if (this.isTerminating) {
return Promise.reject(new Error('ThreadPool is terminating'));
}
return new Promise<R>((resolve, reject) => {
const taskId = `task_${Date.now()}_${Math.random().toString(36).slice(2, 9)}`;
const timeoutTimer = setTimeout(() => {
this.handleTaskTimeout(taskId);
}, this.taskTimeoutMs);
const task: Task<T, R> = {
id: taskId,
data,
resolve,
reject,
timeoutTimer,
};
this.taskQueue.push(task);
this.dispatchNext();
});
}
private dispatchNext(): void {
if (this.taskQueue.length === 0) return;
const availableWorker = this.workers.find((w) => !w.busy);
if (!availableWorker) return;
const task = this.taskQueue.shift()!;
availableWorker.busy = true;
availableWorker.activeTaskId = task.id;
(availableWorker.worker as any).__currentTask = task;
availableWorker.worker.postMessage({
taskId: task.id,
data: task.data,
});
}
private handleWorkerResult(
record: WorkerRecord,
result: { taskId: string; payload: R; error?: string }
): void {
const currentTask = (record.worker as any).__currentTask as Task<T, R> | undefined;
if (currentTask && currentTask.id === result.taskId) {
clearTimeout(currentTask.timeoutTimer);
if (result.error) {
currentTask.reject(new Error(result.error));
} else {
currentTask.resolve(result.payload);
}
(record.worker as any).__currentTask = undefined;
}
record.busy = false;
record.activeTaskId = null;
this.dispatchNext();
}
private handleTaskTimeout(taskId: string): void {
const workerIndex = this.workers.findIndex((w) => w.activeTaskId === taskId);
if (workerIndex === -1) return;
const record = this.workers[workerIndex];
const currentTask = (record.worker as any).__currentTask as Task<T, R>;
if (currentTask) {
currentTask.reject(new Error(`Task execution timed out after ${this.taskTimeoutMs}ms`));
}
record.worker.terminate().finally(() => {
this.workers.splice(workerIndex, 1);
this.spawnWorker();
this.dispatchNext();
});
}
private replaceCrashedWorker(deadRecord: WorkerRecord): void {
const idx = this.workers.indexOf(deadRecord);
if (idx !== -1) {
this.workers.splice(idx, 1);
}
const currentTask = (deadRecord.worker as any).__currentTask as Task<T, R> | undefined;
if (currentTask) {
clearTimeout(currentTask.timeoutTimer);
currentTask.reject(new Error('Worker thread crashed during task execution'));
}
if (!this.isTerminating) {
this.spawnWorker();
this.dispatchNext();
}
}
public async shutdown(): Promise<void> {
this.isTerminating = true;
while (this.taskQueue.length > 0) {
const task = this.taskQueue.shift()!;
clearTimeout(task.timeoutTimer);
task.reject(new Error('ThreadPool is shutting down before task execution'));
}
const terminations = this.workers.map((w) => w.worker.terminate());
await Promise.all(terminations);
this.workers.length = 0;
}
}The ECMAScript specification explicitly forbids invoking Atomics.wait() on the main execution thread of Node.js. Attempting to call this method throws a synchronous TypeError: Atomics.wait cannot be called in this context. Putting the main thread to sleep would block the libuv event loop entirely, freezing network sockets, timers, and active connections.
Always restrict Atomics.wait to background worker threads. On the main thread, use the non-blocking Atomics.waitAsync() API, which returns a Promise.
Creating new Worker('./worker.js') inside an HTTP request handler causes severe performance degradation. Every worker allocates a new V8 Isolate, re-compiles JavaScript scripts, and initializes an event loop instance. Under 100 RPS, this exhausts process RAM and leads to kernel OOM termination.
Always utilize a long-lived worker pool (such as piscina) bounded to physical CPU core count (os.cpus().length).
Passing multi-megabyte payloads (e.g., 50 MB image buffers) via worker.postMessage(data) without a transfer list forces V8 to deeply clone the entire memory structure using the Structured Clone Algorithm. This doubles heap consumption and incurs tens to hundreds of milliseconds of serialization delay.
Always transfer binary structures as ArrayBuffer instances using the second argument: port.postMessage(buffer, [buffer.buffer]).
Writing directly to indices of a SharedArrayBuffer (shared[0]++ or shared[0] = value) from multiple threads is not atomic. In CPU micro-architecture, an increment consists of three separate instructions: read into register, increment register, write back to RAM. Concurrent execution results in clobbered updates and corrupted state.
Always use Atomics.add(), Atomics.store(), or Atomics.compareExchange() for guaranteed thread safety.
Main thread initializes long-lived workers. OS allocates kernel threads, and Node.js spins up isolated V8 engines and libuv loops.
Dispatcher serializes task metadata and transfers binary buffers with zero-copy ownership handover via transferList.
Worker receives message event and performs compute-heavy algorithms on a dedicated CPU core without blocking the main event loop.
Result is posted back to the main thread. A Poll phase callback fires on the primary event loop, resolving the original client Promise.
Worker is kept alive in the pool, instantly ready to receive the next queued task from the dispatch queue.