Node.js v20.20.2 documentation

Overview#

The WHATWG Streams Standard (or "web streams") defines an API for handling streaming data. It is similar to the Node.js Streams API but emerged later and has become the "standard" API for streaming data across many JavaScript environments.

There are three primary types of objects:

• ReadableStream - Represents a source of streaming data.
• WritableStream - Represents a destination for streaming data.
• TransformStream - Represents an algorithm for transforming streaming data.

Example ReadableStream#

This example creates a simple ReadableStream that pushes the current performance.now() timestamp once every second forever. An async iterable is used to read the data from the stream.

import {
  ReadableStream,
} from 'node:stream/web';

import {
  setInterval as every,
} from 'node:timers/promises';

import {
  performance,
} from 'node:perf_hooks';

const SECOND = 1000;

const stream = new ReadableStream({
  async start(controller) {
    for await (const _ of every(SECOND))
      controller.enqueue(performance.now());
  },
});

for await (const value of stream)
  console.log(value);const {
  ReadableStream,
} = require('node:stream/web');

const {
  setInterval: every,
} = require('node:timers/promises');

const {
  performance,
} = require('node:perf_hooks');

const SECOND = 1000;

const stream = new ReadableStream({
  async start(controller) {
    for await (const _ of every(SECOND))
      controller.enqueue(performance.now());
  },
});

(async () => {
  for await (const value of stream)
    console.log(value);
})();copy

API#

Class: ReadableStream#

new ReadableStream([underlyingSource [, strategy]])#

• underlyingSource <Object>
  • start <Function> A user-defined function that is invoked immediately when the ReadableStream is created.
    • controller <ReadableStreamDefaultController> | <ReadableByteStreamController>
    • Returns: undefined or a promise fulfilled with undefined.
  • pull <Function> A user-defined function that is called repeatedly when the ReadableStream internal queue is not full. The operation may be sync or async. If async, the function will not be called again until the previously returned promise is fulfilled.
    • controller <ReadableStreamDefaultController> | <ReadableByteStreamController>
    • Returns: A promise fulfilled with undefined.
  • cancel <Function> A user-defined function that is called when the ReadableStream is canceled.
    • reason <any>
    • Returns: A promise fulfilled with undefined.
  • type <string> Must be 'bytes' or undefined.
  • autoAllocateChunkSize <number> Used only when type is equal to 'bytes'. When set to a non-zero value a view buffer is automatically allocated to ReadableByteStreamController.byobRequest. When not set one must use stream's internal queues to transfer data via the default reader ReadableStreamDefaultReader.
• strategy <Object>
  • highWaterMark <number> The maximum internal queue size before backpressure is applied.
  • size <Function> A user-defined function used to identify the size of each chunk of data.
    • chunk <any>
    • Returns: <number>

readableStream.locked#

• Type: <boolean> Set to true if there is an active reader for this <ReadableStream>.

The readableStream.locked property is false by default, and is switched to true while there is an active reader consuming the stream's data.

readableStream.cancel([reason])#

• reason <any>
• Returns: A promise fulfilled with undefined once cancelation has been completed.

readableStream.getReader([options])#

• options <Object>
  • mode <string> 'byob' or undefined
• Returns: <ReadableStreamDefaultReader> | <ReadableStreamBYOBReader>

import { ReadableStream } from 'node:stream/web';

const stream = new ReadableStream();

const reader = stream.getReader();

console.log(await reader.read());const { ReadableStream } = require('node:stream/web');

const stream = new ReadableStream();

const reader = stream.getReader();

reader.read().then(console.log);copy

Causes the readableStream.locked to be true.

readableStream.pipeThrough(transform[, options])#

• transform <Object>
  • readable <ReadableStream> The ReadableStream to which transform.writable will push the potentially modified data it receives from this ReadableStream.
  • writable <WritableStream> The WritableStream to which this ReadableStream's data will be written.
• options <Object>
  • preventAbort <boolean> When true, errors in this ReadableStream will not cause transform.writable to be aborted.
  • preventCancel <boolean> When true, errors in the destination transform.writable do not cause this ReadableStream to be canceled.
  • preventClose <boolean> When true, closing this ReadableStream does not cause transform.writable to be closed.
  • signal <AbortSignal> Allows the transfer of data to be canceled using an <AbortController>.
• Returns: <ReadableStream> From transform.readable.

Connects this <ReadableStream> to the pair of <ReadableStream> and <WritableStream> provided in the transform argument such that the data from this <ReadableStream> is written in to transform.writable, possibly transformed, then pushed to transform.readable. Once the pipeline is configured, transform.readable is returned.

Causes the readableStream.locked to be true while the pipe operation is active.

import {
  ReadableStream,
  TransformStream,
} from 'node:stream/web';

const stream = new ReadableStream({
  start(controller) {
    controller.enqueue('a');
  },
});

const transform = new TransformStream({
  transform(chunk, controller) {
    controller.enqueue(chunk.toUpperCase());
  },
});

const transformedStream = stream.pipeThrough(transform);

for await (const chunk of transformedStream)
  console.log(chunk);
  // Prints: Aconst {
  ReadableStream,
  TransformStream,
} = require('node:stream/web');

const stream = new ReadableStream({
  start(controller) {
    controller.enqueue('a');
  },
});

const transform = new TransformStream({
  transform(chunk, controller) {
    controller.enqueue(chunk.toUpperCase());
  },
});

const transformedStream = stream.pipeThrough(transform);

(async () => {
  for await (const chunk of transformedStream)
    console.log(chunk);
    // Prints: A
})();copy

readableStream.pipeTo(destination[, options])#

• destination <WritableStream> A <WritableStream> to which this ReadableStream's data will be written.
• options <Object>
  • preventAbort <boolean> When true, errors in this ReadableStream will not cause destination to be aborted.
  • preventCancel <boolean> When true, errors in the destination will not cause this ReadableStream to be canceled.
  • preventClose <boolean> When true, closing this ReadableStream does not cause destination to be closed.
  • signal <AbortSignal> Allows the transfer of data to be canceled using an <AbortController>.
• Returns: A promise fulfilled with undefined

Causes the readableStream.locked to be true while the pipe operation is active.

readableStream.tee()#

• Returns: <ReadableStream[]>

Returns a pair of new <ReadableStream> instances to which this ReadableStream's data will be forwarded. Each will receive the same data.

Causes the readableStream.locked to be true.

readableStream.values([options])#

• options <Object>
  • preventCancel <boolean> When true, prevents the <ReadableStream> from being closed when the async iterator abruptly terminates. Default: false.

Creates and returns an async iterator usable for consuming this ReadableStream's data.

Causes the readableStream.locked to be true while the async iterator is active.

import { Buffer } from 'node:buffer';

const stream = new ReadableStream(getSomeSource());

for await (const chunk of stream.values({ preventCancel: true }))
  console.log(Buffer.from(chunk).toString()); copy

Async Iteration#

The <ReadableStream> object supports the async iterator protocol using for await syntax.

import { Buffer } from 'node:buffer';

const stream = new ReadableStream(getSomeSource());

for await (const chunk of stream)
  console.log(Buffer.from(chunk).toString()); copy

The async iterator will consume the <ReadableStream> until it terminates.

By default, if the async iterator exits early (via either a break, return, or a throw), the <ReadableStream> will be closed. To prevent automatic closing of the <ReadableStream>, use the readableStream.values() method to acquire the async iterator and set the preventCancel option to true.

The <ReadableStream> must not be locked (that is, it must not have an existing active reader). During the async iteration, the <ReadableStream> will be locked.

Transferring with postMessage()#

A <ReadableStream> instance can be transferred using a <MessagePort>.

const stream = new ReadableStream(getReadableSourceSomehow());

const { port1, port2 } = new MessageChannel();

port1.onmessage = ({ data }) => {
  data.getReader().read().then((chunk) => {
    console.log(chunk);
  });
};

port2.postMessage(stream, [stream]); copy

ReadableStream.from(iterable)#

• iterable <Iterable> Object implementing the Symbol.asyncIterator or Symbol.iterator iterable protocol.

A utility method that creates a new <ReadableStream> from an iterable.

import { ReadableStream } from 'node:stream/web';

async function* asyncIterableGenerator() {
  yield 'a';
  yield 'b';
  yield 'c';
}

const stream = ReadableStream.from(asyncIterableGenerator());

for await (const chunk of stream)
  console.log(chunk); // Prints: 'a', 'b', 'c'const { ReadableStream } = require('node:stream/web');

async function* asyncIterableGenerator() {
  yield 'a';
  yield 'b';
  yield 'c';
}

(async () => {
  const stream = ReadableStream.from(asyncIterableGenerator());

  for await (const chunk of stream)
    console.log(chunk); // Prints: 'a', 'b', 'c'
})();copy

To pipe the resulting <ReadableStream> into a <WritableStream> the <Iterable> should yield a sequence of <Buffer>, <TypedArray>, or <DataView> objects.

import { ReadableStream } from 'node:stream/web';
import { Buffer } from 'node:buffer';

async function* asyncIterableGenerator() {
  yield Buffer.from('a');
  yield Buffer.from('b');
  yield Buffer.from('c');
}

const stream = ReadableStream.from(asyncIterableGenerator());

await stream.pipeTo(createWritableStreamSomehow());const { ReadableStream } = require('node:stream/web');
const { Buffer } = require('node:buffer');

async function* asyncIterableGenerator() {
  yield Buffer.from('a');
  yield Buffer.from('b');
  yield Buffer.from('c');
}

const stream = ReadableStream.from(asyncIterableGenerator());

(async () => {
  await stream.pipeTo(createWritableStreamSomehow());
})();copy

Class: ReadableStreamDefaultReader#

By default, calling readableStream.getReader() with no arguments will return an instance of ReadableStreamDefaultReader. The default reader treats the chunks of data passed through the stream as opaque values, which allows the <ReadableStream> to work with generally any JavaScript value.

new ReadableStreamDefaultReader(stream)#

• stream <ReadableStream>

Creates a new <ReadableStreamDefaultReader> that is locked to the given <ReadableStream>.

readableStreamDefaultReader.cancel([reason])#

• reason <any>
• Returns: A promise fulfilled with undefined.

Cancels the <ReadableStream> and returns a promise that is fulfilled when the underlying stream has been canceled.

readableStreamDefaultReader.closed#

• Type: <Promise> Fulfilled with undefined when the associated <ReadableStream> is closed or rejected if the stream errors or the reader's lock is released before the stream finishes closing.

readableStreamDefaultReader.read()#

• Returns: A promise fulfilled with an object:
  • value <any>
  • done <boolean>

Requests the next chunk of data from the underlying <ReadableStream> and returns a promise that is fulfilled with the data once it is available.

readableStreamDefaultReader.releaseLock()#

Releases this reader's lock on the underlying <ReadableStream>.

Class: ReadableStreamBYOBReader#

The ReadableStreamBYOBReader is an alternative consumer for byte-oriented <ReadableStream>s (those that are created with underlyingSource.type set equal to 'bytes' when the ReadableStream was created).

The BYOB is short for "bring your own buffer". This is a pattern that allows for more efficient reading of byte-oriented data that avoids extraneous copying.

import {
  open,
} from 'node:fs/promises';

import {
  ReadableStream,
} from 'node:stream/web';

import { Buffer } from 'node:buffer';

class Source {
  type = 'bytes';
  autoAllocateChunkSize = 1024;

  async start(controller) {
    this.file = await open(new URL(import.meta.url));
    this.controller = controller;
  }

  async pull(controller) {
    const view = controller.byobRequest?.view;
    const {
      bytesRead,
    } = await this.file.read({
      buffer: view,
      offset: view.byteOffset,
      length: view.byteLength,
    });

    if (bytesRead === 0) {
      await this.file.close();
      this.controller.close();
    }
    controller.byobRequest.respond(bytesRead);
  }
}

const stream = new ReadableStream(new Source());

async function read(stream) {
  const reader = stream.getReader({ mode: 'byob' });

  const chunks = [];
  let result;
  do {
    result = await reader.read(Buffer.alloc(100));
    if (result.value !== undefined)
      chunks.push(Buffer.from(result.value));
  } while (!result.done);

  return Buffer.concat(chunks);
}

const data = await read(stream);
console.log(Buffer.from(data).toString()); copy

new ReadableStreamBYOBReader(stream)#

• stream <ReadableStream>

Creates a new ReadableStreamBYOBReader that is locked to the given <ReadableStream>.

readableStreamBYOBReader.cancel([reason])#

• reason <any>
• Returns: A promise fulfilled with undefined.

Cancels the <ReadableStream> and returns a promise that is fulfilled when the underlying stream has been canceled.

readableStreamBYOBReader.closed#

• Type: <Promise> Fulfilled with undefined when the associated <ReadableStream> is closed or rejected if the stream errors or the reader's lock is released before the stream finishes closing.

readableStreamBYOBReader.read(view[, options])#

• view <Buffer> | <TypedArray> | <DataView>
• options <Object>
  • min <number> When set, the returned promise will only be fulfilled as soon as min number of elements are available. When not set, the promise fulfills when at least one element is available.
• Returns: A promise fulfilled with an object:
  • value <TypedArray> | <DataView>
  • done <boolean>

Requests the next chunk of data from the underlying <ReadableStream> and returns a promise that is fulfilled with the data once it is available.

Do not pass a pooled <Buffer> object instance in to this method. Pooled Buffer objects are created using Buffer.allocUnsafe(), or Buffer.from(), or are often returned by various node:fs module callbacks. These types of Buffers use a shared underlying <ArrayBuffer> object that contains all of the data from all of the pooled Buffer instances. When a Buffer, <TypedArray>, or <DataView> is passed in to readableStreamBYOBReader.read(), the view's underlying ArrayBuffer is detached, invalidating all existing views that may exist on that ArrayBuffer. This can have disastrous consequences for your application.

readableStreamBYOBReader.releaseLock()#

Releases this reader's lock on the underlying