Skip to content

feat(core): synthetic_esm extension DSL + node:worker_threads canary - #34038

Merged
bartlomieju merged 4 commits into
mainfrom
feat/synthetic-esm-modules
May 14, 2026
Merged

bartlomieju merged 4 commits into
mainfrom
feat/synthetic-esm-modules

Conversation

@bartlomieju

Copy link
Copy Markdown
Member

Adds a built-in synthetic_esm mechanism in libs/core so an
extension can map a module specifier (e.g. node:worker_threads) to
a backing polyfill script. On first import the loader evaluates the
script (once β€” the result is cached) and synthesizes an ES module
whose named exports are derived from Object.keys of the IIFE's
returned exports object, plus a default export pointing at the
object. This is Node's BuiltinModule.getESMFacade shape, but with
the mapping declared by extensions and the dispatch driven entirely
by libs/core, so it works for any extension that wants a
CJS-shaped polyfill exposed as ESM.

load_ext_script now caches its evaluated result so the
synthetic-esm dispatch and other callers (CJS require through the
JS-side loadExtScript wrapper) share one IIFE evaluation β€”
re-evaluating a polyfill body would clobber registered
internals.__* hooks and produce duplicate class identities. The
cache is snapshot-serialized via SnapshotDataId so the dispatch
can still find values evaluated at snapshot time.

node:worker_threads is the canary because it is the only
*_esm.ts wrapper today that needs live bindings β€” its
worker_threads_esm.ts used export let plus an
internals.__refreshWorkerThreadsWrapper callback to push
parentPort/threadId/workerData/isMainThread/resourceLimits
into the ESM facade after __initWorkerThreads. With this change the
wrapper is gone: the polyfill's tail IIFE returns a data-property
exports object, and __initWorkerThreads mutates those properties.
The synthetic module is constructed lazily on first import, after
bootstrap, so it snapshots the post-init values directly β€” no
refresh op required.

Stacked on #34035.

Drop the separate `02_init.js` module by inlining its content into
`01_require.js`. Along the way:

- Remove the `internals.requireImpl` bridge namespace β€” both files now
  live in the same module, so `setUsesLocalNodeModulesDir` collapses
  to a direct assignment and `nativeModuleExports` is a local
  reference. The dead `setInspectBrk` and `Module` keys go away too.
- Remove the unused `loadCjsModule` helper.
- Replace the nested `internals.node = { closeIdleConnections }`
  handle with a flat `internals.closeIdleConnections`, and update the
  test runner script string accordingly. `closeIdleConnections` is the
  only consumer of that namespace.
- Move the top-level `bindStreamsLazy(console, process)` call into
  `initialize`'s non-warmup branch. It only matters for `node:console`
  consumers, which can't run until after bootstrap anyway.
- Update `esm_entry_point` to `node:module` and drop `02_init.js`
  from the esm list.
Adds a built-in `synthetic_esm` extension mechanism in libs/core that
maps a module specifier (e.g. `node:worker_threads`) to a backing
polyfill script. On first import the loader evaluates the backing
script (once; result cached) and synthesizes an ESM module whose
exports are derived from `Object.keys` of the IIFE's returned exports
object β€” Node's `BuiltinModule.getESMFacade` pattern, but with the
mapping declared by the extension and the dispatch driven entirely by
libs/core.

New pieces:

- `ModuleMap::new_synthetic_module_from_exports_object` builds a V8
  synthetic module from an exports object via `GetOwnPropertyNames`
  plus a `default` export pointing at the object.
- `synthetic_esm = [ "node:x" = "ext:foo/x.js" ]` macro variant carries
  `(module_spec, backing_spec)` pairs through to `ModuleMap` at
  extension init.
- `load_ext_script` now caches its result so the synthetic-esm
  dispatch and any other `loadExtScript` caller share one IIFE
  evaluation (re-eval would clobber `internals.__*` hooks and produce
  duplicate class identities). The cache is snapshot-serialized via
  `SnapshotDataId`.
- Dispatch hooks: `resolve_callback`, the static-import load path in
  `recursive_load`, and the dynamic-import fast path in
  `load_dynamic_import` all consult the new registry.

Applied to `node:worker_threads` as the live-binding canary:

- `worker_threads_esm.ts` wrapper (with its `export let` + refresh
  hook) is gone.
- `worker_threads.ts` switches the tail IIFE return from a
  getter-based object to a data-property `exportsObj` that
  `__initWorkerThreads` mutates at the end. First-import snapshots
  (synthetic) and `require("worker_threads")` reads (live object)
  both see post-init values because the synthetic module is
  constructed after bootstrap.

Verified with `./x test-node worker_threads`, `./x test-unit worker`,
`./x test-spec worker`, `./x test-spec node`, and ad-hoc smoke tests
covering static ESM, CJS `require`, dynamic `import()`, and worker
context.
Base automatically changed from refactor/merge-node-init-into-require to main May 13, 2026 23:51
@fibibot

fibibot commented May 14, 2026 •

Copy link
Copy Markdown
Contributor

node_compat::parallel::test-dns-resolver-max-timeout.js is a known flake (already marked flaky: true in tests/node_compat/config.jsonc), unrelated to this PR's worker_threads/core changes. Workflow still in progress; if it concludes failed, the job needs a re-run.

…dules

# Conflicts:
#	ext/node/polyfills/worker_threads.ts
#	ext/node/polyfills/worker_threads_esm.ts

@fibibot fibibot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The synthetic_esm dispatch is structured cleanly: add_synthetic_esm_module is the registry, build_synthetic_esm_module runs load_ext_script + new_synthetic_module_from_exports_object, and the three hook sites (resolve_callback, recursive_load's register_and_recurse + Loader::load sentinel, load_dynamic_import fast path) all gate on has_synthetic_esm_module. The loaded_script_results cache is the real load-bearing change β€” without it the static-ESM and CJS require paths would re-evaluate the IIFE and produce duplicate class identities / clobber internals.__* hooks, exactly as the commit body calls out.

worker_threads canary: dropping worker_threads_esm.ts plus the __refreshWorkerThreadsWrapper callback and swapping the getter-tail to a data-property exportsObj that __initWorkerThreads mutates is the right shape β€” first-import snapshots happen post-bootstrap so consumers see the resolved values. Existing tests already exercise this path (worker_threads_test.ts, tests/specs/node/worker_threads/*.mjs mix import *, named, and default imports), and CI is fully green across 100+ shards including the full wpt release linux-x86_64 run.

  • nit: new_synthetic_module_from_exports_object does key_str.to_rust_string_lossy(scope) == \"default\" for every own property β€” a per-import UTF-8 conversion. key_val.strict_equals(default_str.into()) after creating default_str once would skip the alloc. Per-import cost so probably noise.
  • nit: the per-property exports_obj.get(scope, key_val).unwrap() will panic if a future synthetic_esm target uses a throwing getter. Fine for worker_threads (data properties only), worth a doc-comment on new_synthetic_module_from_exports_object so future authors know.
  • nit: no test directly exercises the "backing script did not return an object" error path β€” the macro shape makes that hard to misuse, so low priority.

@bartlomieju
bartlomieju merged commit 2d3f3f1 into main May 14, 2026
136 checks passed
@bartlomieju
bartlomieju deleted the feat/synthetic-esm-modules branch May 14, 2026 12:06
littledivy pushed a commit to crowlKats/deno that referenced this pull request Jun 10, 2026
…enoland#34038)

Adds a built-in `synthetic_esm` mechanism in `libs/core` so an
extension can map a module specifier (e.g. `node:worker_threads`) to
a backing polyfill script. On first import the loader evaluates the
script (once β€” the result is cached) and synthesizes an ES module
whose named exports are derived from `Object.keys` of the IIFE's
returned exports object, plus a `default` export pointing at the
object. This is Node's `BuiltinModule.getESMFacade` shape, but with
the mapping declared by extensions and the dispatch driven entirely
by `libs/core`, so it works for any extension that wants a
CJS-shaped polyfill exposed as ESM.

`load_ext_script` now caches its evaluated result so the
synthetic-esm dispatch and other callers (CJS `require` through the
JS-side `loadExtScript` wrapper) share one IIFE evaluation β€”
re-evaluating a polyfill body would clobber registered
`internals.__*` hooks and produce duplicate class identities. The
cache is snapshot-serialized via `SnapshotDataId` so the dispatch
can still find values evaluated at snapshot time.

`node:worker_threads` is the canary because it is the only
`*_esm.ts` wrapper today that needs live bindings β€” its
`worker_threads_esm.ts` used `export let` plus an
`internals.__refreshWorkerThreadsWrapper` callback to push
`parentPort`/`threadId`/`workerData`/`isMainThread`/`resourceLimits`
into the ESM facade after `__initWorkerThreads`. With this change the
wrapper is gone: the polyfill's tail IIFE returns a data-property
exports object, and `__initWorkerThreads` mutates those properties.
The synthetic module is constructed lazily on first import, after
bootstrap, so it snapshots the post-init values directly β€” no
refresh op required.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants