Repository navigation
feat(core): synthetic_esm extension DSL + node:worker_threads canary - #34038
Conversation
Drop the separate `02_init.js` module by inlining its content into
`01_require.js`. Along the way:
- Remove the `internals.requireImpl` bridge namespace β both files now
live in the same module, so `setUsesLocalNodeModulesDir` collapses
to a direct assignment and `nativeModuleExports` is a local
reference. The dead `setInspectBrk` and `Module` keys go away too.
- Remove the unused `loadCjsModule` helper.
- Replace the nested `internals.node = { closeIdleConnections }`
handle with a flat `internals.closeIdleConnections`, and update the
test runner script string accordingly. `closeIdleConnections` is the
only consumer of that namespace.
- Move the top-level `bindStreamsLazy(console, process)` call into
`initialize`'s non-warmup branch. It only matters for `node:console`
consumers, which can't run until after bootstrap anyway.
- Update `esm_entry_point` to `node:module` and drop `02_init.js`
from the esm list.
Adds a built-in `synthetic_esm` extension mechanism in libs/core that
maps a module specifier (e.g. `node:worker_threads`) to a backing
polyfill script. On first import the loader evaluates the backing
script (once; result cached) and synthesizes an ESM module whose
exports are derived from `Object.keys` of the IIFE's returned exports
object β Node's `BuiltinModule.getESMFacade` pattern, but with the
mapping declared by the extension and the dispatch driven entirely by
libs/core.
New pieces:
- `ModuleMap::new_synthetic_module_from_exports_object` builds a V8
synthetic module from an exports object via `GetOwnPropertyNames`
plus a `default` export pointing at the object.
- `synthetic_esm = [ "node:x" = "ext:foo/x.js" ]` macro variant carries
`(module_spec, backing_spec)` pairs through to `ModuleMap` at
extension init.
- `load_ext_script` now caches its result so the synthetic-esm
dispatch and any other `loadExtScript` caller share one IIFE
evaluation (re-eval would clobber `internals.__*` hooks and produce
duplicate class identities). The cache is snapshot-serialized via
`SnapshotDataId`.
- Dispatch hooks: `resolve_callback`, the static-import load path in
`recursive_load`, and the dynamic-import fast path in
`load_dynamic_import` all consult the new registry.
Applied to `node:worker_threads` as the live-binding canary:
- `worker_threads_esm.ts` wrapper (with its `export let` + refresh
hook) is gone.
- `worker_threads.ts` switches the tail IIFE return from a
getter-based object to a data-property `exportsObj` that
`__initWorkerThreads` mutates at the end. First-import snapshots
(synthetic) and `require("worker_threads")` reads (live object)
both see post-init values because the synthetic module is
constructed after bootstrap.
Verified with `./x test-node worker_threads`, `./x test-unit worker`,
`./x test-spec worker`, `./x test-spec node`, and ad-hoc smoke tests
covering static ESM, CJS `require`, dynamic `import()`, and worker
context.
|
|
β¦dules # Conflicts: # ext/node/polyfills/worker_threads.ts # ext/node/polyfills/worker_threads_esm.ts
fibibot
left a comment
There was a problem hiding this comment.
The synthetic_esm dispatch is structured cleanly: add_synthetic_esm_module is the registry, build_synthetic_esm_module runs load_ext_script + new_synthetic_module_from_exports_object, and the three hook sites (resolve_callback, recursive_load's register_and_recurse + Loader::load sentinel, load_dynamic_import fast path) all gate on has_synthetic_esm_module. The loaded_script_results cache is the real load-bearing change β without it the static-ESM and CJS require paths would re-evaluate the IIFE and produce duplicate class identities / clobber internals.__* hooks, exactly as the commit body calls out.
worker_threads canary: dropping worker_threads_esm.ts plus the __refreshWorkerThreadsWrapper callback and swapping the getter-tail to a data-property exportsObj that __initWorkerThreads mutates is the right shape β first-import snapshots happen post-bootstrap so consumers see the resolved values. Existing tests already exercise this path (worker_threads_test.ts, tests/specs/node/worker_threads/*.mjs mix import *, named, and default imports), and CI is fully green across 100+ shards including the full wpt release linux-x86_64 run.
- nit:
new_synthetic_module_from_exports_objectdoeskey_str.to_rust_string_lossy(scope) == \"default\"for every own property β a per-import UTF-8 conversion.key_val.strict_equals(default_str.into())after creatingdefault_stronce would skip the alloc. Per-import cost so probably noise. - nit: the per-property
exports_obj.get(scope, key_val).unwrap()will panic if a futuresynthetic_esmtarget uses a throwing getter. Fine forworker_threads(data properties only), worth a doc-comment onnew_synthetic_module_from_exports_objectso future authors know. - nit: no test directly exercises the
"backing script did not return an object"error path β the macro shape makes that hard to misuse, so low priority.
β¦enoland#34038) Adds a built-in `synthetic_esm` mechanism in `libs/core` so an extension can map a module specifier (e.g. `node:worker_threads`) to a backing polyfill script. On first import the loader evaluates the script (once β the result is cached) and synthesizes an ES module whose named exports are derived from `Object.keys` of the IIFE's returned exports object, plus a `default` export pointing at the object. This is Node's `BuiltinModule.getESMFacade` shape, but with the mapping declared by extensions and the dispatch driven entirely by `libs/core`, so it works for any extension that wants a CJS-shaped polyfill exposed as ESM. `load_ext_script` now caches its evaluated result so the synthetic-esm dispatch and other callers (CJS `require` through the JS-side `loadExtScript` wrapper) share one IIFE evaluation β re-evaluating a polyfill body would clobber registered `internals.__*` hooks and produce duplicate class identities. The cache is snapshot-serialized via `SnapshotDataId` so the dispatch can still find values evaluated at snapshot time. `node:worker_threads` is the canary because it is the only `*_esm.ts` wrapper today that needs live bindings β its `worker_threads_esm.ts` used `export let` plus an `internals.__refreshWorkerThreadsWrapper` callback to push `parentPort`/`threadId`/`workerData`/`isMainThread`/`resourceLimits` into the ESM facade after `__initWorkerThreads`. With this change the wrapper is gone: the polyfill's tail IIFE returns a data-property exports object, and `__initWorkerThreads` mutates those properties. The synthetic module is constructed lazily on first import, after bootstrap, so it snapshots the post-init values directly β no refresh op required.
Adds a built-in
synthetic_esmmechanism inlibs/coreso anextension can map a module specifier (e.g.
node:worker_threads) toa backing polyfill script. On first import the loader evaluates the
script (once β the result is cached) and synthesizes an ES module
whose named exports are derived from
Object.keysof the IIFE'sreturned exports object, plus a
defaultexport pointing at theobject. This is Node's
BuiltinModule.getESMFacadeshape, but withthe mapping declared by extensions and the dispatch driven entirely
by
libs/core, so it works for any extension that wants aCJS-shaped polyfill exposed as ESM.
load_ext_scriptnow caches its evaluated result so thesynthetic-esm dispatch and other callers (CJS
requirethrough theJS-side
loadExtScriptwrapper) share one IIFE evaluation βre-evaluating a polyfill body would clobber registered
internals.__*hooks and produce duplicate class identities. Thecache is snapshot-serialized via
SnapshotDataIdso the dispatchcan still find values evaluated at snapshot time.
node:worker_threadsis the canary because it is the only*_esm.tswrapper today that needs live bindings β itsworker_threads_esm.tsusedexport letplus aninternals.__refreshWorkerThreadsWrappercallback to pushparentPort/threadId/workerData/isMainThread/resourceLimitsinto the ESM facade after
__initWorkerThreads. With this change thewrapper is gone: the polyfill's tail IIFE returns a data-property
exports object, and
__initWorkerThreadsmutates those properties.The synthetic module is constructed lazily on first import, after
bootstrap, so it snapshots the post-init values directly β no
refresh op required.
Stacked on #34035.