Repository navigation
Local CRE: require don_family on all nodesets and scope gateway pairing by family - #22733
Merged
Merged
Conversation
Contributor
|
✅ No conflicts with other open PRs targeting |
cawthorne
force-pushed
the
feature/cre-sharded-gateway-pairing
branch
from
June 4, 2026 12:57
fff3cc1 to
ea10e36
Compare
cawthorne
marked this pull request as ready for review
June 4, 2026 16:56
product-security-plaid-production
Bot
requested review from
chudilka1 and
george-dorin
and removed request for
chudilka1
June 4, 2026 16:56
Tofel
reviewed
Jun 8, 2026
Tofel
reviewed
Jun 8, 2026
Tofel
reviewed
Jun 8, 2026
Tofel
reviewed
Jun 8, 2026
Tofel
reviewed
Jun 8, 2026
Tofel
reviewed
Jun 8, 2026
cawthorne
marked this pull request as draft
June 9, 2026 10:43
RegisterWithContract now requires a DON family; expose DonFamily on Don and default to test-don-family in test helpers when topology omits it.
Legacy single-gateway CI configs (bootstrap-gateway without don_family) keep all-to-all wiring; pairing and validation activate only when a workflow or gateway nodeset sets don_family.
Resolve config.go conflict by combining per-family connector scoping with upstream ToConnectorGateway(). Log pairing via structured logger, default legacy workflow deploy to test-don-family, and add tests for partial don_family validation and pairing summary output.
Tofel
reviewed
Jun 24, 2026
Gateway worker jobs scope via GatewayServiceConfigsForGateway; deploy, cap-reg, and workflow registry setup require explicit don_family. Add don_family to capabilities nodesets in CI TOMLs and inline unit-test fixtures after removing shared topology test helpers.
Drop redundant empty-family guards now that every nodeset requires don_family, log pairing only on gateway topologies, and fix gateway URL wording in state resolver comments.
Remove unused test topology helpers and apply goimports alignment in topology.go.
Gateway connector configs omit Incoming.Host by design; tests and deploy must fall back to infra ExternalGatewayHost like the state resolver already does.
Required after making don_family mandatory on every nodeset.
Filter each gateway service's DON list by don_family instead of replacing it with workflow DON names only. Vault registers on the capabilities DON; replacing with workflow broke gateway→vault OCR routing and caused E2E timeouts.
cawthorne
marked this pull request as ready for review
June 24, 2026 18:20
Tofel
previously approved these changes
Jun 25, 2026
Resolve don_family from local CRE state for run-por-example and env start --with-example so registry registration satisfies the new required field.
|
Tofel
approved these changes
Jun 26, 2026
bolekk
approved these changes
Jun 26, 2026
| [[nodesets]] | ||
| nodes = 5 | ||
| name = "workflow" | ||
| don_family = "test-don-family" |
Contributor
There was a problem hiding this comment.
Technically one DON can belong to multiple families. We will need that for sharding but can be extended later. Also it might be better to use "zone-a" here to reflect what's in prod.
cedric-cordenier
approved these changes
Jun 29, 2026
This was referenced Jun 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Summary
Adds per-DON-family gateway↔workflow pairing for local CRE topologies. Every nodeset declares
don_family; workflow DONs connect only to gateway DONs in the same family, and each gateway worker serves only workflow DONs in that family.The same
don_familyvalue flows through capabilities registry (DonFamilies), workflow registry (SetDONLimitper family), andenv workflow deploy.Previously,
env startinjected all gateway connectors into every workflow DON and registered every gateway worker against all workflow DONs — separate gateway nodesets in TOML did not isolate families.Solution
Every nodeset requires
don_family(workflow, gateway, capabilities, bootstrap, etc.). Gateway topologies validate the workflow↔gateway graph at env start and fail if any family is missing or unmatched.GatewayConnectorsForDonFamilyGatewayServiceConfigsForGatewaydon_family(required)SetDONLimitper workflow family--don-family(+--shard-index), or--workflow-don-name--container-name-pattern{nodesets.name}-nodefrom resolved DONDonFamilyGatewayPairingSummary()Matching is by
don_familystring equality at env start — not index, zone name, or nodeset order. Existing single-family CI topologies usedon_family = "test-don-family"on every nodeset.Example env-start log:
don_familymust match across the stackdon_familyenv start)DonFamilies: [don_family]SetDONLimitper familyUpsertWorkflowWorkflow deploy target resolution
When local CRE state exists, deploy picks the workflow DON in this order:
--workflow-don-name— exact match onnodesets.name(escape hatch for shards and ambiguous families)--don-family— unique workflow DON with that family; use--shard-indexwhen several shard DONs share the family (e.g.shard0/shard1bothtest-don-family)--container-name-patternis copy-only (docker cp substring). When omitted, it defaults to{resolvedDon.Name}-node(matchesfeeds-zone-a-node0, etc.).Deploy guard: when both
--workflow-don-nameand--don-familyare set, deploy errors if they disagree with local CRE state.Multi-zone deploy (one family per zone —
--don-familyis enough):Sharded deploy (same family, multiple shard DONs):
go run . env workflow deploy \ --don-family test-don-family \ --shard-index 0 \ ...Explicit DON name:
go run . env workflow deploy \ --workflow-don-name feeds-zone-a \ ...Deploy without local CRE state requires
--don-familyexplicitly (no default family).Test plan
go test ./system-tests/lib/cre/...go test ./core/scripts/cre/environment/environment/...go build ./core/scripts/cre/environment/...monitoring/local-cre