Skip to content

Add verify-crdify script to detect breaking changes in CRDs - #4830

Merged
k8s-ci-robot merged 19 commits into
kubernetes-sigs:mainfrom
ElenaZvereva:crdify
May 14, 2026
Merged

k8s-ci-robot merged 19 commits into
kubernetes-sigs:mainfrom
ElenaZvereva:crdify

Conversation

@ElenaZvereva

@ElenaZvereva ElenaZvereva commented May 5, 2026 •

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

This PR adds a new verify script hack/verify-crdify.sh, which checks for backward-incompatible changes in CRDs using crdify.

The script compares CRDs in the current branch against a base reference and runs crdify validations for each CRD file under config/crd/standard

Which issue(s) this PR fixes:

Part of #4723

Does this PR introduce a user-facing change?:

NONE

@k8s-ci-robot k8s-ci-robot added do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 5, 2026
@k8s-ci-robot
k8s-ci-robot requested review from rikatz and youngnick May 5, 2026 11:31
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

Welcome @ElenaZvereva!

It looks like this is your first PR to kubernetes-sigs/gateway-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/gateway-api has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 5, 2026
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

Hi @ElenaZvereva. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ElenaZvereva ElenaZvereva changed the title Testing crdify script in pull-gateway-api-verify job Add verify-crdify script to detect breaking changes in CRDs May 5, 2026
@ElenaZvereva

Copy link
Copy Markdown
Contributor Author

/kind cleanup
/area tooling

@k8s-ci-robot k8s-ci-robot added the kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. label May 5, 2026
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

@ElenaZvereva: The label(s) area/tooling cannot be applied, because the repository doesn't have them.

Details

In response to this:

/kind cleanup
/area tooling

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ElenaZvereva

Copy link
Copy Markdown
Contributor Author

Hi @rikatz! Could you please run waiting workflows?
I tested the script locally and in the kubekins-e2e image (which is used by Prow), and it worked as expected. I’d like to verify it in Prow as well.

My plan is:

  1. Verify the script in Prow
  2. Then introduce temporary breaking changes to the CRDs in this PR and re-run pull-gateway-api-verify to confirm that crdify detects it

So I’ll need help triggering jobs twice: once now and once after introducing breaking changes.

I also have two questions:

  1. Should this check cover both config/crd/standard and config/crd/experimental?
  2. Where is the preferred place to install crdify? I installed it directly in hack/verify-crdify.sh, similar to how yamllint is installed in hack/verify-yamllint.sh..

Comment thread hack/verify-crdify.sh Resolved
Comment thread hack/verify-crdify.sh Resolved
Comment thread hack/verify-crdify.sh Resolved
Comment thread hack/verify-crdify.sh Resolved
Comment thread hack/verify-crdify.sh Resolved
Comment thread hack/verify-crdify.sh Resolved
Comment thread hack/verify-crdify.sh Resolved
@rikatz

rikatz commented May 5, 2026

Copy link
Copy Markdown
Member

hi @ElenaZvereva thanks for your contribution!

I've left some comments here to be verified. Also I guess i have answered both of your quiestions:

  • We use go tool for tools installing
  • We should be verifying just standard right now

Thanks!!

@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 7, 2026
Co-authored-by: Ricardo Pchevuzinske Katz <7182341+rikatz@users.noreply.github.com>
@ElenaZvereva

Copy link
Copy Markdown
Contributor Author

/test pull-gateway-api-verify

@rikatz

rikatz commented May 13, 2026

Copy link
Copy Markdown
Member

/label tide/merge-method-squash

@k8s-ci-robot k8s-ci-robot added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label May 13, 2026
Comment thread apis/v1/backendtlspolicy_types.go Resolved
Comment thread apis/v1/gateway_types.go Resolved
Comment thread apis/v1/listenerset_types.go Resolved
@rikatz

rikatz commented May 13, 2026

Copy link
Copy Markdown
Member

@ElenaZvereva thanks. But why are there changes on the stable API? Can we revert it?

Thanks!

@ElenaZvereva

Copy link
Copy Markdown
Contributor Author

@rikatz Hi! Yes, of course, I’ll revert it.

I added these changes only to test how the script behaves with breaking CRD changes in Prow.

One thing I noticed is that because CRDIFY_ENFORCE is currently set to false by default, the job still reports success even when breaking changes are detected. As a result, the overall output shows All tests passed successfully and SUCCESS: hack/../hack/verify-crdify.sh, while the breaking changes are visible only in the script output and can be easily overlooked.

Example run: https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_gateway-api/4830/pull-gateway-api-verify/2054517164261511168

Do you think this behavior is good enough for now, or should we make the warning more visible while keeping the job non-blocking?

@rikatz

rikatz commented May 13, 2026

Copy link
Copy Markdown
Member

hum is there a way we can make the warning more visible on prow without blocking the job? 🤔

if so it would be great, otherwise we can probably enable it by default (or add a github action just for this job that can fail but is not required), wdyt?

@ElenaZvereva

Copy link
Copy Markdown
Contributor Author

Yes, I think we have a few options to make the warning more visible (although it still stays only in prow):

  • Use ANSI colors for the warning output.
  • Update verify-all.sh so that it prints an additional warning if crdify detects breaking changes.

But I think enabling CRDIFY_ENFORCE by default is a good option.

To be honest, I’m not sure I fully understand the concern around BackendTLSPolicy. Do we expect some necessary changes to this CRD in the near future that would be considered breaking changes by crdify?

In other words, why do we need the CRDIFY_ENFORCE flag in the first place?

@rikatz

rikatz commented May 13, 2026

Copy link
Copy Markdown
Member

the flag is more like a guardrail/breaking glass button IMO. Let's enable it by default, if we feel this will break releases or slow us down I can work on disabling it and merging quickly.

Once more, appreciate the effort!

@ElenaZvereva
ElenaZvereva requested a review from rikatz May 13, 2026 16:55
@ElenaZvereva

Copy link
Copy Markdown
Contributor Author

Ok, sounds good. I’ve enabled the flag by default
If there’s anything else around CI and infrastructure, I’d be glad to help

@rikatz

rikatz commented May 13, 2026

Copy link
Copy Markdown
Member

For now this is all, thank you very much @ElenaZvereva !
/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 13, 2026
@k8s-ci-robot

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ElenaZvereva, rikatz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 13, 2026
@rikatz

rikatz commented May 14, 2026

Copy link
Copy Markdown
Member

/release-note-none

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels May 14, 2026
@k8s-ci-robot
k8s-ci-robot merged commit c2e7af0 into kubernetes-sigs:main May 14, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants