Skip to content

Give the CUDA plugin build the CUDA 13 CCCL include path - #32392

Merged
Tianlei Wu (tianleiwu) merged 2 commits into
microsoft:mainfrom
arnej27959:arnej/add-cccl-include-path-for-plugin-build
Sep 2, 2026
Merged

Tianlei Wu (tianleiwu) merged 2 commits into
microsoft:mainfrom
arnej27959:arnej/add-cccl-include-path-for-plugin-build

Conversation

@arnej27959

Copy link
Copy Markdown
Contributor

Description

CUDA 13.0 moved libcu++, CUB and Thrust from /include to /include/cccl. onnxruntime_providers_cuda.cmake compensates for that inside config_cuda_provider_shared_module, but the plugin EP's cmake never did. The plugin target globs the same host .cc files that include CUTLASS headers (contrib_ops/cuda/llm/cutlass_heuristic.cc and others), and those reach <cuda/std/utility>, so on a CUDA 13 toolkit they failed with

cutlass/cutlass.h:40:33: fatal error: cuda/std/utility: No such file or directory

Move the include handling - and the CUDA 13.3 cudafe++ header workaround it depends on - out of onnxruntime_providers_cuda.cmake and into the new shared cmake/onnxruntime_cuda_cccl.cmake, then call it from both provider cmakes.

The include flags of the in-tree provider build are unchanged. The only adjustments made while extracting the code are:

  • The 13.0 version check becomes an early return instead of wrapping the function body.
  • The generated-header directory is pinned to ${CMAKE_BINARY_DIR} rather than ${CMAKE_CURRENT_BINARY_DIR}, which in a function expands in the caller's directory scope. Both are the same directory for the two call sites (cmake/CMakeLists.txt is the top-level list file), but a shared module should not generate files in a caller-dependent location. This matches op_reduction_root in onnxruntime_providers.cmake, the closest analogue: a configure-time generated header tree injected per target.
  • The entry point is named ort_configure_cuda_cccl, since it generates files in the build tree as well as adding include directories.

Comments record the two things that are easy to break later: the SM-specific OBJECT libraries of the plugin target inherit these include directories through a generator expression, which is why calling the function after they are created still covers them; and the CUDA 13.3 workaround is UNIX-only because it is untested on Windows, not because MSVC is known to be unaffected.

Motivation and Context

Trying a plugin build failed with:
cutlass/cutlass.h:40:33: fatal error: cuda/std/utility: No such file or directory

CUDA 13.0 moved libcu++, CUB and Thrust from <toolkit>/include to
<toolkit>/include/cccl. onnxruntime_providers_cuda.cmake compensates
for that inside config_cuda_provider_shared_module, but the plugin EP's
cmake never did.  The plugin target globs the same host .cc files that
include CUTLASS headers (contrib_ops/cuda/llm/cutlass_heuristic.cc and
others), and those reach <cuda/std/utility>, so on a CUDA 13 toolkit
they failed with

  cutlass/cutlass.h:40:33: fatal error: cuda/std/utility: No such file or directory

Move the include handling - and the CUDA 13.3 cudafe++ header workaround it
depends on - out of onnxruntime_providers_cuda.cmake and into the new shared
cmake/onnxruntime_cuda_cccl.cmake, then call it from both provider cmakes.

The include flags of the in-tree provider build are unchanged. The only
adjustments made while extracting the code are:

  * The 13.0 version check becomes an early return instead of wrapping the
    function body.
  * The generated-header directory is pinned to ${CMAKE_BINARY_DIR} rather
    than ${CMAKE_CURRENT_BINARY_DIR}, which in a function expands in the
    caller's directory scope. Both are the same directory for the two call
    sites (cmake/CMakeLists.txt is the top-level list file), but a shared
    module should not generate files in a caller-dependent location. This
    matches op_reduction_root in onnxruntime_providers.cmake, the closest
    analogue: a configure-time generated header tree injected per target.
  * The entry point is named ort_configure_cuda_cccl, since it generates
    files in the build tree as well as adding include directories.

Comments record the two things that are easy to break later: the SM-specific
OBJECT libraries of the plugin target inherit these include directories
through a generator expression, which is why calling the function after they
are created still covers them; and the CUDA 13.3 workaround is UNIX-only
because it is untested on Windows, not because MSVC is known to be
unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings September 2, 2026 14:08
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@arnej27959

Copy link
Copy Markdown
Contributor Author

Note: analyzing the moved cmake config showed that it rewrites the headers on every reconfigure; I have a commit
"Stop rewriting the patched CCCL headers on every cmake reconfigure" if you want it (in this PR or separate).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟑 Changes recommended

The new shared build input is missing from release tracking and no-cuDNN workflow path filters.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Extracts CUDA 13 CCCL setup into a shared helper for both CUDA provider builds.

Changes:

  • Centralizes CCCL include and CUDA 13.3 workaround logic.
  • Applies the helper to the standard and plugin CUDA providers.
File summaries
File Description
cmake/onnxruntime_cuda_cccl.cmake Adds shared CCCL configuration.
cmake/onnxruntime_providers_cuda.cmake Uses the shared helper.
cmake/onnxruntime_providers_cuda_plugin.cmake Configures CCCL for plugin targets.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Balanced

πŸ’‘ Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread cmake/onnxruntime_cuda_cccl.cmake Resolved
The previous commit added cmake/onnxruntime_cuda_cccl.cmake as a build
input shared by the in-tree CUDA provider and the CUDA plugin EP.
Add it to:
plugin-ep-cuda/paths.txt (release-change detection for the plugin),
paths filters of linux_cuda_no_cudnn.yml and windows_cuda_no_cudnn.yml
decide whether those builds run on a PR.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟒 Approval recommended

The extraction preserves existing provider behavior while correctly extending CUDA 13 CCCL support to the plugin target.

Review details
  • Files reviewed: 6/6 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@tianleiwu
Tianlei Wu (tianleiwu) enabled auto-merge (squash) September 2, 2026 16:50
@tianleiwu
Tianlei Wu (tianleiwu) merged commit 44100ea into microsoft:main Sep 2, 2026
92 of 94 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants