Repository navigation
AMD ROCm
AMD ROCm (Radeon Open Compute) is a software platform for GPU computing, designed to support high-performance computing (HPC) and machine learning workloads on AMD GPUs. It provides a set of tools, libraries, and drivers to enable efficient GPU acceleration for various applications.
This guide is a step-by-step instruction for setting up AMD ROCm on Linux for use with SD.Next
For official AMD ROCm documentation, visit AMD ROCm Documentation
AMD ROCm provides torch builds optimized for AMD GPUs via TheRock project
See supported GPUs for a list of AMD GPUs compatible with TheRock torch builds
Tip
SD.Next typically autodetects AMD ROCm
Detection can be forced using --use-rocm flag
However, if autodetect fails, it typically indicates that ROCm is not installed or not properly configured on your system
Note
SD.Next will auto-install torch when not already present on the system
If you want to use a specific version of torch, you will need to install it manually before starting SD.Next
Important
SD.Next installs torch only on first startup
Subsequent startups or updates will NOT reinstall torch
unless its either a) removed by user, b) forced using --reinstall flag
Warning
AMD's level of support for ROCm varies across different platforms and GPU architectures
For example, different versions of torch or rocm may have varying levels of compatibility with specific AMD GPUs: its not guaranteed that the latest version will work with all hardware
Due to large differences between Linux and Windows support for AMD ROCm, this guide provides separate instructions for each platform:
On first use, on first use of a new resolution, or after a PyTorch upgrade, ROCm runs benchmarks to pick efficient kernels
This can make startup slow (up to 5-8 minutes), especially with high-resolution refine passes, but it usually happens once per resolution
If startup time is the priority, set environment variable MIOPEN_FIND_MODE=FAST
If generation performance is the priority, set MIOPEN_FIND_ENFORCE=SEARCH and accept slower first-time startup
For details, see AMD-MIOpen Guide
If you use bf16 (Settings -> Compute Settings -> Execution Precision -> Device precision type), which is auto-detected on RDNA3 and newer cards, VRAM usage can become very high (16+ GB) during final decode and non-latent upscaling
To reduce usage, set Device precision type to fp16 and disable VAE upcasting in Variational Auto Encoder > VAE upcasting
Using fp16 can also improve performance.
On RDNA3 hardware (RX 7000 series), you can enable CK Flash Attention in Compute Settings -> Cross Attention -> SDP Options by toggling CK Flash attention and restarting SD.Next
This requires rocm-hip-sdk, because SD.Next downloads and compiles an additional Python package at startup
In case you want to install it manually, activate the virtual environment then run pip:
pip install --no-build-isolation git+https://github.com/Disty0/flash-attention@navi_rotary_fix