Skip to content

AMD ROCm

Vladimir Mandic edited this page Nov 16, 2026 · 53 revisions

AMD ROCm

AMD ROCm (Radeon Open Compute) is a software platform for GPU computing, designed to support high-performance computing (HPC) and machine learning workloads on AMD GPUs. It provides a set of tools, libraries, and drivers to enable efficient GPU acceleration for various applications.

This guide is a step-by-step instruction for setting up AMD ROCm on Linux for use with SD.Next
For official AMD ROCm documentation, visit AMD ROCm Documentation

AMD ROCm provides torch builds optimized for AMD GPUs via TheRock project
See supported GPUs for a list of AMD GPUs compatible with TheRock torch builds

Tip

SD.Next typically autodetects AMD ROCm
Detection can be forced using --use-rocm flag However, if autodetect fails, it typically indicates that ROCm is not installed or not properly configured on your system

Note

SD.Next will auto-install torch when not already present on the system
If you want to use a specific version of torch, you will need to install it manually before starting SD.Next

Important

SD.Next installs torch only on first startup
Subsequent startups or updates will NOT reinstall torch
unless its either a) removed by user, b) forced using --reinstall flag

Warning

AMD's level of support for ROCm varies across different platforms and GPU architectures
For example, different versions of torch or rocm may have varying levels of compatibility with specific AMD GPUs: its not guaranteed that the latest version will work with all hardware

Platform Guides

Due to large differences between Linux and Windows support for AMD ROCm, this guide provides separate instructions for each platform:

Tips & Advanced Configuration

MIOpen database tuning

On first use, on first use of a new resolution, or after a PyTorch upgrade, ROCm runs benchmarks to pick efficient kernels

This can make startup slow (up to 5-8 minutes), especially with high-resolution refine passes, but it usually happens once per resolution

If startup time is the priority, set environment variable MIOPEN_FIND_MODE=FAST
If generation performance is the priority, set MIOPEN_FIND_ENFORCE=SEARCH and accept slower first-time startup

For details, see AMD-MIOpen Guide

Reduce VRAM consumption

If you use bf16 (Settings -> Compute Settings -> Execution Precision -> Device precision type), which is auto-detected on RDNA3 and newer cards, VRAM usage can become very high (16+ GB) during final decode and non-latent upscaling

To reduce usage, set Device precision type to fp16 and disable VAE upcasting in Variational Auto Encoder > VAE upcasting

Using fp16 can also improve performance.

Composable Kernel (CK) Flash attention

On RDNA3 hardware (RX 7000 series), you can enable CK Flash Attention in Compute Settings -> Cross Attention -> SDP Options by toggling CK Flash attention and restarting SD.Next

This requires rocm-hip-sdk, because SD.Next downloads and compiles an additional Python package at startup

In case you want to install it manually, activate the virtual environment then run pip:

pip install --no-build-isolation git+https://github.com/Disty0/flash-attention@navi_rotary_fix

Clone this wiki locally