Repository navigation
Torch compile backend defaults to "neuron" - #47035
michaelbenayoun merged 4 commits into
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
stevhliu
left a comment
stevhliu
left a comment
There was a problem hiding this comment.
thanks, it would be good to add a sentence or two in https://huggingface.co/docs/transformers/main/en/torch_compile#backend about this! something like:
When no backend is specified, [TrainingArguments] selects one based on your hardware. On most CPUs and GPUs, the default is inductor, which compiles to Triton kernels with AOTAutograd and suits most training workloads. On Intel Gaudi (HPU), the default is hpu_backend. On AWS Trainium and Inferentia (Neuron), the default is neuron.
CI recapDashboard: View test results in Grafana |
* feat: torch compile backend defaults to "neuron" * fix: restore doc back to normal * doc: add information about backend in docs
* feat: torch compile backend defaults to "neuron" * fix: restore doc back to normal * doc: add information about backend in docs
What does this PR do?
When
--torch-compileis set and no backend is specified, defaults to "neuron" when a torch neuron is available.Follows the same spirit as what's done for HPUs.