This document is relevant for: Trn2, Trn3

Install PyTorch for Neuron#

Install TorchNeuron, the native PyTorch backend for AWS Trainium, for training and inference on Trn2 and Trn3 instances.

Supported Instances: Trn2, Trn3

PyTorch Version: 2.13 or 2.14 (torch-neuronx 2.13.3.0 or 2.14.3.0)

Choose installation method#

Method

Best for

Considerations

DLAMI

Getting started quickly, prototyping, single-user development

Pre-configured with tested dependency versions; launch a new AMI to update

DLC

Production deployments, CI/CD pipelines, multi-tenant environments

Requires Docker and Neuron driver on host; portable across EC2, ECS, EKS

Manual

Custom OS images, shared clusters, integrating into existing environments

Full control over versions and dependencies; requires manual dependency management

πŸš€ AWS Deep Learning AMI

Recommended for most users

Pre-configured environment with all dependencies

βœ… All dependencies included

βœ… Tested configurations

βœ… Multiple Python versions

⏱️ Setup time: ~5 minutes

🐳 Deep Learning Container

For containerized deployments

Pre-configured Docker images from AWS ECR

βœ… Docker-based isolation

βœ… Training and inference images

βœ… vLLM-ready images available

⏱️ Setup time: ~10 minutes

πŸ”§ Manual Installation

For custom environments

Install on existing systems or custom setups

βœ… Existing system integration

βœ… Custom Python versions

βœ… Full control over dependencies

⏱️ Setup time: ~15 minutes

Prerequisites#

Before installing, ensure you have:

Requirement

Details

Instance Type

Trn2 or Trn3 instance

Operating System

Ubuntu 26.04, Ubuntu 24.04, or Amazon Linux 2023. For the tested Linux kernel and GLIBC versions, see Neuron Runtime and Driver compatibility reference.

Python Version

Python 3.12, 3.13, or 3.14

AWS Account

With EC2 launch permissions

SSH Access

Key pair for instance connection

What You’ll Get#

After installation, you’ll have:

  • PyTorch 2.13 or 2.14 with the TorchNeuron native backend

  • torch-neuronx package (TorchNeuron) for Neuron-specific operations

  • neuronx-cc compiler for model optimization

  • Neuron Runtime for model execution

  • Neuron Tools for profiling and debugging

Version Information#

Component

Version

PyTorch

2.13.0 or 2.14.0

torch-neuronx

2.13.3.0 or 2.14.3.0

neuronx-cc

2.28.0+

Python

3.12, 3.13, 3.14

Next Steps#

After installation:

  1. Verify Installation: Run verification commands in the installation guide

  2. Try a Tutorial:

  3. Read the TorchNeuron documentation: TorchNeuron: Native PyTorch on AWS Neuron

  4. Explore tutorials: TorchNeuron tutorials

Update an existing installation#

Already have PyTorch Neuron installed and need to update to a newer PyTorch version or Neuron SDK release? Select the guide that matches your installation method.

πŸ”„ Update DLAMI

Update PyTorch and drivers on an existing Deep Learning AMI

πŸ”„ Update DLC

Pull the latest container image and update the host driver

πŸ”„ Update manual install

Update PyTorch packages and drivers on a manual installation

This document is relevant for: Trn2, Trn3