I was able to install PyTorch and train Karpathy's minGPT (small) on a Ryzen MAX APU on Windows machine using official instructions: https://rocm.docs.amd.com/en/7.9.0-preview/install/pytorch-comfyui.html ROCm

TL;DR;

  pip install --index-url https://repo.amd.com/rocm/whl/gfx1151/ "rocm[libraries,devel]"
  hipinfo
  pip install --index-url https://repo.amd.com/rocm/whl/gfx1151/ torch torchvision torchaudio
  python -c "import torch; print(torch.cuda.is_available())"

Be sure to uninstall torch and torchvision if you had CPU-only versions installed.

If you have a different GPU/APU refer to the first link

minGPT: https://github.com/karpathy/minGPT