HPC is very difficult domain to address all settings. With the current diverse hardware and compositional learning needs. Ray is now becoming de-facto framework for compositional learning infrastructure.

Ray over internet is also possible as well as K8 https://docs.ray.io/en/latest/cluster/kubernetes/index.html