GPU Parallelism

Training Large Models on Multiple GPUs

data/ model/ xxx parallelism

https://lilianweng.github.io/posts/2021-09-25-train-large/