close
Skip to content

Latest commit

 

History

History
 
 

Tutorials of Triton-distributed

In this session, we provide a list tutorials for writing various distributed operations with Triton-distributed. It is recommended that you first read the technique report, which contains design and implementation details, and then play with these tutorials.

  1. [Primitives]: Basic notify and wait operation
  2. [Primitives & Communication]: Use copy engine and NVSHMEM primitives for AllGather
  3. [Communication]: Inter-node AllGather
  4. [Communication]: Intra-node and Inter-node DeepSeek EP AllToAll
  5. [Communication]: Intra-node ReduceScatter
  6. [Communication]: Inter-node ReduceScatter
  7. [Overlapping]: AllGather GEMM overlapping
  8. [Overlapping]: GEMM ReduceScatter overlapping
  9. [Overlapping]: AllGather GEMM overlapping on AMD
  10. [Overlapping]: GEMM ReduceScatter overlapping on AMD