Optimizing MoE Communication with One-Sided AlltoAll Over NVLink
This one-sided AlltoAll design for MoE EP communication originated in NVIDIA TensorRT-LLM and has since been ported to FlashInfer. It is currently the fastest available MoE EP AlltoAll implementation for inference on NVLink and has been integrated into vLLM and SGLang. A presentation (in Chinese) is also available.
The original article was published in NVIDIA TensorRT-LLM and is embedded here for convenience.