Optimizing MoE Communication with One-Sided AlltoAll Over NVLink
This one-sided AlltoAll design for MoE EP communication originated in NVIDIA TensorRT-LLM and has since been ported to FlashInfer. It is currently the fastest available MoE EP AlltoAll implementation for inference on NVLink and has been integrated into vLLM and SGLang. A presentation (in Chinese) is also available.
Originally published on the NVIDIA TensorRT-LLM tech blog. Reposted here.