Skip Softmax Attention is a drop-in post-training sparse attention method and the productized implementation of BLASST in NVIDIA TensorRT-LLM. It is available for accelerating both LLM workloads and visual-generation workloads.

The original article was published in NVIDIA TensorRT-LLM and is embedded here for convenience.

This one-sided AlltoAll design for MoE EP communication originated in NVIDIA TensorRT-LLM and has since been ported to FlashInfer. It is currently the fastest available MoE EP AlltoAll implementation for inference on NVLink and has been integrated into vLLM and SGLang. A presentation (in Chinese) is also available.

The original article was published in NVIDIA TensorRT-LLM and is embedded here for convenience.