Accelerating Long-Context Inference with Skip Softmax Attention
Skip Softmax Attention is a drop-in post-training sparse attention method and the productized implementation of BLASST in NVIDIA TensorRT-LLM. It is available for accelerating both LLM workloads and visual-generation workloads.
The original article was published in NVIDIA TensorRT-LLM and is embedded here for convenience.