Accelerating Long-Context Inference with Skip Softmax Attention
Skip Softmax Attention is a drop-in post-training sparse attention method and the productized implementation of BLASST in NVIDIA TensorRT-LLM. It is available for accelerating both LLM workloads and visual-generation workloads.
Originally published on the NVIDIA TensorRT-LLM tech blog. Reposted here.