Publications

TriAttention proposes a novel KV cache compression approach for long reasoning in LLMs. It leverages trigonometric series based on fixed centers in pre-RoPE space to score key importance, achieving 2.5x higher throughput or 10.7x KV memory reduction while matching Full Attention reasoning accuracy.

PSA introduce an efficient attention mechanism to accelerate video understanding and generation. It leverages a multi-level sparse attention strategy, enabling the model to effectively mitigates information loss while preserving computational efficiency under a low compute budget.