Hao Gu Email

Ph.D. student · HKUST / Research intern · Tencent Hunyuan

Hao Gu

I work on model architecture and LLM pre‑training, currently focusing on sparse attention.

HKUSTPh.D. student
Tencent HunyuanResearch intern

Research

Three threads so far — long context, mixture-of-experts and low-bit models — with sparse attention at the centre.

Publications

2026
  1. arXiv First author

    The Gating Mechanism of KV Cache Eviction

    Hao Gu, X. Yang, Z. Li, L. Zhu, J. Yang, Q. Zhu, D. Li, J. Zhang, Z. Kou, H. Wang, et al.

  2. arXiv

    SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking

    Z. Li, L. Zhu, Hao Gu, X. Hu, Y. Wang, H. Mi, S. Han, L. Liang, Z. Guo

  3. arXiv

    Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling

    X. Hu, X. Wei, Hao Gu, M. Zhang, T. Liang, H. Li, L. Zhu, Y. Wang, S. Han, Y. Bai, et al.

2025
  1. arXiv First author

    D²-MoE: Delta Decompression for MoE-based LLMs Compression

    Hao Gu, W. Li, L. Li, Q. Zhu, M. Lee, S. Sun, W. Xue, Y. Guo

News

  • Released The Gating Mechanism of KV Cache Eviction, with code.
  • SAS (attention sparsification) and OMP-MoE (expert pruning) are on arXiv.
  • HiLS-Attention, hierarchical sparse attention toward infinite context, is out — code open-sourced at Tencent Hunyuan.
  • IndexMem: learned KV-cache eviction with latent memory.
  • Four papers accepted to ACL 2026 and one to Findings of ACL 2026, including first-author BTC-LLM and QaRL.
  • Two papers accepted to AAAI 2026: Sub-MoE and Outlier-Guided Model Merging.
  • Released D²-MoE, delta decompression for MoE compression — code.

Blog

All posts

Notes on LLM architecture, pre-training and sparse attention.

The first post is on its way. Subscribe via RSS.

WeChat QR code for MarkusFits
WeChat ID MarkusFits