Research
Three threads so far — long context, mixture-of-experts and low-bit models — with sparse attention at the centre.
Publications
-
arXiv First author
The Gating Mechanism of KV Cache Eviction
-
arXiv
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
-
arXiv
OMP-MoE: Efficient Expert Pruning for Mixture-of-Experts LLMs via Orthogonal Matching Pursuit
arXiv 1 citation
-
arXiv
Fewer Tokens, Smaller Cache: Reward-Coordinated Efficient Reasoning
arXiv 1 citation
-
arXiv
Hierarchical Sparse Attention Done Right: Toward Infinite Context Modeling
arXiv Code★ 150+ 15 citations
-
arXiv
IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference
arXiv 8 citations
-
ACL 2026 First author
BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook
arXiv 9 citations
-
ACL 2026 Findings First author
QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training–Inference Mismatch
arXiv 18 citations
-
ACL 2026
Learning While Staying Curious: Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models
arXiv 11 citations
-
ACL 2026
Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs
arXiv 3 citations
-
ACL 2026
Adaptive Spatial and Temporal Redundancy Optimization for Efficient Reasoning in Large Language Models
Scholar 3 citations
-
AAAI 2026
Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
arXiv 51 citations
-
AAAI 2026
Outlier Matters: Efficient Long-to-Short Reasoning via Outlier-Guided Model Merging
Scholar 5 citations
-
arXiv First author
D²-MoE: Delta Decompression for MoE-based LLMs Compression
-
Preprint
Structured Mixture-of-Experts LLMs Compression via Singular Value Decomposition
Scholar 46 citations
Nothing here yet.
News
- Released The Gating Mechanism of KV Cache Eviction, with code.
- SAS (attention sparsification) and OMP-MoE (expert pruning) are on arXiv.
- HiLS-Attention, hierarchical sparse attention toward infinite context, is out — code open-sourced at Tencent Hunyuan.
- IndexMem: learned KV-cache eviction with latent memory.
- Four papers accepted to ACL 2026 and one to Findings of ACL 2026, including first-author BTC-LLM and QaRL.
- Two papers accepted to AAAI 2026: Sub-MoE and Outlier-Guided Model Merging.
- Released D²-MoE, delta decompression for MoE compression — code.
Blog
All postsNotes on LLM architecture, pre-training and sparse attention.
The first post is on its way. Subscribe via RSS.