Jiaqi Leng

I am an incoming PhD student at NYU and am currently completing my undergraduate studies in Computer Science and Technology at Fudan University.

Under the supervision of Prof. Yucheng Lu, my current research focuses on efficient language modeling, with particular emphasis on byte-level architectures. I previously worked as a research intern at Ant Group, where I studied sparse attention mechanisms for large language models.

My research interests include:

Efficient deep learning and model architectures
Long-context modeling and length extrapolation
Sparse attention mechanisms

In Fall 2024, I was an exchange student at The University of Texas at Austin.

I welcome opportunities for academic collaboration and discussion.

News

Mar 2026	I will be attending ICLR and presenting our work on length-generalizable sparse attention. See you in Brazil!
Mar 2026	I will join NYU as a PhD student in Fall 2026.

Selected Publications

ICLR 2026

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models

Jiaqi Leng^*, Xiang Hu^*, Junxiong Wang, Jianguo Li, Wei Wu, and Yucheng Lu

In Proceedings of the 14th International Conference on Learning Representations, 2026

Abs arXiv Code
We analyze chunk-based sparse attention for long-context generalization and identify its essential ingredients.

Key principles:

non-linear Chunk Encoder with a dedicated CLS token for retrieval

Bypassing Residual Path to integrate global information

enforced selection sparsity in pre-training to close the train–test gap

Evidence: theoretical motivation and unified ablations

Result: state-of-the-art training-free length extrapolation, generalizing 4K-trained models to 32M tokens on RULER and BABILong.
Preprint

Distilling Token-Trained Models into Byte-Level Models

Zishuo Bao, Jiaqi Leng, Junxiong Wang, Bowen Peng, and Yucheng Lu

arXiv preprint, 2026

Abs arXiv
We propose a low-cost distillation pipeline to convert token-trained LLMs into byte-level models without training from scratch.

Method:

Progressive Knowledge Distillation to align byte representations with teacher embeddings

Byte-Level Supervised Fine-Tuning for end-to-end byte generation

Evaluation: Llama, Qwen, and OLMo

Result: comparable performance with only 125B bytes of data.
NeurIPS 2025

Hardware-aligned Hierarchical Sparse Attention for Efficient Long-term Memory Access

Xiang Hu, Jiaqi Leng, Jun Zhao, Kewei Tu, and Wei Wu

In Proceedings of the 39th Conference on Neural Information Processing Systems, 2025

Abs arXiv Code
We introduce Hierarchical Sparse Attention (HSA) to give RNNs efficient long-range random access without losing linear complexity.

Method: chunk inputs, select top-k chunks, and hierarchically aggregate using token-level relevance; a hardware-aligned kernel keeps it efficient

System: Mamba + HSA (RAMba)

Result: perfect passkey retrieval up to 64M tokens from 4K pre-training, plus strong downstream gains with near-constant memory.