Biography

I am a Ph.D. student in Electrical and Electronic Engineering at the University of Hong Kong, supervised by Prof. Ngai Wong. I received my M.Eng. in Big Data Engineering from Tsinghua University in 2025, advised by Prof. Yujiu Yang, and my B.Eng. from Huazhong University of Science and Technology in 2021.

中文学术简历（PDF）   ·   English CV (PDF)

Research Interests

LLM post-training; Knowledge distillation; Efficient reasoning; Low-bit quantization. My work explores model compression and efficient reasoning in large language models.

Publications

Selected published and accepted papers. Full publications and preprints (Chinese PDF).

* indicates equal contribution.

• Revisiting Model Interpolation for Efficient Reasoning
  Taiqiang Wu*, Runming Yang*, Tao Liu, Jiahao Wang, Ngai Wong. ACL 2026 (Main) [Paper]

• ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection
  Tao Liu*, Taiqiang Wu*, Runming Yang, Shaoning Sun, Junjie Wang, Yujiu Yang. ACL 2026 (Findings) [Paper]

• Can We Trust LLMs on Memristors? Diving into Reasoning Ability under Non-Ideality
  Taiqiang Wu*, Yuxin Cheng*, Chenchen Ding*, Runming Yang, Xincheng Feng, Wenyong Zhou, Zhengwu Liu, Ngai Wong. COLM 2026 [Paper]

• Exploring Layer-wise Information Effectiveness for Post-Training Quantization in Small Language Models
  He Xiao, Qingyao Yang, Dirui Xie, Wendong Xu, Zunhai Su, Runming Yang, Haobo Liu, Wenyong Zhou, Zhengwu Liu, Ngai Wong. ACL 2026 (Findings) [Paper]

• Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models
  Taiqiang Wu, Chaofan Tao, Jiahao Wang, Runming Yang, Zhe Zhao, Ngai Wong. COLING 2025 (Oral) [Paper]

• LoCa: Logit Calibration for Knowledge Distillation
  Runming Yang, Taiqiang Wu, Yujiu Yang. ECAI 2024 [Paper]

Industry Experience

Tencent — Research Intern
Nov. 2023 - Jun. 2025

• Built data-synthesis pipelines for image-text content moderation and used the resulting data for model post-training and evaluation.
• Deployed models in tiers in production, fine-tuning large models on the full dataset and distilling smaller models using filtered data, improving throughput and reducing false positives over the previous solution.
• Researched large/small model collaboration and lightweight modeling, contributing to two academic papers and one company patent.

Current Research

On-policy Distillation: Thinking-Mode Transfer and Subsequent RLHKU — 2026.09 - 2026.10

Qwen3 0.6B / 1.7B / 4B students and an 8B teacher, with comparisons of teacher prompting, generation budgets, and stopping behavior. Accuracy figures use MATH-500 avg@4.

• Thinking-mode teachers overcome saturation in standard OPD gains. Gains from non-thinking teachers decline with student size, reaching zero at 4B. With a thinking-mode teacher, 1.7B / 4B still gain 13.3 / 9.5 percentage points over their base models at a 16K budget.
• Behavioral evidence supports transfer of the student's own thinking mode. At 32K, 1.7B / 4B reach 90.9% / 95.2%, approaching their own thinking-mode performance, with similar results from self-distillation at all three sizes. Per-question accuracy profiles align more closely with the student's thinking mode and correlate significantly less with the external teacher, supporting gains from accessing existing reasoning capabilities.
• Distinguished solving a problem from delivering an answer on time. After truncation at 2K, allowing 32 additional tokens for a forced answer raises 1.7B accuracy from 17.8% to 72.8%, showing that OPD raises the non-thinking accuracy ceiling while also causing length inflation. Post-answer p(EOS) and cross-continuation probes show that the teacher still favors continuation after the student's answer, indicating that stopping behavior does not transfer alongside the thinking mode.
• Characterized the benefits of OPD+RL and length control. Under the same downstream RL setup, 1.7B starting from OPD reaches 87.3% at a 16K budget, versus 81.7% for direct RL. Short-run RL shows that rollout length affects the trade-off between length compression and accuracy ceilings. Propose reward adjustment using an adaptive length threshold at fixed rollout length to reduce redundant generation while preserving the reasoning needed for difficult problems.