avatar

Runming Yang
杨润明

Ph.D. student at the University of Hong Kong
yang3121099@126.com

Biography

I am a Ph.D. student in Electrical and Electronic Engineering at the University of Hong Kong, supervised by Prof. Ngai Wong. I received my M.Eng. in Big Data Engineering from Tsinghua University in 2025, advised by Prof. Yujiu Yang, and my B.Eng. from Huazhong University of Science and Technology in 2021.

中文学术简历（PDF）   ·   English CV (PDF)

Research Interests

LLM post-training; Knowledge distillation; Efficient reasoning; Low-bit quantization. My work explores model compression and efficient reasoning in large language models.

Publications

Selected published and accepted papers. Full publications and preprints (Chinese PDF).

* indicates equal contribution.

Industry Experience

Tencent AI Lab — Research Intern, Information Security Department
Nov. 2023 - Jun. 2025

  • Synthesized multimodal data and conducted model post-training and evaluation for image-text content moderation in QQ group chats, Qzone, and short-video comments.
  • Deployed models in tiers and accelerated inference; trained models were deployed in internal risk-control systems, improving throughput and reducing false-positive rates over the previous solution.
  • Researched large/small model collaboration, data distillation, and lightweight modeling; contributed to two academic papers (Shadow-FT and LLM-NEO) and one company patent.

Current Research

CoT Data and Policy Distillation for Reasoning Model Post-Training — Ongoing
OPD-to-RL experiments: studying teacher-distribution supervision on student-generated trajectories as preparation for RL, with analysis of training signals and computational cost. Preliminary results in this setting: Qwen3-1.7B achieved 40.5% mean answer accuracy across three seeds on controlled arithmetic, versus 21.0% for direct RL. A separate single-seed experiment with a larger sampling budget improved accuracy from 39.65% to 49.22%. Separate mechanistic hypotheses (under investigation): exploring selective distribution constraints on residual-error states to reduce error recurrence while preserving correct reasoning branches, with potential applications to OPD, RL, and adaptive thinking.

Projects

Project 1 Image

LoCa (ECAI 24')

  • Identify that misclassification by the teacher during distillation harms student learning.
  • Propose LoCa, calibrating the logits distribution of misclassified samples to ensure target label accuracy.
  • Conduct experiments on image classification and natural language generation tasks; easily extendable to LLMs.
  • Improve accuracy while adding only 1% computational cost as an enhanced plug-in.

    • arxiv
    Project 2 Image

    LLM-Neo

  • Discover similarities between Knowledge Distillation and Low-rank Adapter methods in knowledge transfer for LLMs.
  • Combine both ideas and create a more efficient knowledge transfer strategy, LLM-Neo.
  • Demonstrate better performance on the Llama 3.1 series compared to SFT, LoRA, and KD

    • huggingface
    Project 2 Image

    Kaggle (🥈Silver Medal)

  • Aim to assess juniors' English writing proficiency, which is few-samples and long-text.
  • Utilize techniques like data augmentation, token truncation and dropout regularization.
  • Achieve a silver medal via 8 models (i.e., RoBERTa) ensembled and 5-fold validation.

    • kaggle
    Project 2 Image

    Need for Speed (HUST Undergraduate Course Design)

  • A copy of "Need for Speed" game using pure C language.
  • Pixel style, retro sentiment
  • Implemente key game mechanics, i.e., acceleration, turning, drifting, and collision handling.

    • (Last updated: 2026-10)




    One more thing that excites me is meeting the gradient-color cloud and lonely tree. I'd like to capture them and share them with others!