Lei Li

Master Student

Peking University


Hi, I am a final year master student at School of EECS, Peking University.

Currently, I am a member of Language Computing and Machine Learning Group (LANCO), advised by Prof. Xu Sun.

I am enthusiastic about Natural Lanuge Processing (NLP). My research focuses on applying machine learning techniques for improving the efficiency of pre-trained language models.


  • Multi-modal Pre-training
  • Efficient Methods for NLP
  • Knowledgeable NLP


  • MSc in Computer Science, 2020 - 2023 (Expected)

    Peking University

  • BE in Software Engineering, 2016 - 2020

    Xidian University


Distributional Correlation–Aware Knowledge Distillation for Stock Trading Volume Prediction

Traditional knowledge distillation in classification problems transfers the knowledge via class correlations in the soft label produced …

Model Uncertainty-Aware Knowledge Amalgamation for Pre-Trained Language Models

As many fine-tuned pre-trained language models~(PLMs) with promising performance are generously released, investigating better ways to …

Rethinking the Openness of CLIP

Contrastive Language-Image Pre-training (CLIP) has demonstrated great potential in realizing open-vocabulary image classification in a …

Dynamic Knowledge Distillation for Pre-trained Language Models