Yekun Chai

Baidu NLP


chaiyekun (at)

I am a staff research engineer working on large language models (LLMs) at Baidu NLP. Before that, I was associated with Institute of Automation, Chinese Academy of Sciences (CASIA). I graduated from Edinburgh Informatics in 2018 under the supervision of Adam Lopez and Naomi Saphra.

My research endeavors revolve around the generative pre-training paradigm of NLP, with a particular emphasis on:

  • General language model pre-training, prompting, instruction tuning, and their variants across tasks, languages, and modalities;
  • LLM alignment with human preferences;
  • Augmented LLMs with non-parametric priors.


Feb 20, 2024 One paper on HumanEval-XL, a multilingual code generation benchmark has been accepted to LREC-COLING 2024. We’ve released the code and data! :snowflake:
Jan 16, 2024 One paper on reward models with tool-augmented feedback has been accepted to ICLR 2024 (spotlight):sparkles:. Dive into our research and code now! :fire:
Sep 23, 2023 One paper on XAI has been accepted to NeurIPS 2023 Datasets and Benchmarks Track. Code is available here.
May 02, 2023 ERNIE-Code on multilingual text and code pre-training has been accepted to ACL 2023 Findings. Check our code and models.:snowflake:

selected publications

  1. preprint
    Dual Modalities of Text: Visual and Textual Generative Pre-training
    Yekun Chai ,  Qingyi Liu^ ,  Jingwu Xiao^ ,  Shuohuan Wang , and 2 more authors
  2. preprint
    On Training Data Influence of GPT Models
    Qingyi Liu*^ ,  Yekun Chai*Shuohuan Wang ,  Yu Sun , and 3 more authors
    HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalization
    Qiwei Peng*Yekun Chai* ,  and  Xuhong Li
  4. ICLRSpotlight
    Tool-Augmented Reward Modeling
    Lei Li*^ ,  Yekun Chai*Shuohuan Wang ,  Yu Sun , and 3 more authors
    In The Twelfth International Conference on Learning Representations , 2024
  5. ACLFindings
    ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages
    Yekun ChaiShuohuan Wang ,  Chao Pang ,  Yu Sun , and 2 more authors
    In Findings of the Association for Computational Linguistics: ACL 2023 , Jul 2023