Yekun CHAI
Deep Learning & NLP & Reinforcement Learning
Education
The University of Edinburgh
Aug 2017 - Nov 2018 | Edinburgh
MSc Artificial Intelligence (with Distinction), School of Informatics
Thesis: Discovering spelling variants on Urban Dictionary
supervised by Dr. Adam Lopez and Naomi Saphra
Beijing Information Sci & Tech University
Sep 2013 - Jun 2017 | Beijing
BSc Computer Auditing (Data Science track), School of Information Management
GPA: 88.5/100,   graduate project: rank 1/41
Publications   & Preprints
Yekun Chai, Shuo Jin, Xinwen Hou. Highway Transformer: Self-Gating Enhanced Self-Attentive Networks. under review 2019. [ PDF]
Yekun Chai, Naomi Saphra, Adam Lopez. How to Evaluate Word Representations of Informal Domain?. arXiv 2019. [ PDF|Code]
Work Experience
Machine Learning Research Associate
  • Developed neural models for reinforcement learning (RL) and NLP.
  • Orchestrated and developed Deep RL agents to play human games.
  • Implemented the population-based training (PBT) approach for hyperparameter evolution
  • Participated in the framework design of the national chess AI competition of Wargames
Machine Learning Research Engineer (NLP)
Nov. 2018 - Mar. 2019 | JD AI Research of JD.com, Inc.
Jointly built the NLU component of task-oriented spoken dialogue systems in a pipeline manner for JD DingDong Play, the smart speaker product of JD.com:
  • domain classification (TextCNNs / BERT / FastText -> F1 measure > 90%)
  • slot-value extraction (Bi-LSTM-CRF / Lattice-LSTM -> F1 measure > 85%)
  • intent detection (F1 measure > 90%)
  • leveraged ensemble learning on classifiers, including stacking learning and weighted averaging, resulting in the F1 measure improvement of approx. 2%
R&D Engineer Intern
Feb 2017 - Jun 2017 | Intelligence & Network Security Lab, NSFOCUS
  • Established web crawlers from specified websites with Scrapy, BeautifulSoup4, Selenium, Xpath, etc.
  • Responsible for scraping, extracting and preprocessing the data to populate databases.
Projects
Playing Real-time Strategy Wargames with Deep Reinforcement Learning
at CASIA, Feb. 2020 - present
  • Reproduced agents in Quake III Arena Capture the Flag
Mastering Turn-based Strategy Wargames with Deep Reinforcement Learning
at CASIA, Mar. 2019 - Jan. 2020
  • Developed Advantage Actor-Critic (A2C) RL paradigms to compete on Wargames
  • Reproduced the AlphaStar-like neural architectures, including ResNet, Transformer, Pointer Net, Transposed CNNs, CoreLSTM modules and scatter connections.
  • Implemented multiprocessing parallel training in PyTorch to learn simultaneously on multiple rounds of games.
  • Implemented experience replay buffer queue with TD($\mathbf{\lambda}$) and Monte Carlo methods.
  • Reproduced improvements on Transformers, such as relative positional representation, R-Transformers, etc.
Natural Language Understanding in Task-oriented Spoken Dialogue Systems
at JD AI Research of JD.com, Inc, Nov. 2018 - Mar. 2019
  • Leveraged text classification models with ensemble learning in the NLU component of the spoken dialogue system, such as TextCNNs, BERT, FastText, bi-GRUs;
  • Adopted sequence labeling models like bi-LSTM-CRF, Latice-LSTM for slot-value extraction;
  • Rewrote NER evaluation metrics of SemEval competition in Python.
Discovering spelling variants on Urban Dictionary
MSc thesis, 2018
Supervised by Dr. Adam Lopez, Naomi Saphra
[ PDF|Code]
  • With semi-supervised learning approaches involving weakly-supervised pattern-based bootstrapping and self-training based CRF tagging methods, we extracted and proposed a large list of spelling variant pairs from UrbanDictionary, a crowdsourcing dictionary of slang terms, for use in NLP systems of social media.
  • Like WordSim-353 dataset, such variant pairs can be utilized to evaluate the pretrained word embeddings by comparing the word similarity between word pairs as an intrinsic evaluation metric.
  • Evaluated mainstream word representations that we pretrained on ~3 Gigabyte cleaned English tweets, including Word2Vec, GloVe, FastText, on a downstream task, i.e. Twitter hashtag prediction, as the reference of extrinsic metrics.
  • Developed a web interface for spelling variant search.
Probing word-level and character-level representation on sentiment analysis
Developer; MLP(INFR11132) course project, 2018
[ PDF|Code]
  • Explored word-level (i.e. CBOW, Skip-gram, GloVe, random, tf-idf) and character-level representation on sentiment analysis tasks.
  • Evaluated pretrained word embeddings of different domains (e.g. GloVe trained on wiki and others) on IMDB dataset.
  • Probed different neural net structures, such as TextCNNs, LSTMs, GRUs, CNN+LSTM, combined char-and-word-level models as well as Google FastText.
Courses
Machine Learning & Deep Learning
  • Machine Learning & Pattern Recognition (INFR11130, Dr. Iain Murray)
  • Machine Learning Practical (INFR11132, Prof. Steve Renals)
Natural Language Processing
  • Accelerated Natural Language Processing (INFR11125, Dr. Sharon Goldwater)
  • Natural Language Understanding (INFR11061, Dr. Adam Lopez)
  • Machine Translation (INFR11062, Dr. Rico Sennrich)
Automatic Speech Recognition
  • Automatic Speech Recognition (INFR11033, Prof. Steve Renals)
  • Speech Processing (LASC11065, Prof. Simon King)
Computer Science & Software Engineering
  • Data Structure
  • Computer Networks
  • Operating Systems
  • Database Systems (Teaching Asst.)
  • Programming in C/C#/Java
  • Information System Analysis & Design
  • UML and Modeling tools
Skills
Deep Learning Toolkits
PyTorch | TensorFlow | Keras | Chainer
ML & NLP & ASR & RL
Scikit-learn | spaCy | NLTK | Gensim | Kaldi | HTK
Web crawling
Scrapy | Selenium | requests | BeautifulSoup4 | urllib
Programming Language
Python | Java | Shell | JavaScript/HTML/CSS | C#
Web development
Framework (Django / Flask) | relational database (MySQL / SQL Server / Oracle)
Toolkits
Git | Markdown | LaTeX | Vim | ..
Awards
Honorable Mention of Mathematical Contest in Modeling (MCM), 2015   [view]
(Excellent | Innovative | Social Contribution) Scholarships (4 times), 2013-2017
Top-ten Student Association Chairman (IT Association), 2015
Volunteers
Aug 2018, Edinburgh
Interests
Basketball
Photography