Zeming (Eric) chen
  • Ph.D. candidate at the EPFL NLP LAB. Supervised by Prof. Antoine Bosselut.
  • Goal: to help intelligent machines achieve a broad range of human cognitive abilities.
  • Priors: Research Intern at Allen Institute for AI (AI2); Bachelor in Computer Science and Mathematics, Rose-Hulman, 22.
  • Contact: Please feel free to say hi through my email or LinkedIn!

Featured Posts:

No items found.

Research Directions:

No items found.

Built with Webflow

About me

Short
Long
Honors
Services
Fun

Hello! I am a PhD student at EPFL's NLP Lab, supervised by Dr. Antoine Bosselut. My primary study covers NLP and Machine Learning. Currently, I am working on complex reasoning for Lagre Language Models

I recently completed my first research internship at the Allen Institute for AI (AI2) working with Kyle Richardson. I am currently in collaboration with AI2.

I received my bachelor in 2022, at the Rose-Hulman Institute of Technology, with a double major in Computer Science and Mathematics.

Hello! I am a PhD student at EPFL's NLP Lab, supervised by Dr. Antoine Bosselut. My primary study covers NLP and Machine Learning.

I recently completed my first research internship at the Allen Institute for AI (AI2). I was a member of the Aristo group and supervised by Kyle Richardson and Ashish Sabharwal. I worked on distilling counterfactual knowledge from large general language models (ex. GPT-3). I am currently in collaboration with AI2.

Prior to EPFL, I was a proud member of Rose-Hulman Computer Science and Software Engineering 2022 where I learned to do research in NLP with Larry Moss from Indiana University and Michael Wollowski. At Rose-Hulman, I was also the lead software developer for the Rose-Hulman Mars Rover Team.

My primary research interest are natural language understanding and reasoning. The fundamental goal of my Ph.D. research is to help intelligent machines achieve a broad range of human cognitive abilities such as learning, comprehension, and reasoning. In particular, I am interested in neural-symbolic reasoners, knowledge representation, meta-learning and life-long learning, improving and probing language models. I wish to contribute to human-centered AI technologies that can benefit human life.

In my free time, I enjoy doing many different things! Too me, work-life balance is an important element of everyone's career journey.

I am a musician. I play five different instruments: piano, flute, saxophone, piccolo, and cello. I enjoy playing classic music.

I also love to play tennis and lift weights in the gym.

I love traveling, visiting different places, and experiencing different cultures.

Other

Research Interests

  1. Neuro-symbolic reasoning methods: I am interested in how symbolic methods and statistic learning can jointly teach models to conduct complex reasoning on knowledge and information. This also covers how models can acquire, encode, and apply knowledge to solve various problems.
  2. Large language model NLP: I am fascinated by how large language models like GPT-3 can encode vast information and generate fluent text. I want to explore how these general purpose models can be used in downstream NLP tasks such as open-domain QA and commonsense reasoning. I am interested in building systems that allow general purpose models to be used in a dynamic real-life setting.
  3. Interpretability, benchmarking, and verified AI: I want to develop new tools and theories that help interpret and probe model behaviors. I also want to build benchmarks that can evaluate models' ability and diagnose potential issues in data and learning, especially asses how reliable a model can be in real-life use cases. I recently started to explore the idea of verified AI, where the goal is to have provable assurances of correctness concerning mathematically-specified requirements.
  4. Design new learning algorithms: How machines can learn to understand and reason similarly to human learning is still open for exploration. I want to design new learning algorithms that help models learn continually, actively, comprehensively, and transparently by drawing inspiration from human cognition.
My CV Google Scholar Semantic Scholar

Papers

Presentations

Media

RECKONING: Reasoning through Dynamic Knowledge Encoding

pdf
|
conference

Zeming Chen, Gail Weiss, Eric Mitchell, Asli Celikyilmaz, Antoine Bosselut
Proceedings of The Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023), accepted, to appear


TL;DR: RECKONING is a bi-level learning algorithm that improves language models' reasoning ability by folding contextual knowledge into parametric knowledge through back-propagation. Compared to a fine-tuned in-context reasoning baseline initiated from the same pretrained model, we find that RECKONING shows better performance on multi-hop reasoning tasks, is more robust to distractors, and generalizes better to longer reasoning chains.

DISCO: Distilling Counterfactuals with Large Language Models

pdf
|
code
|
conference

Zeming Chen, Qiyue Gao, Antoine Bosselut, Ashish Sabharwal, Kyle Richardson
Proceedings of The 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)


TL;DR: DISCO is a framework for generating counterfactual data at scale, using a large language model to generate phrasal perturbations and a task-specific teacher model to distill the data into high-quality counterfactuals. Training on DISCO's data leads to a student model that is more robust and generalizes better across distributions, and is also more sensitive in differentiating original and counterfactual examples.

Mitigating Label Biases for In-context Learning

pdf
|
conference

Yu Fei*, Yifan Hou, Zeming Chen, Antoine Bosselut
Proceedings of The 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)


TL;DR: In-context learning (ICL), as a new paradigm for natural language processing (NLP), allows large language models (LLMs) to make predictions based on a few examples. However, ICL is susceptible to bias, which the choice and order of the in-context examples can cause. We propose a simple bias calibration method that significantly improves the ICL performance of GPT-J and GPT-3 on a wide range of tasks.

Curriculum: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language Understanding

pdf
|
code
|
video
|
conference

Zeming Chen, Qiyue Gao
Proceedings of the 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL-HLT 2022)


Oral Presentation

 TL;DR: Curriculum is introduced as a new format of NLI benchmark for evaluation of broad-coverage linguistic phenomena and it is shown that this linguistic-phenomena-driven benchmark can serve as an effective tool for diagnosing model behavior and verifying model learning quality.

Probing Linguistic Information For Logical Inference In Pre-trained Language Models

pdf
|
code
|
conference

Zeming Chen, Qiyue Gao
The 36th AAAI Conference on Artificial Intelligence, 2022


Oral Presentation

 TL;DR: This work proposes a methodology for probing knowledge for inference that logical systems require but often lack in pre-trained language model representations, and demonstrates language models' potential as semantic and background knowledge bases for supporting symbolic inference methods.

NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning

pdf
|
code
|
conference

Zeming Chen, Qiyue Gao, Lawrence S. Moss
The 10th Joint Conference on Lexical and Computational Semantics (*SEM), 2021


TL;DR: This work proposes an inference framework called NeuralLog, which utilizes both a monotonicity-based logical inference engine and a neural network language model for phrase alignment, and shows that the joint logic and neural inference system improves accuracy on the NLI task and can achieve state-of-art accuracy onThe SICK and MED datasets.

Monotonicity Marking from Universal Dependency Trees

pdf
|
code
|
conference
|
award

Zeming Chen, Qiyue Gao
Proceedings of the 14th International Conference on Computational Semantics (IWCS), 2021


Outstanding Paper Award 🏆

 TL;DR: This paper presents a system that automatically annotates monotonicity information based on Universal Dependency parse trees, which utilizes surface-level monotonicism facts about quantifiers, lexical items, and token-level polarity information. Evaluations on shwo that the proposed system achieves SOTA performance.

Attentive Tree-structured Network for Monotonicity Reasoning

pdf
|
Workshop

Zeming Chen
Proceedings of the 1st and 2nd Workshops on Natural Logic Meets Machine Learning (NALOMA 2020, 2021)


TL;DR: An attentive tree-structured neural network that consists of a tree-based long-short-term-memory network (Tree-LSTM) with soft attention designed to model the syntactic parse tree information from the sentence pair of a reasoning task.

Distilling Counterfactual Data from Large Language Models

Final presentation AI2 Internship
|
Oct 2022

Neuro-symbolic Reasoning in Modern AI

Invited talk Rose-Hulman AI Class
|
Oct 2022

Computer Science Duo Publish Four Joint Papers and Awarded Top Paper Honors at Prestigious NAACL Conference

Rose-Hulman News
|
May 2022

Overview: Zeming (Eric) Chen and Qiyue (Bert) Gao, as two computer science majors and Class of 2022 graduates, were research partners in the field of artificial intelligence (AI) and natural language processing and co-authored four conference papers.
Read more