Tianhua Tao - About

Tianhua Tao (陶天骅)

I am a second-year MS student at University of Illinois Urbana-Champaign (UIUC), advised by Prof. Hao Peng. I work on Large Language Models.

Research Interest: Natural Language Processing, Large Language Model Pretraining, Reinforcement Learning, Machine Learning, Computer Systems, Distributed Computing.

Education

University of Illinois Urbana-Champaign - M.S. in Computer Science, 2024
Tsinghua University (Beijing, China) - B.S. in Computer Science, 2022

Experience

Visiting Student - Argonne National Laboratory (2024 - present)
Researcher (remote) - Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) (2023 - present)
Teaching Assistant - CS341 System Programming, University of Illinois Urbana-Champaign (2022 - present)
Reviewer at NeurIPS, ICML, ICLR (2022-2023)

Publication

LLM360: Towards Fully Transparent Open-Source LLMs

Zhengzhong Liu, Aurick Qiao, Willie Neiswanger, Hongyi Wang, Bowen Tan, Tianhua Tao, Junbo Li, Yuqi Wang, Suqi Sun, Omkar Pangarkar, Richard Fan, Yi Gu, Victor Miller, Yonghao Zhuang, Guowei He, Haonan Li, Fajri Koto, Liping Tang, Nikhil Ranjan, Zhiqiang Shen, Xuguang Ren, Roberto Iriondo, Cun Mu, Zhiting Hu, Mark Schulze, Preslav Nakov, Tim Baldwin, Eric P. Xing

preprint

[paper] [website]
SlimPajama-DC: Understanding Data Combinations for LLM Training

Zhiqiang Shen, Tianhua Tao, Liqun Ma, Willie Neiswanger, Joel Hestness, Natalia Vassilieva, Daria Soboleva, Eric Xing

preprint

[paper]
Language Models Meet World Models: Embodied Experiences Enhance Language Models

Jiannan Xiang, Tianhua Tao, Yi Gu, Tianmin Shu, Zirui Wang, Zichao Yang, Zhiting Hu

NeurIPS 2023

[paper] [code]
On the Learning of Non-autoregressive Transformers

Fei Huang, Tianhua Tao, Hao Zhou, Lei Li, Minlie Huang

ICML 2022

[paper]
Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generation

Guangyi Liu, Zichao Yang, Tianhua Tao, Xiaodan Liang, Junwei Bao, Zhen Li, Xiaodong He, Shuguang Cui, Zhiting Hu

NAACL 2022

[paper] [code]