Barry (Xuanyi) Dong
Augment Code; ex Google DeepMind
Address: Palo Alto, California, USA
Email: xuanyi.dxy [at] gmail [dot] com

About Me ([GitHub] [Google Scholar] [Full Publications])

I am a founding member and RL lead at Augment Code, a 1B valuation AI startup for coding. Before Augment Code, I was a research scientist at Google DeepMind and have developed a variety of techniques used in Google Gemini, Google Ads, and Google Cloud. Before that, I spend over five years doing research and engagement at Google, Meta and Amazon. In general, my research interests included large generative models and its application to build a better world.

I received a Ph.D. degree from the School of Computer Science, University of Technology Sydney (UTS) and B.E. degree from Beihang University.

Selected Publications

Gemini: a family of highly capable multimodal models
Gemini Team, 2024
[Google Report]
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
Sang Michael Xie, Hieu Pham, Xuanyi Dong, Nan Du, Hanxiao Liu, Yifeng Lu, Percy Liang, Quoc V. Le, Tengyu Ma, Adams Wei Yu
in NeurIPS 2023
[arXiv] [twitter] [code]
Efficient mixture tuning algorithm for LLM pretraining data domains.
Symbolic Discovery of Optimization Algorithms
Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Yao Liu, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, Quoc V. Le
in NeurIPS 2023
[arXiv] [code]
SoTA performance on vision, LLM, vision-language benchmarks with automatically discovered Lion optimizer.
AutoHAS: Efficient Hyperparameter and Architecture Search
Xuanyi Dong, Mingxing Tan, Adams Wei Yu, Daiyi Peng, Bogdan Gabrys, Quoc V. Le
in NAS@ICLR, 2021
[arXiv] [Slides] [BibTex]
A weight sharing-based hyperparameter and architecture search approach, improving MobileNet/ResNet/EfficientNet/BERT.
NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
Xuanyi Dong, Lu Liu, Katarzyna Musial, Bogdan Gabrys
in IEEE TPAMI, 2021
[arXiv] [IEEE] [API] [Package] [Project] [BibTex] [ICLR]
An algorithm-agnostic NAS benchmark with information of 15,625 neural cell candidates for architecture topology and 32,768 for architecture size on three datasets, and we also provide 13 NAS baselines in a single codebase.

Awards and Honors