Kuang-Huei Lee

Kuang-Huei Lee is a Staff Research Scientist at Google DeepMind in San Francisco. His research interests center around creating general cognitive agents in both physical and virtual worlds, and his current research spans deep generative models, reasoning, planning, reinforcement learning, and robotics. Prior to joining Google in 2019, Kuang-Huei spent 3 years at Microsoft. He received his graduate degree in Computer Science from Carnegie Mellon University, and his undergraduate degree in Mechanical Engineering from National Taiwan University. His research has been widely published, appearing in venues such as NeurIPS, ICML, ICLR, RSS, IROS, CVPR, ECCV, and EMNLP.

Selected Publications (All Publications)

Training-free Diffusion Model Alignment with Sampling Demons.
Po-Hung Yeh, Kuang-Huei Lee, Jun-Cheng Chen.
arXiv.
[paper]

Geometric-Averaged Preference Optimization for Soft Preference Labels.
Hiroki Furuta, Kuang-Huei Lee, Shixiang Shane Gu, Yutaka Matsuo, Aleksandra Faust, Heiga Zen, Izzeddin Gur.
NeurIPS 2024.
[paper]

The Design of the Barkour Benchmark for Robot Agility.
44 authors including Kuang-Huei Lee (last author).
IROS 2024.

A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts.
Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John Canny, Ian Fischer.
ICML 2024.
[paper] [website and demo]

PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs.
Soroush Nasiriany*, Fei Xia*, Wenhao Yu*, Ted Xiao*, Jacky Liang, Ishita Dasgupta, Annie Xie, Danny Driess, Ayzaan Wahid, Zhuo Xu, Quan Vuong, Tingnan Zhang, Tsang-Wei Edward Lee, Kuang-Huei Lee, Peng Xu, Sean Kirmani, Yuke Zhu, Andy Zeng, Karol Hausman, Nicolas Heess, Chelsea Finn, Sergey Levine, Brian Ichter*
ICML 2024.
[paper] [website and demo]

Learning to Learn Faster from Human Feedback with Language Model Predictive Control.
Jacky Liang*, Fei Xia*, Wenhao Yu*, Andy Zeng*, and 46 others including Kuang-Huei Lee.
RSS 2024.
[paper] [website and demo]

Multimodal Web Navigation with Instruction-Finetuned Foundation Models.
Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, Izzeddin Gur.
ICLR 2024.
[paper] [website]

Language to Rewards for Robotic Skill Synthesis.
Wenhao Yu*, Nimrod Gileadi*, Chuyuan Fu†, Sean Kirmani†, Kuang-Huei Lee†, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, Brian Ichter, Ted Xiao, Peng Xu, Andy Zeng, Tingnan Zhang, Nicolas Heess, Dorsa Sadigh, Jie Tan, Yuval Tassa, Fei Xia.
(*: Co-first authors, equal contribution, †: Core contributors)
CoRL 2023 Oral presentation.
[paper] [website] [Google AI Blog]

Barkour: Benchmarking Animal-level Agility with Quadruped Robots.
Ken Caluwaerts*, Atil Iscen*, J. Chase Kew*, Wenhao Yu*, Tingnan Zhang*, Daniel Freeman†, Kuang-Huei Lee†, Lisa Lee†, Stefano Saliceti†, Vincent Zhuang†, ..., (31 others), ..., Vincent Vanhoucke, and Jie Tan.
(*: Co-first authors, equal contribution, †: Core contributors)
Tech Report.
[paper] [website] [Google AI Blog] [YouTube]

Deep RL at Scale: Sorting Waste in Office Buildings with a Fleet of Mobile Manipulators.
{Alexander Herzog, Kanishka Rao, Karol Hausman, Yao Lu, Paul Wohlhart} and 35 others including Kuang-Huei Lee.
RSS 2023 Oral presentation.
[paper] [website] [Google AI Blog] [YouTube]

RT-1: Robotics Transformer for Real-World Control at Scale.
51 authors in alphabetical order including Kuang-Huei Lee.
RSS 2023 Oral presentation.
[paper] [website] [Google AI Blog] [YouTube]

Do As I Can, Not As I Say: Grounding Language in Robotic Affordances.
43 authors in alphabetical order including Kuang-Huei Lee.
CoRL 2022 Special Innovation Award.
[paper] [website] [Google AI Blog] [YouTube]

PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale.
Kuang-Huei Lee*, Ted Xiao, Adrian Li, Paul Wohlhart, Ian Fischer, Yao Lu*
CoRL 2022.
[paper] [website] [supplementary video]

Multi-Game Decision Transformers.
Kuang-Huei Lee*, Ofir Nachum*, Mengjiao Yang, Lisa Lee, Daniel Freeman, Winnie Xu, Sergio Guadarrama, Ian Fischer, Eric Jang, Henryk Michalewski, Igor Mordatch*
(*: Equal contribution)
NeurIPS 2022 Oral presentation.
[paper] [website] [code and pre-trained model] [Google AI Blog]

Deep Hierarchical Planning from Pixels.
Danijar Hafner, Kuang-Huei Lee, Ian Fischer, Pieter Abbeel
NeurIPS 2022.
[paper] [website] [Google AI Blog]

PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representations.
Kuang-Huei Lee*, Ofir Nachum*, Tingnan Zhang, Sergio Guadarrama, Jie Tan, Wenhao Yu
(*: Equal contribution)
IROS 2022 Best Paper Finalist.
[paper] [supplementary video] [Google AI Blog]

Compressive Visual Representations.
Kuang-Huei Lee†, Anurag Arnab†, Sergio Guadarrama, John Canny, Ian Fischer†.
(†: Main contributors)
NeurIPS 2021.
[paper] [code and pre-trained models]

Predictive Information Accelerates Learning in RL.
Kuang-Huei Lee, Ian Fischer, Anthony Liu, Yijie Guo, Honglak Lee, John Canny, Sergio Guadarrama.
NeurIPS 2020.
[paper] [code]

Stacked Cross Attention for Image-Text Matching.
Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, Xiaodong He.
ECCV 2018.
[paper] [website] [code]

CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise.
Kuang-Huei Lee, Xiaodong He, Lei Zhang, Linjun Yang.
CVPR 2018.
[paper] [website] [code] [dataset] [MSR Blog] [Bing Blog]

Software

TF-Agents. [GitHub]
I am a main contributor of the TF-Agents RL library.

Datasets

Food-101N: a dataset for learning to address label noise. [dataset]

Last Update: October 2024
Copyright 2024 Kuang-Huei Lee