Kuang-Huei Lee

Hello, and welcome to my website!

I am a Research Scientist at Google DeepMind. Before Google, I worked at Microsoft.

I received graduate degree from Carnegie Mellon University in computer science and undergraduate degree from National Taiwan University. I also attended the University of Tokyo and University of Illinois at Urbana-Champaign for graduate studies with incomplete degrees.

Selected Publications (All Publications)

A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts.
Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John Canny, Ian Fischer.
arxiv.
[paper] [website and demo]

PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs.
Soroush Nasiriany*, Fei Xia*, Wenhao Yu*, Ted Xiao*, Jacky Liang, Ishita Dasgupta, Annie Xie, Danny Driess, Ayzaan Wahid, Zhuo Xu, Quan Vuong, Tingnan Zhang, Tsang-Wei Edward Lee, Kuang-Huei Lee, Peng Xu, Sean Kirmani, Yuke Zhu, Andy Zeng, Karol Hausman, Nicolas Heess, Chelsea Finn, Sergey Levine, Brian Ichter*
arxiv.
[paper] [website and demo]

Multimodal Web Navigation with Instruction-Finetuned Foundation Models.
Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, Izzeddin Gur.
ICLR 2024.
[paper] [website]

Language to Rewards for Robotic Skill Synthesis.
Wenhao Yu*, Nimrod Gileadi*, Chuyuan Fu†, Sean Kirmani†, Kuang-Huei Lee†, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, Brian Ichter, Ted Xiao, Peng Xu, Andy Zeng, Tingnan Zhang, Nicolas Heess, Dorsa Sadigh, Jie Tan, Yuval Tassa, Fei Xia.
(*: Co-first authors, equal contribution, †: Core contributors)
CoRL 2023 Oral presentation.
[paper] [website] [Google AI Blog]

Barkour: Benchmarking Animal-level Agility with Quadruped Robots.
Ken Caluwaerts*, Atil Iscen*, J. Chase Kew*, Wenhao Yu*, Tingnan Zhang*, Daniel Freeman†, Kuang-Huei Lee†, Lisa Lee†, Stefano Saliceti†, Vincent Zhuang†, ..., (31 other authors), ..., Vincent Vanhoucke, and Jie Tan.
(*: Co-first authors, equal contribution, †: Core contributors)
Tech Report.
[paper] [website] [Google AI Blog] [YouTube]

Deep RL at Scale: Sorting Waste in Office Buildings with a Fleet of Mobile Manipulators.
{Alexander Herzog, Kanishka Rao, Karol Hausman, Yao Lu, Paul Wohlhart} and 35 other authors including Kuang-Huei Lee.
RSS 2023 Oral presentation.
[paper] [website] [Google AI Blog] [YouTube]

RT-1: Robotics Transformer for Real-World Control at Scale.
51 authors in alphabetical order including Kuang-Huei Lee.
RSS 2023 Oral presentation.
[paper] [website] [Google AI Blog] [YouTube]

Do As I Can, Not As I Say: Grounding Language in Robotic Affordances.
43 authors in alphabetical order including Kuang-Huei Lee.
CoRL 2022 Special Innovation Award.
[paper] [website] [Google AI Blog] [YouTube]

PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale.
Kuang-Huei Lee*, Ted Xiao, Adrian Li, Paul Wohlhart, Ian Fischer, Yao Lu*
CoRL 2022.
[paper] [website] [supplementary video]

Multi-Game Decision Transformers.
Kuang-Huei Lee*, Ofir Nachum*, Mengjiao Yang, Lisa Lee, Daniel Freeman, Winnie Xu, Sergio Guadarrama, Ian Fischer, Eric Jang, Henryk Michalewski, Igor Mordatch*
(*: Equal contribution)
NeurIPS 2022 Oral presentation.
[paper] [website] [code and pre-trained model] [Google AI Blog]

Deep Hierarchical Planning from Pixels.
Danijar Hafner, Kuang-Huei Lee, Ian Fischer, Pieter Abbeel
NeurIPS 2022.
[paper] [website] [Google AI Blog]

PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representations.
Kuang-Huei Lee*, Ofir Nachum*, Tingnan Zhang, Sergio Guadarrama, Jie Tan, Wenhao Yu
(*: Equal contribution)
IROS 2022 Best Paper Finalist.
[paper] [supplementary video] [Google AI Blog]

Compressive Visual Representations.
Kuang-Huei Lee†, Anurag Arnab†, Sergio Guadarrama, John Canny, Ian Fischer†.
(†: Main contributors)
NeurIPS 2021.
[paper] [code and pre-trained models]

Predictive Information Accelerates Learning in RL.
Kuang-Huei Lee, Ian Fischer, Anthony Liu, Yijie Guo, Honglak Lee, John Canny, Sergio Guadarrama.
NeurIPS 2020.
[paper] [code]

Stacked Cross Attention for Image-Text Matching.
Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, Xiaodong He.
ECCV 2018.
[paper] [website] [code]

CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise.
Kuang-Huei Lee, Xiaodong He, Lei Zhang, Linjun Yang.
CVPR 2018.
[paper] [website] [code] [dataset] [MSR Blog] [Bing Blog]

Software

TF-Agents. [GitHub]
I am a main contributor of the TF-Agents RL library.

Datasets

Food-101N: a dataset for learning to address label noise. [dataset]

Last Update: November 2022
Copyright 2022 Kuang-Huei Lee