Ryan Hoque

I am a Robotics Research Scientist at Apple working on foundation models for robot manipulation. I received my PhD from UC Berkeley, where I was advised by Ken Goldberg in the Berkeley Artificial Intelligence Research (BAIR) lab. I have also been fortunate to collaborate with researchers from Google DeepMind, NVIDIA SRL, Meta FAIR, Honda Research Institute, Siemens AI, and Uber ATG (acq. Aurora). My research primarily investigates scalable imitation learning for robot manipulation.

Email  /  CV  /  Google Scholar  /  LinkedIn  /  Twitter

The goal of my research is to build toward a future where robots are capable of human-level dexterity. I have investigated this goal mostly in the context of scaling up robot imitation learning. This includes learning dexterous manipulation from large-scale egocentric human video (EgoDex, Humanoid Policy ~ Human Policy), imitation learning algorithms that scale to large robot fleets (Fleet-DAgger, ThriftyDAgger), synthetic data generation (IntervenGen), robot-free data collection via augmented reality (ARMADA), and community-wide efforts to train large vision-language-action models (RT-X). In addition to selected publications below (see my CV for the full list), check out:

• My dissertation talk, a 45-minute summary of my PhD research accessible to both researchers and the general public.
• My written dissertation, the entirety of my PhD research in 197 pages.
• My talk at CoRL 2022 in Auckland, New Zealand and BAIR Blog post on Interactive Fleet Learning.
• Issue #238 of the Import AI newsletter featuring my Master's research on manipulating deformable objects.
• University courses that have featured my research, including CS 16-881: Deep Reinforcement Learning for Robotics at Carnegie Mellon University, CS 6960: Human-AI Alignment at the University of Utah, and CS 343: Artificial Intelligence at the University of Texas, Austin.

We introduce the largest and most diverse dataset of dexterous human manipulation to date: 829 hours of egocentric video and paired 3D hand poses across 194 tasks. We also train and systematically evaluate imitation learning policies for hand trajectory prediction with new metrics and benchmarks.

(In collaboration with Google DeepMind and 33 academic labs) Cross-embodiment fleet learning with heterogeneous robots. An open-source dataset of 1M+ robot trajectories from 22 robot embodiments, and results with robot foundation models trained on this data.

(In collaboration with UCSD, CMU, UW, and MIT) We co-train a human-humanoid behavior policy on large-scale egocentric human video and small-scale robot teleop data and deploy it on a humanoid robot performing manipulation tasks.

A state-of-the-art robot-gated interactive imitation learning algorithm that reasons about both state novelty and risk to actively query for human interventions more efficiently than prior algorithms.

We introduce new formalism, algorithms, and open-source benchmarks for "Interactive Fleet Learning": interactive learning with multiple robots and multiple humans.

(In collaboration with NVIDIA SRL) A data generation system extending MimicGen to the interactive imitation learning setting, improving policy robustness up to 39x with a data budget of only 10 human interventions.

(In collaboration with Honda Research Institute) A novel model-based reinforcement learning technique that trains a visual dynamics model for sequentially manipulating fabric toward a variety of goal images, entirely from random interaction RGBD data in simulation.

I'm a Bay Area native and a lifelong bear: I received my B.S., M.S., and Ph.D. in EECS at UC Berkeley in a single stretch of 8 years. Outside of research, I enjoy playing piano covers of heavy metal bands, reading and writing about philosophy, traveling, and exploring the outdoors. For the philosophically inclined, I am steadily working on a book on metaphysics and mysticism, but it's still in progress - stay tuned!

Website template from BAIR alum Jon Barron.