Ted Xiao

DIAL consists of three stages: 1) finetune pretrained CLIP with robotics data 2) relabel existing datasets 3) train an instruction following agent with imitation learning.

We apply DIAL to a challenging robotics domain with 80,000 demonstrations, of which only 3.5% contain crowd-sourced language labels.

We formulate an LLM inner monologue by continually adding information from various sources of feedback into the language model prompts during robotic planning.

LLMs are able to act as reactive and roboust planners by incorporating textual feedback sources from success detectors, scene descriptors, visual question-answering, and human feedback.

Value Function Spaces is a simple approach that produces useful representations by using the value functions corresponding to lower-level skills.

These value functions capture the affordances of the scene, thus forming a representation that compactly abstracts task relevant information and robustly ignores distractors

This project analyzes deep continuous-time reinforcement learning in the context of concurrent action environmnts.

We introduce a new class of concurrent action deep reinforcement learning algorithms with theoretical learning guarantees.

Evaluations on simulated benchmark tasks and large-scale robotic grasping tasks show that our methods are smoother and faster by over 37%.

Presented at the International Conference on Learning Representations, 2020.

This literature review surveys existing work in quadrotor control at a broad level, and then examines two specific approaches in real-time control scenarios: learning-based model predictive control (LBMPC) and reachability-based safe learning.

Real-time safety-constrained scenarios encapsulate challenges that extend to many other research areas.

I founded a student-run non-profit machine learning organization focusing on workshops, industry projects, and research.

I grew the organization to more than 80 undergraduate and graduate students in 18 months.

I served as Vice President in Fall 2015. I worked with clients, determined strategic project directions, and lead consultants.

Projects I worked on: Valve, National Basketball Assocation, Twitch.tv, Warby Parker

In less than 36 hours, we built a parser and tokenizer to create a programming language out of Tweets.

As a Technology Advisor, I was part of the founding team and saw the organization grow and mature. I designed DiversaTech's website that has been used since Spring 2016.

Kaffeine is an open-source application that pings Heroku sites to keep them from idling.

I added features to update the application to comply with Heroku's policy changes in 2015.

Our model is inspired by the seminal Neural Turing Machine, and has an external memory consisting of a set of tokens which summarise the previous history (i.e., frames). This memory is efficiently addressed, read and written using a Transformer as the processing unit/controller at each step.

TTM outperforms other alternatives, such as other Transformer models designed for long sequences and recurrent neural networks, on two real-world sequential visual understanding tasks.

SayCan combines the reasoning and planning capabilities of large language models with grounding through affordance estimation.

The robot can act as the language model’s “hands and eyes,” while the language model supplies high-level semantic knowledge about the task

In this paper, our aim is to test the scalability of prior IL + RL algorithms and devise a system based on detailed empirical experimentation that combines existing components in the most effective and scalable way.

Our complete method, which we call AW-Opt, combines elements of advantage-weighted regression and QT-Opt, providing a unified approach for integrating demonstrations and offline data for robotic manipulation.

Presented at the Conference on Robot Learning, 2021.

This project proposes learning on top of unlabeled teleoperated human play data as a way to scale up multi-task skill learning

We introduce Play-LMP, a self-supervised method that learns to organize play behaviors in a latent space and then reuse them at test time to achieve specific goals.

Presented at the Conference on Robot Learning, 2019.

Term Paper for CS294-112 Deep Reinforcement Learning, Spring 2017

This project uses Generative Adversarial Networks to learn dynamics models used for Tree Search methods with Deep Value Networks.

Term Paper for CS281A Statistical Learning Theory, Fall 2016

We leverage expectation-maximization, graphical methods, and Recurrent Neural Networks to achieve state-of-the-art results in building baseline power prediction.

I implemented the motion planning module for a Pioneer robot for an experiment investigating human-friendly task planning algorithms.

I created an MVP for a safety and navigation application.

We won multiple pitch competitions, culminating in presenting at IBM Interconnect 2016 in Las Vegas.

A guessing game inspired by GeoGuessr, personalized for UC Berkeley.

It was the top post on the UC Berkeley subreddit and saw 8GB of traffic in 48 hours.

Job Description

I designed the Asian American Association's website that was used between Spring 2015 and Spring 2016.