Imitation Learning Explained: How Robots Learn by Watching Humans

Imitation learning is how we teach robots by showing them what to do. Think about teaching a child to tie a shoelace. You do not hand over a list of rules. You sit next to them, tie the lace slowly a few times, and let them copy you. Imitation learning brings that same idea to machines. Instead of writing every rule by hand or letting a robot stumble through millions of random attempts, we record a person doing the task and train the robot to repeat it.

This approach sits behind some of the most impressive robot demos of the last few years: arms that fold laundry, cook shrimp, load a dishwasher or tie a knot. This guide covers how imitation learning works, why it breaks, the methods researchers use to fix it, and how you can try it in your own robotics projects.

What is imitation learning?

Imitation learning is a machine learning method for training a robot or software agent from examples of an expert doing a task. The expert is usually a person, though it can be another program or a planner that is too slow to run on the robot. You will sometimes see it called learning from demonstration or apprenticeship learning.

Each example is called a demonstration. A demonstration records two things at every moment in time:

• What the robot sensed (the observation): camera images, joint angles, gripper position, force readings.
• What the expert did (the action): how far each joint moved, how fast the wheels turned, whether the gripper opened or closed.

From these recordings we learn a function called a policy. The policy looks at a new observation and outputs the action the expert would most likely have taken. Once trained, the robot runs that policy on its own, many times per second.

Imitation learning vs reinforcement learning vs hand written rules

There are three common ways to get a robot to do something useful.

Hand written control. An engineer writes the logic directly: if the object is here, move the arm there. This works well for repeatable factory work where nothing changes. It falls apart when objects vary, lighting shifts, or the task needs a soft touch, like folding a towel. Nobody can write rules for every wrinkle.

Reinforcement learning. The robot tries actions, receives a reward when it does well, and slowly improves. If you are new to it, my guide to reinforcement learning covers the basics, and my guide to Markov decision processes covers the math underneath. It has produced remarkable results in games and simulation. On real hardware it is slow and risky. A robot arm cannot try a million random grasps without breaking things, and writing a reward that captures “fold the shirt neatly” is surprisingly hard.

Imitation learning. A person shows the task 50 to a few hundred times, and the robot learns from those recordings. There is no reward to design and no dangerous random exploration. For many household and lab tasks, this is the fastest route from idea to a working robot.

The three work well together. Many modern systems start with imitation learning to get a decent policy quickly, then use reinforcement learning to polish it.

How demonstrations are collected

The quality of your demonstrations sets a ceiling on how well the robot can perform. There are a few standard ways to record them.

Teleoperation. The person controls the robot remotely with a joystick, a VR headset, a phone or a second “leader” arm that the robot copies joint for joint. The ALOHA system from Stanford uses this leader and follower setup with low cost arms, and it made fine two handed tasks like threading a cable tie possible to record.

Kinesthetic teaching. The person physically grabs the robot and guides it through the motion while the robot records its own joint positions. This is simple and intuitive for arms with compliant joints.

Human video. The robot watches a person do the task with their own hands. This is the cheapest source of data, as there are millions of such videos online, but it is the hardest to learn from. A human hand and a robot gripper look and move differently, and video does not record the exact forces or joint angles.

A good rule from practice: be consistent. If you sometimes grab the cup from the left and sometimes from the right, record plenty of both, or the policy may average the two into a motion that misses the cup entirely.

Behavior cloning: the simplest form of imitation learning

The most direct form of imitation learning is called behavior cloning. It treats the problem as ordinary supervised learning, the same kind of learning used to classify images.

The recipe goes like this:

1. Record demonstrations as pairs of observation and action.
2. Split them into a training set and a validation set.
3. Train a neural network to predict the action from the observation, using a loss that measures how far the predicted action is from the expert’s action.
4. Put the trained network on the robot and let it act.

Behavior cloning has a long history. In 1989, Dean Pomerleau at Carnegie Mellon built ALVINN, a small neural network that learned to steer a van by watching a human driver. It took in a low resolution road image and produced a steering direction. With very modest compute, it drove on real roads. The core idea has not changed much since then. The networks and the datasets have simply grown.

A minimal behavior cloning example in PyTorch

Here is what a basic behavior cloning training loop looks like. The observation is a small vector of sensor readings and the action is a vector of joint commands.

import torch
import torch.nn as nn
import torch.nn.functional as F

# demos: observations (N, obs_dim) and expert actions (N, act_dim)
obs = torch.load("demo_observations.pt")
acts = torch.load("demo_actions.pt")

policy = nn.Sequential(
    nn.Linear(obs.shape[1], 256), nn.ReLU(),
    nn.Linear(256, 256), nn.ReLU(),
    nn.Linear(256, acts.shape[1]),
)
optimizer = torch.optim.Adam(policy.parameters(), lr=0.001)

for epoch in range(200):
    for idx in torch.randperm(len(obs)).split(64):
        predicted = policy(obs[idx])
        loss = F.mse_loss(predicted, acts[idx])  # how far from the expert?
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

torch.save(policy.state_dict(), "bc_policy.pt")

# on the robot: read sensors, ask the policy, send the command
# action = policy(torch.tensor(current_observation))

That is the entire algorithm. If your demonstrations are good and the robot only ever sees situations close to the ones in your recordings, this can work surprisingly well.

Compounding errors: why behavior cloning fails

Behavior cloning has a well known weakness, and it shows up the moment the robot makes its first small error.

During training, the network only sees states the expert visited. The expert never drifted toward the edge of the road, so there are no examples of how to recover from being near the edge. When the trained policy makes a tiny mistake, it ends up in a slightly unfamiliar state. Its next prediction is a little worse there, which pushes it into an even less familiar state, and so on. The errors stack up until the robot is somewhere the expert never went and it has no idea what to do.

Researchers call this compounding error, or distribution shift. Stéphane Ross and Drew Bagnell showed that for plain behavior cloning, the expected error can grow with the square of the task length. A task twice as long can be roughly four times as likely to fail. This is why a cloned policy might handle a five second grasp perfectly and still fall apart on a two minute assembly task.

DAgger: fixing distribution shift with expert corrections

The best known fix is an algorithm called DAgger, short for Dataset Aggregation, published by Ross, Gordon and Bagnell in 2011. The idea is simple: let the robot drive, and have the expert label what it should have done in the states the robot actually reached.

1. Train a policy with behavior cloning on the expert demonstrations.
2. Run that policy on the robot and record the states it visits, including its mistakes.
3. Ask the expert what the correct action would have been in each of those states.
4. Add these new labeled examples to the dataset.
5. Retrain on the combined data and repeat.

After a few rounds, the dataset contains examples of recovering from the very errors the policy tends to make. In theory, this removes the snowball problem and brings the error back to growing in a straight line with task length.

The practical difficulty is the expert. Labeling the right action for a state you are only watching, not controlling, is hard for a person. Several variants make this easier. HG DAgger lets the human take over the controls only when the robot is heading for trouble, then hands control back. Many teams use this “intervene when needed” style today, as it feels natural to the operator.

A cheaper trick that works surprisingly well is noise injection. While recording demonstrations, you add small random disturbances to the robot so that it drifts a little, and the expert naturally corrects it. The dataset then contains recovery behavior without any extra rounds of labeling. This is the idea behind a method called DART.

Inverse reinforcement learning and GAIL: learning the goal

Behavior cloning copies actions. A different family of imitation learning methods tries to work out why the expert acted as they did, then plans its own actions to reach the same goal.

Inverse reinforcement learning (IRL). Here the robot assumes the expert is trying to maximize some hidden reward, and it tries to recover that reward from the demonstrations. Once it has a reward, it can use reinforcement learning to find a policy. Pieter Abbeel and Andrew Ng called this apprenticeship learning, and with colleagues at Stanford they used related ideas to teach a model helicopter to fly aerobatic maneuvers by watching an expert pilot. A recovered reward often generalizes better than copied actions, as it captures intent. If an obstacle appears that the expert never saw, a robot that knows the goal can find a new path around it.

Generative Adversarial Imitation Learning (GAIL). Introduced by Jonathan Ho and Stefano Ermon in 2016, GAIL borrows the setup used to generate realistic images. A discriminator network learns to tell expert behavior apart from the robot’s behavior. The policy is trained to fool the discriminator. When it succeeds, its behavior looks like the expert’s. GAIL needs far fewer demonstrations than behavior cloning in many simulated tasks, but it still needs the robot to interact with its environment a lot, which makes it more common in simulation than on real hardware.

Recent advances: ACT, Diffusion Policy and large robot datasets

For a long time, behavior cloning was seen as a quick baseline that serious methods would beat. That view has shifted. A few developments made plain imitation work far better than expected on real robots.

Predicting chunks of actions. Action Chunking with Transformers (ACT), from Tony Zhao and colleagues in 2023, predicts a short sequence of future actions at once instead of one action at a time. Committing to a short plan cuts down the number of decision points where errors can creep in, and it smooths out jittery motion. Paired with the ALOHA hardware, it learned tasks like opening a translucent condiment cup and slotting a battery from about ten minutes of demonstrations each.

Handling more than one right answer. When a person demonstrates a task, there is often more than one valid way to do it. Go around the obstacle on the left or on the right; both are fine. A simple network trained to minimize average error will average the two and drive straight into the obstacle. Diffusion Policy, from Cheng Chi and colleagues in 2023, solves this by borrowing the method behind modern image generators. It starts from random noise and gradually refines it into an action sequence, which lets it represent several distinct good options and pick one cleanly.

Much bigger and more varied datasets. The Open X Embodiment project pooled demonstrations from 22 different robots across 21 institutions into one shared dataset. Policies trained on this mix can transfer some skills between robots, the same way large language models transfer knowledge between topics. Newer vision language action models take a camera image and a typed instruction such as “put the apple in the bowl” and output robot actions directly, trained largely through imitation.

Cheap, portable data collection. Tools like the Universal Manipulation Interface use a handheld gripper with a camera so people can record demonstrations in their own kitchens without a robot present. More data from more places means policies that cope better with new rooms and objects.

Real world applications of imitation learning

• Household and service robots: folding clothes, clearing tables, loading dishwashers.
• Warehouse picking: grasping varied items from bins where every object looks different.
• Autonomous driving: early stages of many driving systems learn from huge logs of human driving, then add safety layers and planning on top.
• Surgical robotics: research systems learn suturing and tissue handling from recordings of expert surgeons.
• Drones and mobile robots: learning to fly through forests or follow trails from pilot demonstrations. My own Veg drone project and AI rover used hand written planners, and imitation learning is a natural next step for both.
• Games and simulation: game agents often start by copying human players before improving on their own.

How to get started with imitation learning

You do not need a research lab. A low cost arm such as the SO 100, a webcam and the open source LeRobot library from Hugging Face are enough to record demonstrations and train ACT or Diffusion Policy on a single consumer GPU. A few lessons that save a lot of time:

• Start in simulation. Gymnasium and robosuite environments let you test the whole pipeline before touching hardware.
• Record more than you think you need, and record it consistently. Fifty clean demonstrations beat two hundred sloppy ones.
• Show recoveries. Deliberately start some demonstrations from awkward positions, or nudge the object mid task and correct it. This is the cheapest defense against compounding error.
• Keep the scene the same at test time. Same camera angle, similar lighting, similar objects. Policies trained on small datasets are sensitive to these details.
• Normalize your inputs and outputs. Scale joint angles and actions to similar ranges, or the network will struggle to learn.
• Watch the robot, not only the loss curve. A low validation loss does not guarantee task success. Test on the real task often.

Limitations of imitation learning

Imitation learning is not a complete answer. A policy can only be as good as the person teaching it, so it rarely surpasses the expert. It struggles in situations far from the demonstrations. Collecting large amounts of good robot data is still slow and costly compared with scraping text from the web. And a cloned policy has no built in sense of safety; it will happily repeat a dangerous motion if that motion appeared in the data.

For these reasons, most serious systems combine imitation with other tools: reinforcement learning to improve beyond the expert, classical planning and safety checks to keep the robot inside safe limits, and human oversight while the system is new.

Frequently asked questions

Closing thoughts

Imitation learning turns the hardest part of robotics, describing exactly what you want, into something anyone can do: show it. The simple version, behavior cloning, is easy to build and works well on short, consistent tasks. Its weakness with compounding errors led to DAgger, inverse reinforcement learning and adversarial methods, and more recently to action chunking and diffusion based policies that make plain imitation work on real hardware. With cheap arms and open source tools now available, this is one of the most rewarding areas of robotics to experiment with at home.

If you build something with it, I would love to hear about it. Reach out through the contact page.

References

1. Pomerleau, D. A. (1989). ALVINN: An Autonomous Land Vehicle in a Neural Network. Advances in Neural Information Processing Systems.
2. Abbeel, P., and Ng, A. Y. (2004). Apprenticeship Learning via Inverse Reinforcement Learning. International Conference on Machine Learning.
3. Ross, S., and Bagnell, J. A. (2010). Efficient Reductions for Imitation Learning. AISTATS.
4. Ross, S., Gordon, G. J., and Bagnell, J. A. (2011). A Reduction of Imitation Learning and Structured Prediction to No Regret Online Learning. AISTATS.
5. Ho, J., and Ermon, S. (2016). Generative Adversarial Imitation Learning. NeurIPS.
6. Laskey, M., Lee, J., Fox, R., Dragan, A., and Goldberg, K. (2017). DART: Noise Injection for Imitation Learning. Conference on Robot Learning.
7. Kelly, M., Sidrane, C., Driggs Campbell, K., and Kochenderfer, M. J. (2019). HG DAgger: Interactive Imitation Learning with Human Experts. ICRA.
8. Zhao, T. Z., Kumar, V., Levine, S., and Finn, C. (2023). Learning Fine Grained Bimanual Manipulation with Low Cost Hardware. Robotics: Science and Systems.
9. Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., Tedrake, R., and Song, S. (2023). Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. Robotics: Science and Systems.
10. Open X Embodiment Collaboration (2023). Open X Embodiment: Robotic Learning Datasets and RT X Models.
11. Chi, C., Xu, Z., Pan, C., Cousineau, E., Burchfiel, B., Feng, S., Tedrake, R., and Song, S. (2024). Universal Manipulation Interface: In The Wild Robot Teaching Without In The Wild Robots. Robotics: Science and Systems.