Robots Learning From Human Video: The Biggest Imitation Learning Shift of 2026

Every imitation learning method I have written about on this site so far starts the same way: someone drives the robot. A person holds a teleoperation rig, the robot copies what they did, and after a few hundred runs you have enough data to train a policy. It works, and it is slow. One trained operator, one robot, one task at a time.

In 2026 that picture started to change. A run of papers from NVIDIA, UC Berkeley, the University of Maryland and several other labs all pointed the same way: video of people doing ordinary work with their own hands, filmed from a camera on their head, can teach robots to manipulate objects. In some tests it beat data collected on the robot itself.

This post covers what changed this year, the three main ways labs are turning human video into robot actions, the scaling law that made people pay attention, and what still does not work. If imitation learning is new to you, my overview of imitation learning and the DAgger explainer are good places to start.

Why robot data became the bottleneck

Behavior cloning, the core of most modern robot policies, is supervised learning on pairs of what the robot saw and what it did. Large language models had the whole internet to read. Robots have nothing like that. Every robot demonstration has to be recorded on real hardware by a person in the loop, and each one only shows a single body doing a single task in a single room.

That leaves two problems. The first is cost: thousands of hours of teleoperation means paid operators, robots that break and labs that run all day. The second is variety. A robot dataset tends to come from a handful of labs, with the same tables, the same lighting and the same objects. A policy trained on it often struggles the moment it sees a new kitchen.

Human video has the opposite profile. People already fold shirts, open jars and sort parts in millions of real places, and a small head mounted camera can record all of it. The catch is that video has no action labels. It shows a hand moving, not the joint commands a robot would need to repeat the motion.

Three ways to turn human video into robot actions

Most of the 2026 work falls into one of three recipes.

Recipe	What it does	Example from 2026
Retarget the hand	Track the wrist and the 21 finger keypoints in the video, then map them onto the robot hand with an optimizer	EgoScale (NVIDIA, Berkeley, Maryland)
Learn latent actions	Train a network to describe the change between two video frames as a hidden "action", and pretrain the policy on those	LAPT-Scale
Rebuild the scene for the robot	Retarget the motion and redraw the frames so a robot arm appears in place of the human arm	Ego2Robot

A fourth idea sits between them: describe each demonstration as the interaction between the hand and the object, which both a person and a robot can share, and train the policy on that. HumanEgo takes this route.

In all four cases the plan ends the same way. Pretrain on lots of human video, then fine tune on a small amount of real robot data so the policy learns how its own body moves.

EgoScale: the scaling law that got attention

The paper most people point to is EgoScale, posted in February by Ruijie Zheng, Linxi Fan and colleagues from NVIDIA, UC Berkeley and the University of Maryland.

They gathered about 20,854 hours of egocentric human video, mostly recorded in the wild, including 829 hours from Apple's EgoDex dataset. Off the shelf SLAM recovers how the camera moved, a hand tracker finds the 21 keypoints of each hand, and the arm motion is stored as the change in wrist pose from one frame to the next. That last choice matters: the action no longer depends on where the camera was, so videos from different people and rooms line up.

The finger keypoints are retargeted onto a 22 DoF robot hand through an optimization that respects joint limits, then smoothed over time. A flow matching vision language action model, close in design to NVIDIA's GR00T N1, is pretrained on these wrist and hand actions. A short middle stage with about 50 hours of matched human data and 4 hours of robot data ties the model to the real robot before task training.

The headline finding is a clean scaling law. Validation loss on wrist and hand action prediction follows

L = 0.024 − 0.003 · ln(D)

where D is the amount of human data in hours, with R² = 0.9983 across the 1,000 to 20,000 hour range they tested. More important for anyone building robots, that loss tracked real robot performance. Average task completion went from 0.30 with 1,000 hours to 0.71 with 20,000 hours.

Some other results from the paper:

• Pretraining on human video improved average task completion over training from scratch by more than 55 percent.
• With a single robot demonstration plus human video, the robot folded shirts with 0.88 success.
• On a Unitree G1 with a three finger hand, human pretraining added more than 30 points of absolute success on both tasks they tested.

GR00T N1.7: human video in an open model

EgoScale did not stay a paper. In April, NVIDIA released Isaac GR00T N1.7 on Hugging Face as an early access model. The previous version, N1.6, was trained on a few thousand hours of robot teleoperation. N1.7 was pretrained on the same 20,854 hours of human egocentric video from EgoScale, covering more than 20 kinds of work from factories and shops to homes and hospitals.

It is a 3 billion parameter vision language action model with a reasoning backbone for planning and a 32 layer diffusion transformer for motor control. NVIDIA tested it on a Unitree G1, a bimanual YAM arm setup and an AGIBot Genie 1, and released it under a commercial license, so a startup or a student can fine tune it without a research only restriction.

HumanScale: can human video beat robot data?

In June a team of 22 researchers led by Juncheng Ma and Daquan Zhou asked the question directly. HumanScale trained the same model twice with equal amounts of pretraining data: once on teleoperated robot trajectories, once on filtered and labelled egocentric human video. Everything after pretraining was kept identical.

The human video version came out ahead:

• 24 percent lower validation loss on real robot action prediction.
• 52.5 percent higher success on real robot tasks similar to the training data.
• 90 percent higher success on tasks outside the training data.

That last number is the one I find most telling. Robot data is precise, and human video is varied. When the test moves away from the training setup, variety wins. The authors' suggested recipe matches EgoScale: pretrain on human video for a broad picture of the world, then align the actions with a small amount of labelled robot data.

LAPT-Scale: learning actions without labels

Retargeting depends on a good hand tracker and a robot hand that looks like a human hand. LAPT-Scale, posted on October 9 by Zhiyuan Xiang and colleagues, tries to skip that step.

Its first part, LAPT-Scale-LAM, looks at two neighbouring frames of human video and learns a continuous latent action that explains the change between them. Earlier latent action models squeezed motion into a small set of discrete codes. Keeping the latent continuous holds on to finer detail, and the training adds optical flow and semantic signals on top of pixel reconstruction. The second part is a vision language action policy pretrained on those latent actions, aligned to the target robot, then trained on the task.

They pretrained on 23.3 thousand hours of egocentric video. As the annotated human video grew from 1,000 to 5,000 hours, real robot success rose from 63.3 percent to 85.0 percent, and the policy held up in visual conditions it had not seen.

HumanEgo and Ego2Robot: from minutes to millions of frames

Two more papers show how wide the range has become.

HumanEgo (May, from Zhi Wang, Yiannis Aloimonos and colleagues) goes small. It turns each human demonstration into a description of how the hand and the object interact, and trains a flow matching policy with extra dense training targets that pull more signal out of every clip. With 30 minutes of human video per task and no robot data at all, it reached 92.5 percent average success on four real tasks, and 75 percent with only 15 minutes. Against robot teleoperation given the same amount of collection time, it came out 41 percent ahead.

Ego2Robot (August) goes big. It retargets human motion, redraws the frames so a robot arm appears where the human arm was, and filters the result at several levels. The output is 18,561 hours of synthetic robot data across 15 robot body types, which the authors call the largest dataset of its kind so far. Pretraining on it alongside real robot data improved generalization in simulation and carried over to real robots.

A small Python check of the scaling law

A log law has a simple consequence: every doubling of data buys the same fixed drop in loss. The fitted slope of 0.003 is all you need to see it, since the gap between two data sizes depends only on their ratio.

import math

SLOPE = 0.003  # from the EgoScale fit, L = 0.024 - 0.003 * ln(D)


def loss_drop(hours_from, hours_to):
    """How much the fitted validation loss falls between two amounts of human data."""
    return SLOPE * math.log(hours_to / hours_from)


steps = [1_000, 2_000, 5_000, 10_000, 20_000]
for a, b in zip(steps, steps[1:]):
    print(f"{a:>6} to {b:>6} hours: loss falls by {loss_drop(a, b):.5f}")

print(f"\nany doubling:          {loss_drop(1, 2):.5f}")
print(f"1,000 to 20,000 hours: {loss_drop(1_000, 20_000):.5f}")

Running it prints:

  1000 to   2000 hours: loss falls by 0.00208
  2000 to   5000 hours: loss falls by 0.00275
  5000 to  10000 hours: loss falls by 0.00208
 10000 to  20000 hours: loss falls by 0.00208

any doubling:          0.00208
1,000 to 20,000 hours: 0.00899

Going from 1,000 to 2,000 hours lowers the loss by exactly as much as going from 10,000 to 20,000. The first doubling costs a few weeks of filming. The last one costs ten thousand hours. That is the honest reading of any log law: progress keeps coming, and each step costs more than the one before. The authors themselves do not extrapolate past 20,000 hours.

What still does not work

Human video is not a free lunch, and the papers are open about the gaps.

• The body gap. A human hand is soft, has five fingers and feels everything it touches. Most robot hands have fewer joints and no skin. Every result above still needed some robot data, or a robot hand that looks a lot like a human one, to close that gap.
• Force and touch. A video shows where a hand went, not how hard it pushed. Tasks like tightening a screw or peeling tape depend on force. Startups are already selling sensor gloves for this; in October, Maxinsights announced MaxGlove, which records touch and force alongside video.
• Labels are only as good as the trackers. Retargeting relies on SLAM and hand tracking. When a hand is hidden behind an object, the label is a guess. HumanScale only beat robot data after careful filtering.
• Long tasks. These policies do well on short skills. Chaining many of them is still hard. Physical Intelligence's π0.7, shown in April, can combine skills it learned separately, but needs step by step coaching for longer jobs.

What this means for robot learning

For years the advice for a new robot learning project was: buy a teleoperation rig and start recording. In 2026 the advice is shifting toward: start from a model pretrained on human video, then collect a small amount of robot data for your own body and task.

The research community is treating this as a field of its own. At CoRL 2026 in Austin there is a workshop called Scaling H2R, short for human to robot, on November 12. Its framing sums up where things stand: the open question is no longer whether robots can learn from people on video, it is how far that scales and which kinds of variety matter most.

For my own course work, the link to older imitation learning ideas is clear. DAgger taught us that a policy has to see the states it will meet. Human video is one more answer to the same question: show the policy far more of the world than any lab could ever record on a robot.

Frequently asked questions

Closing thoughts

The quiet change in 2026 is about where robot skills come from. The teleoperation rig is not going away, but it is turning into the last step of training rather than the first. If you are starting a manipulation project, it is worth checking what an open model like GR00T N1.7 already knows before you record your first demonstration.

If you have tried training on human video yourself, I would like to hear how big the gap to your robot was.

References

• R. Zheng, D. Niu, Y. Xie, J. Wang, M. Xu, Y. Jiang, F. Castañeda, F. Hu, Y. L. Tan, L. Fu, T. Darrell, F. Huang, Y. Zhu, D. Xu and L. Fan, EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data, 2026. arXiv:2602.16710
• J. Ma, J. Bi, Y. Deng and others, HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining, 2026. arXiv:2606.20521
• Z. Xiang, P. Liu, M. Li, J. Li, W. Li, W. Ding and X. Deng, LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos, 2026. arXiv:2610.13732
• Z. Wang, B. He, K. Yu, S. Lee, R. Gao, F. Huang and Y. Aloimonos, HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos, 2026. arXiv:2605.24934
• Y. Wang, P. Lin, X. Chen and others, Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data, 2026. arXiv:2608.02580
• NVIDIA, Isaac GR00T N1.7: open reasoning VLA model for humanoid robots, Hugging Face, April 2026. Hugging Face blog
• Scaling H2R workshop at CoRL 2026. scaling-h2r-corl.github.io
• TechCrunch, Physical Intelligence says its new robot brain can figure out tasks it was never taught, April 16, 2026. techcrunch.com