$2 / month

Comment and reply across TML.

Start or manage membership

Most pulsed

Latest from other desks

tml.the modern leap
Tech / Future Tech

Apple Teaches a Humanoid Robot Entirely in Simulation

HALO scored 91 percent across 360 real-world trials after training on about 296 hours of simulated motion and no teleoperation.

TML Tech DeskNov 11, 5:12 PM ET · 2 min read

Edited and fact-checked by Stephen Sprague

A Unitree G1 humanoid robot, the model Apple used to validate HALO. Photo: Sayanesy / CC0, via Wikimedia Commons.
Summary

Apple's HALO system taught a Unitree G1 humanoid to follow instructions in new rooms using training data generated entirely in simulation.

Tap a passage. React to the part that matters.

Apple researchers taught a humanoid robot to follow spoken instructions in rooms it had never seen, without a single hour of human demonstration.

The system, HALO, ran on a Unitree G1 carrying a Mac mini and scored 91 percent success across 360 real-world trials. Its nearly 300 hours of motion training were generated in simulation, matching the 91 percent the same tasks scored there.

Eight Apple authors published the paper on arXiv on November 4. HALO stands for Humanoid Autonomy, Learned in simulation, running Onboard, and it stacks three layers. A vision-language model, Qwen3.6-35B-A3B, reads the instruction and breaks it into subtasks about every two seconds.

Below it, a motion layer pairs a frozen Qwen3.5-2B vision encoder with a 49-million-parameter diffusion transformer to plan the next five seconds of whole-body movement, about three times a second. A tracking policy trained with reinforcement learning in Isaac Lab executes the plan at 50 hertz.

The compute rides on the robot. An M4 Pro Mac mini with 64 gigabytes of memory sits on the G1's back and runs the first two layers, while the tracking layer runs on the robot's built-in Jetson Orin NX. The training data came from a motion model called Kimodo and 1,000 procedurally generated rooms: 162,698 clips, about 296 hours, rendered in Isaac Sim. None of it came from a human operator.

The real-world results held up room by room. Across six environments and six tasks, crawling under a table succeeded every time, passing through a half-open door scored 93 percent, walking to an object 92 percent, pushing a box 88 percent and sitting on a chair 85 percent.

The authors list the limits themselves. The robot has no sense of touch and none of the six skills involves grasping anything. Each new skill still needs a person to write prompts and keyframes. And because the paper doesn't test rival systems on the same tasks, the 91 percent can't be ranked against anyone else's number.

The code is public on Apple's machine learning GitHub, without the trained weights or the 300 hours of data. Apple hasn't announced a robot product. What it has published is a recipe for skipping the most expensive part of humanoid development: the years of human demonstration everyone else is collecting.

Sources

The conversation

A little more perspective.

Reading is free. Verified members can comment and reply for $2 a month.

The conversation.

Loading comments…

Keep your curiosity going.

Some links here are affiliate links. If you buy through one, we might make a commission — it never changes what you pay. As an Amazon Associate I earn from qualifying purchases.