Attune Vector Level 1: Results

Attune Vector Level 1: Results

Attune's first challenge, Vector Level 1, ended with the crown at 87.9% success on unseen scenes across 16 two-arm robot tasks, each learned from a single demonstration. A fine-tuned pi0.5 scored 87.2% on the same benchmark.

In-context learning (ICL) in robotics is frontier territory. A robot is shown a task once and must do it again in a scene it has never seen. Until now, there has been no open competition for it. Vector is the first.

Vector Level 1 results: from one task to sixteen

Vector Level 1 started from the BPP paper. Its method was tested on a single-arm Franka robot on libero-gen with one pick-and-place task, and scored 79.1% on unseen scenarios of known tasks. In that setting, pi0.5 reached 86.6%, a 7.5-point lead over the specialist.

Vector made the problem harder on every axis:

The score rose 8.8 points while the problem widened from one task to sixteen, and the gap to pi0.5 flipped from 7.5 points behind to 0.7 ahead. The benchmarks differ, so the pi0.5 comparison is the cleaner one: in each setting, both models were measured on the same tasks.

Level 1 ran for 68 duels. The crown's score jumped from about 49% to nearly 80% in the first two duels, passed 90% by duel 8, and held around 90% for the rest of the level, peaking near 94%. After the final duel, the crown stood at 87.9%.

The crown's score after each Level 1 duel. Each dot marks a challenger that took the crown.

One demonstration is the whole prompt

Ten of the tasks need one arm, and the scene decides which. Six need both, such as lifting a skillet with one arm while the other drops bread in.

Each test unit pairs two scenes of the same task. In the first, a scripted expert does the task once, recorded from a head camera and two wrist cameras. That recording is the prompt. In the second, the objects have moved and sometimes the other arm must act. The policy has twice the expert's steps to succeed, and it is never told the task's name or what counts as success.

King of the hill

Vector has one crown. Each new entry duels the reigning champion on 240 units, 15 per task, and takes the crown only by beating its score by at least 3.0 points. The four most recent champions share Vector's 30% of the subnet's miner emissions, split 40%, 30%, 20% and 10%, newest first.

An entry is nothing but weights for one fixed 781M-parameter network, vector_v1.1. It pairs a CLIP ViT-B/16 encoder per camera with a transformer decoder that reads the demonstration as prompt tokens, and a diffusion head that outputs actions for both arms. No participant code runs, so the contest is decided by training alone: the data, the curriculum, the objective, and which demonstrations prompt which episodes.

Built so the only way to win is a better model

• Unseen scenes. Units are dealt from the hash of a chain block finalized after the entry is committed, using seeds above the training range. Nobody can choose them or train on them.
• Identical inputs. Both sides play the same scenes, prompted with the same demonstration bytes.
• Nothing privileged. The policy never sees the task name, the scene seed or the success condition.
• A public record. Every duel is published with per-unit outcomes and clips, and anyone can recompute the verdict. Validators run duels on their own GPUs, and the code is on GitHub.

What's next

Today, only object positions and the acting arm change between demonstration and test. Vector will widen one factor at a time, so every score change has one cause: lighting and background next, then object appearance, new object categories, clutter, and finally a different robot. Clutter is where ICL really counts. Once the table holds distractors, the demonstration is the only way to know what to do.

Vector Level 2 is running now, with 38 duels played so far. On top of the new object layouts, Level 2 varies the lighting, wall and table textures, and table height, testing whether a model still reads its demonstration when the scene looks different.