Skip to content
RobotensorRobotensor
Blog

Attune Vector Level 2: Results, and What's Next for Level 3

The Robotensor team4 min read

Attune Vector Level 2 is a wrap. On a harder test with changing lighting, textures and table heights, the king hit 91.3%, up from 87.9% in Level 1, across 124 duels. In a separate eval, it beat a fine-tuned π0.5 baseline 89.3% to 86.0%. Level 3 is underway, adding new objects, categories, sizes and colors to test generalization.

Robotensor Level 3 Thumbnail

The last king: 88.8%

Vector Level 2 is in the books. The competition closed with a leaderboard king score of 88.8%, beating Level 1's closing score of 87.9%, on a task set that was deliberately harder.

That is the headline. The more important story is what it says about the network: when we made the world messier, Attune's miners didn't just keep up. They got better.

What made Level 2 harder

Level 1 asked whether a model could do the task. Level 2 asked whether it could still do the task when the room around it changed. We introduced three kinds of environmental variation:

  • Changing lighting conditions
  • Different wall and tabletop textures
  • Variable table heights

To a person, none of these change the job. To a robot policy, they can change everything: shadows shift, surfaces look unfamiliar, and the target sits higher or lower than it did in training. Robots in homes, warehouses and labs face exactly this every day, so robustness here is not optional. It is the gap between a demo and a deployment.

Competition highlights

The harder test drew a bigger fight.

  • 124 duels, up from 68 in Level 1, roughly 1.8x the head-to-head activity

  • About 14 days of continuous competition

Every duel is a challenger trying to dethrone the reigning king. Nearly doubling the duel count on a tougher task means challengers kept coming, iterating faster and pushing the ceiling higher. That is the Attune flywheel working as designed: open competition, measurable results, and a king that only holds the crown by being the best.

Beyond the leaderboard: king vs. baseline

A leaderboard tells you who won. It doesn’t tell you how good the winner really is. To find out, we ran a separate evaluation outside the competition: 50 scenarios per task, pitting the current Level 2 king against a strong baseline, π0.5 fine-tuned on 200 scenarios per task.

The king completed the task in 89.3% of scenarios. The baseline managed 86.0%.

That’s a 3.3-point win. Just as important, the king’s 89.3% on fresh scenarios is close to its 91.3% leaderboard score. It isn’t gaming the competition. Its skill carries over.

This is the result that matters most to us. An open, decentralized competition on Bittensor produced a model that beats a fine-tuned version of one of the leading open robot foundation models, in conditions built to trip it up.

What's next: Vector Level 3

Level 2 tested whether a model could handle a changing environment. Level 3 tests whether it can handle a changing object. It keeps Level 2's environmental randomization and adds object-level variation:

  • Different object instances and models

  • Alternative object categories

  • Randomized object sizes

  • Randomized object colors

The question is no longer "can the robot repeat what it was shown?" It is "can the robot transfer what it learned to things that look and behave differently?" A mug that is taller, a different shade, or a different kind of container entirely should not stop a capable policy.

The dual-arm setup and all Level 2 variations remain in place. Object generalization is the next frontier, and it is one of the hardest open problems in robotics.

Building the intelligence layer for robots, in the open

Each Vector level removes one more crutch. Level 1 proved the models could learn the tasks. Level 2 proved they could hold up when the environment shifted. Level 3 will show whether they can generalize to the objects themselves.

That progression is how Robotensor is building the intelligence layer for robots: open competition on Attune (Bittensor Subnet 58), real benchmarks, and results anyone can check.

If you're a builder working on robot foundation models, Level 3 is your shot at the crown. Follow @robotensor_ai and visit attune.robotensor.ai to get involved.

Attune and Robotensor

Build and mine

Community

Contact