What in-context robot learning asks of a model

Almost every robot policy in wide use today learns before it is deployed and never again. If it meets a task that was not in its training corpus, the remedy is to collect data and train again — a loop that takes weeks, and the reason a policy that looks excellent in a paper can be useless in a room it has not seen.

This competition tests the other thing: a model that improves at an unfamiliar task inside a single deployment, with its parameters frozen. No gradient step, no fine-tune, no test-time optimisation of any kind. Whatever the model works out about the task, it works out in its context window, and it forgets when the context is cleared. That constraint is not an aesthetic preference. It is what makes a result legible: if no weights moved, then whatever adaptation happened is a property of the model's forward pass, and it is the same forward pass a robot would run in the field.

Where the idea comes from

The oldest version of the argument is RL² (Duan et al., 2016), which represented the fast learning algorithm as a recurrent network whose activations persist across episodes within one task. The network receives everything an RL algorithm would receive — observations, actions, rewards, termination flags — and the slow outer algorithm only shapes the weights. On randomly generated bandits and finite Markov decision processes, its behaviour on new problems came close to hand-designed algorithms with optimality guarantees.

Algorithm Distillation (Laskin et al., ICLR 2023) sharpened it into a recipe: do not distil the policy an RL algorithm converges to, distil its entire training history. A causal transformer trained to predict actions from an across-episode context continues the improvement curve, including on goals it never saw. The paper's decisive ablation is the one this competition is built around — "multi-episodic contexts of 2-4 episodes are necessary to learn a near-optimal in-context RL algorithm". Shorten the context below one episode and you get Expert Distillation instead: a model that correctly identifies which task it is in, and then stops improving. Those two behaviours look identical on a single episode and completely different over a run of 20.

The Decision-Pretrained Transformer (Lee et al., NeurIPS 2023) took the opposite route — supervised pretraining to predict the optimal action given a query state and an in-context interaction dataset — and got exploration online and conservatism offline "despite not being explicitly trained to do so", along with a proof that the resulting procedure is an efficient implementation of Bayesian posterior sampling and can learn faster than the algorithm that generated its data.

The strongest existence proof remains AdA (Bauer et al., ICML 2023): online meta-RL with Muesli across XLand 2.0's pool of 25 billion tasks, a Transformer-XL memory and a regret-based curriculum. Given more trials it improves on over 80% of a held-out task set. Its scaling result is the part worth carrying into robotics: "the slope is steeper when evaluating higher numbers of trials, showing that scaling the model is particularly effective at encouraging stronger adaptation, as opposed to stronger zero-shot generalisation." Adaptation and zero-shot generality are different capabilities. They scale differently, and a benchmark that scores only the first attempt cannot tell them apart.

Two tracks, because there are two abilities

Robotics reached the same idea from imitation rather than from RL. One-Shot Imitation Learning (Duan et al., 2017) put a single demonstration into the network as input and acted with frozen weights. The line runs through Keypoint Action Tokens (RSS 2024), where an off-the-shelf text transformer with no additional training performs few-shot in-context visual imitation on real everyday tasks; ICRT (ICRA 2025), a causal transformer prompted with teleoperated trajectories that reaches 65.0% on unseen pick-and-place and 93.3% on unseen poking where baseline overall averages sit between 7.5 and 20.0; and Instant Policy (ICLR 2025), which learns a task from one or two demonstrations by graph diffusion, trained on arbitrary trajectories generated in simulation.

Most generalist policies, though, do not have this ability at all. RICL (CoRL 2025) says so directly: policies trained by imitation lack in-context learning despite its emergence in language and vision, and putting it back takes targeted retraining — after which the system accepts 10 to 20 demonstrations of a new task and reaches 31.25% complete task success with no parameter updates at all. Vintix (ICML 2025), the first cross-domain in-context action model, is honest in the same direction: "overall, we observed that Vintix is not yet capable to handle significantly new tasks."

The language literature has a name for the gap. What In-Context Learning "Learns" In-Context (Findings of ACL 2023) separates task recognition — inferring which task this is and applying a prior formed during pretraining — from task learning — acquiring a mapping that pretraining did not contain. Task recognition does not improve with more demonstrations. Task learning does. The two tracks are built to pull those apart.

Few-shot imitation gives a model 5 expert demonstrations of a task it has never seen. Each demonstration starts from a different position; all of them share one hidden goal. The model is then scored from a held-out start. Because the goal dimensions are not in the observation and the evaluation start is not one of the demonstrated starts, no demonstrated trajectory can simply be replayed. The demonstrations reveal the goal and the skill; using them requires composing something new.

In-context RL gives a model nothing at all to begin with. It runs 20 consecutive episodes on one unseen instance, accumulating its own observations, actions and rewards in context, and is scored on the mean across all 20 of them. Adapting on the third episode is worth far more than adapting near the end, and holding back early to make a late episode look impressive costs more than it can ever recover.

The environment is not a formality

Both tracks run on Meta-World 3.1.1, with 9 task families in the imitation track and 10 in the reinforcement-learning track, at 500 steps per episode. Meta-World (Yu et al., CoRL 2019) is the standard meta-RL manipulation benchmark, and it carries a warning from its own authors: "special care needs to be taken to ensure that the task cannot be inferred directly from the image, else meta-learning algorithms will memorize the training tasks." Meta-World+ (NeurIPS 2025 Datasets and Benchmarks) documents how much undocumented drift accumulated in the benchmark over five years and reports that on the ML45 test split "there is statistically no difference between MAML-V1, MAML-v2, and RL2-V2". The environment version is pinned in the specification for exactly that reason.

Two concessions follow from the same warning. The goal dimensions are removed from the observation the model sees. And one task family — door-close-v3 — is excluded from the imitation track altogether, because its start variation cannot be made to hide the goal, which would turn it into a replay test rather than a few-shot test.

What a strong result would actually prove

It is worth being precise, because manipulation benchmarks have a bad recent record. What Are We Actually Benchmarking in Robot Manipulation? (2026) builds a roughly 90M-parameter probe with no language encoder and no robotics pretraining, which looks up a per-task embedding by index instead of reading the instruction — and it lands within one percentage point of the best published result on three of four LIBERO splits. LIBERO-PRO reports that models scoring above 90% under standard evaluation "collapse to 0.0%" once objects, initial states, instructions and environments are generalised. Shortcut Learning in Generalist Robot Policies (CoRL 2025) traces the pattern to limited diversity inside sub-datasets and distributional disparity across them.

So: a strong result here means that on a freshly drawn set of instances the entrant had never seen, in an environment where the goal is not visible in the observation, with frozen weights and a parameter budget capped at 400M parameters, a model produced more successes than the reigning model on identical seeds — and did so on at least 60% of the units where the two disagreed.

That is a narrow claim, on purpose. It is not evidence of competence on real hardware, on tasks outside this benchmark's tabletop, or under camera and lighting variation this simulator does not produce. What it is evidence of is the specific capability everything above has been circling for a decade: a frozen network that gets better at something because of what is in its context, and not because anyone trained it again.