Early-stage project · started October 2026; this page describes the plan and the groundwork it builds on.

Space-VLA: Vision-Language-Action Models for Manipulation Aboard the Space Station

Asif Amin
University of California, Riverside
Project in progress · contact: ramin019@ucr.edu
Zero-gravity project (groundwork) Code planned
Scene mock-up, not yet the ISS environment. Recorded episodes of the zero-gravity experts (PickCube, top; LiftPeg, bottom) replayed inside the Japanese Experiment Module (JEM) from NASA’s astrobee_media models, seen from a station camera, the robot’s exterior camera and its wrist camera. Each clip ends just before the gripper first touches the object: after that, most of the original tasks switch gravity on and finish on a table, which would be wrong in the station. I kept only episodes where nothing else touches the object first. The module is scenery only, the blue-gray work plate is my addition, and the physics are still those of the original tabletop tasks. Green ring: PickCube’s goal. Ray-traced, 0.5× speed.
Groundwork: robot experts trained in zero gravity on six tasks (from my zero-gravity project). Objects float in mid-air until the robot touches or grasps them. The ISS environment replaces the table with a station module and keeps gravity off for the whole task. Ray-traced, 2× speed.
TL;DR — Robots already work inside the International Space Station: NASA’s Astrobee, DLR’s CIMON and JAXA’s Int-Ball (Yamaguchi et al., 2026). Yet the vision-language-action models (VLAs) now used for manipulation on Earth (RT-2, OpenVLA, π0) have, to my knowledge, never been trained or tested for it. Their training data, such as Open X-Embodiment, was collected under Earth gravity, where objects fall and stay where they are put. I am building an ISS-interior manipulation environment, with the station’s real geometry and appearance and exact microgravity physics, and using it as a benchmark and training ground for VLAs. It builds directly on my zero-gravity robot-learning work, which already provides the microgravity physics, the expert policies and the demonstration-to-VLA pipeline.

Motivation

Space robots have almost no data

A recent survey of AI for space robot operations names scarce mission data and microgravity dynamics as open problems (Survey, 2026). Simulation is the practical place to gather manipulation experience at scale.

Earth VLAs assume gravity

VLA pretraining data was recorded on Earth (Open X-Embodiment), where released objects settle and nudged objects stop. In microgravity, nothing stops a nudged object until it hits something. Whether Earth-learned habits help or hurt in orbit has, to my knowledge, not been measured.

No benchmark for it yet

Space simulators exist for reinforcement learning (Space Robotics Bench), free-flyer control (APIARY, Int-Ball2) and cargo transport (pyastrobee), but to my knowledge none offers language-conditioned manipulation of free-floating objects inside a station, with demonstrations to train VLAs on.

The station is also a good proving ground for robot learning in general. It is a fixed, well-documented interior: Astrobee image sequences with ground-truth camera poses and 3D CAD models of four modules are public (Astrobee ISS datasets), and the Japanese Experiment Module has already been reconstructed with Gaussian splatting from about 500 Astrobee images (Kim et al., 2026). Its tasks are concrete: free-flyers have been built and tested to move the station’s cargo transfer bags (pyastrobee, Su et al., 2026). And the dominant physical effect, the absence of gravity, can be simulated directly.

Current progress

The ISS environment itself is just starting. What already exists is the groundwork from my zero-gravity manipulation project, which trains robots with gravity switched off in simulation; there, zero gravity is a training aid for tasks on Earth, while here it is the place the robot will actually work.

  • done
    Microgravity physics in a GPU simulator. Per-environment gravity control in ManiSkill3: thousands of parallel environments, each of which can have gravity off, on, or switch on at contact or grasp. Objects float without damping and keep any momentum the robot gives them.
  • done
    Containment volumes. Thin, camera-invisible walls that stop a floating object from drifting out of reach but let the robot arm pass through — the simulated equivalent of a station rack bay.
  • done
    Zero-gravity expert policies on eight tasks (reaching, pushing, pulling, picking, rolling, peg lifting, YCB objects), trained with reinforcement learning (PPO), 3 seeds each. On Earth-gravity test tasks they reach objects at heights where gravity-trained experts almost never do.
  • in progress
    Demonstration → VLA pipeline. Expert demonstrations collected and released as datasets (zero gravity, regular gravity); distillation into a VLA (π0.5) is underway in the zero-gravity project and carries over unchanged.
  • done
    Literature review and gap analysis (October 2026), summarized under Related work.
  • in progress
    ISS scene prototype. First step done: a visual mock-up (top of the page), with the robot arm mounted on a deck rack of the Japanese Experiment Module (JEM), the main module of the Astrobee ISS datasets and reconstructed by Kim et al., using the CAD model from NASA’s astrobee_media station models. Next: the first floating-object tasks built in the station itself.

Future directions

The approach

Real ISS appearance

Station geometry from public CAD models first, then Gaussian splats from posed ISS images, as Kim et al. have shown for one module.

Source: Astrobee ISS datasets

Exact microgravity physics

From the simulator, not learned from video: free-floating objects, momentum transfer, contact.

Source: our GPU force mechanism

Tasks from real operations

Move a cargo transfer bag, as in pyastrobee; proposed additions: catch a drifting item, stop a tumbling one, stow into a rack slot, hand over a tool.

Source: published ISS free-flyer work, ISS footage

Experts → data → VLAs

Reinforcement-learning experts generate language-labeled demonstrations; VLAs are trained and evaluated on them.

Source: zero-gravity project pipeline

Splitting it this way keeps each part on solid ground: removing gravity is simple to model, so the physics come from the simulator, while ISS imagery provides what simulators are worst at — realistic appearance and realistic tasks.

Research questions

  1. Do Earth-pretrained VLAs carry a gravity bias? Evaluate them zero-shot in the ISS scene and catalog how they fail: releasing objects that then drift, pushing too hard, never stopping a moving object.
  2. What is the best starting point for space? Compare adaptation from Earth-gravity, gravity-randomized and zero-gravity experts and VLAs, with few microgravity demonstrations.
  3. How much does visual realism matter? Compare CAD-rendered and reconstructed ISS visuals for policies that must work from images.
  4. Can a world model of the station replace the hand-built scene? Longer term: learn an action-conditioned world model of the ISS interior from real footage, so VLAs can practice in it. Video world models can already infer actions for unlabeled video (DreamGen), and a world model has been trained for ISS docking from simulated flights (Out-of-this-World-Model).

Roadmap

  1. 1 · ISS scene prototype — now

    A station module in the simulator, an arm mounted on a station rack, and two or three floating-object tasks.

  2. 2 · Gravity-bias probe

    Zero-shot evaluation of existing VLAs in the ISS scene, with a failure-mode taxonomy.

  3. 3 · Experts, demonstrations and VLA training

    Expert policies for each task, language-labeled demonstrations, VLA fine-tuning, and the adaptation study.

  4. 4 · Realism and release

    Reconstructed ISS visuals, a public benchmark (tasks × lighting × objects) and dataset.

  5. 5 · Beyond a fixed arm

    A free-flying robot with an arm, where every push also moves the robot, and real-world checks on planar air-bearing analogues of microgravity, which now carry robot arms (PINGU, 2026).