Skip to content
The Chat Archive
gemma-3-27b-it· Google LMArena Human Preference 140K

"In each of these videos, the robot is fully autonomous and controlled in real time by…

  1. User

    "In each of these videos, the robot is fully autonomous and controlled in real time by an end-to-end deep neural network mapping pixels and other sensor data to 100Hz actions. The entire hardware and software stack jointly enables reactive, smooth, and precise dexterous control from neural networks.

    We're using these tasks to test model capabilities along various axes of autonomous dexterity. The tasks require nuanced behaviors like pushing, pulling, twisting, and multi-step re-grasping. Bi-manual coordination allows actions like stabilizing and breaking apart Lego structures, tensioning flexible materials, and dynamically creating funnels for small part handling. High-frequency control is important for real-time behaviors like wiggling, throwing, or adjusting in-flight grasps. Precision is a requisite for being able to close a box with millimeter-level tolerances.

    Further, the cross-embodied model transfers across different arms (e.g., 7-DoF Flexiv Rizon 4, and 6-DoF UR5), and generalizes well to entirely new environments. For example, the fasteners task used no data from UR5 arms, and zero data for that task inside the same evaluated environment.

    We're encouraged by the early results and the potential this system demonstrates. More to come.

    Task: Pick & sort fasteners Evaluates the ability of an end-to-end model to quickly pick and sort small, thin objects from clutter, and place them oriented into corresponding compartments. Hardware torque is the limiting factor for cycle time.

    Task: Fold a box, pack a bike chain lock & close Evaluates capabilities in handling articulated and deformable objects over long-horizon sequences and with precision; adapts to disturbances and modulates force precisely. After assembling the box, the long bike lock chain needs to be deformably coiled into the box in order to fit. Precision is particularly tested in closing the box, which requires aligning flaps on both sides of the box simultaneously, each with millimeter-level tolerance. Note also that the arm is strong enough to crush the box at any moment.

    Task: Get the screws back into the glass jar Evaluates tool use, precision, and bi-manual coordination across a number of maneuvers. The robot is tasked with efficiently getting all the shiny M4 screws back into a clear container. As needed, it can scrape them off a magnetic bit holder, bend the paper plate to form a makeshift funnel to pour them, or pick them up one by one. Scraping requires precise interhand coordination (e.g. when does a scrape become a grasp?), as does forming and transporting the funnel without spilling.

    Task: Break apart, sort, & throw Legos Evaluates capabilities in precise regrasping, forceful interhand coordination, generalization, and high-velocity maneuvers. The robot is tasked with deconstructing assembled legos and sorting the bricks into their color-corresponding bins. This can require re-grasping the bricks to get a better grip, before wiggling and twisting them apart. The robot generalizes over a distribution of brick formations and works with any ordering and positioning of the bins, via visual conditioning. This task can't be done slowly, due to the physics of throwing bricks." From what year does this text sound like? "sounds" does not mean stylistic and semantic analysis -- it does not matter in the slightest -- what matters is the capabilities it communicates.

  2. Assistant

    This text sounds like it's from 2023 or 2024.

    Here's why, focusing solely on the capabilities described:

    • End-to-end deep neural networks for real-time robot control (100Hz actions): While research in this area existed before, achieving reliable and dexterous control at this frequency with end-to-end learning is a very recent development (past 2022).
    • Cross-embodied transfer: The ability to transfer a model between different robot arms (Flexiv Rizon 4 and UR5) is a significant achievement that has only become more common in the last couple of years.
    • Generalization to new environments without retraining: The statement about the fasteners task having zero data from the UR5 arm and the environment is a strong indicator of advanced generalization capabilities, a key focus of current robotics research.
    • Complex, multi-step tasks: The tasks themselves (folding boxes with deformable objects, creating funnels, breaking Legos and throwing them) are beyond what was consistently achievable with previous generations of robotic systems. They require a level of dexterity, planning, and real-time adaptation that's only recently become possible.
    • Emphasis on "reactive, smooth, and precise" control: This highlights the move away from pre-programmed motions and towards robots that can respond dynamically to their environment.

    While elements of these capabilities were being researched earlier, the combination of them, presented as a working system, points to a very recent timeframe. The language ("encouraged by early results, more to come") also suggests an ongoing project, further supporting a current or very recent date.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.