Why Robotics Still Isn't Solved - But Could Be Soon | YC Paper Club

The Unsolved Promise of Robotics [00:00:00]

• For the past decade, there have been recurring predictions that robotics would be "solved" within the next year.
• Early optimism was fueled by advancements like AlphaGo and the ability to train robots to walk in simulations with thousands of iterations.
• The emergence of systems like Aloha, capable of tasks like watering plants and making coffee, further fueled expectations.

  "Clearly, next year, robotics is going to be solved."

The 2026 Robotics Prediction [00:01:00]

• A specific prediction for 2026 suggested it would be the year of the robots.
• Papers like "Diffusion Policy" and advancements in Vision-Language Models (VLAs) initially seemed to confirm this.
• The idea was that with teleoperation data, robots would become highly capable.

  "I would say honestly, we're halfway through 2026. And you still can't yet you can pre order Neo one x, I can't buy a pie or figure robot just yet."

The Reality of Demos vs. Deployment [00:02:00]

• Despite impressive demos, readily available, general-purpose robots are not yet on the market.
• While work cells have seen some success, widespread domestic robots are still absent.
• The current year is characterized as the "year of the demos" rather than the year of widespread robotic deployment.

  "But it's definitely the year of the demos. I know that for certain."

The Challenge of Teleoperation Data [00:02:30]

• Collecting data through teleoperation is remarkably difficult and finicky, especially with limited tools like grippers and wrist cameras.
• Relying on this type of data for scaling up robotic capabilities presents a significant bottleneck.

  "Is it easy or hard? I mean, it's remarkably hard."

Four Walls to Scaling Robotics [00:03:00]

• Scaling robotics faces at least four major challenges that have yet to be overcome.
• These challenges represent fundamental limitations in current approaches.

  "And so I kind of think about it in these four, there's probably more, but at least these four walls that we have to scale and get over."

1. Physical Real-World Modeling and the Sim-to-Real Gap [00:03:30]

• Video models trained in simulation often fail to respect real-world physics when deployed.
• Simulated environments can allow unrealistic behaviors, like driving through buildings without crashing, highlighting the "sim-to-real" gap.
• This gap makes it difficult to transfer learned behaviors from simulation to the real world.

  "In the real world, it really doesn't respect physics."

Deformable Objects and Action Conditioning [00:04:30]

• Modeling the transition function between states (ST to ST+1) is even harder with deformable objects.
• Conditioning these models further on actions requires a lot of data and is not yet well-solved.

  "Deformable objects, even worse is, that's just determining the transition function from ST to ST plus one, if you condition further on the action, then it really doesn't work."

2. Representation for Action Space [00:05:00]

• Creating an effective representation for the action space is a completely unsolved problem for rapid learning.
• Even with techniques like feature pyramid networks, managing the complexity of action spaces remains a challenge.

  "So representation for your action space is actually like a completely unsolved thing, if you want to learn quickly."

3. The Sensory-Motor Issue [00:05:30]

• Humans possess incredibly rich sensory feedback through nerve endings, allowing for detailed perception of forces, moisture, temperature, and friction.
• Robots typically have very limited sensory input, such as a single force-torque sensor on a fingertip and perhaps a wrist camera.
• This lack of rich sensory data prevents robots from performing tasks that humans do intuitively through touch alone, like finding an object in a backpack without sight.

  "We have these nerve endings that can do so much... And these robots don't have that."

The Importance of Tactile Sensing [00:06:30]

• The ability to sense normal and tangent forces, moisture, temperature, and friction is crucial for dexterity.
• Without an "epidermis" like humans have, robots cannot replicate this level of environmental understanding.
• This limitation is evident even in simple tasks like tying skates in the cold, where tactile feedback is essential.

  "There's no way we have robots that can do that now, because we don't have an epidermis."

4. Embodiment Drift [00:07:00]

• Embodiment drift refers to the degradation of a robot's physical components over time, affecting actuator performance.
• Dust, corrosion, and wear cause actuators to function less reliably, impacting the force applied for a given action.
• This drift is a real issue, even in self-driving cars, where battery power output can vary over time.

  "The actuators get dust in them, they get corroded... and they just don't work as well."

Retraining and Data Staleness [00:07:30]

• Embodiment drift necessitates retraining the entire Vision-Language Action (VLA) model because the teleoperation data becomes stale.
• The mapping from actions to outcomes shifts, requiring new data collection.

  "And so like, because it's not mapping to the teleops data is almost stale, and you have to recollect it."

Introducing MEM: Multi-Scale Embodied Memory [00:09:00]

• Marcel presents MEM (Multi-Scale Embodied Memory), a system designed to address long-horizon tasks in robotics.
• Current policies often lack memory, leading to repetitive or incomplete tasks (e.g., washing dishes forever, burning grilled cheese).
• Adding memory is challenging due to effectiveness issues (performance degradation) and efficiency issues (increased resource intensity).

  "My claim is that in order to obtain all of these insights, we actually need to add memory into our policies."

Decomposing Memory for Robotics [00:11:00]

• MEM decomposes policies into a high-level and low-level policy, and memory into short-term and long-term components.
• Short-term memory (dense frames) is for dexterous manipulation (low-level policy).
• Long-term memory (compressed language representation) is for high-level planning.

  "And then we're going to decompose the memory into two different types."

Short-Term Visual Memory [00:11:30]

• A new encoder based on Vision Transformer (ViT) with temporal attention layers is used.
• This compresses image sequences, retaining necessary information for dexterous manipulation.
• Benefits include easy ViT initialization, compression, and fast inference.

  "Our idea here was to design a new encoder that is based on the VIT."

Long-Term Language Memory [00:13:00]

• A high-level policy predicts a memory scratchpad, which is a compressed textual representation of past events.
• This compressed representation is more efficient than images and less prone to distribution shifts.
• It allows for tracking task progress and remembering past events over longer durations.

  "This actually, like much more, much more compressed representation than images, since text uses way less stock tokens and images."

In-Context Adaptation with Memory [00:15:00]

• Memory enables "in-context adaptation," allowing robots to learn from mistakes and react to new scenarios.
• Policies without memory repeat the same errors, while those with memory can correct their actions (e.g., adjusting grip, switching sides to open a fridge).

  "It's actually something that is really lacking right now into VLAs. And I think it can be super promising in the future."

Self-Supervised Bootstrapping of Embodied Reasoning [00:18:00]

• Milan discusses "Self-supervised Bootstrapping of Action-Predictive Embodied Reasoning" (R&B Encore).
• The goal is to enable robots to teach themselves reasoning, addressing the scarcity of robotics data.
• Key challenges are the "grounding problem" (what to reason about) and the "oracle source" problem (where the reasoning comes from).

  "The idea is that there's not a lot of data in robotics. Um, it's data scarce. And so any form of signal, um, that you can use to augment your data set, um, is very valuable."

R&B Encore: Refine and Bootstrap Embodiment-Specific Chain of Thought Reasoning [00:20:00]

• R&B Encore is a self-improving pre-training cycle for embodied reasoning VLAs.
• It treats reasoning as an unobserved latent variable, using variational inference.
• Components include a "reasoning proposer" and a "reasoning validator" that scores reasoning traces based on concision, non-triviality, and action predictiveness.

  "This is a self-improving pre-training cycle for embodied reasoning VLA's."

Generalizing Embodied Reasoning [00:22:00]

• The approach is tested across embodiments like manipulation, legged locomotion, and self-driving.
• Findings suggest that for manipulation, move and gripper position reasoning are useful, while perceptual reasoning is less so.
• For legged locomotion, structural affordances and movements are more important than terrain reasoning.

  "Selective reasoning is way more important than exhaustive reasoning."

SimToolReal: Dexterous Tool Manipulation via Goal Reaching [00:26:00]

• Tyler presents SimToolReal, a single policy trained entirely in simulation for dexterous tool manipulation.
• The policy learns to reach arbitrary goal poses, unifying various tasks without task-specific training.
• It uses a human video demonstration to specify the sequence of goal poses at inference time.

  "Our key insight is that we can unify dexterous tool manipulation as goal reaching."

Zero-Shot Dexterity and Generalization [00:27:00]

• The SimToolReal policy works zero-shot across novel tools and tasks unseen during training.
• A new task is defined by a sequence of goal poses, allowing rapid adaptation.
• Evaluation shows substantial progress across various tool families, though performance varies with object weight and occlusion.

  "This single policy works zero shot across novel tools and tasks never seen during training."

Recovering from Failures [00:29:00]

• The policy demonstrates strong recovery behaviors, such as re-grasping a dropped object.
• This capability is partly attributed to random forces applied to objects during simulation training, exposing the policy to failure scenarios.

  "The policy demonstrates really strong recovery behaviors."

Rerun: A Data Layer for Physical AI [00:33:00]

• Nico introduces Rerun, a unified data layer for physical AI, providing tools for data collection, visualization, and training.
• He advocates for the rise of "robotics application companies" that focus on end-to-end business problems rather than general AI.
• These companies should start with teleoperation and off-the-shelf hardware, iterating rapidly based on real-world feedback.

  "At Rerun, we are building this sort of unified data layer for physical AI."

The Robotics Application Company Model [00:34:00]

• This model involves solving a specific customer problem, starting with teleoperation, and gradually incorporating learned models.
• Real-world deployment reveals critical business requirements and failure modes that cannot be anticipated in a lab.
• The physical world is "brutal," and rapid iteration based on customer needs is essential for success.

  "My personal belief is that this kind of category of company is going to be the new SaaS."

World Action Models (WAMs) vs. VLAs [00:42:00]

• Bill discusses the shift from Vision-Language Actions (VLAs) to World Action Models (WAMs).
• WAMs use diffusion transformers to predict future states and kinematics, aiming for better generalization.
• However, WAMs like NVIDIA's Dreamer are computationally heavy and expensive to run.

  "Basically, you have a VLM that's trying to predict an action through an action head. What a word action model is, however, is you have a central diffusion transformer that's trying to predict what the future looks like and future kinematics at the same time."

Optimizing WAMs for Robotics [00:43:00]

• General Instinct focuses on optimizing WAMs for real-time robotics applications.
• They employ techniques like distillation, smaller models, and cross-attention to reduce computational load.
• The goal is to achieve faster inference times without sacrificing performance, making WAMs economically viable for robotics.

  "So economically, for robotics as an industry, this is not scalable."

Latent World Action Models [00:44:00]

• Latent WAMs predict future states in a compressed latent space rather than decoding full future frames.
• This approach, exemplified by "ImageRAM" and "Fast1," is more efficient.
• The focus shifts from predicting future video to predicting future kinematics or end states.

  "But for the latent word action model, it's like you look at the navigation of your Google Maps and then think about what the model the policy is heading to and what kind of action you're going to produce in the future."

General Instinct's Infrastructure [00:46:00]

• General Instinct provides infrastructure for running physical AI models efficiently.
• They have optimized WAMs to run significantly faster, making them practical for real-time applications.
• Their approach involves distillation and architectural changes to speed up the inference process.

  "And for us, since we are doing the infra thing, so we did all those optimization on our infra."

Future of Robotics and Data [00:49:00]

• The discussion touches on the challenges of data collection, model generalization, and the need for efficient infrastructure.
• The future likely involves a combination of advanced models, robust data pipelines, and practical application-focused companies.
• The "death by a thousand cuts" nature of robotics requires a holistic approach, addressing both AI and operational challenges.

  "The physical world is brutal, right? Everything that you do is going to break."