98. A Paper-by-Paper Guide to Classic Work on Robot Foundation Models and VLA—“Humans Are the Most Intelligent VLA”

Summary

• 陈建宇 sees the decisive variable in this embodied-intelligence startup wave as the arrival of large language models, which made general-purpose robots look attainable for the first time. Previously, “100 scenarios, 100 tasks, and I had to redevelop 100 robots”; neither code nor hardware could scale. ChatGPT showed that one class of AI methods might generalize across tasks, while humanoid robots offered a relatively general-purpose body. Together, they opened a new cycle.
• The industry has yet to find a genuinely scalable robot-model architecture, and the technology path will converge before the application and business models do. 陈建宇 points to an end-to-end robot foundation model centered on vision, language, and action, with touch, sound, depth, and other modalities added over time. Traditional pipelines and systems that call existing LLMs can expand the range of scenarios in the short term, but “cannot take us to a truly general-purpose robot.”
• RT-1, RT-2, RT-X, and OpenVLA have all shown early signs of scaling, but robotics has not yet had the language-model equivalent of an “aha moment.” RT-1 used 13 robots over 17 months to collect 130K episodes covering 700 tasks, reaching nearly 100% success on seen tasks and about 75% on unseen ones. The more important lesson was that diversity matters more than size: cross-task, cross-object, and cross-embodiment data is worth more than repeatedly piling up similar examples.
• VLMs gave robots visual-language understanding and reasoning, while exposing a bottleneck: action requires dedicated modeling and cannot simply be represented by language tokens. RT-2 could identify the German flag, Taylor Swift, and the Miami Heat, but its 1–3 Hz inference rate was too slow for dynamic, fine-grained manipulation. HRT, Figure Helix, π0, and NVIDIA GR00T therefore all add a dedicated action policy or expert after the VLM, coordinating larger and smaller models or separate modules.
• The next architectural competition will center on whether action, prediction, and understanding can be unified, and whether reinforcement learning can train the full VLA directly. 陈建宇’s team has models jointly generate actions and future images, depth maps, and even tactile signals because “being able to predict means you understand the temporal relationships of the physical world.” In its VLA reinforcement-learning experiments, some tasks moved from below 50 toward 100, but directly training end to end with PPO could instead make performance deteriorate.
• 星动纪元 is taking a staged to-A, to-B, and to-C route instead of waiting for a general-purpose household robot to mature in one leap. To A serves universities, corporate researchers, and innovators; its dexterous hands are already sold in volume and have formed an initial commercial loop. To B works with manufacturing and service operators to provide scenarios and data, while household to C comes later because it requires the highest level of generalization. 陈建宇 sees “defining the robot around the scenario” as the previous paradigm—one that has at least proved unable to scale.
• Robotics has a harder endgame than language models, but commercialization along the way may arrive earlier. Industrial robots have almost no intelligence yet already ship at the 10K scale and support public companies. A new model only needs to reach human-level performance on one action to unlock B2B demand. 陈建宇 hopes that within 5 years robots will be common enough that “you’ll often see one wherever you go, and some households will start to have one,” while judging that capabilities will keep improving over the next several years: “Every capability upgrade could unlock a new commercial opportunity.”

Deep dive

1. 陈建宇 is betting his research and startup on the same embodied-intelligence stack

• 陈建宇 is an assistant professor at Tsinghua University’s Institute for Interdisciplinary Information Sciences and the founder of 星动纪元. He studied at Tsinghua as an undergraduate and earned his PhD at UC Berkeley researching robotics, control, and autonomous vehicles, before continuing to work on robot AI after returning to China.
• 星动纪元 began substantive R&D in 2022 and was formally incorporated in August 2023. It is currently at the Pre-A stage, with a new fundraising round underway; its valuation has not been disclosed.
• The company has iterated through multiple generations of humanoid hardware, reinforcement-learning locomotion-control models, and higher-level end-to-end embodied foundation models. 陈建宇 says his MBTI oscillates between INTP and INTJ.

2. Large language models are the key variable in this robot revolution

• 陈建宇’s technology timeline begins with deep learning entering robot perception through computer vision. Autonomous vehicles adopted neural networks for vision relatively early, but robot decision-making and control remained fragmented.
• Deep reinforcement learning, represented by AlphaGo, was the next inflection point. Go’s enormous state space showed that neural networks and reinforcement learning could handle continuous spaces, while “a robot is making behavioral decisions in a continuous space.”
• Reinforcement learning alone still struggled to support general-purpose robots. ChatGPT changed expectations: although it was a pure language model, it made people intuitively feel that “one AI method could be general enough.”
• The current wave therefore broke out broadly in the second half of 2023, shortly after the large-language-model boom began spreading into robotics.

3. Specialized robots cannot scale because neither code nor bodies can scale

• The traditional model involved not only “100 scenarios and 100 sets of code,” but also redesigning 100 different bodies for 100 different tasks. The two dimensions compounded the problem, making it difficult for robotics to replicate the scale of cars, phones, or PCs.
• Even industrial robots, the highest-volume segment, remain “negligible” compared with the automotive and smartphone industries. 陈建宇 attributes the root cause to specialization, not to the industry’s lack of history.
• Humanoids became hot at the same time because they offer a relatively general-purpose body, reducing the need to redesign the robot’s form for every task.

4. The technology path will converge before the business path takes shape

• Looking at the dozens of startups that emerged over 2 years, 陈建宇 believes the technology path is “gradually converging, but not fully converged,” with technology likely settling before scenarios and commercial applications.
• The next wave of commercialization must rest on new technical capabilities. The commercial opportunities accessible to traditional robots have already been mined by a large number of incumbents.
• Business and technology will continue to shape each other: technical direction and data sources must connect to a commercial loop, while technical progress will determine which product forms and business models are viable. The primary driver, however, remains model architecture.

5. VLA is a broad end-to-end embodied model, not a closed list of 3 letters

• 陈建宇’s definition is direct: V is vision, L is language, and A is action. In his usage, VLA must be an end-to-end model that can scale and generalize.
• Once vision and language enter the model, the robot must directly output action as well. Touch, sound, and other modalities should also be included, but “let’s not keep extending the letters forever”; VLA serves as a broader shorthand.
• Existing VLA systems may not be the final architecture, but the broad objective will remain: unify different modalities in one model, with inputs and outputs trained directly rather than assembled from multiple external modules.
• The more distant end state is to unify language models, autonomous-driving models, and robot models into one embodied model: “A human is a standard general-purpose VLA model. A human is AGI.”

6. The hardest problem is finding a genuinely scalable model architecture

• 陈建宇 reduces the primary unsolved problem to one question: “A model architecture that can scale.” The industry is moving quickly, but has not yet found an answer capable of supporting general-purpose robots.
• This is both a scientific and an engineering problem. Scientifically, researchers must explain the essence of embodied intelligence and how humans learn; engineering still has to solve data, compute, hardware, training, and deployment.
• He expects progress to come incrementally, much like autonomous driving moved from Level 2 and Level 3 toward Level 4. Model size will also have to be determined by the robot’s body and available compute.

7. Robot foundation models replicate the working method of “large models”

• Traditional AI trains separate models for segmentation, classification, detection, VQA, and other tasks. Foundation models pretrain on large-scale multimodal data, then adapt through fine-tuning or prompting—effectively “unifying the brain.”
• Robot learning still largely follows the pattern of “one body, one task, one model.” Even advanced systems such as ALOHA mostly remain within that paradigm, replacing hand-written programs with imitation or reinforcement learning.
• The ultimate goal is for one robot foundation model to generalize zero-shot to new tasks and bodies through prompting. The practical path will first involve pretraining plus fine-tuning, then gradually reduce task-specific adaptation.

8. SayCan first connected language models to high-level planning

• Traditional robots can be roughly divided into perception, decision/planning, and actuation. The most natural first step was to replace high-level planning that can be expressed in language with an LLM, while retaining existing perception and low-level control.
• Google’s SayCan, published in 2022, targeted long-horizon natural-language tasks. The LLM proposed actions that might achieve the goal, while a separately trained value function judged which actions were actually executable from the robot’s current state.
• After matching the two, the robot chose an action that was both useful for the goal and “able to be done.” When 张小珺 asked, “If it can’t do something, doesn’t it just get stuck?”, 陈建宇 acknowledged that SayCan does not guarantee an optimal solution and must have failure cases that were not shown.

9. Closed-loop feedback turns robots from plan-followers into adaptive systems

• The key contribution of Inner Monologue was not another round of planning, but feeding environmental feedback back into the reasoning chain. If a key did not fit a lock, the system could infer that it was the wrong key and try another.
• Unlike SayCan, which might list steps 1 through 5 and execute them sequentially, this approach allows the robot to reassess after an action changes the environment, creating a robot feedback loop resembling chain-of-thought.
• 陈建宇’s team also moved feedback into task execution, using a VLM detector at roughly 10 Hz to identify anomalies in real time. If a box fell while the robot was carrying it, the robot could immediately squat down and pick it up instead of discovering the problem only at the destination.
• In this architecture, the VLM “looks at the environment” and checks whether the task is proceeding normally, while the LLM handles high-level planning and replanning after anomalies. The two cooperate, but this is still not a unified robot foundation model.

10. VoxPoser lets language models write control code, but remains tool assembly

• Fei-Fei Li’s team developed VoxPoser to use a VLM for spatial information, an LLM for task understanding and reasoning, and automatic robot-program generation, with a low-level controller handling arm trajectory planning.
• To open a drawer, for example, the model could write an affordance map marking the target location, then a constraint map representing obstacles, and finally generate collision-free motion through trajectory optimization.
• Code as Policies and Microsoft’s work on having ChatGPT write robot code follow the same idea: use the language model’s coding ability to automate control logic previously written by engineers.
• 陈建宇’s reservation is that these models have not actually learned embodied capabilities; they have merely “been connected to some robot tools.” They are useful in the short term, but unlikely to reach a truly general-purpose robot foundation model.

11. ALOHA showed that low-cost hardware can perform fine-grained bimanual manipulation

• ALOHA’s first-generation system, released in 2023, emphasized low-cost hardware and fine-grained bimanual manipulation. The dual-arm hardware, teleoperation method, and algorithms were all open source. It remained one task, one model, but substantially lowered the barrier to fine manipulation.
• Its core model, ACT—Action Chunking Transformer—is an encoder-decoder Transformer. Cameras from multiple views are encoded by CNNs, and the model outputs a sequence of future actions at once.
• Rather than executing only the latest planned point, ACT takes a weighted average of predictions for the current action generated at different historical moments, creating a temporal ensemble. This borrows the MPC idea and produces a smoother trajectory.
• Demonstrations such as putting a battery into a box showed that low-cost hardware could perform highly precise bimanual coordination. But 陈建宇 emphasized that “although it is very capable,” it technically has no language and cannot be considered a complete VLA.

12. Mobile ALOHA links teleoperation, mobility, and bimanual manipulation

• Mobile ALOHA, which gained attention in 2024, added a mobile base to the dual-arm system. A person could push the system around, operate while walking, or move to a target location before starting the task.
• ALOHA’s teleoperation maps a master arm to a follower arm. The operator directly grips the robot arm, whose movements are copied synchronously and recorded as imitation-learning data.
• Mobile ALOHA fixed the operator onto the system. 陈建宇 considers the design valuable: the tactile feedback from pushing and gripping directly may be more precise than using a remote controller.
• The complete system costs a little over $30K, or more than RMB200K. Building it yourself can reduce the cost further.

13. Gato proposed “one model governing every modality” too early

• DeepMind’s Gato, released in 2022, attempted to use one Transformer for vision, language dialogue, VQA, and robot control. It was an early prototype of a “generalist agent.”
• The method was extremely direct: feed different modalities into one unified Transformer. Because ChatGPT had not yet appeared, the demos were relatively simple and performance was not impressive.
• 张小珺 felt the work was somewhat “forcing the issue.” 陈建宇 says he had the same doubt at the time: could a generalist really emerge simply by continuously increasing model parameters and data? ChatGPT’s arrival materially changed his view of the approach.
• Gato’s value therefore lay less in its metrics than in raising the question of the end state. The team then narrowed its focus to robotics and evolved the unified Transformer into RT-1.

14. RT-1 pushed the robot Transformer to the edge of practical use with engineered data

• RT-1 focused on robot control. Its front end processed images and language, its core remained attention-based, and it ultimately output actions directly, forming an end-to-end VLA.
• Google and DeepMind used 13 robots over 17 months to collect 130K episodes covering 700 tasks. Success on seen tasks was close to 100%.
• On unseen tasks, RT-1 still achieved about 75% success. The tasks were mostly simple grasping tasks, but the results demonstrated the generalization unlocked by large-scale robot data.
• Asked how a robot Transformer differs from a large-language-model Transformer, 陈建宇 said the underlying attention mechanism is not fundamentally different. The main differences lie in visual encoding, action output, and the full network architecture.

15. Data diversity matters more than repeatedly piling up volume

• 陈建宇 summarizes RT-1’s key lesson as follows: for robot data, diversity matters more than size.
• Diversity means different tasks, objects, positions, and states—not repeating demonstrations of the same task with the same object in the same position.
• Expert-driver data in autonomous driving shows the same problem. Normal drivers spend most of their time in the middle of the road, while the dangerous states and corner cases that determine safety are rare.
• Model architecture sets the ceiling for capability, while data quality, volume, and distribution affect actual performance. Unfiltered and undesigned data may consist mostly of redundant samples.

16. Octo and CrossFormer begin sharing one brain while retaining different small brains

• Octo can be viewed as an open-source version of RT-1. Images and language are encoded separately before entering a Transformer, and downstream fine-tuning can replace the action space to support controls such as end-effector poses or joint angles.
• CrossFormer goes further in pursuing cross-embodiment learning, allowing arms, dual-arm robots, wheeled robots, quadrupeds, and even aircraft to share a core model.
• It does not force every output into one format. Instead, it configures a different action head for each embodiment. 张小珺 described this as “sharing one brain while having different small brains,” a metaphor 陈建宇 endorsed.
• In experiments, joint training across multiple embodiments outperformed training on a single embodiment for some tasks, showing that tasks and bodies share transferable commonalities and that data can benefit one another across embodiments.

17. GR-1 and GR-2 add future prediction to the VLA training objective

• ByteDance’s GR-1 first pretrained on video: given a current image and language description, it generated an image from a future point in time. Robot action data was then added during fine-tuning so the model could output robot actions.
• 陈建宇 explains the mechanism this way: “Being able to predict means you understand the temporal relationships of the physical world.” 张小珺 cited predicting whether a child might run into the road while driving and worrying that a ceramic cup might fall from the edge of a table; 陈建宇 agreed that prediction feeds back into better decisions.
• GR-2 kept the basic approach but expanded the model and training data. Experiments showed that larger parameter counts produced lower training loss and higher success rates, offering direct evidence of scaling.
• The approach still trains a robot Transformer from scratch, but future-image supervision means the model no longer asks only, “What should I move next?” It also learns, “What will the world look like after I move this way?”

18. PaLM-E brought reasoning, but remained a hierarchical system

• Many VLMs are built on pretrained language models. Compared with training from scratch, they can leverage stronger language understanding, visual semantics, and internet knowledge.
• Google’s PaLM-E combined PaLM with a visual encoder to create an embodied multimodal language model. Its parameter count reached 562B at the time, and it could answer questions from a robot’s perspective and plan relatively long tasks.
• Sorting blocks of different colors into corresponding corners, for example, requires understanding color, the meaning of a “corner” in physical space, and the logic of categorization. This kind of reasoning is difficult for a pure robot Transformer to learn from limited demonstrations alone.
• PaLM-E itself still output a language plan, then called RT-1 or another controller to execute the actions. The VLM and controller were not trained end to end, so the architecture remained hierarchical.

19. RT-2 connected internet knowledge directly to robot actions

• 陈建宇 sees RT-2 as one of the most representative foundational VLA works. It absorbed RT-1’s control capabilities into a VLM, allowing one model to output actions directly from vision and language.
• The model still output tokens, but the tokens were mapped to robot actions such as gripper positions. During training, RT-1 robot data was fine-tuned jointly with the original VLM data, preventing the model from weakening its existing vision-language capabilities after learning actions.
• The robot could therefore execute commands such as “put the banana on Germany” and “put the Coke can on Taylor Swift,” recognizing flags, people, and sports teams that might not have appeared in the robot demonstrations.
• RT-1 was already close to saturation on seen tasks. RT-2’s advantage appeared mainly in unseen scenarios, where the pretrained VLM connected internet knowledge to robot actions.

20. RT-X showed that cross-lab data can help, but quality remains a hard constraint

• Open X-Embodiment/RT-X aggregated roughly 60 datasets contributed by about 60 laboratories, covering multiple skills and embodiments. Some labs contributed existing data, while others collected new data for the project; Google also released the data used in the RT-1 and RT-2 work.
• After its release, the dataset became a common resource for training VLA models. With many authors participating, it effectively established a reusable robot-data foundation for academia.
• The limitations are equally clear: despite differing embodiments, most systems were still arms with grippers, and data-collection standards varied widely across labs. Overall quality was not especially high.
• Even so, RT-2-X trained on multiple datasets outperformed models trained on a single dataset, providing another early indication that a general model can improve specialized tasks.

21. OpenVLA handed the RT-2 approach to the open-source community

• Google did not open-source RT-2. 陈建宇 therefore describes OpenVLA as the “open-source version of RT-2,” likewise using a VLM backbone to predict robot actions directly.
• Both its model and training method were released publicly, and the team performed substantial tuning. In comparisons with RT-1 and RT-2, OpenVLA outperformed the original RT-2 on some metrics and achieved SOTA performance, lowering the barrier to reproduction and downstream modification.

22. Pure-VLM approaches stall on action frequency and representation

• After reproducing RT-2, 陈建宇’s team concluded that VLMs have knowledge and reasoning but lack dedicated treatment of the action layer. Representing continuous control as tokens does not mean action has been properly modeled.
• The more practical limitation is speed. RT-2 runs at roughly 1–3 Hz—only 1 to 3 actions per second—which is too slow for dynamic tracking, fine contact, and stable closed-loop control.
• The team proposed HRT: a low-frequency, large VLM provides semantics and latent representations, while a high-frequency action policy with tens of millions of parameters specializes in action and directly receives visual feedback.
• The approach maintained a success rate similar to, or slightly better than, RT-2 while running faster. When a person moved the target object, the high-frequency action policy could track it more promptly and complete the grasp.

23. Helix, π0, and GR00T all strengthen the action module

• Figure’s Helix combines System 1 and System 2. A pretrained VLM sits at the top, while a lower-level Transformer with roughly 80M parameters handles action, with the two operating at different frequencies.
• PI’s π0 likewise adds an action expert after the VLM, but uses diffusion/flow matching for the action module. It does not use the same frequency split, yet its demonstrations—folding clothes and removing laundry from a washing machine—are precise.
• NVIDIA GR00T also places a Diffusion Transformer after the VLM and adds an action encoder and decoder for more detailed action modeling, while using simulation to generate augmented data automatically.
• 陈建宇 sees a converging direction: the VLM handles understanding and reasoning, while a dedicated smaller model handles high-frequency continuous action, rather than one language decoder managing every aspect of control.

24. Diffusion Policy treats action trajectories as a generation problem

• Diffusion learns a distribution by gradually adding and removing noise. It is a training and generation method; the denoising network inside it can still use a CNN or Transformer, so diffusion and Transformer are not mutually exclusive.
• Diffusion Policy transfers the logic of image generation to robot action sequences. Conditioned on the current observation, it gradually generates a future trajectory, then executes the most recent action in a rolling process similar to MPC.
• It is particularly suited to complex, long-horizon, fine-grained tasks, such as placing a trapezoidal object in a designated location or continuously spreading sauce evenly on a pancake while a person keeps moving the target.
• These models can be small and have no language capability, but they show that continuous action itself deserves dedicated generation and modeling rather than being treated as an accessory numerical output from a VLM.

25. RDT scales Diffusion Policy to roughly 1B parameters

• The team led by 朱军 at Tsinghua scaled Diffusion Policy to roughly 1B parameters in RDT, pretraining on about 1 million examples, much of it from Open X-Embodiment, before fine-tuning on proprietary data.
• Images and language are encoded separately before entering a Diffusion Transformer, which repeatedly denoises to generate actions. It remains an end-to-end VLA with vision and language inputs and action output.
• RDT also proposes a unified action space, using different segments of a long vector to carry the actions of single-arm, dual-arm, wheeled, and other embodiments.
• 陈建宇 remains cautious: the range of future embodiments cannot be exhaustively enumerated in advance, and a unified vector could become longer and eventually insufficient. Sharing a brain while adding a small action head for each new embodiment may be more flexible.

26. Joint prediction and action give the VLA an internal world model

• 陈建宇’s team’s Prediction with Action combines two capabilities of diffusion: generating an action while generating how the world will change after that action.
• The model can take images, language, robot state, depth, or tactile input, and output actions alongside future images, future depth, or even a prediction of “what I will be able to feel in the future.”
• The core ideas are joint distribution and joint denoising. Rather than bolting on a separate world model, the system jointly trains prediction and control inside one unified Diffusion Transformer.
• In tests with more unseen objects and harder tasks, the joint-prediction model performed better. The team also observed success rates rising as parameter counts increased.

27. World prediction works, but the model can still “guess wrong” plausibly

• In a refrigerator-opening task, the model could predict how the door would open and what might be behind it. The generated result showed signs of AI synthesis but broadly followed the physical relationships of the motion.
• A partially occluded object made the boundary clearer: the model predicted a pear behind the obstruction when the real object was a banana. It guessed the specific object incorrectly, but the relationship among robot movement, gripper closure, and the object’s emergence remained physically reasonable.
• Beyond RGB video, the model can also predict depth maps. 陈建宇 calls this a world model integrated into the VLA: given an action, it represents how the environment will change next.

28. Pretrained video models accelerate learning complex tool use

• The first-generation joint model trained from scratch on robot data. Its successor, Video Prediction Policy, reuses a generative model pretrained on large-scale video to leverage visual and physical knowledge.
• The team first fine-tunes the video model on the robot domain so it understands robot instructions and scenes, then deeply integrates it with Diffusion Policy through latent representations and cross-attention.
• In a zero-shot task—“use the spoon to scoop the red liquid into the blue bowl”—the model knows what a spoon is and understands that it must first grasp the spoon before scooping. The initial success rate is low, but a small amount of task-specific fine-tuning quickly stabilizes performance.
• The same architecture learned dexterous-hand tasks such as hammering a nail and using a screw gun to press a switch and drive a screw, moving beyond standard pick-and-place into multi-step tool use.

29. Understanding, prediction, and action are moving toward one unified model

• 陈建宇’s team has proposed UP-VLA, jointly training VLM understanding, video-model prediction, and robot action in one architecture.
• The same model can describe a current image, answer VQA questions, predict what will happen in the next few seconds after a task is executed, and directly perform the action.
• The goal is close to Gato’s, but the foundation has changed. Pretrained language models, VLMs, and generative models are now available, so the system no longer has to train one foundational Transformer entirely from scratch.

30. Reinforcement learning may break through imitation-learning ceilings, but direct PPO does not work

• Most earlier VLA systems relied on supervised or imitation learning. Drawing an analogy to RLHF, 陈建宇 argues that robots also need reinforcement learning after supervised fine-tuning reaches a plateau.
• In the team’s experiments, pure imitation policies scored below 50 on some tasks. A specially designed reinforcement-learning procedure steadily improved them, in some cases approaching 100, with especially clear gains on unseen tasks.
• Standard PPO applied directly to the full VLA can instead make performance “get worse the more you train.” A workable compromise is to freeze the VLM, train only the action head with RL, and save successful trajectories.
• The second step unfreezes the VLM and uses the successful trajectories for supervised training, indirectly transferring the improvement back into the large model. 陈建宇 acknowledges that this is not the final state; ideally, RL would train the entire model end to end.

31. 陈建宇’s path moved from hardware and MPC toward end-to-end learning

• As an undergraduate in Tsinghua’s Department of Precision Instruments, he studied mechanics, electronics, control, and embedded systems, and his undergraduate thesis already involved bipedal robot gait planning. The department had begun researching bipedal robots more than 20 years earlier.
• During his PhD at Berkeley, he worked with a Japanese professor, initially studying mechanical-system control and spending his first 2 years focused on MPC and optimizers.
• After AlphaGo appeared, he shifted toward reinforcement learning and robot learning. Deep learning had already entered vision but had not truly entered robot control, which became the turning point in his research direction.
• Around 2017–2018, he began exploring end-to-end autonomous driving with reinforcement learning. The technical foundation was not yet mature, so the work mainly used simulation and similar environments; deploying end-to-end control on real vehicles came later.

32. End-to-end methods can make complex robot systems simpler

• Traditional robot pipelines divide the system into perception, cognition, prediction, decision, planning, control, and other subsystems. The more subsystems there are, the more complex the coupling and the harder the system is to analyze.
• For 陈建宇, who has a control background, a unified network is not an extra burden but a simpler system: “If it is one system, one end-to-end network, I’m actually happier.”
• 星动纪元 works not only on the “brain,” but also on humanoid bodies, bipedal systems, dual hands, reinforcement-learning locomotion control as the “small brain,” and core components such as joints, motors, and dexterous hands.
• The full-stack approach is not simply an effort to maximize in-house development. The supply chain is still immature and the market does not offer satisfactory components. Hardware and software must be iterated together to handle their coupling around the final task.

33. Moving from autonomous vehicles to robots meant betting on a larger, less-defined industry

• When 陈建宇 went to the US for his PhD in 2015, he arrived during the previous autonomous-vehicle boom, and his doctoral work was largely related to autonomous driving. By the time he graduated in 2020, he believed much of the autonomous-vehicle industry’s structure had already taken shape.
• He chose robotics because, at the model level, robotics would eventually encompass autonomous driving: one embodied model could control a vehicle, an arm, or a humanoid, while their dynamics could also be described in a unified form.
• His decision to return to China reflected a desire not to join a large company and to pursue research more freely. Changes in the China-US environment and the pandemic in 2020 reinforced that inclination; after receiving an opportunity at Tsinghua, he returned to teach.
• The startup came from “not wanting to wait.” Universities can explore technology but struggle to cover manufacturing, products, sales, and commercialization. After Tesla and Xiaomi released robots in 2022, he also worried that a small lab would not be able to race large companies indefinitely.

34. Commercialization starts with to A, then follows technical maturity toward to B and to C

• 星动纪元 divides the market into to A, to B, and to C, with A standing for academia. During the introduction of a new category, the earliest buyers are universities, corporate researchers, technically minded users, and startups or large companies exploring new applications.
• Its dexterous hands are already being sold in volume to universities and companies in China and abroad. 陈建宇 says this part has “already started turning into a commercial loop,” although the product will continue to iterate because the technology has not yet converged.
• In the to-B phase, the company works with manufacturing and service operators to provide scenarios, some data, and commercial understanding, turning capabilities into products that meet specific requirements. Some partnerships are already underway.
• Household to C comes later because home environments demand the highest level of generalization. The robot end state is harder than the language-model end state, but B2B customers only need human-level performance on selected actions, allowing earlier deployment.

35. Scenario customization is the old logic; general-purpose capability is the new cycle’s through line

• Asked whether robots should be defined around scenarios, 陈建宇 answered: “That was the previous generation.” The approach cannot be called a total failure, but it has at least proved unable to scale.
• The new path does not reject specific scenarios. It allows more general models and bodies to be reused across scenarios, followed by the necessary adaptation.
• He also does not advocate waiting for the technology to be finished before commercializing. Commercial scenarios provide data and feedback, while every capability improvement expands the range of tasks the model can serve; the two should form a loop.
• The university and company therefore divide the work. The company handles engineering architecture, products, and the commercial loop; students focus more on frontier AI research while using the company’s hardware, infrastructure, and some data.

36. Scaling laws are showing signs, but the “aha moment” has not arrived

• 陈建宇 points to evidence including the diversity gains in the RT series, cross-embodiment joint training, GR-2, and the lower loss and higher success rates produced by larger models in his team’s work.
• But he insists on calling these only “signs.” He has not declared robot scaling laws fully proven, and robotics has not produced the powerful “aha moment” that made everyone intuitively rethink language models.
• Even so, the industry has found ways to keep improving robot capability. He expects capability to continue rising “every year, every month” over the next several years.
• Commercial opportunities therefore do not need to wait for ultimate AGI. Industrial arms have almost no intelligence yet ship at the 10K scale; each new capability tier could unlock another batch of applications.

37. Robots will “lay eggs along the way” before reaching the end state

• 张小珺 summarized 陈建宇’s bet as believing that model capabilities will keep improving and that the company can “lay product eggs along the way.” 陈建宇 agrees, and believes robotics may unlock applications along the way more smoothly than language models.
• Language products usually need to approach human-level language ability before they become genuinely useful; weak early models are difficult to use as substitutes for work. Robots only need to reach human-level performance on one narrowly defined process.
• A fully general-purpose robot is of course much harder. It must generalize actions, possess human-level vision, language, and reasoning, and handle arbitrary situations in the open physical world.
• 陈建宇’s 5-year vision is therefore not “a few robots in every household,” but “you’ll often see one wherever you go, and some households will start to have one.” He considers that level of ubiquity highly achievable.