Inside a Startup Revolutionizing Robot Intelligence for a Quantum Leap in Technology

In San Francisco, freshly brewed coffee is being crafted by robots in a state-of-the-art facility, showcasing the rising integration of robotic technology in our everyday lives. Although robots have been serving coffee for years, the advanced AI behind this process offers a broad skill set beyond just brewing. These robots are capable of performing various tasks, such as folding laundry, peeling vegetables, and even kitchen cleaning, which is remarkable for technologies still in their infancy.

Founded in 2024, Physical Intelligence is leading the charge towards a future where robots seamlessly integrate into daily routines. The startup focuses on creating versatile control systems that enhance productivity across multiple tasks and different machines, similar to the humanoid robots developed by Tesla and Boston Dynamics, as well as Amazon’s industrial robots.

The concept of general robotic intelligence has long been an aspiration within the robotics community. Yet just as large language models (LLMs) revolutionized AI chatbots in the early 2020s through advanced computing techniques, breakthrough innovations in physical intelligence promise to elevate robotics to new heights.

Sergei Levin from the University of California, Berkeley, a co-founder of Physical Intelligence, remarked, “In many fields, having more data complicates matters. However, in AI, it facilitates learning from a diverse array of information, making the development process smoother.”

The evolution of LLMs has given rise to the Vision-Language-Action (VLA) model, significantly influencing Physical Intelligence’s research direction. Instead of learning tasks individually, VLA capitalizes on LLMs to convert general commands into actionable steps, empowering robots to execute a multitude of tasks. According to Ingmar Posner from Oxford University, “[VLAs] represent an exciting frontier in robotic learning, as they predict necessary movements instead of simply anticipating the next word in a conversation.

One of the critical obstacles in programming robots lies in the vast array of real-world scenarios that require adaptability. Conventional methods often struggle to amass sufficient data for learning effectively. Levine notes that while automating learning seems ideal, developers commonly avoid it due to the substantial data-gathering efforts required: “In theory, automation could simplify the process, but in practice, obtaining enough application-specific data often outweighs the manual work needed.”

By leveraging VLA capabilities, Levine and his team aim to minimize the data required for robots to thrive. In a boardroom setting, staff members were instructing robots on mundane tasks like folding shirts and organizing gifts. Adjacent to their main lab are two extensive warehouses designed like a faux supermarket and living spaces, facilitating real-world training scenarios for the robots. Additionally, they have begun introducing robots to actual homes to evaluate their capabilities in unpredictable environments.

This immersive training environment is crucial for progress, with robots learning generalization techniques that allow them to tackle tasks they’ve never encountered before. For instance, a recent AI model named π0.7 successfully prepared sweet potatoes using an air fryer, simply by following step-by-step verbal directions—including methods the AI had never been explicitly trained on.

Levine expressed astonishment at the rapid advancements made in just two years since launching Physical Intelligence. “The progress has exceeded our expectations,” he noted.

Interest from other companies is growing, with many well-funded startups and industry giants like Amazon and Google DeepMind working on their own general-purpose robotic solutions.

Although the field is advancing quickly, predicting the speed of future developments remains challenging. While AI entities such as OpenAI and Anthropic are notably progressing, robotics innovation typically occurs at a more gradual pace. This is exemplified by Moravec’s paradox: while robots can excel at strategizing in games or IQ tests, they often struggle to acquire fundamental perceptual and motor skills akin to those of a toddler.

Posner remains cautious, suggesting that the amount of data needed for practical robot deployment in real-world settings is still an open question. “We see early signs indicating potential breakthroughs, but whether this will yield viable applications in the near future is uncertain.”

Prominent researchers like Posner acknowledge the intrinsic challenges posed by human interaction with robots. “Humans tend to push robots to their limits, largely for entertainment,” he stated. “Is a scalable business model for such technology feasible? At this stage, it seems highly improbable.”