Why safety will decide the future of humanoid robots

In this episode of Lexicon, we spoke with Rick Balzano, Vice President of Go-to-Market for AI Services, Safety and Robotics at TaskUs, about the less visible work needed to turn humanoid robots into dependable machines.

Balzano works with companies developing robotics, autonomous vehicles, and other AI-powered systems. Much of that work involves collecting, annotating, and validating the data used to train machines to operate in the physical world.

Also, subscribe to IE+ for premium insights and exclusive content!

From language models to embodied AI

Large language models (LLMS), Balzona told us, work primarily from digital information. Robots, on the other hand, face a messier problem: they need to perceive their surroundings, decide what to do, and then perform an action without damaging themselves, nearby equipment, or the people around them.

“An LLM is working with patterns in text, images, and other digital inputs,” Balzano explained. “Embodied AI has to perceive the physical environment and determine how to act within it,” he added.

Unlike text and images gathered from the internet, much of the data needed to train a robot needs to be captured in the real world. This includes first-person video, sensor readings, demonstrations of physical tasks, and information about how objects reacted when touched or moved.

“Physical behaviour doesn’t scale easily,” he said. “Generalization requires environmental diversity and high-signal egocentric data.”

The gap between demos and reality

As Balzano explained, humanoid robots are regularly marketed as flexible, general-purpose workers. But he offered a more cautious assessment.

“Today, they’re probably still highly specialized robots packaged in a human-like form,” he said.

That doesn’t mean that progress is, for want of a better word, fake. Companies have achieved genuine breakthroughs in locomotion, perception and manipulation. However, online videos rarely reveal the controlled conditions, remote assistance and extensive preparation behind a successful demonstration.

“There’s a very realistic gap between demo and reality,” Balzano said, pointing to “the teleoperator that might be controlling the robot behind the scenes.”

Some robots that appear to be roaming freely through factories were actually operating inside tightly defined zones.

“What you see online is these robots looking like they’re free-roaming a factory, when in reality they’re actually in a 40-by-40 area, with safeguards and teleoperators in place,” he added.

The humanoid form, he added, is not suitable for all tasks in all industries. Some robots might need hands and feet, while others would work better with claws, wheels, or task-specific tools. The eventual goal is to develop a sufficiently capable “brain” that could be paired with different physical configurations.

Training for an unpredictable world

“Humanoid robots operate within an unpredictable environment pretty much entirely,” Balzano explained. “That makes safety more complex than traditional industrial robotics,” he added.

The hardest data to collect for training of this kind often involves rare but dangerous events. Deliberately making an expensive robot fall down stairs or collide with equipment, he explained, is hardly an attractive proposition.

But systems still had to learn how to react when something went wrong. Fine motor control presents another major challenge. A robot needs to know how much force to apply when grasping an object, folding laundry or handling a fragile dish.

Balzano said that, despite growing interest in machines that learned independently, humans were still doing much of the teaching. “The idea is that, in the future, they learn for themselves,” he said. “But right now, we’re doing a lot of the learning for them.”

Simulation can only take robots so far

Simulation offers companies a cheaper and safer way to repeat tasks at enormous scale. Using it, robots can practise movements, encounter synthetic hazards and fail without destroying costly hardware.

“Simulation is essential for any of these businesses to succeed,” Balzano said. “It’s extremely cost-efficient, and there’s zero risk in scaling.”

Simulated environments are particularly useful for creating edge cases that would be difficult or dangerous to reproduce physically. However, a simulated factory can never capture every irregular surface, hardware fault, unexpected obstruction, or unpredictable human action.

Balzano described this as the “90–10 rule.”

“If simulation gets a robot to 90 percent, does solving the final 10 percent take 90 percent of the actual budget?” he said. “We’re saving money today, but what are we doing tomorrow?”

Factories also differ from one another, meaning a robot trained for one site could not simply be dropped into another and expected to work.

“You’re not going to train in a simulated environment and just say, ‘We’re ready to go,’” he said. “You can’t throw ten of these into ten different factories and expect them to figure out what’s going on,” he added.

Safety will determine whether robots scale

Before humanoids can become ordinary factory or warehouse workers, facilities need reliable connectivity, remote intervention systems, physical recovery tools and clear procedures for the humans working around them.

“It’s our responsibility alongside our partners to collect, label and validate the data so that robots learn how to interact safely with the physical world,” Balzano said.

Training data also has to reflect genuinely diverse environments. Differences in lighting, room layouts, objects and working practices, for example, can all affect a robot’s performance.

“There has to be some serious distributional coverage, or models risk overfitting to a narrow range of scenarios,” he warned.

Balzano expects automotive assembly, warehousing and logistics to provide some of the first convincing financial returns. These sectors already use extensive automation and offer relatively structured environments.

Nevertheless, in his view, cost and productivity alone will not drive widespread adoption. “Reliability and integration are what will generate revenue,” he said. “But ultimately, safety is going to lead to mass deployment and scale.”

From viral vids to useful technology

For Balzano, humanoid robots will cross the line from compelling demonstrations to useful technology when they can operate safely at scale in varied, real-world environments.

“The more diverse, high-quality, real-world data these robots have access to, the better they can generalize to situations they haven’t necessarily encountered before,” he explained.

He estimates that this transition could happen within five to ten years, although the industry was changing too quickly for confident predictions.

“I already know that in two weeks something cool is going to happen, and everything we talked about today will become a thing of the past,” he joked.

Source: https://interestingengineering.com/ai-robotics/why-safety-will-decide-the-future-of-humanoid-robots