IA · 27 July 2026 · 4 min read
Beyond Vision: Why Brain Waves Are the Next Frontier for Physical AI
In brief: Training the next generation of humanoid and industrial robots requires more than just video recordings. A collaboration between data infrastructure startup Encord and German neuroscience firm Zander Labs incorporates operators' brain signals into robotics training datasets. By measuring cognitive load and intent via EEG headsets, the initiative aims to unlock high-density data for physical AI.
by Team Mocchi's
The Bottleneck in Physical AI Has Shifted Away From Models
In the race toward advanced, autonomous robotics, the primary hurdle is no longer raw compute power or neural network architecture, but a severe scarcity of real-world physical data. While large language models thrived on the vast expanse of public internet text, physical AI systems designed to operate humanoid robots or industrial manipulators require rich, granular data on how humans navigate physical force, friction, gravity, and spatial awareness.
To address this data bottleneck, data tooling startup Encord has launched a pioneering experiment at its San Leandro, California facility: capturing not just an operator's physical movements and eye tracking during task execution, but also their real-time brain waves. As reported by TechCrunch, the project aims to convert neurological reactions into high-density labels for training physical AI models.
The Jenga Experiment: Turning Brain Signals Into Datasets
During trial runs conducted by Encord, human robotic trainers perform high-precision physical tasks, such as carefully disassembling a teetering tower of Jenga blocks. While executing these maneuvers, the operator wears a camera headset to track eye focus alongside a specialized electroencephalography (EEG) headset developed by German neuroscience startup Zander Labs.
The EEG sensors track brain activity in real time, detecting critical cognitive states such as hesitation, anticipation of an error, surprise, or spikes in mental effort. Vineeth Velmurugan, Encord’s head of robot learning and a veteran of OpenAI’s robotics lab, points out that conventional vision-based teleoperation captures what an operator’s hands are doing, but remains completely blind to what their mind is calculating at that exact microsecond.
By decoding brain waves, the data pipeline learns when a human trainer experiences heightened cognitive strain or makes an on-the-fly trajectory correction. This "neural telemetry" allows data engineers to signal to the underlying AI model precisely when higher-effort inference or strategy re-evaluation is required.
Moving Beyond Basic Teleoperation
Until now, training physical AI models has relied heavily on standard teleoperation or synthetic data generated in physics simulators. Both approaches come with distinct limitations: visual teleoperation is slow and lacks intent context, while physics simulations often fail to capture the messy, unpredictable nuances of real-world physical interactions.
Integrating neural telemetry provides an implicit supervisory signal. According to Lucas Gehrke, a neuroscientist at Zander Labs involved in the project, measuring brain activity helps deduce the exact moments when an operator senses that control or stability is deteriorating. Transferring this awareness into foundational robotics models could enable machines to develop an intuitive sense of caution or dynamic adaptability when handling fragile objects or navigating uncertain environments.
Nevertheless, incorporating neural signals into physical AI pipelines poses technical challenges. EEG data is notoriously noisy, susceptible to muscular interference, and highly variable across individuals. Encord is currently conducting a trial phase to evaluate whether brain wave-tagged datasets tangibly improve robot model performance and convergence rates before deciding whether to scale up production.
Mocchi's take
The experiment by Encord and Zander Labs marks a conceptual shift for AI in the physical world: overcoming the limitations of current robotics models requires moving data ingestion from external observation to internal cognitive understanding. For manufacturing, mechatronics, and robotics leaders in Italy and Europe, this evolution underlines that the core competitive differentiator will not be hardware alone, but the semantic richness of the training pipelines behind it. While neural telemetry is still in its experimental phase, embedding deeper operational context into data collection is already becoming the primary pathway to building safe, flexible robotic systems capable of seamless human collaboration on the factory floor.