Tesla's VP of AI, Ashok Elluswamy, has released a 30-minute presentation detailing progress in Full Self-Driving (FSD) software, AI development, and the team's latest innovations. Delivered via the International Conference on Computer Vision (ICCV), it highlights Tesla's vast data resources and new simulation tools. The talk underscores efforts toward global robotaxi scaling and full fleet autonomy.
In the presentation, Elluswamy emphasized Tesla's unparalleled data advantage, stating that the company's vehicle fleet generates '500 years of driving data every single day.' He described this as a 'Niagara Falls of data,' comprising hundreds of years' worth of collective fleet driving, captured through smart data triggers for rare corner cases like complex intersections and unpredictable behavior. This approach allows Tesla to extract only essential data for efficient model training, addressing the 'curse of dimensionality' where eight high-frame-rate cameras produce billions of tokens per 30 seconds of driving context.
Despite the end-to-end neural network system, Tesla maintains interpretability for debugging. Engineers can prompt the model to output auxiliary predictions such as 3D occupancy, road boundaries, objects, signs, and traffic lights, which do not control the vehicle but aid safety checks. Natural language querying enables questions like why a certain decision was made. Additionally, Tesla developed a custom, ultra-fast Gaussian splatting system for reconstructing crisp 3D scenes from limited camera views, outperforming standard NeRF and splatting methods for visual debugging.
Evaluation remains a key challenge, with models excelling offline but struggling in real-world edge cases. To counter this, Tesla created a learned world simulator—a neural network-generated video engine that simulates eight Tesla camera feeds simultaneously in fully synthetic environments. It supports testing, training, reinforcement learning, adversarial event injection (e.g., a pedestrian cutting in), and replaying past failures, running near real-time for simulated drives.
Looking ahead, Elluswamy outlined plans to scale robotaxi services globally, achieve full autonomy across the Tesla fleet, and introduce the Cybercab—a next-generation two-seat vehicle optimized for robotaxi use, aiming for transportation costs lower than public transit. The same neural networks will power the Optimus humanoid robot, with the video generation system now applied to simulate and plan robot movements.