The Underwhelming Reality of FSD

As a proud owner of a 2024 Tesla Model 3, I acknowledge my bias. Despite this, I have yet to activate the FSD mode, as it is both costly and underwhelming. Why is that? The autonomy level is not as advanced as other manufacturers.

Currently, Tesla's FSD operates at SAE Level 2, while competitors like Mercedes have already introduced Level 3 vehicles. The Society of Automotive Engineers categorizes autonomy from Level 0 (no automation) to Level 5 (full automation). Level 2 means the vehicle cannot independently monitor its environment.

Consequently, despite a recent price reduction, FSD remains unpopular, mirroring the general skepticism surrounding autonomous vehicles. Data from AAA indicates only 9% of Americans trust these systems. Adding to the controversy, Tesla's reliance on camera-based technology instead of LiDAR has sparked debate, with many experts believing LiDAR is superior for navigating complex scenarios.

Tesla's FSD relies on detecting objects and predicting their movements, while other companies are exploring occupancy-based systems that assess potential collisions. There is no consensus on which approach is superior.

Tesla's FSD currently faces challenges due to overhyping, low trust levels, and inadequacy in handling complex situations.

Grok 1.5 Vision: A Potential Game Changer

For those unfamiliar, Elon Musk also heads an AI company called xAI, which recently unveiled Grok 1.5 Vision. This MLLM claims to compete with leading models like GPT-4, Claude 3, and Gemini in various benchmarks. However, it’s important to remain cautious as these results have yet to be validated in real-world applications.

So, what exactly is an MLLM? Simply put, it refers to Large Language Models that can also process various data forms, such as images and audio.

This capability is crucial for enhancing Tesla's FSD, particularly in handling unexpected, complex scenarios. Traditional camera-based FSD systems struggle to deal with edge cases—situations that the model hasn't encountered during training, where it may fail.

Consider a scenario where a Tesla approaches a stop sign. The system must decide to stop, but edge cases require a different kind of thinking.

Driving Dynamics: System 1 vs. System 2

When driving normally, actions like shifting gears or braking are instinctive, a phenomenon described as "System 1 thinking" by psychologist Daniel Kahneman. In contrast, unfamiliar or complicated situations necessitate "System 2 thinking," where planning and reasoning come into play.

Current FSD systems excel at System 1 tasks but falter in System 2 scenarios. However, if MLLMs could enhance the reasoning capabilities of vehicles, it could transform their decision-making.

A Practical Example: Wayve's Lingo Models

UK-based Wayve is already exploring this idea with their Lingo model, which enables self-driving cars to explain their decisions.

Unlike its predecessor, Lingo-2 can actually drive while commenting on its actions, functioning as a vision-language-action model. It integrates video input with text to enhance decision-making.

This model uses a spatiotemporal Transformer to analyze both time and space, while also employing an adapter to convert video information into text.

The potential for MLLMs in FSD systems is vast. These models can follow verbal commands and could enhance decision-making in complex situations that are often unforeseen.

The Synergy Between Tesla and xAI

Tesla possesses a wealth of real-world driving data, which could significantly boost Grok's performance. This symbiotic relationship could elevate Tesla's FSD capabilities while furthering xAI's goals.

As Elon Musk aims to develop Artificial General Intelligence (AGI), real-world video data is increasingly recognized as a vital resource for achieving this goal, a sentiment echoed by leading researchers.

Addressing the Challenges Ahead

While the concept of talking cars may seem futuristic, several hurdles must be overcome before MLLM-enhanced FSD becomes a reality:

1. Current Limitations: MLLMs have not yet achieved effective reasoning and planning, an area under intense research.
2. Methodology Debate: The effectiveness of camera-based systems versus LiDAR is still contested within the industry.
3. Latency Issues: FSD systems require rapid decision-making, and while Grok shows promise, latency remains a critical factor.
4. Model Size: The considerable size of Grok may necessitate cloud-based solutions, which could hinder real-time performance.

In conclusion, combining FSD with MLLMs could enhance reasoning in complex cases and add an element of explainability, potentially boosting public trust in autonomous systems. The collaboration between Tesla and xAI represents a crucial bet for the future of both entities, as they strive to navigate the evolving landscape of autonomous technology.

If you found this exploration insightful, I share similar thoughts in a more accessible format on my LinkedIn. Feel free to connect with me on X as well.

Thank you for engaging with the In Plain English community! Don't forget to follow us on our various platforms for more content!