Tesla Releases FSD v12.4: New Vision Attention Monitoring, Improved Strike System With Update 2024.9.5 (2024)

Tesla has just rolled out its latest FSD software, v12.4 to employees. Elon Musk announced that this update would be available to employees this past weekend, with plans to release it to the public in small numbers later this week.

Surprisingly, the new update is version 2024.9.5, which is likely based on the earlier 2024.8 branch and not Tesla's latest 2024.14. The spring update (2024.14) brings various new features such as a new media player, a new parked visualization, Audible support, and a Preview of Sentry Mode events, among others.

However, FSD v12.4 brings itsownexcitement with two new major changes.

Vision-Based Attention Monitoring

The release notes show a new Vision-Based Attention Monitoring feature that replaces the steering wheel nag as Musk previously mentioned.

However, as we predicted, Tesla will still leverage the steering wheel to detect attentiveness when the cabin camera is inconclusive.

The car can only rely on the vehicle's cabin camera, and therefore remove the steering wheel nag under certain conditions:

• the camera is not occluded

• there is sufficient lighting

• the driver is looking forward

• the driver is not wearing sunglasses

• the driver is not wearing a low-brim hat or another object that covers their eyes

If any of these situations occur, or if the vehicle doesn't have a cabin camera, then the vehicle will continue to use the steering wheel to determine driver attention.

Tesla is careful to state that images and video from the cabin camera are not saved or transmitted unless you enable data sharing.

Updated Strike System

With FSD v12.4, Tesla has also updated its Autopilot Suspension feature which is designed to enforce the responsible use of FSD.

The current system lets the driver receive up to five strikes (three strikes for vehicles without a cabin camera)before Autopilot and FSD become unavailable. If that happens, then FSD is unavailable for one week. Strikes are only removed once the driver has accrued five strikes, or when Tesla wipes out strikes for everyone, which happens about twice a year.

The new system is more gracious about removing strikes. The vehicle will continue to issue strikes whenever the driver isn't paying attention, however, now the vehicle will gradually remove strikes for the driver after a certainperiod of time.

Tesla states that one strike will be removed foreach 7-day periodthe driver goes without receiving a strike. So if FSD gets disabled due to strikes, the driver will still go one week without FSD, although now strikes are removed on an ongoing basis. This new strike system is expected to apply tovehicleswith and without a cabin camera.

Other New Features

Other new features areexpected in FSD 12.4 as well, which we outlined in our look at Tesla's FSD v12.4 article.

They include a focus on improved driver comfort by reducing the amount of hard braking, automatically seeking a parking spot when arriving at a destinationandmore. Driver interventions are also expected to be drastically reduced with Musk stating that Tesla expects to see a 5-10x improvementin interventions.

Eligibility

Unfortunately, due to FSD v12.4 being on branch 2024.9, it's expected to only be available to owners on update 2024.8 and below, which includes everyone currently on update 2024.3.25.

If no major issues are found with FSD v12.4, we could see it start to roll out to the public later this week.

Tesla’s Autowipers have been a long-standing issue for customers, as well as Tesla’s engineers. Elon Musk has often mentioned that improvements to Autowipers have been around the corner for years, but they appear to have taken a rain check.

Tesla owners often complain that their wipers are running against a completely dry windshield on a sunny day, or not running at all in light rain. Moderate to heavy rain performance seems to be improved for the most recent update to Auto Wipers.

In our test, we cover the vehicle's windshield with Post-It notes to find out what the vehicle’s cameras can see.

The Water Problem

Tesla vehicles lack a rain sensor. This is a result of Tesla’s strategy to rely upon Tesla Vision to replace all sensors – which they’ve successfully used most recently to replace the ultrasonic parking sensors (USS) on their vehicles. Tesla believes that vision is the solution to achieving all self-driving capabilities, and this includes the elimination of extraneous sensors such as USS and radar.

Of course, many other vehicles employ simple rain sensors that do function fairly well, and effectively detect the presence of water on the windshield. So, what’s stopping Tesla Vision from doing the same thing?

The problem isn’t Tesla’s detection methods, it’s the view of the windshield Tesla has. Water droplets are small and the three front cameras only capture a small portion of the entire windshield, making it difficult for them to capture and acknowledge the fact that there is water on the windshield that needs to be wiped away. This is in addition to the fact that the cameras are focused further away, to drive and navigate the vehicle effectively, rendering the visuals close to the camera blurry – leaving a rather technical challenge.

Our Test

Even Tesla’s front-facing, wide-angle camera can barely see the yellow Post-In notes in our test, let alone the blue Post-It notes spread through the windshield. Tesla’s other front-facing camera, the “main” camera, see only see a trace of the Post-Its along the bottom. In reality, Tesla has no idea what’s on the vehicle's windshield and can only make assumptions based on a few square inches near the top of the windshield.

Tesla’s third front-facing camera, the “narrow” view, is only available on hardware 3 and it’s just a zoomed-in version of the main camera, which causes it to miss our Post-It notes completely. With hardware 4, Tesla removed the narrow camera and now determines the narrow view by cropping the main camera, which has a higher resolution in hardware 4.

Tesla’s Engineering Magic

Tesla is able to detect drops on the windshield when they land in front of the camera cluster, but again, since the cluster is mounted high above most of the windshield, this can lead to the windshield filling up with water droplets before a wipe is triggered.

They’ve also tried using the vehicle’s other cameras, such as the repeater or B-pillar cameras to detect falling rain – which can help the Autowiper system build confidence that there is water on the windshield. Once it has enough confidence, it triggers a wipe – or changes the wipe modes. However, in reality, it has very little knowledge of what’s actually on the windshield.

Wiping Away Challenges

The transition to entirely vision has left a considerable challenge for Tesla’s very capable engineering teams. The current Autopilot hardware suite lacks a good view of the windshield itself, but it can detect falling rain. Given that Tesla’s software can detect and display other vehicles using turn signals, it is also possible that they could detect other vehicles using wipers or use other cameras to see rain on the vehicle, such as the B-pillar cameras, to increase the system’s confidence that a wipe is required.

However, these solutions are likely to not be perfect, as they don’t provide an accurate representation of what the driver sees in the windshield, but instead an accurate representation of what the vehicle itself can see.

When engaging FSD on a lightly rainy day, your car may drive perfectly, but you’ll have a windshield full of rain when supervising!

Of course, there are many other reasons to run the windshield wipers beyond simple rain – spray and debris from vehicles in front, water falling from trees or being kicked up by deep puddles, or even snow settling on the windshield. Another challenge is bugs – they leave sticky smears at higher speeds when impacting the windshield, causing spots that can impede either vehicle vision or driver vision.

Future Solutions

Tesla likes to avoid solutions that only solve a single problem – such as adding a rain sensor. It is an additional manufacturing complication, adds additional cost to vehicles, and segments Tesla’s vehicles between model years.

On the other hand, cameras have solved many problems over the years – such as Tesla Vision taking over from the USS system that was present on pre-2022 Tesla vehicles. While it did take until FSD V12 for Autopark to become available to vision-only vehicles, the upgraded Vision-based Autopark is a marked improvement over the older USS solution.

Another such problem has been auto-high beams, which were solved with better oncoming vehicle detection and improved AI – which has led to a feature that is much better than its original implementation. This updated implementation has also led to improvements in the use of Matrix headlights – a feature that for now is only available in Europe and China due to legislative issues in the US.

Tesla is most likely going to be pushing another hardware revision – Hardware 5 – in the coming years. Some of the indications of this include the Cybertruck’s bumper camera – which is also expected to arrive in the Model X and Model S in the future.

There is a chance that Tesla may include a forward-facing interior at some point – to solve the fact that Vision is unable to currently see the windshield. However, Tesla also strongly believes they’ll achieve level 4 or 5 of vehicle autonomy in the future, so they could also be banking on autonomy removing the need for good auto wipers.

A Personal Example

If you get into your Tesla on a rainy day or add some drizzle via a watering jug, you can see how Tesla’s Autowipers will not activate unless water is placed directly in front of the camera cluster. You can also spray some water onto the camera housing and take a seat inside and use the camera preview to view what the cameras can see. It’ll provide you with a blurry, limited view of what the cameras see.

We may see further software improvements, but it may be a difficult engineering problem to solve with the limited camera capabilities that currently exist on hardware 3 and hardware 4 vehicles.

Tesla recently fired the entire Supercharger team, including Tesla’s head of charging – Rebecca Tinucci, after she pushed back on the extreme layoffs that took place right before the cut.

The Supercharger team consisted of over 500 employees, at least after the initial layoffs. In the following days and weeks, Tesla began to rehire some of the employees that it had fired.

Some Damage Done

In the immediate aftermath of the firing of the Supercharger team, contractors and site planners were left bewildered, with no contact from the Supercharger team that was responsible for payment, planning, and decision-making.

As this has played out, new Supercharger deployments have been reduced – stations that were already being built are being completed, but no new announcements have been made since t

It was dire news at the time - but it isn’t all bad.

Returning Employees

Now, more and more of the employees that were fired are beginning to return to Tesla, some of whom are announcing that they were asked to return to Tesla in their previous capacities.

George Bahadue, Senior Manager of Site Acquisition and Business Development commented on LinkedIn:

“Two weeks ago, I was asked to return to Tesla in my previous capacity heading up the business development and site acquisition for Tesla charging – I accepted.”

His reasoning to accept the position was a quote from Rebecca Tinucci:

“You work at Tesla because you hope to have at least a small impact on our collective future – aspirationally, to leave the world better for our children and grandchildren and their children and grandchildren – by accelerating the transition to sustainable energy. And that mission is too important to allow any distractions.”

New Stations Could be Coming Soon

With the restaffing of the Supercharger team, especially with the return of George Bahadue, we can expect that new Supercharger sites may be announced in the coming weeks, as the ripple effect from the layoffs begins to settle.

The rehiring of experienced staff suggests that Tesla and Elon Musk are still committed to the vision of maintaining and expanding its Supercharger network – the largest and most reliable charging network in North America, which is crucial for the mission to move the world to renewable energy.