Tesla Introduces Audio and Vision Capabilities in FSD V13.4

With the rollout of FSD V13.2.1 to HW4/AI4 vehicle owners this week, we are thrilled to witness the debut of exciting new features such as Park, Unpark, and Reverse in action for the first time.

But that's not all—there's more on the horizon. We previously mentioned that Tesla is gathering audio input to enhance its neural networks, and we’ve now learned that this functionality will be introduced in FSD V13.4.

Better Audio Handling

Ashok Elluswamy, Tesla's Vice President of AI, stated that enhanced audio input processing will be introduced in FSD V13.4. This is an exciting development, especially considering that the current management of emergency vehicles in version 13.2.1 is already quite effective.

We are confident that the ability to detect emergency vehicles through sound will enhance both the speed and accuracy of detection. Just like with visual inputs, the Full Self-Driving (FSD) system will begin to analyze all audible sounds and identify indicators of emergency vehicles.

FSD will effectively assess whether a siren is approaching or merely reflecting off nearby surfaces by utilizing the Doppler effect. This principle states that the frequency of a sound wave increases as the source moves closer to the observer and decreases as it moves away.

Interestingly, Tesla plans to utilize the internal microphone for this purpose, as there are currently no external microphones installed in any Tesla vehicles. This internal microphone is adequate because sirens are loud enough for occupants to hear them even inside a moving car.

Better Than a Human

Some users have questioned how the vehicle will differentiate between sirens heard on the radio and those in real life. While many of us have likely been misled by a siren playing on the radio, Teslas are designed to be more discerning.

Tesla can filter out the audio being played through the radio from the sounds picked up by the internal microphone, effectively eliminating sirens from the recorded audio. Moreover, by measuring the intensity and direction of the sound, the vehicle should be able to accurately identify emergency vehicles even before a human can.

Opt-In Audio Sharing

Tesla has introduced an option for Full Self-Driving (FSD) users to voluntarily share audio data. This feature is available in FSD versions 12.5.6.4, 13.2, and 13.2.1, and it is anticipated to be included in the forthcoming Hardware 3 version, FSD 12.6.

It's important to highlight that the release notes for Tesla between versions V13.2 and V13.2.1 underwent a slight modification regarding audio sharing. Initially, Tesla indicated that the vehicle would record 10-second audio clips upon detecting a siren.

In FSD V13.2.1, Tesla has revised the data sharing feature, informing users that audio recordings will now be captured when the vehicle identifies an emergency vehicle, rather than solely detecting a siren. Additionally, the duration of the audio clips is no longer restricted to 10 seconds.

By opting into audio sharing, users will allow the collection of microphone recordings along with the other data that Tesla routinely gathers for its FSD training. However, if you prefer not to participate, you can choose to opt out of the audio sharing specifically. Tesla’s privacy policy also states that the data collected will be anonymized and sanitized during both collection and processing.

While vision plays a much larger role, expect Tesla to deal with the capturing and analyzing of audio data in a very similar manner.

We’ve already seen improved handling for school buses on V13.2, so we’re excited to see what else Tesla does in the next few months. Perhaps handling school zones would be the next big item to tackle.

🎄Make your Christmas shopping even merrier! 🎅 Enter code XMAS at checkout to unlock your exclusive discount! 🎁

https://arcoche.com/collections/for-new-model-3-highland

https://arcoche.com/collections/tesla-model-y-aftermarket-accessories

https://arcoche.com/collections/for-cybertruck

(if any problems, pls do not hesitate to contact us via "arcochehsop@gmail.com")