Ashok Elluswamy who is the Director of Autopilot Software at Tesla presented Full Self-Driving (FSD) session on March 2, 2023.

• What can FSD do for sustainable future?

  • Cars are rarely used just sitting idly in parking lots now.
  • But with complete FSD, these cars would be used by other customers.

• Three main parts to build FSD

  • AI
  • Data
  • Compute

1. AI

• Approach

  • Past: Single Camera, Single Frame Neural Networks

    • brittle, not leading to great success

  • Recent: Multi Camera, Video Neural Networks

    • from 8 cameras
    • produce a single unified 3D output space

• Modeling Component
  Some tasks such as Lane connectivity are complicated to model using native vision methods. So Tesla use techniques in other areas such as language modeling and reinforcement learning like LLM.
  

  • Vision
    • RegNet: https://arxiv.org/pdf/2003.13678
    • BiFPN: https://arxiv.org/pdf/1911.09070
    • Transformer
    • Video Module
  • Map
  • Language

• Planning

---

2. Data

• Label data to train

  • Data from human labeling would be too tiny.
  • Automated labeling is necessary.
  • 3D reconstruction by collecting various clips from different cars in the fleet and assembling them into a single unified representation.
    
  • Base construction → Various challenging simulations for the corner cases

• Example of why data is critical
  

  • precautiously breaking because of the parked car in image
  • after adding similar case data, the model understands that car is parked.

• Data Engine
  

  • Triage Shadow Testing: good post

---

3. Compute

---

📙 참조

• Documents and Events | Tesla
• 2023 Investor Day Presentation | Tesla
• 2023 Investor Day | Tesla