Photons In, Controls Out: What Tesla FSD Actually Learns, and Where Its Ethics Actually Live

A special-edition deep dive on Tesla Full Self-Driving as a case study in machine-learning systems ethics: how an end-to-end vision-only driving network actually learns, where ethical preferences live in a stack with no system prompt, and why a convergence of practitioners and researchers now argues that the Trolley Problem is the wrong question for the engineering reality. Sober comparative ground from Waymo, Mobileye, and Wayve keeps it from being a Tesla monologue.

In this episode

• Why a striking convergence of AV practitioners (Volvo), academic researchers (NC State, Frontiers in Robotics and AI), and policy analysts (Brookings) now argue the Trolley Problem is the wrong frame for AV ethics — and what they think the right one is.
• What "photons in, controls out" actually means inside FSD v14 — the imitation-learning foundation, the v14 release cadence (v14.1 through v14.3.3, the silent vision-encoder swap, the MLIR compiler rewrite), and the gap between user-visible changelogs and actual model evolution.
• The Dojo post-mortem: Tesla's pivot from custom D1 silicon to a conventional GPU cluster called Cortex, and what that does to the vertical-integration-wins-AI-compute thesis.
• The adversarial-robustness picture for vision-only perception — a 76% patch attack success rate, 73–91% cross-architecture transfer, the box-truck physical attack at NDSS, and why shared CLIP encoders defeat architectural diversity as a defense.
• What Waymo, Mobileye, Wayve, and Cruise have publicly disclosed about where safety constraints live in their stacks — and why Waymo's December 2025 Demonstrably Safe AI post is the clearest answer the field has produced.
• The regulator and courtroom tracks: NHTSA's Engineering Analysis EA26002 covering ~3.2M vehicles, the $243M Autopilot jury verdict, UNECE R157's extension to heavy vehicles and 130 km/h ceiling, and Germany's Ethik-Kommission auditability and non-discrimination constraints.
• The honest synthesis — ethics in an ML driving stack is distributed across training data, loss design, inference-time validators, operational scoping, and external accountability. The "system prompt" mental model is wrong because it asks for one location; the Trolley Problem is wrong because it asks for one decision.

Sources & References

Primary / originating sources

Zero-anchor theme — no operator-provided primary sources. The corpus plan's editorial commitments (Trolley Problem as wrong frame; "system prompt" as wrong mental model for an end-to-end driving net) functioned as the framing artifact, evidenced by the sources below.

Research & critique — ML methodology and AV safety architecture

• arXiv 2503.05997 — Imitation Learning vs. Reinforcement Learning in AV Policy Training — peer-reviewed preprint identifying the structural ceiling of imitation learning and proposing a three-stage IL→surrounding-vehicle-IL→RL pipeline.
• Waymo Research — "Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning" — 38% reduction in hard-scenario failures from hybrid IL+RL.
• Waymo Blog — "Demonstrably Safe AI for Autonomous Driving" (December 2025) — the clearest public account of how a learned policy and hard inference-time validators coexist in production.
• NeurIPS 2025 — RAD: Closed-Loop RL in 3D Gaussian Splatting Simulation — ~3× lower collision rate vs. IL baselines.
• ACM Computing Surveys — "RLHF Deciphered" — failure modes (reward misspecification, annotator bias, reward hacking) and their implications outside language-model settings.
• Mobileye — Responsibility-Sensitive Safety (RSS) — explicit kinematic-inequality safety framework.
• Wayve — "A Global Regulatory Breakthrough for Assisted and Automated Driving" — training-objective and ODD-led safety architecture.
• ACM Digital Library — End-to-End AV Architecture Survey (2025) — characterizes pure IL as no longer frontier for robustness.
• arXiv 2603.16050 — Large Driving Model Survey — classifies FSD v12–v14 alongside Wayve and NVIDIA COSMOS, notes internal modularity.

Ethics — the Trolley Problem reframe

• Frontiers in Robotics and AI — AV Ethics Expert Survey (2025) — peer-reviewed evidence that practitioners emphasize rule-following and community-wide risk reduction over utilitarian calculus.
• Volvo Autonomous Solutions — "The Misguided Dilemma of the Trolley Problem" (January 2024) — practitioner rebuttal.
• NC State — "Ditching the Trolley Problem" (December 2023) — methodological critique of trolley-style AV ethics research.
• Brookings Institution — "The Folly of Trolleys" — policy-level reframe.
• Harvard Kennedy School — "It Is Time to Change the Autonomous Vehicles Regulatory Approach" (October 2024) — comparative regulatory analysis.

Security — adversarial robustness of camera-only perception

• SAE 2026-01-0170 — "Understanding Adversarial Transferability in Vision-Language Models for Autonomous Driving" — 76% patch success rate; 73–91% cross-architecture transfer; CLIP-encoder correlated risk.
• IEEE Transactions on Intelligent Vehicles — T. Wei et al., Survey of Adversarial Attacks on Camera-Only AV Perception (2024) — full-stack vulnerability across BEVDet/BEVDet4D, sign recognition, vehicle detection, lane detection.
• NDSS Symposium — Physical Adversarial Patch on Box-Truck Rear Panel — fully physical attack against camera-based obstacle detection.
• arXiv 2603.08897 — Adversarial Perception → Unsafe Driving Decisions — closes the loop from corrupted perception to unsafe vehicle behavior; documents temporal amplification.
• U.S. DOT / BTS ROSAP Technical Report — Adversarial Robustness Gaps in AV Perception — government-level acknowledgment of the gap.

Industry & implementation — Tesla FSD and Dojo

• TechCrunch — "Tesla Dojo: The Rise and Fall of Elon Musk's AI Supercomputer" (September 2025) — Cortex GPU cluster takes over from Dojo.
• Electrek — FSD v14.1 Release Notes (October 2025) — start of the v14 family.
• Not A Tesla App — FSD v14.3.2 Release Notes — RL training upgrades, MLIR compiler rewrite.
• Not A Tesla App — FSD v14.3.3 Release Notes — intervention-reason logging interface.
• Tesla Oracle — FSD v14.2.2.3 — silent vision-encoder upgrade shipped with identical user-visible release notes.

Regulation & enforcement

• Electrek — NHTSA Upgrades Tesla FSD Visibility Investigation, 3.2M Vehicles (March 2026) — EA26002.
• NHTSA — Standing General Order on Crash Reporting — under-reporting exposure.
• Senator Markey — FTC Investigation Request on Tesla Marketing — congressional ask.
• Automotive World — $243M Tesla Autopilot Jury Verdict Context — civil-liability benchmark.
• UNECE — UN Regulation 157 Extended to Trucks — heavy-vehicle extension and 130 km/h ceiling.
• UNECE R157 Full Text (December 2023) — binding international standard.
• Connected Automated Driving EU — German Ethik-Kommission Overview — non-discrimination and auditability soft-law constraints.

Have questions about this episode? Reach out.