GPT-4 Vision for self-driving:
-Excels: scene understanding, causal reasoning
-Potential: out-of-distribution scenarios, recognizing intentions, making informed decisions
-Struggles: direction discernment, traffic light recognition, vision grounding, and spatial reasoning tasks
Cool paper applying GPT-4V to driving.
Obviously, this wasn't built to drive, and LLMs are very different than the AIs used in self-driving. It fails, but also does some interesting things that other AIs can't do, like "infer" motives & handle edge cases. arxiv.org/pdf/2311.05332…




