Computer-Vision-Informed Visual Explanation Cards for Autonomous-Driving Traffic-Sign Alerts: Localization, Classification, and Retrieved Evidence on GTSDB

Abstract

This paper develops a computer-vision-informed visual explanation card for traffic-sign alerts in autonomous-driving and driver-assistance interfaces. The card organizes a detected sign crop, predicted class, confidence, default semantic display-priority tier, retrieved visual precedents, scene-location cue, and concise action prompt. The empirical study uses the complete German Traffic Sign Detection Benchmark (GTSDB), comprising 900 road scenes in the standard 600-scene development and 300-scene evaluation portions. The same scenes support localization, crop classification, retrieval, calibration, and end-to-end analysis. The first 600 scenes were divided at the scene level into training and validation subsets; the 300 evaluation scenes were held out until model choices, retrieval depth, fusion weight, and detector threshold had been fixed. A learned class-agnostic localizer filters color-connected-component proposals with a histogram-of-oriented-gradients and color classifier. Five crop classifiers and four nearest-neighbor settings were evaluated, with retrieval treated primarily as example-based explanation support. On the held-out scenes, the selected localizer achieved an AP at IoU 0.50 of 0.211, an AP averaged over IoU 0.50–0.95 of 0.098, and a recall of 0.260 at the validation-selected operating point. The selected crop classifier achieved 0.784 accuracy and 0.574 macro-F1 on ground-truth crops. With predicted crops, correct-class end-to-end coverage was 0.177, and correct-tier end-to-end coverage was 0.244. These results define the information that the proposed card can receive from the evaluated vision pipeline. They do not measure driver comprehension, glance behavior, response time, trust, usability, or deployment safety, which require separate human-centred evaluation.