返回首页
arXiv AI··论文与技术

Metonymic Circuits for Abstract Concept Grounding in Vision Transformers

中文摘要

研究表明视觉Transformer利用具体概念(如火)作为锚点,通过转喻机制将视觉信号映射为抽象概念(如愤怒),实现抽象概念的语义关联。

English Summary

Vision Transformers ground abstract concepts by using concrete anchor concepts as metonymic bridges, enabling models to connect visual signals to complex semantics even with limited training data.

原文节选

arXiv:2610.06928v1 Announce Type: new Abstract: We study how Vision Transformers ground abstract concepts (e.g., angry) when training data provide limited direct referential evidence. We hypothesize a metonymic grounding mechanism in which abstract predictions are driven by concrete, interpretable anchor concepts (e.g., fire) that bridge visual signals to abstract semantics. By applying Transcoders on CLIP and DINO vision encoders, we recover intermediate features that can be associated with semantic labels for more concrete concepts, and trace their contributions in circuits underlying abstract concept recognition. Experiments on a carefully curated icon dataset reveal structured metonymic circuits, in which perceptual primitives dominate early layers and object-like anchors precede abstract targets. Images containing rendered text instead recruit a distinct perceptual-to-textual route. Causal interventions further validate that metonymic intermediates are functionally involved in grounding abstract concepts.