MiClip: Learning to Interpret Representation in Vision Models

With pretrained CLIP models, MiClip avoids relying on manual heuristics and directly maps the internal representations of vision models into the human-understandable visual-language semantic space of CLIP.

Specifically, we project the input image $x_i \in \mathcal{X}$ and the model's predicted label $\hat{c}_i$ with CLIP's vision encoder $\mathrm{E}_{\text{i}}(\cdot)$ and text encoder $\mathrm{E}_{\text{c}} (\cdot)$ into the embedding space. We then learn a neuron encoder $\mathrm{E}_{\text{n}}(\cdot; \theta_{\text{n}})$ that maps the model's hidden representations $\mathbf{a}_i \in \mathcal{A}$ to the same space. Finally, we optimize the symmetric InfoNCE loss to align the model's internal space with the visual-language semantic space.

$$ \mathcal{L}_{\mathrm{alignment}} = \underbrace{\mathcal{L}_{\mathrm{CLIP}}^{\text{out}} \left(\mathrm{E_n}(\mathcal{A}; \theta_{\text{n}}), \mathrm{E_c}(\{\hat{c}_i\}_{i=1}^N) \right)}_{\text{neuron-concept loss}} + \underbrace{\mathcal{L}_{\mathrm{CLIP}}^{\text{in}} \left(\mathrm{E_n}(\mathcal{A}; \theta_{\text{n}}), \mathrm{E_i}(\mathcal{X}) \right)}_{\text{neuron-image loss}} $$

Once trained, MiClip enables both Concept-to-Mechanism Localization with open-vocabulary concepts, and Mechanism-to-Concept Description for either activation neurons or learned Sparse Autoencoder (SAE) features. This is achieved by comparing the cosine similarities $\operatorname{sim}(\cdot,\cdot)$ between embeddings.

For a given concept $c$, we collect its corresponding units indexed by $\mathbf{L}_c$ (neurons or SAE features), and adjust their activations to suppress or amplify the concept’s influence on the model.

For each representation unit $u$ ($u=a_i \cdot e^{(i)}$ or $u=f_i$) indexed within $\mathbf{L}_{c}$, we either apply a scalar multiplication or add an additive bias as

$$ \tilde{u}_i = \beta u_i \text{ (Scaling)} \quad \text{or} \quad \tilde{u}_i = u_i + \beta\text{ (Adding)}, \quad \forall i \in \mathbf{L}_{c}, \beta \in \mathbb{R}. \quad $$

By applying a distinct parameter $\beta$, we can suppress or amplify the target feature to adjust the model.

We verify our localization by intervening on the top-5 neurons or features for each ImageNet-1k concept. We measure the change in classification accuracy $\Delta Acc$ after applying either enhancement ($\times 2$ scaling) or removal ($\times 0$ scaling) after interventions on the activations of the following layers: the 10th layer for ViT-B/16 and CLIP/ViT-B-16, and the stages.3.layers.1.shortcut layer for ResNet-50.

Results show that MiClip enables precise localization of the mechanisms that govern model classification. Baselines, like Act-Values, even exhibit an inconsistent responses, as shown in red.

We investigate the spatial grounding of the learned features by examining their activations within the model's attention maps. To show the generalization of MiClip, we test both high-level concept like "kit fox" and low-level concepts like colors and textures.

Experimental results highlight that MiClip can identify feature consistently activates around the ears of the "kit fox", a key visual identifier for this class. Also, the visualization for concept "green" confirms that our method can ground foundational visual concepts, like specific colors, to their corresponding spatial locations within an image.

We leverage MiClip to diagnose failures in visual reasoning in the CLIP/ViT-B-16 model by tracing the semantic trajectory of an image's internal representations across layers.