What a vision transformer learns, layer by layer
Research into how efficient vision transformers organise the concepts they learn. It builds on the Visual Concept Connectome (CVPR 2024) and takes it to the SHViT family.
- Concept discovery and inter-layer connections for SHViT.
- Double- and triple-head variants that share the early stages of the network.
- Evaluation on ImageNet-1K and ImageNet-R, trained on an HPC cluster (Slurm).
- PyTorch
- SHViT
- UMAP
- Slurm



