Pronouns: he/him
I am primarily interested in the rich interplay of language and perception in AI, both from a technical and a philosophical/cognitive angle. Recently, I am adopting methods from interpretability to study these phenomena (recent blog post on why I pivoted to interp). Other directions I enjoy working on: 1) Diagnostic benchmarking, 2) Interpretability of various complex LLM phenomena like reasoning, introspection, latent thinking.
▶ Expand to see examples of questions I currently think about (Summer 2026)
At the end of my undergrad, why did I choose to do a PhD? I was fascinated by language grounding (see: the original Symbol Grounding Problem), and specifically the position paper Experience Grounds Language had a strong influence.
Before coming to Mila, I graduated from LMU Munich in computational linguistics where I studied symbolic reasoning in Transformers and machine translation.
Aside from research, science communication/organization is fun and important: I currently organize the Mila Tea Talks (institution-wide academic talks), have organized the NewInML workshop at NeurIPS 2023 and ran the reading group on language grounding at Mila for three years. I gave a 3-minute summary of my research for a broader audience at Mila's speed science competition ( YouTube). And toogether with my friend Tomas Vergara Browne we talk to fellow scientists on our podcast Behind the Research of AI. In my undergrad, I also founded a philosophy society which is still thriving to this day. I strongly believe these kinds of activities next to publishing papers are crucial to a healthy and vibrant research community.
▶ Outside of research (expandable)
You can email me to chat about research or life in general: benno.krojer@gmail.com
Updates
Publications
Would you still call this Dax? Novel Visual References in VLMs and Humans
Ada Defne Tür, Gaurav Kamath, Joyce Chai, Siva Reddy,
Preprint | Demo | Code | Dataset
LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs
, Shravan Nayak, Oscar Mañas, Vaibhav Adlakha, Desmond Elliott*, Siva Reddy*, Marius Mosbach*
ICML'26 | Code/Library | Demo
A Shortcut-aware Video-QA Benchmark for Physical Understanding via Minimal Video Pairs
, Mojtaba Komeili, Candace Ross, Quentin Garrido, Koustuv Sinha, Nicolas Ballas, Mido Assran
TMLR'25 | Transactions on Machine Learning Research
Learning Action and Reasoning-Centric Image Editing from Videos and Simulations
, Dheeraj Vattikonda, Luis Lara, Varun Jampani, Eva Portelance, Christopher Pal, Siva Reddy
NeurIPS'24 Spotlight (Datasets & Benchmarks) | Conference on Neural Information Processing Systems
Are Diffusion Models Vision-And-Language Reasoners?
, Elinor Poole-Dayan, Vikram Voleti, Christopher Pal, Siva Reddy
NeurIPS'23 | Conference on Neural Information Processing Systems
Image Retrieval from Contextual Descriptions
, Vaibhav Adlakha, Vibhav Vineet, Yash Goyal, Edoardo Ponti, Siva Reddy
ACL'22 | Association for Computational Linguistics
Improving Automatic VQA Evaluation Using Large Language Models
Oscar Manas, , Aishwarya Agrawal
AAAI'23 | AAAI Conference on Artificial Intelligence
Pragmatic Inference with a CLIP Listener for Contrastive Captioning
Jiefu Ou, , Daniel Fried
ACL Findings'23 | Findings of the Association for Computational Linguistics
Are Pretrained Language Models Symbolic Reasoners Over Knowledge?
Nora Kassner*, *, Hinrich Schütze
CoNLL'20 | Conference on Computational Natural Language Learning
(* = Equal Contribution)
ContraCAT: Contrastive Coreference Analytical Templates for Machine Translation
Dario Stojanovski*, *, Denis Peskov*, Alexander Fraser
COLING'20 | Conference on Computational Linguistics
(* = Equal Contribution)
Blogposts
Behind the Scenes: LatentLens
Jul 22nd, 2026
An excerpt from the appendix of the LatentLens paper: the pivot into interpretability, the Mosaic detour, and why you should test your assumptions early.
Inspecting Visual and Audio Token Representations in Gemma4-12B
Jun 17th, 2026
Behind the Scenes: Five years of PhD
Jun 11th, 2026
An excerpt from the appendix of my PhD thesis where I summarize the PhD journey: applying, the slumps, the pivots, lessons learned, and the original statement of purpose.
List of papers with great writing, story, figures, ...
Apr 1st, 2026
Better late than never: Getting into interpretability in 2025
Dec 13th, 2025
Tips on what to do in Montreal for attendees of the COLM 2025 conference (written by Marius, me and Michael)
Sept 19th, 2025
Collection of good advice and blog posts, etc
Sept 19th, 2025
Behind the Scenes: MVP
Jun 11th, 2025
An excerpt from the appendix of the MVP paper: the frustration with video benchmarks that started it, and lessons from curating minimal video pairs at FAIR.
Behind the Scenes: AURORA
Oct 17th, 2024
An excerpt from the appendix of the AURORA paper — the first one I wrote such a section for: the discarded datasets, and what I learned about scoping a research project.
Things I like
Jan 2nd, 2024
Elevator Pitch & ELI5 of my first PhD project: Image Retrieval from Contextual Descriptions
May 30th, 2022
Explaining my research to someone outside the field.
What made me do a PhD far away?
April 28th, 2021
Sometimes people ask me why I left Europe when there is good opportunities there, too.
[DRAFT]: Defining the most basic concepts in Language Grounding
Jan 22nd, 2021
We rarely think about what the most basic words mean.
Explain like I am 5: Language Grounding
Nov 21st, 2020
How would I explain the research field of language grounding to a novice.
Attending my first conference! ACL 2020 from an undergrad's perspective
Jul 2nd, 2020
Networking, learning about language grounding and mentally prepping for the PhD.
Why the current Corona situation makes memories blurry
May 8th, 2020
Applying knowledge from reading about neuroscience
Moral Pluralism through the lense of optimization
Apr 29th, 2020
Some reflections after a philosophy society meeting