Publications

In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Heads
Charlotte Pouw, Hosein Mohebbi, Afra Alishahi, Willem Zuidema
preprint 2026
paper

Tracking the emergence of linguistic structure in self-supervised models learning from speech
Marianne de Heer Kloots, Martijn Bentum, Hosein Mohebbi, Charlotte Pouw, Gaofei Shen, Willem Zuidema
preprint 2026
paper

Gender Disambiguation in Machine Translation: Diagnostic Evaluation in Decoder-Only Architectures
Chiara Manna, Hosein Mohebbi, Afra Alishahi, Frédéric Blain, Eva Vanmassenhove
In Proceedings of LREC 2026
paper

Disentangling Textual and Acoustic Features of Neural Speech Representations
Hosein Mohebbi, Grzegorz Chrupała, Willem Zuidema, Afra Alishahi, Ivan Titov
preprint 2024
paper | code | video

What Do Self-supervised Speech Models Know about Dutch? Analyzing Advantages of Language-specific Pretraining
Marianne de Heer Kloots, Hosein Mohebbi, Charlotte Pouw, Gaofei Shen, Willem Zuidema, Martijn Bentum
In Proceedings of Interspeech 2025
paper | model checkpoints

On the Reliability of Feature Attribution Methods for Speech Classification
Gaofei Shen, Hosein Mohebbi, Arianna Bisazza, Afra Alishahi, Grzegorz Chrupała
In Proceedings of Interspeech 2025
paper

How Language Models Prioritize Contextual Grammatical Cues?
Hamidreza Amirzadeh, Afra Alishahi, Hosein Mohebbi
In Proceedings of BlackboxNLP 2024
paper

DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers
Anna Langedijk, Hosein Mohebbi, Gabriele Sarti, Willem Zuidema, Jaap Jumelet
In Findings of NAACL 2024
paper

Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers
Hosein Mohebbi, Grzegorz Chrupała, Willem Zuidema, Afra Alishahi
🏅 Outstanding Paper Award
In Proceedings of EMNLP 2023
paper | data | code

Quantifying Context Mixing in Transformers
Hosein Mohebbi, Willem Zuidema, Grzegorz Chrupała, Afra Alishahi
In Proceedings of EACL 2023
paper | code | blog | demo

AdapLeR: Speeding up Inference by Adaptive Length Reduction
Ali Modarressi, Hosein Mohebbi, Mohammad Taher Pilehvar
In Proceedings of ACL 2022
paper | code | blog

Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations
Mohsen Fayyaz, Ehsan Aghazadeh, Ali Modarressi, Hosein Mohebbi, Mohammad Taher Pilehvar
In Proceedings of BlackboxNLP 2021
paper

Exploring the Role of BERT Token Representations to Explain Sentence Probing Results
Hosein Mohebbi, Ali Modarressi, Mohammad Taher Pilehvar
In Proceedings of EMNLP 2021
paper | code | blog