Analysis and visualisation of linguistic structures in neural language models: neural representations of verb-particle constructions in BERT

KISSANE H, Schilling A, Krauß P (2026)

Publication Type: Journal article

Publication year: 2026

Journal

Language, Cognition and Neuroscience Psychology Press

DOI: 10.1080/23273798.2026.2648588

Abstract

This study examines how transformer-based neural language models, particularly BERT and a construction-aware variant (CxG-BERT), represent verb–particle constructions (e.g., “agree on,” “come back,” “give up”) across layers. Using data from the British National Corpus, we extract model activations and analyse them via multidimensional scaling (MDS), generalized discrimination value (GDV), and permutation tests. Results show that BERT's middle layers best capture syntactic structure, with notable variation across verb types. These findings challenge assumptions of uniform linguistic processing in neural networks, revealing a more complex interaction between architecture and linguistic representation. Overall, the study advances understanding of how neural models encode language and highlights both their strengths and limitations for linguistic analysis, while suggesting directions for optimizing architectures in theory-driven research.

Authors with CRIS profile

Hassane KISSANE Chair of English Linguistics Patrick Krauß Lehrstuhl für Informatik 5 (Mustererkennung)

Involved external institutions

Universitätsklinikum Mannheim / University Medical Centre Mannheim (Universitätsmedizin Mannheim) DE Germany (DE)

How to cite

APA:

KISSANE, H., Schilling, A., & Krauß, P. (2026). Analysis and visualisation of linguistic structures in neural language models: neural representations of verb-particle constructions in BERT. Language, Cognition and Neuroscience. https://doi.org/10.1080/23273798.2026.2648588

MLA:

KISSANE, Hassane, Achim Schilling, and Patrick Krauß. "Analysis and visualisation of linguistic structures in neural language models: neural representations of verb-particle constructions in BERT." Language, Cognition and Neuroscience (2026).

BibTeX: Download