Résumés des masterclasses et des ateliers

Guillaume Desagulier

Masteclass: An introduction to distributional semantics and its application to cognitive linguistics

Distributional semantic models (henceforth DSMs) are computational implementations of the distributional hypothesis: words that occur in similar contexts tend to have similar meanings (Harris 1954; Firth 1957; Miller and Charles 1991). Initially developed in the field of cognitive psychology to model memory acquisition (Landauer and Dumais 1997; Lund and Burgess 1996), DSMs have been used extensively in NLP in the wake of Turney and Pantel (2010).

DSM is a cover term for a great number of methodologically related yet distinct approaches. Three features influence shape the kind of distributional semantic modeling that you do:

• context types: document-based (e.g., LSA) vs. word-based
• level of analysis: type (e.g., word2vec) vs. token (BERT)
• computational representation: explicit vectors (e.g., count-based models) vs. implicit vectors (e.g., predictive models).

In any case, DSMs embrace the ‘Bag-of-Words’ approach. On the one hand, semantic modeling works well at the lexical level (Hamilton et al. 2016) but not so much at more complex levels. We shall see why briefly, and I will propose a workaround based on previous works. The approach I propose taps into the computational force of NLP and the methodological intuitions of corpus linguistics. It combines the assets of collocational analysis and DSMs.

Another related issue has to do with diachronic linguistics. In quite general terms, diachronic linguistics is the study of language change. The methodological implications are not that simple because doing diachronic linguistics depends on your theory of language and on what the linguist considers is a relevant linguistic unit for the study of change (morphemes? lexemes? syntactic patterns? etc.)

I will address these issues via a combination of theoretical reflections and practice (i.e. with R, drawing upon Susanne Flach’s seminar). After introducing the foundations of DSMs, I will move on to a review of their applications in diachrony, comparing one NLP approach to corpus-linguistic approaches. I will conclude with a case study of Captain Kirk’s split infinitive, hoping to boldly go where no linguist has gone before.

References

Firth, J. R. 1957. “A Synopsis of Linguistic Theory 1930-55.” In Studies in Linguistic Analysis (special volume of the Philological Society), 1952-59:1–32. Oxford: The Philological Society.

Hamilton, William L, Jure Leskovec, and Dan Jurafsky. 2016. “Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change.” In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 1489–1501.

Harris, Zellig S. 1954. “Distributional Structure.” Word 10 (2-3): 146–62.

Landauer, Thomas K, and Susan T Dumais. 1997. “A Solution to Plato’s Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge.” Psychological Review 104 (2): 211.

Lund, Kevin, and Curt Burgess. 1996. “Producing High-Dimensional Semantic Spaces from Lexical Co-Occurrence.” Behavior Research Methods, Instruments, & Computers 28 (2): 203–8.

Miller, George A., and Walter G. Charles. 1991. “Contextual Correlates of Semantic Similarity.” Language and Cognitive Processes 6 (1): 1–28.

Sagi, Eyal, Stefan Kaufmann, and Brady Clark. 2009. “Semantic Density Analysis: Comparing Word Meaning Across Time and Phonetic Space.” In Proceedings of the Workshop on Geometrical Models of Natural Language Semantics, 104–11.

------------. 2011. “Tracing Semantic Change with Latent Semantic Analysis.” Current Methods in Historical Semantics, 161–83.

Turney, Peter D, and Patrick Pantel. 2010. “From Frequency to Meaning: Vector Space Models of Semantics.” Journal of Artificial Intelligence Research 37: 141–88.

Hands-on Session: Sketching shifting constructional networks with distributional semantic models

This session builds upon the first one and adopts a hands-on approach to reproduce the findings of a paper that I published recently:

"Changes in the midst of a construction network: A diachronic construction grammar approach to complex prepositions denoting internal location" Cognitive Linguistics, vol. 33, no. 2, 2022, pp. 339-386.

The participants will have access to the datasets and the R code and will proceed with me through the major methodological steps until the construction network of the Internal Location construction (see below) is sketched. The purpose is twofold: (a) to acquaint the participants with ongoing discussions in corpus-based Diachronic Construction Grammar, and (b) to introduce the participants to a methodological path that they can replicate and apply to their own research topics. The session ends with a discussion on competition models in cognitive linguistics.

Linguists have debated whether complex prepositions deserve a constituent status, but none have proposed a dynamic model that can both predict what construal a given pattern imposes and account for the emergence of non-spatial readings. This paper reframes the debate on constituency as a justification of the constructional status of complex prepositional patterns from a historical perspective. It focuses on the Prep NPIL of NPlm construction, which denotes a relation of internal location between a located entity (a trajector) and a reference entity (a landmark).

Four subschemas of the Internal Location construction are examined: MIDDLEcxn (in the middle of), CENTERCXN (in/at the center of), HEARTCXN (in/at the heart of), and MIDSTCXN (in the midst of).

1.         He stops suddenly in the middle of the stage and seems to consider. (1815-FIC-FalseShame)

2.         Marvin walked to the chalk mark in the center of the ring. (1934-FIC-Captain Caution)

3.         (…) we were in the heart of Norwalk. (1827-FIC-Novels)

4.         We see St Eustace praying in the midst of the river. (1980-FIC-RiddleyWalker)

All occurrences are extracted from the COHA, along with their co-occurring landmark NPs. Using vocabulary growth curves, all patterns are shown to be productive over the whole period covered by the corpus, although at different levels. Using word2vec, a semantic vector space with the landmark collocates of each pattern is made. Curves indexed on association scores are plotted to see how densely semantic areas have been populated across four consecutive periods: 1810s–1860s, 1870s–1910s, 1920s–1970s, and 1980s–2000s.

Two divisions of labor emerge. MIDSTCXN and HEARTCXN are in complementary distribution and operate mostly at the level of abstract locations whereas MIDDLECXN and CENTERCXN are in parallel distribution and operate at the level of concrete locations.

Resources

Article (open access): https://doi.org/10.1515/cog-2021-0128.

All primary data and code used in this session are available via the Open Science Framework, and can be retrieved from https://osf.io/x32jn/

Dagmar Divjak

Masterclass: Of Principles and Practice: what emerges when we learn from usage?

Usage-based linguistics is predicated upon the premise that languages are dynamic systems that emerge from usage and are shaped by usage in a process that is mediated by general cognitive abilities and functional considerations. Although the concept of “emergence” plays a key role in usage-based theories, cognitive processes or functions that would enable a system to “emerge” from use have, however, been conspicuously absent from consideration. In my talk I will present the work I am doing with the Out Of Our Minds team [https://outofourminds.bham.ac.uk] to operationalize emergence through learning. Using data from our work on English and Polish nominal and verbal systems, I will show how using computational techniques that implement principles of learning allows us to study what kinds of patterns might be picked up if usage is learned from exposure to the ambient language alone. I will argue that a learning-driven approach provides valuable insights into the kinds of generalizations or abstractions that can lay claim to cognitive plausibility and what information these categories might contain, which may ultimately change the way in which we think about language and and how it ought to be taught.

References

Divjak, D., Milin, P., Ez-zizi, A., Józefowski, J. & Adam, C. (2021) What is learned from exposure: an error-driven approach to productivity in language, Language, Cognition and Neuroscience, 36:1, 60-83, DOI: 10.1080/23273798.2020.1815813

Divjak, D., Romain, L., P. Milin (Under review). From their point of view: the article category as a hierarchically structured referent tracking system.

Divjak, D., I. Testini, P. Milin. (Under review). On the nature and organisation of morphological categories:  insights from modelling the learnability of verbal aspect from exposure to usage. 

Ez-Zizi, A., D. Divjak, P. Milin (2023, Early View). Error-correction mechanisms in language learning: modelling individuals. Language Learning. https://onlinelibrary.wiley.com/toc/14679922/0/0

Romain, L., Ez-zizi, A., Milin, P. and Divjak, D. (2022) What makes the past perfect and the future progressive? Experiential coordinates for a learnable, context-based model of tense and aspect. Cognitive Linguistics, vol. 33, no. 2, 2022, pp. 251-289. https://doi.org/10.1515/cog-2021-0006

Romain, L., D. Hanzlikova, P. Milin & D. Divjak. (In press). Ruled by Construal? Framing article choice in English. Constructions and Frames.

Romain, L., P. Milin, D. Divjak (Registered Report). Order effects in L2 learning. Language Learning.

Additional recommendation

Participants are asked to install Python (Anaconda distribution) before the masterclass

The following videos can be used as they will guide participants through the steps:

https://github.com/ooominds/Masterclasses_WidrowHoff/blob/main/00_Install_Python.pdf

To get the most out of the hands-on session, it would help if they could read the following 3 blogposts:

https://outofourminds.bham.ac.uk/blog/8

https://outofourminds.bham.ac.uk/blog/9

https://outofourminds.bham.ac.uk/blog/10

Susanne Flach

The node and its links: Modal auxiliaries from a combinatorial perspective

A current debate in Construction Grammar centres on the relationship between nodes (constructions) and the links between the nodes. In this talk, we'll revisit this debate by looking at modal auxiliaries. Modals are difficult to model in a CxG framework, because they defy core definitions for constructionhood. An quantitative-empirical perspective which focuses on the modals’ relationship with their surrounding lexical material, such as verbs and adverbs, showcases how factoring in combinatorial properties sheds new light on the debate, but also where corpus measures such as collostruction strength and experimental behaviour converge.

Hands-on session: Basics of R & Collostructional Analysis

For almost 20 years, the family of methods now known as Collostructional Analysis has been part of the quantitative corpus linguist's toolkit. Based on the logic of collocations of mutual attraction, CA measures the degree of attraction (or repulsion) between two units, such as constructions and the words that occur in it. In a way, CA directly operationalises a core interest in Construction Grammar, namely lexis-syntax interaction. CA has also been found to be a much better predictor of experimental behaviour than raw frequency. In this session, we will discuss the different flavours of CA (e.g., simple, distinctive, and co-varying), what it can and can't tell us, and illustrate how CA can be used in R/RStudio based on data from construction grammar, sociolinguistics, and discourse analysis. (https://www.sfla.ch/tutorials/R)

Maarten Lemmens (Univ. Lille, France)

Masterclass 1. Introduction to Cognitive and usage-based linguistics

In this session, we will give a more general (theoretical) background to cognitive linguistic, focussing in particular on the usage-based perspective that this model gives and why this is a more viable alternative than more formal approaches, as it does not entail any dualism (learned vs. innate) nor does it assume a strict division between grammar and lexicon. Instead, grammar is seen as emergent from children operating on linguistic input that leads to frequency-biased abstractions giving rise to grammaticality. As such, there is no need to posit any innate structure, only the capacity to learn a meaningful symbolic system.

Selective bibliography

Lemmens, M. 2021. Usage-based perspectives on lexical and constructional semantics. [Series: New Directions in Cognitive Linguistics],Shanghai Foreign Languages University Press, Ch. 1 & 2.

Lemmens, M. 2019. In defense of frequency generalizations and usage-based linguistics. An answer to Frederick Newmeyer’s “Conversational corpora: when big is beautiful”. CogniTextes 19 (https://journals.openedition.org/cognitextes/1616)

Lemmens, M. 2017. “A cognitive, usage-based view on lexical pragmatics. A reply to Hall” In: I. Depraetere & R. Salkie Semantics and Pragmatics: Drawing a Line, Springer, 101-114.

Lemmens, M. 2015. “Cognitive semantics”. Routledge Handbook of Semantics. Editor: Nick Riemer, London & New York: Routledge, 90-105.

Masterclass 2. Typological perspective on locative expressions

In this session, we will present our typological research on locative expressions. The starting point of our discussion will be the Talmian typology distinguishing verb-framed and satellite-framed languages even if it only partially applies to locative expressions. We will illustrate this via the use of posture verbs (SIT, LIE, STAND) in Dutch, contrasting their use to what is found in English and in French, on the basis of solicited descriptions. In a second part, we will extend the perspective to constructional differences and co-verbal gestures. In the last part, we will briefly talk about recent experimental research (Lesuisse 2022a,b, Lesuisse & Lemmens, subm.) that incorporates memorization and eye gazing.

Selective bibliography

Lemmens, M. 2021. Usage-based perspectives on lexical and constructional semantics. Shanghai Foreign Languages University Press, Ch. 5-9.

Lemmens, M. & J. Perrez. 2012. “A quantitative analysis of the use of posture verbs by French-speaking learners of Dutch”, CogniTextes 8, http://cognitextes.revues.org/609

Lemmens, M. & J. Perrez. 2010. “On the use of posture verbs by French-speaking learners of Dutch: a corpus-based study”, Cognitive Linguistics 21, 315-347.

Lesuisse, Mégane. 2022a. Cognitive Representations and Spatial Language: Views from French, English and Dutch. Unpublished PhD dissertation, Univ. de Lille, France.

Lesuisse, Mégane. 2022b. “Exploring the conceptualisation of locative events in French, English and Dutch: Insights from eye-tracking on two recognition tasks. Yearbook of the German Cognitive Linguistics Association, Vol. 10, 121-158.

Lesuisse, M. & M. Lemmens. (subm.) “Looking differently at locative events: the cognitive impact of linguistic preferences”. Language and Cognition.

Lesuisse, M. & M. Lemmens. 2018. “Grammaticalisation cut short: a diachronic constructional view on English posture verbs” In: Coussé, E., P. Andersson, & J. Olofsson (eds.) Grammaticalisation meets Construction Grammar. Amsterdam/Philadelphia: John Benjamins, 43-73.