For best experience please turn on javascript and use a modern browser!
You are using a browser that is no longer supported by Microsoft. Please upgrade your browser. The site may not present itself correctly if you continue browsing.

Diego Marconi (University of Turin): The role of visual imagery in language understanding

The September SMART Lecture was presented by Diego Marconi, Professor of Philosophy of Language at the University of Torino, with an introduction by Franz Berto.

Advocates of the ideational theory of meaning (e.g., John Locke) claimed that language comprehension consists in the generation of "ideas in the mind" as an effect of language perception. As a semantic theory, the ideational theory was discredited in the 20th century. Nevertheless, there is evidence that visual imagery -the generation of mental images- does play a role in comprehension and is not just a possible side effect of it. High imageable words (bananasmilechair), as opposed to low imageable words (deductioneventdemocracy) appear to facilitate several tasks related to language comprehension. Moreover, HIW turn out to selectively activate visual-related areas in the human cortex, to the difference of LIW. Visual imagery is often seen as one form of simulation, the "partial re-enactment of perceptual, motor and introspective states acquired during experience with the world, body and mind" (Barsalou 2008). Proponents of the strong simulation theory of understanding hold that understanding in general just consists in such simulation by activation of dedicated areas in the brain (visual, motor, olfactory etc.). This view entails that even processing of "abstract" words ought to correlate with modality-specific activations.  However, in a recent experiment we found significantly different patterns of activation for HIW vs. LIW: it thus seems that the processing of "abstract" [=LI] words does not involve the same kind of simulation as characterizes imageable words (if it involves simulation at all).  


Paul Boersma (University of Amsterdam): Linguistic category creation with deep learning

The October SMART Lecture was presented by Paul Boersma, Professor of Phonetic Sciences at the University of Amsterdam.

 Inspired by recent successes with deep learning models that show humanlike behavior, we applied deep belief networks to unsupervised phonological category creation solely on the basis of auditory input. When trained with sounds drawn from a marginal formant distribution like that of Spanish (i.e. three peaks representing the vowels /a/, /o/ and /u/), the shallowest possible network, which consists of two stacked restricted Boltzmann machines, exhibits a learning process that goes through four stages when confronted with an input sound: (1) reflecting noise, (2) reflecting the marginal distribution, (3) reflecting the closest of the three peaks, and (4) reflecting the input sound. The third of these stages is the most relevant one: it exhibits categorization behavior, by showing that at the deepest (i.e. "highest") level there are only three possible activation patterns. We conclude that deep networks can help us model linguistic category creation.


Aditi Lahiri (University of Oxford): The Art of Listening: Processing alternatives and asymmetry

The November SMART Lecture was presented by Aditi Lahiri, Professor of Linguistics at the University of Oxford, Fellow of Somerville College and the Director of the Language and Brain Lab. Her talk was introduced by Silke Hamann.

A speaker-listener relationship is fraught with difficulties. The listener has no influence over a speaker’s utterances which could be as variable as (s)he pleases. Nevertheless, language comprehension amidst native speakers seem straightforward in normal conversation. Models of speech perception and word recognition vary considerably in their assumptions about how words are represented in the mental lexicon, how much detail is stored, and how the speech signal is mapped on to the lexicon. Furthermore, models differ in symmetry or asymmetry in representation and in recognition. Since languages are replete with asymmetries, representational and processing hypotheses assuming asymmetries are not unreasonable. In this presentation, Aditi Lahiri discussed a model assuming segmental as well as tonal asymmetries in synchronic phonological systems.  In support of the model, she provided evidence from language change as well as from a series of psycholinguistic and neurolinguistic experiments on German, English, Swedish and Bengali. 


Marc Leman (Ghent University): Musicology as cognitive science: An interdisciplinary paradigm for humanities?

The December SMART Lecture was presented by Marc Leman, Methusalem research professor in systematic musicology at Ghent University, with an introduction by Henkjan Honing.

Everybody is an expert in music, as musician, as listener or dancer, but hardly anyone seems to understand how music works, and why and how it may empower people, or simply said, make them better. This question is the research topic of a musicology.

In that respect, musicology provides a good example of what the humanities could contribute to cognitive science: (i) clarification of the foundational concepts that should allow us to comprehend how the enormous complex domain of music empowers people, (ii) steps towards studying music in ecological settings (rather than laboratory settings) as a basis for empirical testing and evidence-gathering, and (iii) a focus on modelling that turns humanistic knowledge into devices that can be beneficial for the wellbeing of humans.

In his talk, Marc Leman presented the main trends in our research and he showed concrete examples (videos, devices) of how musicology influences cognitive sciences in its paradigm shift from cognition to action and interaction. Musicology is on its way to develop a dynamic systems approach to embodied human-music interaction, recent contributions from anthropology and music therapy adopt the embodied interactive paradigm for understanding how music works in ecological settings, and last but not least, musicology promotes the development of assistive music-based bio-feedback systems, to be used in different application domains, including the arts, the sports and medical sector.


Martijn Meeter (VU University): Reading and the brain

The January SMART Lecture was presented by Martijn Meeter, Professor in education sciences at Vrije Universiteit Amsterdam (VU), with an introduction by Gerard Steen.

The brain is a massively parallel information processing organ, in which processes seem to have fluid boundaries. But language is serial (one word at a time), it is symbolic, and it has been argued to follow rules. All these characteristics were once seen as antithetical to parallel and fuzzy processing. However, new theories of language comprehension rely more strongly than previous ones on parallel and fuzzy processing.

Here, Martijn Meeter discussed the example of research on reading. Reading seems to progress one word at a time, and at some level that is an accurate description of what is going on. At another not. New paradigms have shown that our brain integrates information about letters without caring particularly much about what words those letters are in. That is puzzling: if that is the case, why don’t we read “the car” as “che tar” or any other recombination of letters in those words? He discussed a computational model that describes what is going on in the brain while we read words and present a framework that extends this model to semantic and syntactic processing during reading.


Janet Pierrehumbert (University of Oxford): The Cognitive and Social Lives of Words

The February SMART Lecture was presented by Janet Pierrehumbert, Professor of Language Modelling in the University of Oxford e-Research Centre and Adjunct Professor at the New Zealand Institute of Language Brain and Behaviour, University of Canterbury. Her talk was introduced by Rens Bod.

People learn words from instances of words as they are produced by different speakers and in different contexts. The words they know in turn provide the foundation for generalizations about words, supporting lexical productivity. A large and productive lexical system is a hallmark of the remarkable linguistic capabilities of human beings.

In this talk, Janet Pierrehumbert reviewed several projects that shed light on learning of words and word formation patterns. These projects use a diverse methodology, bringing together data from lab experiments, on-line experiments that resemble computer games, and analysis of large corpora. Results indicate that lexical representations are both phonologically abstract and phonetically detailed. They include socio-indexical information. Statistical patterns matter, but sometimes in surprising ways: more is not always better or more productive. Competition within the lexicon, social factors, and individual differences all play important roles. She concluded by drawing some connections to historical change.


Vincent F. Hendricks (University of Copenhagen): Bubbles Go Bust

The March SMART Lecture was presented by Vincent F. Hendricks, Professor of Formal Philosophy at The University of Copenhagen and Director of the Center for Information and Bubble Studies at the University of Copenhagen. Sonja Smets introduces his talk.

Besides champagne, bubbles are typically associated with situations in finance in which assets trade at prices far exceeding their fundamental value. That was the way of the Dutch tulip bulb frenzy in the 1630s and the subprime crises in 2008. The market overheated, bubbles went bust with catastrophic consequences. But bubbles are not confined to the world of finance. In fact one may today speak of information bubbles, status bubbles, bullying bubbles, political bubbles, news bubbles, even science bubbles.

Vincent F. Hendricks walked us through bubble studies using a wide range of entertaining, thought-provoking and disconcerting examples from the world of finance, social media, politics, populism, fake news and post-factual democracy. On the way he punctured some of the most inflated tendencies of today's public debate especially on social media.


Marco Caracciolo (Ghent University): Narrative Beyond Anthropocentrism: Embodying the Nonhuman

The May SMART Lecture was presented by Marco Caracciolo, Assistant Professor of English and Literary Theory at Ghent University, with an introduction by Stephan Besser.

Narrative is a practice geared toward what psychologist James J. Gibson called the “intermediate world”—that is, the human-scale world of everyday perception. One of the upshots of this idea is that storytelling has, in Monika Fludernik’s (1996, 13) term, an “anthropomorphic bias.” It is not just that narrative understanding is embodied, as psychologists and psycholinguists have persuasively shown; at a very fundamental level, narrative implies human forms of embodiment.

This talk engaged with stories that resist this bias, putting the reader in touch with a wide array of nonhuman realities—including the experience of nonhuman animals, the “deep” temporality of evolution, and a cosmic perspective on human affairs. To explore these narratives, Marco Caracciolo drew on work on the embodied basis of narrative comprehension in fields such as psycholinguistics and cognitive linguistics. His central claim was that, when approached creatively, embodiment becomes an opportunity for this kind of narrative: through a process of what he calls “bodily defamiliarization,” readers’ imagination may be elevated—temporarily, of course, and tentatively—beyond the human. In this way, the talk demonstrated how cognitive literary studies is not just a productive framework in itself, but one that can make a significant contribution to other areas of discussion, particularly the environmental humanities and human-animal studies.


Courtenay Norbury (UCL): Non-verbal cognitive ability does not influence rate of language in the first years of formal schooling

The June SMART Lecture was presented by Courtenay Norbury, Professor of Developmental Disorders of Language and Communication at Psychology and Language Sciences, University College London, with an introduction by Jeannette Schaeffer.

The role of non-verbal cognitive ability in the diagnosis and development course of ‘developmental language disorder’ (DLD) remains controversial, despite recent recommendations that non-verbal ability should not be included in diagnosis criteria. A question of theoretical and practical import is whether non-verbal cognitive deficits yield slower rates of language growth, perhaps signaling limited biological potential for language change. 

In this lecture Norbury presented data from SCALES, a population study of language development and disorder from school entry. SCALES employed sample weighting procedures to estimate growth in core language skills, including measures of vocabulary, grammar and narrative. A stratified subsample (n = 529) received comprehensive assessment of language, non-verbal ability, and behavioral difficulties at 5-6 years of age and 95% of the sample (n = 499) were assessed again at ages 7-8. Language growth was measured using both raw and standard scores in children with typical development, children with DLD, and children with language impairment associated with other developmental disorders.

Across the first three years of school there was strong individual stability of language (estimated ICC = .93). Linear mixed effects models highlighted steady growth in language (raw scores), and parallel growth in standard scores across all three groups. Adjusted models indicated that while non-verbal ability, socio-economic status and behavioural skills predicted initial language score (intercept), none influenced rate of language growth (slope).

From school entry, rate of language growth was remarkably similar in three groups of children with diverse language and cognitive profiles. During this developmental window, language growth was not influenced by non-verbal ability; neither group with language impairment showed evidence of substantially narrowing the gap with typical peers. Importantly though, those with non-verbal cognitive deficits did not fall further behind. Findings suggest limited potential to change language trajectories after school entry, though the potential for intensive, well-targeted interventions to accelerate growth in children with DLD remains to be tested.