Hi, I’m Ivan.

I am a third-year doctoral researcher at the Center for Doctoral Training in NLP at the University of Edinburgh. I work at the intersection of machine learning and cognitive science, focusing on few-shot learning and generalization in machine learning systems using insights from analogical reasoning. I am supervised by Alex Doumas and Siddharth Narayanaswamy.

Publications

• Behavioural vs. Representational Systematicity in End-to-End Models: An Opinionated Survey
  Ivan Vegner, Sydelle Souza, Valentin Forch, Martha Lewis and Leonidas A. A. Doumas
  To appear in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)(2025)
  Abstract, PDF
  A core aspect of compositionality, systematicity is a desirable property in ML models as it enables strong generalization to novel contexts. This has led to numerous studies proposing benchmarks to assess systematic generalization, as well as models and training regimes designed to enhance it. Many of these efforts are framed as addressing the challenge posed by Fodor and Pylyshyn. However, while they argue for systematicity of representations, existing benchmarks and models primarily focus on the systematicity of behaviour. We emphasize the crucial nature of this distinction. Furthermore, building on Hadley's (1994) taxonomy of systematic generalization, we analyze the extent to which behavioural systematicity is tested by key benchmarks in the literature across language and vision. Finally, we highlight ways of assessing systematicity of representations in ML models as practiced in the field of mechanistic interpretability.
• What does memory retrieval leave on the table? Modelling the Cost of Semi-Compositionality with MINERVA2 and sBERT
  Sydelle Souza, Ivan Vegner, Francis Mollica and Leonidas A. A. Doumas
  The SIGNLL Conference on Computational Natural Language Learning(2025)
  Abstract, PDF
  Despite being ubiquitous in natural language, collocations (e.g., kick+habit) incur a unique processing cost, compared to compositional phrases (kick+door) and idioms (kick+bucket). We confirm this cost with behavioural data as well as MINERVA2, a memory model, suggesting that collocations constitute a distinct linguistic category. While the model fails to fully capture the observed human processing patterns, we find that below a specific item frequency threshold, the model’s retrieval failures align with human reaction times across conditions. This suggests an alternative processing mechanism that activates when memory retrieval fails.
• FoVAE: Reconstructive Foveation as a Self-Supervised Variational Inference Task for Visual Representation Learning
  Ivan Vegner, Siddharth N and Leonidas A. A. Doumas
  NeuRIPS 2023 Workshop on Gaze Meets ML(2023)
  Abstract, PDF
  We present the first steps toward a model of visual representation learning driven by a self-supervised reconstructive foveation mechanism. Tasked with looking at one visual patch at a time while reconstructing the current patch, predicting the next patch, and reconstructing the full image after a set number of timesteps, FoVAE learns to reconstruct images from the MNIST and Omniglot datasets, while inferring high-level priors about the whole image. In line with theories of Bayesian predictive coding in the brain and prior work on human foveation biases, the model combines bottom-up input processing with top-down learned priors to reconstruct its input, choosing foveation targets that balance local feature predictability with global information gain. FoVAE is able to transfer its priors and foveation policy across datasets to reconstruct samples from untrained datasets in a zero-shot transfer-learning setting. By showing that robust and domain-general policies of generative inference and action-based information gathering emerge from simple biologically-plausible inductive biases, this work paves the way for further exploration of the role of foveation in visual representation learning.
• What can MINERVA2 tell us about 'killing hope'? Investigating L2 Collocational Processing with a Memory Model
  Sydelle Souza, Ivan Vegner, Leonidas A. A. Doumas and Francis Mollica
  Proceedings of the Annual Meeting of the Cognitive Science SocietyVolume 45(2023)
  Abstract, PDF
  Collocations are semi-productive word combinations with one word used literally and one other figuratively, characterized by an arbitrary restriction on substitution (kill hope, #murder hope). They are notoriously difficult for L2 speakers to acquire, yet there is no processing model specific to collocations. The present study attempts to explain trends in L2 collocational processing as memory retrieval. Modifying MINERVA2, a frequency-based memory model, we simulate reaction times and compare them to data from 99 L1 and 230 L2 (L1 Portuguese) English speakers involving free combinations (eat cake, ‘comer bolos’), congruent (read minds, ‘ler mentes’) and incongruent collocations (kick habits, no equivalent translation in Portuguese), and nonsense baselines (#read cakes). Under the assumptions that the L2 lexicon develops conditioned on the L1 and that the L2 lexicon is sensitive to L1 frequencies, we report that MINERVA2 can predict processing trends in both L1 and L2 collocational processing.

Contact Me

I’m reachable on my Twitter/X, and you can also email me at <myfirstname> @ <thisdomain> with just about anything!