Marie-Catherine de Marneffe is a FNRS research associate and professor at UCLouvain. She obtained her PhD under the supervision of Chris Manning at Stanford University and worked 10 years in the Linguistics department at The Ohio State University as assistant then associate professor. Her main research interests are in computational pragmatics, building models that capture what people infer “between the lines”. She is also one of the principal developers of the Universal Dependencies framework. Her research work has been funded by Google Inc., the National Science Foundation and the FNRS.
Recently, NLP researchers have increasingly begun to acknowledge that humans often diverge in their interpretations of various NLP tasks, and that such variation should be captured if robust language understanding is to be achieved. In this talk, I will focus on analyzing human label variation in the Natural Language Inference (NLI) task, in which - given a premise - one identifies whether a hypothesis sentence is true, false, or undetermined. For instance, if one says: “My friend often travels with a heavy suitcase”, can it be inferred that “My friend often travels with a light suitcase”? I will examine the various sources of NLI label variation and investigate whether or not they can be captured by current LLMs - arguing that, in the presence of variation, labels without explanations are not sufficiently meaningful.