Semanlink - Learning Visually Grounded Domain Ontologies via Embodied Conversation and Explanation

Tags:

About This Document

sl:bookmarkOf : https://scholar.google.com/citations?view_op=view_citation&hl=en&user=fVRKVf4AAAAJ&sortby=pubdate&citation_for_view=fVRKVf4AAAAJ:MLfJN-KU85MC
sl:creationDate : 2025-05-28
sl:creationTime : 2025-05-28T14:14:47Z

File info

Bookmark of: https://scholar.google.com/citations?view_op=view_citation&hl=en&user=fVRKVf4AAAAJ&sortby=pubdate&citation_for_view=fVRKVf4AAAAJ:MLfJN-KU85MC

Documents with similar tags (experimental)

Learning neuro-symbolic dialogue strategies for interactive symbol grounding - University of Edinburgh Research Explorer

Tags:

> **Interactive task learning** studies situations in which a teacher interacts with a learner to help them perform a novel task in an embodied environment. To successfully interpret the teacher’s utterances, the learner has to perform **interactive symbol grounding**: it must **update its prior beliefs about the mapping from symbols to referents**, given their visual features, each time the teacher speaks. Interactive symbol grounding is even more challenging if the learner starts out unaware of concepts that are critical to task success. In that case, the learner must use the embodied conversation to discover and adapt to unforeseen possibilities, and so must cope with a continuously expanding hypothesis space and hence a non-stationary domain model. In this paper, **we propose a neuro-symbolic model for learning dialogue strategies for achieving interactive symbol grounding**. In particular, we study the effects of enriching the model with symbolic reasoning that captures the valid consequences of quantifiers (e.g., both, every). Our hypothesis is that utilizing such reasoning makes interactive task learning more data efficient. We test this empirically via a task of interactive reference resolution, in which the learner must jointly learn a grounding model and a policy for querying the teacher to enhance its accuracy in grounding. Our results show that a learner that exploits such symbolic reasoning for both decision-making and grounding is more data efficient than learners that ignore such linguistic insights.

2025-05-28 About