You have interrupted me again!: making voice assistants more dementia-friendly with incremental clarification - Addlesee, Angus; Eshghi, Arash (Frontiers in Dementia, 2024)
In spontaneous conversation, speakers seldom have a full plan of what they are going to say in advance: they need to conceptualise and plan incrementally as they articulate each word in turn. This often leads to long pauses mid-utterance. Listeners either wait out the pause, offer a possible completion, or respond with an incremental Clarification Request (iCR), intended to recover the rest of the truncated turn. The ability to generate iCRs in response to pauses is therefore important in building natural and robust Everyday Voice Assistants (EVA) such as Amazon Alexa. This becomes crucial with People with Dementia (PwD) as a target user group since they are known to pause longer and more frequently, with current state of the art EVAs interrupting them prematurely, leading to frustration and breakdown of the interaction. In this paper, we first use two existing corpora of truncated utterances to establish the generation of CRs as an effective strategy for recovering from interruptions. We then proceed to report on, analyse and release \dataset: a new corpus of 3000 crowdsourced, human-produced iCRs; the first of its kind. We use this corpus to probe the incremental processing capability of a number of state of the art Large Language Models (LLM) by evaluating: (i) the quality of the model's generated iCRs in response to incomplete questions; and (ii) the ability of the said LLMs to respond correctly after the user’s response to the generated iCR. For (i), our experiments show that the ability to generate contextually appropriate iCRs only emerges at larger LLM sizes, and only when prompted with example iCRs from our corpus. For (ii), our results are in line with (i), i.e. that larger LLMs interpret incremental clarificational exchanges more effectively. Overall, our results indicate that autoregressive LMs are, in principle, able to both understand and generate language incrementally; and that LLMs can be configured to handle speech phenomena more commonly produced by PwD, mitigating frustration with today's EVAs by improving their accessibility.