Why So Serious? Exploring Humor in AAC Through AI-Powered Interfaces

We explore the role of humor in Augmentative and Alternative Communication (AAC) and how to design interfaces to support timely humorous expressions. (a)~Here an AAC user tries to make a joke in an ongoing conversation. (b)~as the conversation continues, the user frantically tries to type the joke (c)~unfortunately they trigger text-to-speech (TTS) too late to effectively land the joke causing confusion (d)~we explore how to design interfaces to speed up the response through AI powered tools, while minimizing the cost of agency. (e)~Leading to design that allows AAC users to meaningfully express themselves through humor in ongoing conversations. Note: AAC input types and interfaces vary widely across users. One of our authors is an AAC user who commonly interacts with his phone as displayed here. Only for illustration purposes. Our actual interfaces were used on a range of devices in our studies.

Abstract

People with speech disabilities may use speech generating devices to facilitate their speech, aka Augmentative and Alternative Communication (AAC) technology. This technology enables practical conversation; however it remains challenging to deliver expressive and timely comments. In this paper, we study how AAC technology can facilitate such speech, through AI powered interfaces. We focus on the least predictable and most high-paced type: humorous comments. We conducted seven qualitative interviews with people with speech disabilities, and performed thematic analysis to gain in-depth insights in usage and challenges of AAC technology, and the role humor plays for them. We designed four simple AI powered interfaces to create humorous comments. In a user study with five participants with speech disabilities, these interfaces allowed us to study how to best support making well-timed humorous comments. We conclude with a discussion of recommendations for interface design based on both studies.

Video

Demo

Try out the interfaces used in the paper in your own web browser (or on your personal AAC device)

Citation

@article{weinberg2024whysoserious,
title={Why So Serious? Exploring Humor in AAC Through AI-Powered Interfaces},
author={Weinberg, Tobias and Kadoma, Kowe and Penuela, Ricardo E Gonzalez and Valencia, Stephanie and Roumen, Thijs},
journal={arXiv preprint arXiv:2410.16634},
year={2024}
}

Acknowledgements

Thanks to Zekai Shen, Samee Chandra and Maria Tane for helping in the develpment of the prototypes and Roy Zunder for helping analysing the data and wrinting. Thank you guys we couldn't have done without you!
The website template was borrowed from Michaël Gharbi.