Nina Beguš, current lecturer at UC Berkeley’s School of Information and Department of History and former CSTMS Postdoctoral Scholar, has published an article, “Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytelling,” in Humanities and Social Sciences Communications. Her study, released today, October 28th, compares storytelling by human writers and AI models like GPT-3.5 and GPT-4, revealing insights into generative AI’s limitations and its approach to cultural themes such as the Pygmalion myth.

Abstract: The paper proposes a framework that combines behavioral and computational experiments employing fictional prompts as a novel tool for investigating cultural artifacts and social biases in storytelling both by humans and generative AI. The study analyzes 250 stories authored by crowdworkers in June 2019 and 80 stories generated by GPT-3.5 and GPT-4 in March 2023 by merging methods from narratology and inferential statistics. Both crowdworkers and large language models responded to identical prompts about creating and falling in love with an artificial human. The proposed experimental paradigm allows a direct and controlled comparison between human and LLM-generated storytelling. Responses to the Pygmalionesque prompts confirm the pervasive presence of the Pygmalion myth in the collective imaginary of both humans and large language models. All solicited narratives present a scientific or technological pursuit. The analysis reveals that narratives from GPT-3.5 and particularly GPT-4 are more progressive in terms of gender roles and sexuality than those written by humans. While AI narratives with default settings and no additional prompting can occasionally provide innovative plot twists, they offer less imaginative scenarios and rhetoric than human-authored texts. The proposed framework argues that fiction can be used as a window into human and AI-based collective imaginary and social dimensions.

In addition to her recent publication, Nina Beguš, alongside colleagues from Denmark and Joe Dummit from the University of California, Davis, has received funding from the Danish National Research Foundation to establish TEXT: Center for Contemporary Cultures of Text at Aarhus University. Set to run from 2025 to 2031, this center will focus on research into Large Language Models (LLMs) and their impact on contemporary text and culture.

For more information regarding her publication, please visit the Berkeley News or The Independent.