Research: ChatGPT stories rated higher by readers than texts by human authors

Short stories written by ChatGPT were found to be more engaging and of higher quality by readers on average than those penned by human authors. According to a compilation by Euronews Turkish, the study by researchers at Villanova University in the US was published in the academic journal *Judgment and Decision Making*.

One of the prominent findings of the research was that participants struggled to distinguish the source of the texts. In the experiments, many participants could not reliably identify which story was written by artificial intelligence.

WHAT WAS EXAMINED IN THE EXPERIMENTS?

In the first experiment, 1,682 adult volunteers were asked to read one of six short stories. Three of these stories were written by humans, and three were written by ChatGPT. Each text produced by the AI covered the same theme as one of the stories written by humans.

Researchers told participants whether the story they were reading was written by a human or an AI; however, this information was not always accurate. Some participants were intentionally given false information. Readers were then asked to evaluate the text in terms of engagement, interest, and quality.

According to the results, stories written by ChatGPT outperformed human-authored texts in average scores. It was also noted that participants tended to give higher ratings to a text when they were told it was written by a human.

The study also indicated that personal attitudes toward artificial intelligence could influence evaluations. Participants with more positive views on AI tended to give higher ratings to stories they were told were written by ChatGPT.

The researchers then conducted two additional experiments with 905 people. At this stage, participants were presented with two stories covering the same theme: one written by a human and the other by ChatGPT. Participants were asked to guess which of the texts belonged to the AI.

In the first additional experiment, the correct guess rate remained at 40 percent. In the second experiment, this rate was measured at 52 percent. The researchers stated that these results show that people generally cannot reliably distinguish between AI-written and human-written stories.

The study also stated that participants' self-assessments regarding fiction literature were not linked to prediction success, whereas those who claimed to have more knowledge about AI systems were able to identify ChatGPT texts more accurately.

News Source: 12punto