A couple of months ago, a research group released a paper on the ArXiv titled “The emotional arcs of stories are dominated by six basic shapes.” In it, they replicate results similar to those first described by Matt Jockers, using a somewhat different technique.
I’ve written a jupyter notebook that raises doubts about their argument. They claim that their work has shown that there are “basic shapes” that dominate human stories, but the results they’ve described provide no basis for such generalizations. Given what we know so far, it’s much more likely that the emotional arcs that these techniques reveal are, in general, noise. The notebook is available for examination and reuse as a github repository.
What does it mean to say that these plot shapes are “noise”? The notebook linked above focuses on technical issues, but I want to write a few brief words here about the broader implications my argument has — if it’s correct. Initially, it may seem that if sentiment data is “noise,” then these measurements must be entirely meaningless. And yet many researchers have now done at least some preliminary work validating these measurements against responses from human readers. Jockers’ most recent work shows fairly strong correlations between human sentiment assessments and those produced by his Syuzhet package. If these sentiment measurements are meaningless, does that mean that human assessments are meaningless as well?
That conclusion does not sit well with me, and I think it is based on an incorrect understanding of the relationship between noise and meaning. In fact, according to one point of view, the most “meaningful” data is precisely the most random data, since maximally random data is in some sense maximally dense with information — provided one can extract the information in a coherent way. Should we find that sentiment data from novels does indeed amount to “mere noise,” literary critics will have some very difficult questions to ask themselves about the conditions under which noise signifies.