Our CSC 790 term project, by James R. Brown and me, analyzed Goodreads reviews from the UCSD Book Graph (Wan and McAuley, 2018; Wan et al., 2019). Alongside searching the reviews and predicting their star ratings, the project clustered the people who wrote them. The question was whether reviewers fall into recognizable types, distinguished by how much they write, how they rate, and how often they review. The clustering was my partner's part of the project, and this article describes what it did and what it found, using the concepts of this topic. The code is in the Goodreads repository.
What was clustered
The objects clustered are reviewers, not reviews or documents. Each reviewer is summarized by four numbers computed from their reviews:
- the number of reviews they wrote;
- their average star rating;
- the variance of their ratings, which is low for a reviewer who gives every book the same score and high for one who uses the whole scale;
- the average length of their reviews, as recorded in the dataset, most likely in characters.
The clustering used K-means with $K = 5$, Euclidean distance on the four features, starting centroids chosen at random from the reviewers, and up to 100 iterations. A two-dimensional projection of the clusters by principal component analysis was used to visualize them.
The clusters
| Cluster | Reviewers | Average review length | Average rating | Reviews per reviewer | Rating variance |
|---|---|---|---|---|---|
| 3 | 7,483 | 417 | 3.84 | 6.8 | 0.59 |
| 1 | 4,712 | 1,104 | 3.78 | 8.8 | 0.69 |
| 2 | 2,304 | 2,056 | 3.77 | 9.2 | 0.65 |
| 0 | 779 | 3,442 | 3.74 | 7.8 | 0.60 |
| 4 | 116 | 6,323 | 3.41 | 3.6 | 0.36 |
The 15,394 reviewers are sorted here by the average length of their reviews, and the clusters fall into clear bands of length. Nearly half write short reviews averaging a few hundred characters. Smaller groups write progressively longer reviews, and a group of 116 writes reviews averaging over 6,000.
Why length shaped the clusters
The four features were used in their original units. Review lengths run to thousands, while average ratings lie between 1 and 5 and rating variances below 2. Euclidean distance adds up squared differences in each feature, so it is dominated by whichever feature varies most in absolute terms. Between the centroids of clusters 3 and 1, the squared difference in average length is about 470,000, and the squared differences in the other three features together come to about 4. Length accounts for more than 99.99 percent of the squared distance. The clustering is therefore, in effect, a division of the reviewers into bands of review length. The other three features can then be compared across those bands.
That is a legitimate and readable result. Grouping by the dominant feature produces bands with clear boundaries, and the averages of the other features within each band show how they vary with length. It does mean that the clusters should be read as length bands rather than as types defined by all four features together. K-means in practice describes the alternative. Rescaling each feature, for example to zero mean and unit variance, before clustering puts all four on comparable scales, so that each can influence the result.
What the clusters show
Reading the table across the length bands:
- Average rating falls as reviews get longer. It falls only slightly through the first four bands, from 3.84 to 3.74, then drops to 3.41 for the 116 reviewers who write the longest reviews. The writers of very long reviews are, on average, the most critical. The difference is concentrated in a small group, and no test of its statistical significance was made.
- The longest reviewers rate most consistently. Their rating variance, 0.36, is the lowest of any cluster, and they review the fewest books each.
- Prolific reviewers write mid-length reviews. The clusters with the most reviews per reviewer, about nine, are the middle length bands.
A plausible reading is that very long reviews are often written by readers with a considered, critical view, and that critical readers write at length to explain it. The clustering cannot establish why, only that the pattern is there.
Next steps
The natural extensions use methods from this topic:
- Standardize the features before clustering, so that rating behavior and review counts shape the clusters as well as length.
- Choose K from the data, by comparing the residual sum of squares or another criterion across values of K, as described in K-means in practice.
- Run several restarts and keep the clustering with the lowest residual sum of squares, since K-means finds only a local minimum (Manning, Raghavan, and Schütze, 2008, §16.4).
- Cluster reviewers by what they write, using the vocabulary of their reviews as TF-IDF vectors, which would group reviewers by the kinds of books they discuss and the language they use.
References
- Manning, C. D., Raghavan, P., and Schütze, H. (2008). Introduction to Information Retrieval, chapter 16. Cambridge University Press.
- Wan, M., and McAuley, J. (2018). Item recommendation on monotonic behavior chains. In Proceedings of RecSys 2018, 86–94.
- Wan, M., Misra, R., Nakashole, N., and McAuley, J. (2019). Fine-grained spoiler detection from large-scale review corpora. In Proceedings of ACL 2019, 2605–2610.