Very long blog post, but here's some selected quotes:
Really interesting thought process and unusual case.
In our recent post on User Reviews Revisited we covered our process for identifying off-topic review bombs on games, and how you can decide for yourself whether or not you'd like to see them included in Review Scores throughout the store. Back in 2017, we defined a review bomb as an event where players post a large number of reviews in a very compressed time frame, aimed at lowering the Review Score of a game. At the time, we wondered if there'd be "positive review bombs", but there were no examples of one. We decided to wait and see.
A few weeks ago, in response to the Notre Dame tragedy, Ubisoft did something great for their fans by making Assassin's Creed: Unity available for free on Uplay, and committing funds towards rebuilding the monument. This led to a significant spike in players of AC: Unity on Steam, and a large number of positive reviews for the game. This led us, and members of the community, to wonder if this was finally a positive review bomb, and whether it should be considered off-topic.
Data-wise, it doesn't quite fit the pattern of negative review bombs: in the case of AC:Unity there was a significant increase in actual players alongside the increase in reviews. That isn't necessarily the case with a typical off-topic review bomb (but, to be clear, we have seen some negative review bombs with that characteristic).
Without reading the actual reviews, the data here all looks very much like a game that's gone on sale, or received an update. It's seen a spike in players, and as many people have come to realize, there's a fairly good correlation between player count and user reviews - if you get more players, you're going to get more reviews.
But we also went and read a large chunk of the reviews. Some reference Notre Dame or the giveaway. But most just look like standard reviews of a new player, or a player that's returning to a product they bought a while ago. Ubisoft has released significant updates to AC:Unity since launch, and it appears that some players who bounced off it at launch have returned, and found themselves enjoying the game more.
In this case, the Notre Dame tragedy has made it so that AC:Unity happens to now include the world's best virtual recreation of the undamaged monument. That's a context change that could be increasing the value players are getting from the game, so perhaps the game really is better than it was before? Or maybe that's unrelated, and it's actually players feeling good about Ubisoft's significant donation to rebuilding the monument? Irrespective of the reason, perhaps this is a short-term temporal effect? Should temporal effects even be included in Review Scores? If a game was heavily focused on a time of the year, like Christmas, we suspect we'd see it have Review Score fluctuations around Christmas-time, as more people bought it and thought it was better on average than people who bought it at other times of the year.
If visiting the virtual Notre Dame is a reason players have reviewing the game more positively, we'd expect the Review Score to continue to reflect it in the future, albeit at a lower volume. But that's still the case even if it's not the reason - the future Review Score would revert to where it was prior to this event.
So, we're not really sure what to do here. It doesn't actually seem to be a review bomb in the way we've previously defined them, but maybe that's just our definition being wrong. But even if we define it as one, we're not sure whether it should be off-topic or not. The overall Review Score would decrease by 1.3% if we marked it, which wouldn't have any significant effect on its visibility in the store (see the FAQ below for more on that). So either way, the game itself wouldn't be affected by our decision.
As a result, we've decided we're just going to leave it alone. But hopefully, this post has helped you understand that thinking behind why we've ended up there. If you have any thoughts on how we should approach this case, or similar cases in the future, we'd love to hear from you in the comments below.
Really interesting thought process and unusual case.
