Steam, the largest digital PC game storefront in the West, continues to struggle with user-contributed game reviews. Valve launched the feature in 2013, and since then, it has seen various updates to deal with issues such as false and gamed reviews.
But none of those updates were much comfort to the game Firewatch last week. Its Steam review page was swarmed with negative reviews after its developer Campo Santo denounced the hateful speech of game streamer PewDiePie and issued DMCA challenges to that streamer’s videos about Firewatch.
Maybe it’s a coincidence that Valve not only announced a new Steam user review feature on Tuesday but also tied it in a huge way to the issue of “review bombing.” Either way, Steam store pages now come with a lot more data in the form of “review histograms.” What are they? How will they affect reviews going forward? And most importantly, is that enough action to deal with a noticeable rise in irrelevant and poisonous use of Steam’s storefront?
What, you don’t believe the scores, do you?
The Tuesday blog post, written by Valve staffer Alden Kroll, calls review bombing out by name and defines it: “where players post a large number of reviews in a very compressed time frame, aimed at lowering the Review Score of a game. At the same time, they upvote each other’s reviews and downvote all the other reviews.”
Kroll says that these coordinated bursts of review activity often focus on issues “outside of the game.” The post mentions a few hypothetical examples, including differences between a Steam version and other platforms or “the developer’s political convictions,” which is as close as Kroll gets to directly referencing last week’s Firewatch bombing.
Perhaps more interesting is Valve’s own data-driven verdict on the practice. Namely, the company says that these bursts of user-submitted reviews rarely have a significant statistical impact on a game’s reviews going forward—meaning, Valve looked at game review averages and found that bombs were very rarely followed with a legitimate shift in customer opinion in either direction. This discovery, Kroll claims, is what led Valve to fight the practice: “review bombs make it harder for the Review Score to achieve its goal of accurately representing the likelihood that you’d be happy with your purchase if you bought [it].”
The post mentions a few strategies that Valve considered as a response when its system recognizes a review-bomb incident. One would have shifted a game’s “review score” (a conversion of a game’s positive/negative game review ratio to a “positive” or “negative” descriptor) to only score “more recent” data, but the post didn’t clarify exactly what that meant. Another would have temporarily frozen out all game reviews when a spike in review activity occurred, much like Wikipedia’s page-locking feature uses in the case of a newsworthy or controversial topic.
Perhaps the most effective practice would be for Valve to remove those user-generated scores at the top of each game’s store listing—especially when they can be gamed, so long as users are allowed to contribute and up-vote reviews to the service. Then users would have to read reviews to understand their context before letting them sway their purchase.
But Kroll shot this down in a curious way. “Unfortunately, we’re pretty certain that this isn’t really an option,” the post explains. “Scores were added in response to player demand in the past, and that demand for a summary of some kind is likely to still be there, even if players know it isn’t always accurate.”
In other words: review-bombing is a problem, but, like, review scores are sometimes fake?
Interesting data—if you can scroll to it
Instead, Valve has opted to add “review histograms” to every Steam store listing, effective immediately.
Steam’s new review histograms allow potential game buyers to filter game reviews based on either the month or day they were posted and also by their positivity (as blue bars above a line) or negativity (orange bars below a line). The idea being, if you scroll to a review bar chart and see a giant red line appear on a certain day, you can click it and see that day or month’s similarly minded reviews grouped together. Maybe they all mention the brand-new addition of obtrusive copy-protection software or a new online mode that has somehow dramatically changed the game. Or maybe they all mention a certain misappropriated cartoon frog. You could judge from there.
This follows efforts in September 2016 to segregate the service’s game reviews between “Steam purchases” and “others.” The latter are primarily game owners who unlocked the game using a 25-digit code, which sometimes comes from legitimate purchases but often comes from free copies given out by game developers. These were sometimes used by developers, Steam says, to puff up review scores.
The first problem with today’s new histograms is in their looks. Quite frankly, they’re hideous and hard to parse. The histograms are designed in such a way that their blue and orange bars take up an entire maximized Steam window, yet the bars are itty-bitty in terms of pixel size, so they’re simultaneously hard to click. Plus, numbers and other data points are tossed onto the X and Y axes without any lines or other design cues. They look like they were designed in Microsoft Word.
Should you successfully click on the bar you want, you then need to scroll your entire window down to see the relevant reviews that load (and if you don’t do so quickly enough, you don’t even see the reviews load or propagate). This interface still also has a confusing, separate column of “recent reviews,” which will fill up with different reviews no matter which month or year of data you might be looking at.
Worst of all, because Steam’s store listing pages are such a cluttered mess, you have to scroll quite a ways before even finding these charts. It’s a gol-danged journey: past the official description, any Early Access explainer box, the “buy now” buttons, the DLC options, the “recent news” boxes, an “About This Game” text box, a “system requirements” box, a “more like this” ad for other games, and the “what curators say” box.
And after all of that, Steam thinks we want to pick and prod at a massive, unclear line chart?
Origins of hate
The idea, of course, is sound: give users a better look at spikes in review activity to ascertain what the heck caused them. But there has to be a better way. Yelp, for example, is clever about pulling out specific buzzwords in any restaurant’s hundreds of reviews, either about specific dishes or restaurant features. Surely Steam could do the same for whatever terms reappear in various reviews: “Denuvo,” “no 21:9 support,” or “Jim Sterling actually liked it.”
The bigger issue, of course, is that this is another example of Steam passing the “users have gone amok” workload buck. As Zoe Quinn reported in her book Crash Override, Steam has said that it employs neither a department nor an employee whose job it is to “handle abuse or user safety” and that it blamed its choice not to remove offending content from Steam review or forum pages due to what it called the “Streisand effect.” Quinn learned this after trying in vain to get Steam to take action against users who’d posted personally identifying info or links to nude photos to the reviews and forum pages attached to her game Depression Quest.
Ubisoft learned this, as well, in a recent explosion of racist and disturbing comments, images, and ASCII art on the forum pages for the upcoming game Assassin’s Creed Origins. This comment bombing on August 28 revolved around a racist meme and remained on the page until Ubisoft moderators swooped in to delete posts and ban offending users from the forum. Neither Ubisoft nor Valve responded to Ars’ questions about this incident. But it put a spotlight on a glaring issue for Steam: that its community pages are all solely policed by game makers, as opposed to Steam itself. For tiny independent game-development teams, that means they may be one misunderstood blog post away from being overwhelmed by poisonous users—and being stuck playing comment clean-up.
If those issues aren’t enough, Steam’s gatekeepers are currently letting in drivel like The Mexican Dream, a game in which players must murder Mexicans as they approach an American border wall.
Histograms may expose data for one small portion of the Steam experience, but the shop as a whole has begun to show serious cracks in its content-management armor. And without an apparent outcry from Valve or a pledge to go so far as to wholesale ban user accounts and IP address ranges for repeat TOS offenders (and thus risk losing paying customers), there’s no reason to believe the Internet’s worst trolls and abusers won’t keep on creating new accounts and raising hell wherever they’re allowed on Steam.