Paying online community members to write product reviews backfires badly among the socially influential

CATONSVILLE, MD, May 1, 2017 – Online user reviews have become an essential tool for consumers who increasingly rely on them to evaluate products and services before purchase. The business models of online review platforms such as Yelp and TripAdvisor, and e-commerce sites such as Amazon and Expedia critically depend on them. Should such sites pay users to encourage them to write reviews? According to a forthcoming study in the INFORMS journal Marketing Science, a leading academic marketing journal, that is a bad idea. Paying users suppresses the number of reviews on social platforms, especially among those users who are socially well-connected and likely to be more influential.

The study, “Motivating User Generated Content: Social Connectedness Moderates the Effects of Monetary Rewards,” is authored by Yacheng Sun of University of Colorado and Tsinghua University, and Xiaojing Dong and Shelby McIntyre of Santa Clara University. The authors examine user response from a sample of customers following the introduction of a monetary payment program for user reviews by a social shopping platform in China. The payment was roughly the equivalent of 25 cents per review in credit for purchases from sellers affiliated with the platform. To the company’s surprise, the number of user reviews declined by over 30 percent in the month after the payment program was introduced, relative to the month before. “The familiar “Law of Supply,” that implies supply increases in response to higher prices, does not seem to hold true when it comes to paying for reviews on a social platform,” said Sun.

The paper explores why reviews drop in response to the monetary payments. The authors conjecture that the drop in reviews could be the result of community members’ concerns that their honest reviews - motivated by an intrinsic motive to either help others with relevant information or to present themselves as knowledgeable about the product or service - may now be interpreted by the community as simply driven by the less honorable extrinsic motivation of making money. If this were true, the drop in user reviews would be greater among users who had more friends on the social network, who could potentially misinterpret the user’s motivation for writing reviews.

The authors empirically test their conjecture by comparing the change in reviewing behavior among “socialites” (more than five friends on network) against “loners” (no friends on network) after the introduction of the payment scheme. Indeed, the reviews from socialites drop 85 percent, from just over 0.4 reviews a month to just under 0.06 reviews a month. In contrast, the loners who had little to lose in terms of social capital increase their reviews from close to zero to about 0.03 per month. The increase in reviews from the loners however does little to offset the massive drop among the socialites, who are the heavy contributors overall. Hence the aggregate drop of 30 percent.

“Nobody wants to be seen as a paid shill for brands, so the users with more friends and followers, who were likely more influential and wrote more originally, are the ones who stop writing. A real double-whammy,” said Dong.

 “Our results support the approach of industry leaders like Yelp or Amazon, who do not compensate for reviews. In fact, they tap into the intrinsic motive for social recognition through status badges for frequent contributors,” said McIntyre. “There may be still ‘under the radar’ ways to pay only the less socially active users for their reviews, but such targeting can be risky as the heavy reviewers may perceive it to be unfair and therefore stop writing reviews, if and when they learn about it.” 

The complete paper is available at: http://pubsonline.informs.org/doi/abs/10.1287/mksc.2016.1022

About INFORMS and Marketing Science

Marketing Science is a premier peer-reviewed scholarly marketing journal focused on research using quantitative approaches to study all aspects of the interface between consumers and firms. It is published by INFORMS, the leading international association for operations research and analytics professionals. More information is available at www.informs.org or @informs.

About ISMS

The authors are members of ISMS, the INFORMS Society of Marketing Science. ISMS is a society of scholars focused on describing, explaining and predicting market phenomena at the interface of firms and consumers. 

####