Consumers rely on crowdsourced reviews to make decisions on many products and services. It should be no surprise that many patients use the experiences of others to select their doctors. Choosing a competent and experienced physician, however, may be more important than buying a pair of shoes or a set of stereo speakers.
A recent study of the online ratings of urologists finds that online reviews are scant and overwhelming positive. Moreover, protecting the anonymity of the reviewer may question the veracity of the reviewer.
The study, published online in the Journal of Urology, analyzed the online ratings and reviews of 500 randomly selected urologists. The researchers created composite ratings from 10 popular physician review websites (PRWs) and categorized the comments from the most popular PRW. Extremely negative or extremely positive comments were defined as being overly flattering or overly rude, non-constructive comments.
The researchers reported that nearly 80% of the urologists had at least one rating (range, 0-64), although composite scores were based on just 2.4 ratings per physician, on average. Eight-six percent of the ratings were positive, of which, 36% were highly positive. Just more than half of the written comments (53%) were positive or extremely positive.
Given that composite ratings were based, for the most part, on a relatively small number of submitted reviews, the researchers conclude that average ratings are highly volatile to single reviewers. In addition, some of the PRW review scores were based on other dimensions of patient satisfaction, such as accessibility and office wait time, not physician competence. Moreover, the anonymity of the crowdsourced reviews gave the researchers some pause on their veracity:
Ratings can be submitted anonymously on most PRWs, so many physicians fear that angry patients or competitors may abuse these sites.
Quoted on National Public Radio, Chandy Ellimoottil, the lead author of the study suggests that with so few reviews, a single angry patient can ruin a doctor’s reputation. The anonymity of these reviews also makes it possible for doctors (or their office staff) to self-evaluate.
The results of this study are somewhat disturbing but not altogether surprising. Similar studies of crowdsourced post-publication ratings and reviews (e.g. article ratings, journal comments, tweets, book reviews, Facebook likes) provide similar results — crowdsourced reviews are few, mostly positive, potentially fraudulent, and provide little detail.
When trusting the wisdom of the crowds, its important to understand what is meant by “crowd.”