Ruth Given has written a paper entitled, MD Rating Websites: Current State of the Space and Future Prospects (PDF), that was recently published on THCB. It’s a 39-page informal analysis (with an emphasis placed on informal) that takes a fairly good and comprehensive look at the space of doctor rating sites as they exist today.
I think that the main issue Given hit upon in the report (but I’m not sure she recognizes as the primary challenge of doctor rating sites) is the numbers issue. With over 700,000 physicians in the U.S., a ratings database of 10,000 or even 20,000 is pitifully and woefully small.
And not only is such a number small, it is statistically troublesome, likely painting an inaccurate picture of providers listed. If most sites only have one or two ratings for any given physician (and such data is statistically useless, if the average family physician is covering between 2,000 to 2,500 patients), then the sites themselves provide little added value outside of the directory listing. Which, as Given notes, is inaccurate or nonexistent more often than not.
For a number to have meaning in research, one needs to look at where the number comes from. In terms of rating sites (any rating site), ratings are more likely to be valid and reliable if:
- They come from a significant number of people compared to the number of people using the service or product
- They come from a randomized set of people who have used the product or service
- They represent both positive and negative opinions
- They objectively assess certain characteristics of the product or service to arrive at an objective rating number
You’ll notice most ratings sites don’t have much concept of statistics or statistical significance and virtually every physician rating site fails this simple set of minimal requirements for a rating site to have even minimal face validity.
An example of a rating site that meets most (but still not all) of the above requirements is Amazon.com. Because of the sheer number of consumers who use the site to buy stuff from, virtually every popular product has a significant number of reviews. Because it attracts such large numbers, it is likely that it is a fairly randomized sample (as randomized as one can hope for online). And virtually all popular products carry both positive and negative reviews. The one thing Amazon.com doesn’t do well is trying to rate products objectively, instead relying mainly upon a well-written narrative by the reviewer.
So it doesn’t really matter what a physician rating site’s business model looks like in terms of whether it will succeed or fail. Most user-generated content sites online today have an unrealistic, non-existent or unachievable business model that goes something like this (with apologies to South Park’s underwear gnomes):
- Gather together a massive amount of users and get them to generate content for our for-profit, commercial site
- ???
- Profit!
It matters only if under Step 1, these physician ratings sites can generate enough ratings to make their data valuable and worthwhile to most users who visit their site. Because if they only satisfy 1 or 2 percent of their users (or present users with inaccurate, invalid data based 2 or 3 user ratings), people will simply not revisit the site in the future. And more importantly, they’ll fail to recommend the site to others (where most successful social networking sites succeed).
There are other validity problems with the data generated by these sites, which I’ve covered previously last year. None of these problems have been solved, and the fact remains that rating a person on their entire livelihood based on your usually brief interaction with them once is rife with concerns.
Given’s report is a valuable addition to the e-Health landscape, but it also has a few flaws. I think that with a good editor, the report could’ve been half as long and twice as good. And frustratingly, sometimes I found the author seemingly contradicting herself. For instance, on page 14 she writes,
Search ads are indeed cheap. The MD rating websites using them to promote themselves appreciate this. But their low cost means these ads are not particularly lucrative for the sites displaying them.
And then in the next paragraph, she notes the premium that pharmaceutical advertising garners — the main source of revenue for virtually any consumer-focused medical or health website:
One source I came across indicated that pharmaceutical display ads typically pay between $50 and $100 per 1,000 page views, while other advertisers pay at most $15. Unfortunately, we haven’t seen many drug ads on these sites thus far.
Which is fine as far as an observation goes while randomly looking at a particular site a few times over a 2 or 3 month period. But it’s not really contributing to the understanding of the underlying business model in this space unless you’ve done an actual, objective analysis (or simply asked them in the survey conducted, which wasn’t done).
And then some opinions are just in contradiction to other professional forecasts and data:
And the opportunity for profiting from generous pharmaceutical industry spending may decline significantly in the future as more brand name drugs go off patent, since brand competition is what drives advertising outlays.
Yet eMarketer, which tracks trends such as online pharmaceutical and health spending, actually reports that 2008 ad spending will again rise by approx. 20% from 2007 (when it also rose about 19%), and is expected to grow another 20 to 25% in 2009. This objective data doesn’t jive with such opinion.
Overall, however, I found the report a must-read for anyone who wants to understand how these kinds of sites are doing and whether they provide any real value to e-Patients and other consumers. I have my doubts as they exist today, because they still lack the volumes necessary to make their data of value (and valid).
Health Grades and other similar sites rank high on Search engines such as Google. Enter a providers name, and put MD after it, and 9/10 times Healthgrades is in the first two ranked.
When one goes to Health Grades you will find some provider listings and details that are ‘free’.
Obviously these providers ‘buy their ratings (grades(
Others require you to buy the details for 12.95.
Organized medicine such as the AMA and the ABMS ought to be able to have a public data base that offers up credible and accurate information. No one should have to buy this type of information.
Other site. locate a doc. offers to track your referrals (for a fee)
No we have not attained true transparency, yet
Worse is that the gullible patient or consumer knows not what or what not he is obtaining.
Profiteering is what I call it, and any provider who lends his name to this type of endeavour only makes it worse for all of us in the long run.
Now you have to be ‘credentialled by the internet, insurers, CAQH, hospitals, state licensing boards, etc. Where will it end?
Besides everything that John wrote, none of the existing services are of ANY help for the millions of Americans who are experiencing the long tail of medicine.
Nothing can come close to the social search effect of specialized online communities for rare diseases.
The fact that for a very large percentage of people suffering from cancer these websites are unable to provide ANY information or only incorrect data represents, IMO, a real public health issue. At a minimum these sites should be required to explain clearly what are the strict limits where they can be of use.
I would just like to correct the impression that the largest MD ratings sites now have only 10,000 – 20,000 ratings. RateMDs has over 600,000 ratings (all health care providers, but mostly MDs) in the US and Canada. Per Scott Shapiro, Healthgrades has about 600,000 (US only I assume.) That said, the volume of consumer generated opinions and scores are not nearly has high as they need to be to create true value for consumers, at least IMO.
Also, I appreciate the comments on the more favorable future prospects for online DTC prescription drug advertising now and in the future. This is exactly the type of feedback I was hoping to get by posting this analysis.
It’s a start, but look at the RateMDs website — it doesn’t even have any information about the company (no “About Us” that I could find) nor any real-world contact information. Really, consumers are going to feel like this is a trustworthy service worth their time? (I did finally find something buried in their “FAQ” about who runs it.)
HealthGrades has been doing this for 9 years and apparently doctor ratings from at least 2005. They trumpeted in a Nov. 2005 press release they had in-depth ratings on 600,000 physicians. Given that number has seemingly not moved much in 3 years, I’m not sure where that leaves us.
If you get 12 ratings on a doctor who has 2,000 patients, you’re at a whopping 0.6% sample. You would need, at minimum, 4,200,000 ratings to get 12 ratings on a simple majority of physicians in the U.S.
So when one of these ratings sites gets over 4,000,000 ratings, wake me up. Until then, they are providing something even worse than ratings — they are actively promoting mis-information about doctors based upon simplistic and inaccurate statistical models.
That’s the real story here.
I’d also hazard to guess, based upon my experiences in the e-Health care industry (sorry, call me cynical), that many of these sites don’t really care about their long-term goals. Their only purpose is to grow their database as quickly as possible to get to a M&A event. That’s how they plan on making money (or at least for the executives running the company).
Another issue in the development of the doctor-rating sites is that many people don’t have a wide range of choices when it comes to their providers AND once one is chosen, most people won’t need the service again for years. It’s an example of the point Stead Burwell made to me when we met at Connected Health in Boston: health is a topic that many people engage with (80% of internet users) but few on a daily basis (7% on a typical day).
The Pew Internet Project’s surveys have found (in 2004 and 2006) that 29% of internet users have looked online for info about a doctor or hospital. Our upcoming survey will separate those two and get a clean read on how many internet users are looking up doctors. The measure will be more of a “have you ever” question, though, not a frequency question.
The only site I found that has ratings for every physician 850,000 in the U.S and Canada is mdnationwide.org. I ordered two reports and they each had the doctors ratings, and a very detailed report compared to some other sites I visited.
The ratings were based on a proprietary system they designed, with the implementation of their methodologies.
Healthgrades and choicetrust are other choices consumers have regarding a doctors rating, or researching their doctors.
Good luck.
Karen
Informative article I was not aware of MD rating sites. Being in the industry I never really needed to look up doctors but I can really see how this could be a problem. Thanks for the info.
I agree that these rating sites are pretty much worthless. Part of the problem is that these businesses are taking a global approach to a local need. If someone were to focus on only one city at a time, develop a true database of docs in that city and develop their brand and expand from there, it could work.
Reality is that I am less concerned with the publics opinion about a doctor’s bedside manner than I am concerned that the doctor is properly credentialed, hasn’t had any complaints with the authorities, is who they say they are, and can be trusted.
The fact is, there are hundreds of patients of Gerald Barnes that would say he was a great doctor, despite the fact that the guy isn’t a doctor and has spent more time in prison than doctoring.
One final note on that theme is a story I heard from a doctor friend of mine. He said that he has a doctor working in his clinic that has done 3 years in prison. The governor’s husband was coming in to see him about a procedure. My friend’s comment was that you would think the Governor would have access to information to determine whether the doc working on her husband was a felon or not.
I think that says it all.
Ruth Given has given a fair assessment of the ratings site. Vitals.com is proud to earn the highest score in the report. Our data is the most robust with over 60 pieces of data available on a large segment of doctors. Vitals goes way beyond rankings and reviews to provide awards, sanctions, procedures, insurance, special expertise and the list goes on and on. Since the analysis, our data is even better and we wish Ruth Given would present a new analysis report to show the full power of Vitals.
I think that Dr.Given is, if anything, too kind in her evaluation. Many of the sites are difficult to navigate, many provide little more information than can be found in the yellow pages. Even one of the better ones, Vitals, provides the detailed information (e.g. patient evaluations) on only very few of the listed doctors. Overall, it is a deplorable situation.
Greetings : )
Vitals.com is an edited, physician pleaser. This site edits out what the djoc wants. The site also will only allow what the doc wants the public to see. That in my book is not beneficial but deceitful and misleading to the public. What’s the point of having a doctor ratings review site if the patient/public can’t leave reviews so that the public can see them and then go from there. DR. Anthony Leazzo a D.O. out of Plainfield Illinois is one doc is particular who has abused this system. He has posted his own reviews on vitals.edited out all and any other reviews and all you can see is :”It was tough, but I found him” It’s obvious that this was put in by the doc himself Hence you can’t see anything else but this post. WEll here’s the deal with this dude.He’s a wanna -be sugeon without the surgical license. He has a woman at the fjront desk(clerical) who is taking vitals on patients. He has an untrained medical assistant giving injections,assisting him in his :”Procedures” which is probably not a good seeing he isn’t a suurgeon. He has made life/death blunders with meds and he just isn’t knowledgeable nor does he treat patients with dignity. Example : Call in to be seen (feeling sick etc). The day comes, you wait, wait, wait and wait in waiting room, and then after 2hour wait his untrained staff take your vitals, he walk s in and SITS DOWN. Yep he doesn’t get up. He does not examine you . He does not listen to your heart, your lungs or examine you. This is a poor example of what a doctor is. This person wants the glory, the title but unwilling to be a doctor who actually examines his patients. No other words other than he is an attrocity to this field.
Hello anon. Newsflash, This docotor is not unique. He’s banking on ignorance and what better way to get rich quick when you have a minimal overhead for stff. He sounds like a real con to me. Hopefully you get the word out and he will not hurt anyone. We have had a few of those too. I agree about the sites like vitals. It’s a joke because they edit all the stuff that will show the truth and only keep in what the doc puts in. I don’t go to vitals just for that reason. It’s not a site that is helpful to the public but only serves the sef-serving purposes of these crummy docs. One has to wonder what the beep is wrong with the country when some dr. is running around with all of these false crudentials and getting away with it. Come on people! I’ll never go to an Anthony Leazzo type. Thanks anon!
i have different ideology i think its the best practice for doctors to patient to make sure the eligibility and the points of doctors to be scored.
We have a function on our site where doctors rate other doctors. This should take a lot of pressure off the doctor and allow them to focus on their practice. Peer reviews go a long way.
My biggest issue with these 3rd party marketing sites (really, that’s all they are) is that they hog all the valuable search result space and push the websites of legitimate physician practices to page 5 or 9 or 20. (They don’t teach SEO and SEM in med school.)
Transparency is a huge problem with these so-called medical sites – and I found another one here. I read CareRally’s post (they didn’t post with 1st or last name) and was intrigued. Immediately checked them out. There was no owner info, address, or phone number on their website. Their “About Us” was vague and nameless: “…was founded in 2011 by consumer advocates, health insurance industry veterans, physicians, internet innovators and other professionals passionate about leveraging the power of the internet…” On the “Denver” page a bold headline reads: “Recent Care Tips in Denver” yet none of the tips are from Denver or Colorado physicians. In fact, I recognized the 3rd “tip” because it was my kids’ pediatrician when we lived in Pasadena, CA. The “tip” was a vaccination pamphlet obviously aggregated from another site (possibly from the owner’s other dental/medical site). Not only was there no real info or link to the peds practice site, but a disclaimer read: “This practice is not yet Verified – please check back soon.” Huh? Another example of confusing the consumer/patient – and using a physician’s name to market a “product” without their consent.
Hi Susan –
I personally have a mixture of thoughts on your comment.
I certainly agree that transparency is a big issue – specifically who publishes it. That’s my first go-to question on any source.
An ironic aspect is that even well credentialed sources are wrong far more than I’d expect. (On MedHelp.org, the info about the treatment I got 5 years ago is still wrong, even though it was approved by a board-certified physician.) But that doesn’t change the importance of the issue.
I tend to disagree, though, that a ratings site is nothing but third party marketing. Today’s incarnations of the idea may tend in that direction, because there’s no other income to run them (I’m speculating). But I remember decades ago where skeptics in the auto and retail electronics industries said Consumer Reports was a scam, they were on the take, etc.
To me (as a non-medical person, just a mediocre business person) one of the conspicuous differentiating factors about medicine is that consumers have virtually no info about what their options are regarding cost, quality, and customer satisfaction; in fact consumers who ask such questions are sometimes insulted for asking, or treated as if they were being offensive.
Having said that, I agree again (strongly) that as with all online info, ya gotta be careful about who’s publishing it. But I wouldn’t reject the whole idea – we just need to improve it.
And here’s hoping some big big foundation will support creation of a truly independent ratings resource, NOT controlled by medical professionals, because sometimes the establishment is wrong.
Flappy Bird Hack v1.4
Download link: http://bit.ly/1lPcXGs
Instructions:
1. Download and install.
2. Connect the mobile device using USB cable.
3. Selectt the platform (iOS/Android) annd click on “Apply cheats” button.
4. Play the game.