Summary:<\/em><\/strong> High-quality product reviews will be an important part of this new journal, with its focus on supporting and encouraging people to participate in their own health care. But how should we go about evaluating various interactive applications and devices that bill themselves as “health tools”? Health design advocate Amy Tenderich of DiabetesMine.com explores the definition of a “health tool,” and lays out parameters for a new kind of participatory product review process.<\/p>\n Keywords: <\/em><\/strong>Participatory medicine, medical devices, self-care, health tools, Health 2.0, patient networks, health social media, health design, ePatients, product reviews, patient advocacy.<\/p>\n Citation: <\/em><\/strong>Tenderich A. Reviewing health tools: a community matter. J Participat Med. 2009(Oct);1(1):e9.<\/p>\n Published: <\/em><\/strong>October 21, 2009.<\/p>\n Competing Interests: <\/em><\/strong>The author is working with startup company Keas Inc. to introduce a new set of online health tools beginning in November 2009.<\/p>\n The Journal of Participatory Medicine<\/em> (JoPM) is about people taking a more active role in managing their own health conditions. That’s not to suggest supporting the concept of unaided self-medication; we still need doctors. But a new era of technology-based tools is enabling patients to care and advocate for themselves, and to research their own medical conditions in ways never before imaginable.<\/p>\n For example, self-testing and monitoring devices have revolutionized the daily existence of people living with diabetes. Programs for online health education and record-keeping are teaching people about self-care, and giving them new confidence to take a proactive role in their own treatment. Every day, networks for interacting with others online are getting better at affording patients the opportunity to collectively communicate with medical providers and with the drug industry that caters to them.<\/p>\n Without these evolving communication-enabling tools for individual patients, “participatory medicine” would be quite a vacant term indeed. But, in all fairness, not every tool is a useful one. Thus, publishing high-quality product reviews will be an important part of this new journal, with its focus on supporting and encouraging people to participate in their own health care.<\/p>\n Scores of people\u2014health professionals and patients alike\u2014are struggling to identify which of these “tools” really matter. Which ideas change lives? Which ones have staying power? Which ones require too much manual data entry? Which ones are simply faulty?<\/p>\n As many of my colleagues know, I’m a huge proponent of this great wave of new health applications and personal medical gadgets. I’ve been fortunate to review many, and have campaigned for greater innovation in diabetes tools through the 2009 DiabetesMine\u2122 Design Challenge<\/a>, on my blog, Diabetes Mine<\/a>. Once upon a time, I personally reviewed technology products for corporations; now, living with chronic illness, I find I have a much grittier perspective on the promises of new products. How then should we go about evaluating various interactive applications and devices that bill themselves as “health tools”?<\/p>\n This is a health journal, not an engineering publication. Therefore, I believe the focus should consistently be on whether\u2014and how\u2014the tool in question can improve a person’s health<\/em> or change that person’s life for the better<\/em>. This should be our basis for evaluation.<\/p>\n The next question we should ask is whether the so-called “tool” is something really useful in a person’s daily life, or does it simply make the user’s life more complicated by demanding an undue burden of time and attention? In other words, was it designed with the user (patient) in mind, or was it assembled by technology experts and other \u201cspecialists\u201d with a more abstract interpretation of what they believe <\/em>patients want or need?<\/p>\n At recent conferences, I’ve seen demonstrated a number of PC-based “decision support programs” and other “solutions” that require a huge amount of data entry, but don’t offer much in terms of value coming out the other end. For example, one online “personal health management” program that enables users to track their health by using eight different “tabs,” namely, information (pharmacy and insurance details), food, exercise, body measurements, chemistry (blood and urine tests), mood ratings, medical history, and a graphs page that “makes it easy to… look for trends and view your progress.” Clearly, a user would have to spend many long hours painstakingly inputting data here\u2014whereas the only output from this \u201ctool\u201d is pie charts and other fancy representations of the data that has been tracked.<\/p>\n Likewise, another example: why would a person with diabetes spend hours entering glucose values and food choices if the “tool” fails to do anything actionable with this information?<\/p>\n In a more distressing example, a recent list of “5 online health tools” promoted a website called BodyBio.com<\/a>. Here, visitors are encouraged to “simply get a blood test from your doctor, go to the website, and enter the data from the blood test results sheet.” After which, the user can obtain “a personalized printout that details the nutrients you’re getting and those you need.” And what can users do with this information? The answer: “You can talk to your doctor about the results\u00a0and the best ways to\u00a0improve your vitamin and mineral intake.” Meanwhile, the website very prominently sells a range of nutritional supplement products, from ammonium chloride to sunflower oil. Clearly, this is not an actual tool, but a marketing scheme. This is a pretty banal example, but it certainly illustrates the point.<\/p>\n Whether it’s a weight loss aid aimed at improving eating habits, or a beeper that reminds elderly patients to take their blood pressure meds, the best “health tools” are interactive, mobile, and provide the user some tangible value. Without these attributes, it\u2019s a non-tool. And the need remains to expose the opportunistic imposters who build a website, populate it with some basic health information, use it for some entrepreneurial purpose, and try to pass it off as a “tool.”<\/p>\n Two well-known major “sins” of technology reviews are (1) reviewing the product in a lab environment (synthetic) rather than a “real-world” setting, and (2) overlooking the fact that the products themselves were built without the end-user in mind.<\/p>\n In fact, reviews of computing equipment are notorious for focusing on benchmarks, and leaving out the practical question of how the tool would perform in the real world.<\/p>\n The lack of standardized, widely accepted performance benchmarks for online health tools will make objective measuring difficult. Therefore, our reviews will, of necessity, be subjective evaluations by the people doing the testing.<\/p>\n But this is quite appropriate, given that “participatory” medicine focuses on the user experience\u2014subjectivity is the point! One might argue that objectivity does not exist in social media; the good information is that which is best-liked by the most people, and therefore bubbles to the top.<\/p>\n Thus, we should look at a review not as the definitive word on the usefulness of any given product, but rather as the start of a conversation. The reviewer inevitably voices an opinion, and others are invited to chime in.<\/p>\n The widely venerated 1999 publication, The Cluetrain Manifesto<\/em><\/a> by Rick Levine and colleagues, brought to light the concept that “markets are conversations”; which is to say that conversation<\/em> is the essential element needed to reach and serve clients and other stakeholders. According to the authors, the participants in a market (community or industry) “communicate in language that is natural, open, honest, direct, funny, and often shocking. Whether explaining or complaining, joking or serious, the human voice is unmistakably genuine. It can’t be faked. [1<\/a>]<\/sup>”<\/p>\n When we publish a review in the JoPM, our goal will be to initiate a community review process in which many others will have an opportunity to offer comment, feedback, and dissenting opinion. Moreover, similar to many health blogs, when rich discussion occurs, our reviewer can later summarize that discussion in a future article. So the reviewer becomes the facilitator of a conversation.<\/p>\n Naturally, some guidelines and rules of engagement will be necessary to make the process a success.<\/p>\n First, the reviewer should be mindful of the concepts introduced above: namely, that we are not<\/em> looking for quantitative “lab” reviews conducted in a sterile environment that overemphasizes the underlying technology. Rather, we seek to reiterate the question, “Does this product recognize the true needs and real life context of the people intended to use it?”<\/p>\n As for the nuts and bolts, I personally would like to see reviews that offer a mix of three elements:<\/p>\n Drilling down, here are some key criteria for the reviewer to explore:<\/p>\n For participants in the resulting conversation, some basic ground rules are in order:<\/p>\nWhat Makes a Health Tool?<\/h3>\n
Getting Real<\/h3>\n
Reviews as Conversations<\/h3>\n
Reviewer Guidelines<\/h3>\n
\n
\n
\n
Participant Guidelines<\/h3>\n