Medical Device Link.

Originally Published MX January/February 2004

INFORMATION TECHNOLOGIES

A method of analyzing scientific and clinical literature can help manufacturers develop messages that support product marketing.

Lawrence E. Liberti and Sarah Conaway

Almost 30,000 scientific journals are published, giving researchers ample opportunity to communicate not only the results of their scientific studies but also their personal practical experiences with, and opinions regarding, the state of the art of a particular technology or therapy. The most prolific authors appearing in these publications include those affiliated with academic institutions, government research organizations, and the healthcare industry. Although the information published in medical journals is widely diverse, each article communicates the author's thoughts to a targeted readership. Presentations may be compelling enough to convince leaders in the field to change standards of practice. Or, authors may intend to disseminate key information about a compound, device, or treatment approach to a general audience. In some cases they use the publication process to establish a repository of knowledge about a topic.

Medtech manufacturers can, and should, take advantage of this publication phenomenon. It is important that they be able to properly assess the value of this published knowledge. To position their products positively and to understand whether this objective has been achieved, they need to rate the utility of the information sources (that is, the publications), monitor the clinical information being published, and determine the value of the knowledge record. Their objective should be to gain a correct understanding of how the manufacturer's product or technology is perceived by its potential end-user.

Creating a Product Position

The medtech industry uses publication in professional journals (and other technical media) to attain a variety of goals, including:

• Creating marketing value from information derived from clinical studies.
• Educating users on the pros and cons of using a technology.
• Providing balanced information.
• Influencing users' product selection tendencies.
• Ensuring consistency in the market positioning, or messaging, of a product or technology.
• Highlighting a product's benefits.
• Targeting specific audiences.

All medtech companies should strive to disseminate product or technology information in a practical, unbiased fashion that reflects the quality of the scientific method. Publications should provide sufficient data to allow readers to draw their own conclusions regarding the quality of the information, the value of the findings, and the merit of the presentation. All of the clinical information published on a particular healthcare method or device in the aggregate influences opinions, attitudes, and, ultimately, treatment practices.

With respect to a health professional attempting to assimilate the wealth of knowledge available on the subject, this information may result in any of the following:

• Positive positioning, where the reader sees valid scientific evidence that leaves an overall positive impression of the product or technology.
• Negative positioning, caused by the presentation of facts strongly detrimental to the case for the product's utility.
• Zero-positioning, where an inadequate quantity of valid information is available to generate significant awareness of the technology.

Positive positioning based on the publication of accurate, well-balanced, and thoughtfully evaluated information can lead to increased awareness and use of a medtech company's product or technology. Negative positioning results when problems with a product arise, such as significant safety issues, high costs, or patient-related obstacles. But perhaps more frustrating is when a paucity of published data means little or no awareness among target audiences. Zero-positioning often results from poor publication strategy planning. This critical component of the research process is too often overlooked or undervalued.

A company developing a new technology for noninvasive red blood cell monitoring, for example, first needs to stimulate professional awareness of its technology. By disseminating solid, promising information about the monitor and its advantages over other methods of taking red blood counts, the manufacturer produces positive positioning of its device. The company must target its presentation at the proper audience for the technology and repeat consistent, positive messages.

Attracting Targeted Readers

Positive product positioning is readily achieved by medtech company leaders who recognize that reading journals is a primary way that clinicians and researchers keep up-to-date with advances in their field. Recent surveys of doctors revealed that internists dedicated an average of 4.4 hours per week and surgeons 14 hours a month to reading journals.^1,2 This is often insufficient time for remaining fully informed of key advances, especially given the finding that physicians read only the abstract of about two-thirds of the articles they find of interest (see sidebar 1, page 56).¹

Under these circumstances, the difficulty for clinicians is to identify the information most relevant to their practice of medicine and to keep track of it usefully. The challenge for technology providers is to understand, quantitatively as well as qualitatively, what effect on clinicians' perception of a product information presented in the published literature is having, particularly whether it is positively, negatively, or zero-positioning that product. Medtech executives will find that being able to determine the value of scientific publications is critical to properly assessing a product's positioning.

Given time and logistical constraints, a clinician or researcher cannot possibly read all of the published information about a certain product or therapeutic area. The professional has to ignore some of the extraneous information in order to concentrate on the most relevant and reliable material. Several studies have demonstrated how this process of selection can be managed effectively by busy readers.^3–5 Their results also offer guidance to medtech manufacturers wishing to disseminate information about their new products through publication of clinical reports.

One study showed that clinical trials whose reports were published in languages other than English were less likely to identify clear primary outcomes, include a placebo group, or justify sample-size estimation.⁴ Another found that excluding non-English articles in performing a metaanalysis did not introduce errors into the analysis or compromise its validity.⁵ The most valuable scientific publications usually turn up in systematic reviews of English-language literature that can be accessed through major databases.

Research or clinical study reports must be accessible to be effective. Publications that are hard for trained searchers to find are nearly impossible for clinicians to locate. Articles appearing in obscure journals cannot have a great influence on the positioning of a product if only a few target readers know they exist. And in most therapeutic areas other than HIV/AIDS and oncology, studies that go unpublished or appear only as abstracts have less of an impact than published data.⁵ In addition, those publications that have limited circulation are often of lower quality and are more likely to introduce bias due to miscommunication and misinterpretation of the data than to prevent it through careful editing and peer review.

The investigations also discovered that studies conducted using an open-label design are more likely to exhibit highly positive results and are often poorly designed.

The manufacturer with the new noninvasive red blood cell monitoring technology should bear these findings in mind when choosing journals through which to disseminate knowledge of its product. The company should concentrate on English-language literature and on literature that its target audiences can easily obtain and understand. This will help it to acquire the important understanding of how target audiences are likely to interpret all the published information about the new technology.

Rating Publication Quality

Guidelines for ensuring the desired quality of scientific articles are available.^6,7 Rating checklists are particularly useful for critiquing the value of these publications. Many people have developed and studied that method of quality assessment.^8–15

Developing a rating checklist is important for giving the assessment process a basic consistency, for validating methodologies, and for ensuring a credible interrater reliability—consistency among different scorers of a study report—that can be duplicated. A rating checklist can be used to evaluate a variety of study designs. Well-designed checklists can reflect how a typical healthcare professional will be inclined to interpret a publication.¹⁶

Even well-designed checklists have limitations, however, because each checklist uses slightly different assessment parameters. Interrater reliability may be difficult to determine with some checklists. The lack of a gold standard in this area makes demonstration of interrater reliability difficult, leaving this statistic virtually unreported.¹⁰

Objective data interpretations that do not vary from reader to reader are often difficult to achieve, especially among readers with different levels of expertise. A study comparing the rating skills of general journal subscribers, peer reviewers, and specialized experts observed a wide variability in rating; 84% of general subscribers, 79% of peer reviewers, and 61% of experts rated a group of manuscripts in the experts' therapeutic area as acceptable.¹⁷ Several attempts have been made to create rating systems that minimize this variability.

The READER method (an acronym standing for relevance, education, applicability, discrimination, and overall evaluation) is a simple, consistent technique of evaluating articles. It was designed to help general practitioners appraise and assimilate information from scientific publications. This method succeeds in enabling evaluators to perform a more rigorous analysis of clinical methodology, and has been shown to be both accurate and replicable.¹⁸

Getting the Message

Scientific publications are based on data, but besides the recital of facts and findings they deliver key take-away messages that ultimately leave the reader with an impression of the technology studied. A message can be defined as an important concept or theme intended to be memorable. It may be highly related to the data, well supported by the logic of the presentation, expressive of an author's opinion, or emphatic about the novelty of something. Messages are placed in articles to educate readers and influence their opinions about a product or technology. They can illuminate various aspects of a product, including safety, efficacy, and ease of use.

Figure 1. Factorial analysis showing that well-supported messages relating to important issues have the greatest effect on attitudes.19 (click to enlarge)

A medical technology provider that can assess the value, or strength, of a message in an article about its product can know how much of an impact, positive or negative, the message will have on positioning that product. Message identification and strength assessment are important because, in the case of a topic of high interest, the presence of a strong message creates a high probability of the publication changing attitudes (see Figure 1). The strength of a message can be related to and influenced by the amount and quality of supportive data, its relevance to current research, or its clarity of presentation.¹⁹

Messages about the new noninvasive blood-monitoring technology—that it is reliably accurate, that results are obtained quickly, that patients regard it with excitement—are the elements from which a positive product position is built. They have a greater positive impact on general understanding when they are supported by data. Messages supported by data certainly are stronger than those based on speculation. Also, article readers tend to remember clear, consistent messages that are repeated several times more than they do many different messages conveyed only once.

The medtech company should identify the set of messages it is interested in disseminating to the prospective users of its technology. Then it should accumulate sound data to support the messages and place the reinforced messages in scientific journals that can be obtained easily by the target audience.

Message Mapping

Message mapping takes message assessment to a high level of analytical sophistication (see sidebar 2, page 58). This system of rating the value of publications appearing in the scientific literature is a dynamic process that identifies key published messages about a product. It is an objective technique for identifying, categorizing, and analyzing essential concepts found in articles, abstracts, monographs, and other communication vehicles. Message mapping minimizes subjective misinterpretation and can help make the importance of the published data clearer to the reader. It is designed to provide an interpretation of the message presentation as perceived by the technology users encountering it in the literature.

The objective tool also detects changes in the quality and scope of information about a product over time. It helps assess the landscape of information about competing products as a support to the process of developing a new product or technology. When these functions incorporate the perspective of healthcare professionals, the consumers of the literature, message mapping is a means of interpreting how real-time trends in messaging could be influencing them.

Message mapping can be used to plan and monitor a medtech business's strategic approach to market analysis, clinical development, and marketing program management. Providing a central knowledge base of information published about a product, the process maintains message consistency, verifies that it does, and determines whether messages reach specific target audiences. It monitors the effects of new communication initiatives, helping to determine whether these were helpful or harmful. Most important, message mapping closes the gap between development and marketing groups working on the same product.

Message mapping can also be used by a manufacturer to monitor competitor activity and to identify strategies used in research and marketing by competitors. It can identify the strengths and weaknesses of competing products, and determine the nature of the information that competitors are using to disparage the manufacturer's own product.

The medical technology company developing the noninvasive red blood cell monitor could, then, use message mapping to assess its product's positioning objectively in relation to that of competitive technology. It could use the knowledge gained from message mapping before the monitor was fully developed to strategically analyze the market and plan clinical development approaches in response to trends in the technology area. Message mapping could serve to support and monitor all marketing activities involving the developing product.

Assessing Message Value

Figure 2. Messages in articles are often poorly supported by the underlying data. This chart of an analysis shows that 70% of nearly 6500 rated messages were found to have a strength score of 20–70%. (click to enlarge)

The presentation in the professional literature of strong messages backed by supportive data can be critical to successful product development and marketing. An analysis of nearly 6500 mapped messages across 10 therapeutic areas found that 70% of these messages had a strength score of 70% or below (see Figure 2). The objective proprietary scoring system took into account numerous aspects of the data source, including the type of article or data presentation, the specific data associated with the message, and the clarity of the message. The analysis suggests that data often do not support messages in the literature well and that manufacturers should try to improve information presentation so as to create a compelling story and project a sense of clinical relevancy.

Figure 3. Message strength versus age in years versus number of citations (all). A bubble chart displays message quality in terms of three variables and provides an at-a-glance perspective of the likely influence of published information about a product on that product's target audience. (click to enlarge)

The value of a message for product positioning can be evaluated by means of many different tools, scoring systems, and analytical procedures. One analysis quantified three variables to assess each message pertaining to a product as well as its relationship to other messages. The result was an information-rich, easily comprehended bubble chart (see Figure 3). The bubbles display:

• The message quantity—that is, how many times the message has been repeated in the literature—shown by the size of the bubble.
• The message quality, signified by the bubble's center on the vertical axis; higher message strength means greater influence on clinical practice.
• The age of the message, where a newer message, appearing toward the left side of the horizontal axis, may be presumed to reflect recent advances in the knowledge base.

Messages that are repeated often (large bubbles), supported by strong data (positioned high in the graph), and recently published (close to the vertical axis)—strong messages—are likely to play the greatest role in positioning a product, as indicated in Figure 1. Bubble chart analysis also makes apparent which messages should be strengthened, renewed, or contradicted.

Figure 4. The product credibility index gives an indication of product positioning success in terms of how clinicians encountering messages in scientific literature perceive a product's value in relation to that of a competing product. (click to enlarge)

The analysis can be enhanced by aggregating and weighting the strength scores of a product's messages, both positive and negative, to arrive at a product credibility index. This score suggests the overall impression made on a typical reader by the body of literature on the product. The index can be used to assess the positioning strength of a product in comparison with that of an analyzed competing product (see Figure 4). In the hypothetical case depicted in the figure, the manufacturer of the noninvasive red blood count monitor can take satisfaction that the presentation of information about its product should have a greater positive impact on healthcare professionals than that pertaining to its competitor's invasive monitor. The credibility index, compounded of many message scores, tells, among other things, that negative messages about the noninvasive monitor that may have been issued by competitors have done little to diminish the strength of the positive messages about the product, and certainly have had no detrimental impact on its relative positioning.

Evaluating message strength in terms of particular audiences is also important in developing positive positioning for a product. By targeting product information at specific audiences, a company can design cost-effective marketing communication programs.

Conclusion

Well-supported communication plans can strengthen the research and marketing programs of a medtech company. Message analysis applied to the published literature about a particular product or technology provides a dynamic assessment of what the target market of technology users perceives as its strengths and weaknesses, based primarily on the presentation of supportive scientific data. Using an appropriate analytical system to measure the success of the messaging effort enables the company to optimize the positioning of its technology.

This type of real-time clinical knowledge assessment assists in designing new studies to address perceived weaknesses in a product or technology. Findings from these studies can then buttress the messages used to position and market the product, as well as inspire new ideas for research. Messages can be disseminated through various types of media, including symposia and continuing medical education programs, as well as journal articles.

A number of methods are available to analyze and display this information. Message mapping is one technique that integrates different types of information and displays the analyzed data in practical formats. This tool facilitates the development and distribution of strong product messages, helping to keep the focus on issues that are the most important for product development and market positioning.

References

1. S Saint, "Journal Reading Habits of Internists," Journal of General Internal Medicine 15 (2002): 881–884.

2. M Schein et al., "What American Surgeons Read: A Survey of a Thousand Fellows of the American College of Surgeons," Current Surgery 57 (2000): 252–258.

3. Egger et al., "How Important Are Comprehensive Literature Searches and the Assessment of Trial Quality in Systemic Reviews? Empirical Study," Health Technology Assessment 7 (2003): 1–76.

4.D Moher et al., "Completeness of Reporting Trials Published in Languages Other Than English: Implication for Conduct and Reporting of Systematic Reviews," Lancet 347 (1996): 363–366.

5. D Moher et al., "What Contributions Do Languages Other Than English Make on the Results of Meta-Analyses?" Journal of Clinical Epidemiology 53 (2000): 964–972.

6. D Moher et al., "The CONSORT Statement: Revised Recommendations for Improving the Quality of Reports of Parallel-Group Randomized Trials," Journal of the American Medical Association 285, no. 15 (2001): 1987–1991.

7. The Asilomar Working Group, "Checklist of Information for Inclusion in Reports of Clinical Trials," Annals of Internal Medicine 124, no. 8 (1996): 741–743.

8. S West et al., Systems to Rate the Strength of Scientific Evidence, Evidence Report/Technology Assessment no. 47, publication no. 02-E016 (Rockville, MD: Agency for Healthcare Research and Quality, 2002).

9. SH Downs et al., "The Feasibility of Creating a Checklist for the Assessment of the Methodological Quality Both of Randomized and Non-Randomized Studies of Healthcare Interventions," Journal of Epidemiology and Community Health 52 (1998): 377–384.

10. MK Cho et al., "Instruments for Assessing the Quality of Drug Studies Published in the Medical Literature," Journal of the American Medical Association 272, no. 2 (1994): 101–104.

11. JS Reisch et al., "Aid to the Evaluation of Therapeutic Studies," Pediatrics 84, no. 5 (1989): 815–827.

12. KF Schulz et al., "Assessing the Quality of Randomization from Reports of Controlled Trials Published in Obstetrics and Gynecology Journals," Journal of the American Medical Association 272, no. 2 (1994): 125–128.

13. TC Chalmers et al., "A Method for Assessing the Quality of a Randomized Control Trial," Controlled Clinical Trials 2 (1981): 31–49.

14. A Liberati et al., "A Quality Assessment of Randomized Control Trials of Primary Treatment of Breast Cancer," Journal of Clinical Oncology 4, no. 6 (1986): 942–951.

15. R Harbour et al., "A New System for Grading Recommendations in Evidence-Based Guidelines," British Medical Journal 323 (2001): 334–336.

16. JE Tyson et al., "An Evaluation of the Quality of Therapeutic Studies in Perinatal Medicine," The Journal of Pediatrics 102, no. 1 (1983): 10–13.

17. AC Justice et al., "Do Readers and Peer Reviewers Agree on Manuscript Quality?" Journal of the American Medical Association 272, no. 2 (1994): 117–119.

18. D MacAuley et al., "Randomised Controlled Trial of the READER Method of Critical Appraisal in General Practice," British Medical Journal 316 (1998): 1134–1137.

19. M Jordan, "Factorial ANOVA: Two Factor Designs" [on-line] (Florence, SC: Francis Marion University, n.d. [cited 4 August 2003]); available from Internet: www.fmarion.edu/mjord/helpfile/psy395/anova_2.htm.

Lawrence E. Liberti is executive vice president and Sarah Conaway is group leader for research and development at Astrolabe Analytica Inc. (Philadelphia).

Illustration by DIGITAL VISION

Copyright ©2004 MX