Are Advertisement Claims in Anaesthetic Journals based on High-Quality Evidence?

Christian Zedler

Join Us \| Latest Articles \| Contact

Journal Home

Editorial Board

International Journal of Anesthetics and Anesthesiology

DOI: 10.23937/2377-4630/3/1/1043

Are Advertisement Claims in Anaesthetic Journals based on High-Quality Evidence?

Christian Zedler¹, Alexander Schnabel², Peter Kranke² and Leopold Eberhart^3*

¹Department of Anaesthesiology and Intensive Care, University Halle-Wittenberg, Germany
²Department of Anaesthesiology, University of Würzburg, Germany
³Department of Anaesthesiology and Intensive Care Medicine, Philipps-University Marburg, Germany

^*Corresponding author: Leopold Eberhart, Department of Anaesthesiology and Intensive Care Medicine, Philipps-University Marburg, Baldingerstrasse, D-35033 Marburg, Germany, Tel: +49 6421 58-62945 Fax: +49 6421 58-65898, E-mail: eberhart@staff.uni-marburg.de
Int J Anesthetic Anesthesiol, IJAA-3-043, (Volume 3, Issue 1), Original Article; ISSN: 2377-4630
Received: January 17, 2016 | Accepted: February 24, 2016 | Published: February 26, 2016
Citation: Zedler C, Schnabel A, Kranke P, Eberhart L (2016) Are Advertisement Claims in Anaesthetic Journals based on High-Quality Evidence? Int J Anesthetic Anesthesiol 3:043. 10.23937/2377-4630/3/1/1043
Copyright: © 2016 Zedler C, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Abstract

Background: Many medical journals contain advertisements for pharmaceuticals products. While the WHO demands that claims in pharmaceutical advertisements should be based on scientific evidence, past investigations demonstrated that some advertisements fail to fulfill these demands. As there is currently no investigation dealing with advertisements in anaesthetic journals, we aimed to find out whether claims in these journals are accompanied by references to external evidence, and whether these claims are supported by the evidence cited.

Methods: We analysed all drug advertisements placed in four international and two German language anaesthetic journals published in the years 2000, 2004 and 2008, regarding presentation and occurrence of claims. For all claim-citing references to external evidence we tried to obtain these references, ranked their grade of evidence, and verified whether the claim was actually supported by evidence given. The analysis was primarily descriptive.

Results: We found 5453 advertisements for 1063 pharmaceuticals. The advertisements contained 3797 marketing claims and 2489 references, of which 64% referred to articles published in a journal listed in MEDLINE. In total, 17% of all claims were supported by a high-quality study or guideline. This portion rose from 9% in 2000, up to 37% in 2008. Overall, 49% of the claims were not accompanied by a reference to an external proof, and 6% referenced a study which did not support the claim.

Conclusions: These results imply that drug advertisements in anaesthetic journals are not yet sufficiently supported by high-quality studies, and medical professionals therefore need to critically appraise the claims in advertisements.

Keywords

Advertising as topic, Anesthesiology, Drug industry, Evidence-based medicine

Introduction

Many medical journals contain advertisements placed by pharmaceutical companies [1]. While they are an important source of funding for publishers [2], there is an ongoing discussion on how advertisements influence doctors' prescription behavior [3-5]. Advertisements may also interfere with the scientific independence of a journal [6].

Pharmaceutical advertisements do have some effect on doctors' prescription behavior [7,8], and not all of them are aware of this influence [9]. Supporters of pharmaceutical advertisements argue that they are a way to inform doctors about new products in a fast and direct fashion, and do not necessarily interfere with the need for balanced information [10,11].

Different authors as well as the WHO demand that claims in pharmaceutical advertisements should be based on scientific evidence [12-14]. Past investigations demonstrated that a number of advertisements fail to meet these demands [7,15-19], and ask journals to abstain from publishing pharmaceutical advertisements [20]. At present there is no investigation dealing with advertisements in journals specializing in anaesthesia. As drugs commonly used in anaesthesia are quite specific, results from other investigations might not apply to them.

The aim of this study was to evaluate the quality of pharmaceutical advertisements in anaesthesia journals published between 2000 and 2008, and to identify fields for improvement. Furthermore, the influence of time and origin of the scientific journal will be investigated.

Methods

Material

We selected the four international, and the two German-language anaesthetic journals with the highest impact factor in the year 2000, respectively. Details concerning these journals are shown in table 1.

Table 1: Characteristics of the investigated journals. View Table 1

In order to detect any trends in advertising practice, we included all volumes published in the years 2000, 2004 and 2008, and searched each regular issue of these three years for advertisements. Journals only dealing with subspecialties (for example intensive care), as well as supplements of the above mentioned journals were not included. If a page could not clearly identified as paid advertisement (for example pages containing congress announcements or special offers of the journal's publisher), we decided in accordance with the design of the page and the impression on an unbiased reader whether to include or to exclude it from our analysis of advertisement. We classified each advertisement according to product advertised. For pharmaceutical advertisements, we recorded the drug group and the size of the advertisement.

The evaluation of the advertisements followed a predefined protocol.

We searched for all promotional claims within each pharmaceutical advertisement. Phrases citing known medical facts without giving information about the promoted drug were classified as general knowledge statements, and were assessed separately. Text passages giving legally required information such as excerpts of the summary of product characteristics were not recorded.

We also did not further assess short slogans accompanying the drugs’ or the companies’ names.

Each promotional claim was classified as unambiguous clinical outcome, vague outcome, non-clinical outcome or emotive/immeasurable outcome. This classification has been used by other authors before [21,22].

In addition, for all pharmaceutical claims we categorized the kind of sales argument used [18,23]. The sales argument "emotional statement" corresponds with the outcome category "emotive/ immeasurable".

For all cited references we stated the kind of publication. We then attempted to obtain all cited references by searches via MEDLINE, general internet search engines, including Google Scholar or inter-library loan. We did not search for references to summary of product characteristics and other legally required documents, as they are less suitable to verify the advantages of a drug. Likewise we did not obtain any data on file, as we believe it is very unlikely that the average reader of an advertisement will contact the manufacturer in order to get an extract of the cited data. These circumstances were recorded with the corresponding references, and the references were subsequently marked as not available.

All references were classified by their level of evidence. Three different [15] levels were used for this process. Blinded, randomized-controlled trials, meta-analyses, incidence/prevalence studies, guidelines, and official publications were rated as high-quality evidence. Non-interventional studies, congress abstracts, and trials not blinded or not randomized were rated as low-quality evidence. All major medical journals are listed in MEDLINE; articles from journals not listed may be difficult to obtain for the regular reader of an advertisement. Therefore we rated any article published in a journal not listed in MEDLINE as low-quality evidence, regardless of the study type. References to monographs, non-systematic reviews, citations of the introduction of an article, letters and editorials were rated as expert opinion.

Evaluation

For each promotional claim accompanied by at least one reference we evaluated each reference separately. If some of the references were not available, we just assessed the available ones. In order to rate the claim as evidence-based or not evidence-based, we used a best-case scenario, [24] and rated it as evidence-based if at least one of the references substantiated the claim. Only if a reference of a higher level of evidence contradicted another substantiating reference of a lower level we considered the claim as not substantiated. For each claim we stated the level of evidence of the highest quality reference.

The primary assessment was performed by one investigator (CZ). If a claim was primarily found to be insufficiently substantiated, a second investigator assessed the claim independently. In case of disagreement between second and first investigator the claim was again evaluated by two senior investigators (LE, PK).

If at least one of the senior investigators found the claim inadequately substantiated, we regarded it as at least ambiguous, and therefore rated it as not supported by the given references. The two primary investigators were medical students close to their final exams, and the two senior investigators were consultants in anaesthetics.

As all medical claims should be based on high-quality evidence, we only regarded claims substantiated by at least one reference rated as high-quality evidence as sufficiently evidence-based.

Statistics

The statistical analysis was primarily descriptive. Furthermore, we compared the share of claims supported by high-quality evidence in 2000 with those supported in 2008, and performed a chi-square test. An additional explanatory analysis was performed using ordinal logistic regression analysis using JMP statistical software (version 8.0.1; SAS Institute Inc., Cary, NC, USA 27513). Scientific rating (high level of evidence, low level of evidence, expert opinion and no reference or reference not assessable) was used as the dependent variable and country of publication, year of publication and impact factor of the journal were used as independent variables.

Results

General findings

We found 5453 advertisements (Figure 1), with 1063 advertising drugs or pharmaceutical companies (33%). Fewer pages were devoted to advertisements for scientific events (14%), medical equipment (10%) and classifieds marketplace (10%). The portion of journal pages occupied by pharmaceutical advertisements ranged between 2.2% and 9.6% (average 5.5%) depending on the journal.

.
Figure 1: Number of advertisements, claims and supported claims. View Figure 1

Within the 1063 advertisements for pharmaceutical products, we identified 231 unique copies. Sixty-four advertisements were published in the chosen sample exactly once and three advertisements were published more than 20 times (up to 33). Pharmaceutical advertisements for pain medication (n = 452), especially opioids (n = 254), were the most common, followed by inhalational anaesthetics (n = 114), antiemetic drugs (n = 85), volume replacement fluids (n = 67) and muscle relaxants (n = 52).

The average size of a pharmaceutical advertisement was 1.8 standard pages (210 mm × 297 mm; SD 0.96, range 1/8 to 5 pages).

We found 3797 promotional claims within the 1063 pharmaceutical advertisements, resulting in a ratio of 3.6 claims per advertisement. There was no marked difference between advertisements published in 2000, 2004 and 2008 (with 3.4, 3.8 and 3.5 claims per advertisement).

The number of general knowledge statements was 655 in total, rising from 0.2 per advertisement in 2000 up to 2.0 in 2008.

Just 41% (n = 1538) of all promotional claims stated an unambiguous clinical outcome, 27% (n = 1037) a vague outcome, 21% (n = 783) an emotional/ immeasurable outcome and 12% (n = 439) a non-clinical outcome (Table 2). Within the subgroup of unambiguous clinical outcome most claims gave numbers or facts (53%). Stating the superiority above a competing drug (31%), non-inferiority compared to a competitor (9%), and comparison with placebo (7%) were less common.

Table 2: Frequency of the different kinds of claims depending on the year of publication. View Table 2

A broad variety of sales arguments were used in the pharmaceutical advertisements. Apart from emotional statements (21% of all claims) most advertisements focused on aspects relevant to daily practise. Alongside financial aspects (2%) official recommendations (like guidelines) for a distinct treatment (1%) were the least used arguments. The portions of sales arguments used are presented in table 3.

Table 3: Sales arguments used as promotional claims. View Table 3

Type of reference

On all 1063 pages containing pharmaceutical advertisements we found altogether 2489 references accompanying either general knowledge statements, or more frequent promotional claims. Sometimes one reference was used by more than one claim. On the other hand, we found one advertisement without a discrete claim with 12 references. More than 50% of all claims cited at least one reference.

We were able to obtain 1846 of these references. In 217 cases, data on file was referenced, which we did not generally consider for further analysis, apart from 18 references where substantial information was published within the advertisement itself. In 206 cases, references cited the summary of product characteristics, which were also excluded. Despite serious research efforts, 102 references could not be obtained. However, 45 of these may have been accessible at the time of publication of the advertisement (e.g. congress abstracts, official online resources).

We found 64% (n = 1602) of all references referring to a paper in a MEDLINE-listed journal. Of these 62% (n = 988) cited a randomized-controlled trial as the most common type of referenced study. Non-systematic reviews (12%), incidence/prevalence studies (10%), non-RCT trials (10%) and others (6%) were cited less often. The average impact factor of the journals publishing the cited studies was 3.0 (SD 5.4). Only 42% (n = 676) of the papers published in a MEDLINE-listed journal was written without any industrial financial support.

Comparing the advertisements published in 2008 to those published in 2000, the incidence of cited MEDLINE-papers rose from 51% to 79%. The ratio of congress abstracts and publications in NON-MEDLINE-listed journals inversely decreased from 21% to 6%.

Promotional claims supported by high-quality evidence

Of all 3797 claims just 17% (n = 657) were supported by high-quality evidence such as randomized-controlled trials or guidelines. Forty-nine percent (n = 1876) of the claims were not accompanied by a reference to an external proof, 19% referenced data on file, data not useful as evidence or untraceable sources, and 8% were supported by low-quality studies, experts’ opinions or similar.

We found 241 claims (6%) citing evidence not supporting or even contradicting the claim. Examples for non-supporting evidence are shown in table 4. While the senior reviewers coincided in most of the cases, they disagreed on six claims, which were therefore rated as not supported as well.

Table 4: Examples for claims not supported by the cited evidence. View Table 4

Comparing the three investigated years of publication, we found that the amount of claims supported by high-quality studies rose from 9% in 2000 up to 37% in 2008 (Table 5; p < 0.005 when applying the χ²-test). In contrast the amount of claims not supported by their references or not stating useful references declined. Assuming that all references to sources not evaluated in our study (data on file, data not useful as evidence or untraceable sources) were actually high-quality evidence, the amount of claims supported by high-quality evidence would still increase significantly from 24% in 2000 up to 48% in 2008 (p < 0.005 when applying the χ²-test).

Table 5: Proportion of advertisement claims supported or not supported by different kinds of evidence, distinguished by year. View Table 5

Dependency on the journal

By comparing the advertisements of the different journals we found considerable differences between the journals, depending on the country of publication. In the two German journals we found more claims without any reference (A&I 79%, Anaesthesist 84%, on average 49%) when compared to the international journals. The advertisements in the American journals most often cited data excluded in our study (as data on file and summary of product characteristics), or data not obtainable (Anesthesiology 23%, A&A 28%, on average 19%). The two British journals had the highest ratio of references which did not prove the underlying claim (BJA 15%, Anesthesia 12%, on average 6%).

Ordinal logistic regression analysis revealed that references in German journals had significantly less quality (p < 0.0001) compared to British or US journals with no difference between the latter both (p = 0.21). This analysis also confirmed the results that quality improved over time in British and US journals (p < 0.0001) but to a lower extend in German journals (p < 0.04). Also impact factor of the journal was significantly associated with better reporting quality. However, additional graphic interaction analysis revealed that this effect was mainly due to the fact that both German journals had lower impact factors than the four international journals. Thus, limiting this analysis only to the British and US journals could not confirm the significant effect of the impact factor of a journal on reporting quality of published advertisements.

The funding source of the journal in which the cited study had been published had no influence on the fact if the study did or did not support the claim.

Dependency on the kind of claim and the promoted drug

We classified the claims depending on their characteristics, and evaluated how they were supported by references. Claims classified as emotional statements did not refer to any references. If a claim advertised a non-clinical outcome approximately an half of the cases pointed to a reference, but most often to data not freely available. Claims stating unambiguous or vague clinical outcomes were supported by clinical studies in about one third of the cases, but then often also referenced studies inappropriate for supporting the claim.

The portion of claims supported by a high-quality clinical study also depended on the kind of medication advertised by the claim. Advertisements for antiemetic drugs (27%), pain medication (24%) and drugs influencing the cardiovascular system (23%) were more often supported by high-class references than advertisements for other drugs (2-11%).

Evidence of general knowledge statements

In comparison to promotional claims we found general knowledge claims to be substantiated by high-quality evidence twice as often (30% vs. 17%) An expert’s opinion was cited nearly five times more often (19% vs. 4%). Fewer general knowledge claims are lacking any reference (36% vs. 49%).

Discussion

We could show that 17% of all promotional claims in pharmaceutical advertisements were supported by high-quality scientific evidence. Furthermore, 6% stated evidence which did not support or even contradicted the claim. However, we found a temporal positive trend with an increase in both the number of claims accompanied by references and the quality of these references. Evaluating the reason for this trend is beyond the scope of our study. We suppose this effect is partly caused by an increased awareness for evidenced based medicine and partly caused by an easier access for medical professionals to original studies via electronic media.

In previous investigations even fewer claims were supported by good evidence [15,16,18]. This partly may be the result of different study protocols; for instance one study did not consider monographs as a possible form of evidence [15] and another investigation only included references citing randomized controlled trials [16]. The type of drug advertised may be also a reason. In 75% of advertisements we included in our study, narcotics, analgesics drugs or relaxants were promoted. Effects of these drugs can usually be observed directly and long term adverse effects are often just of secondary importance.

As we showed a positive trend towards fewer poorly supported claims the publication date of the advertisements included in the different studies may explain the differing results as well.

Limitations

Every year only a few new drugs are released. Since most of the advertisements refer to these few drugs and the same key selling arguments are used repeatedly, the influence of one advertising campaign on the results of our investigation is considerable.

New drugs, legal regulations and differing journal topics did not permit any predictions about future drug advertisements.

Conclusion

A majority of all drug advertisement claims in anaesthetic journals are not supported by high-quality evidence. Pharmaceutical companies could improve their credibility and increase the acceptance of pharmaceutical advertisements by substantiating any claim made within advertisements. Until then medical professionals dealing with advertisements need to critically appraise all given claims in advertisements. This is why academic training in reading and interpreting clinical studies is important. Likewise, it is equally important to draw attention to this issue with publications like the one you have just read here.

Competing Interests

CZ: No interest declared; AS: No interest declared; PK: He worked as consultant for MSD, Fresenius and Acacia. He received payment for lectures from MSD, Fresenius, Baxter and ratiopharm; LE: He is a board member of Grunenthal GmbH and ratiopharm GmbH. He received payment for lectures from Baxter GmbH, Fresenius GmbH, Grunenthal GmbG, and Sintetica GmbH.

Acknowledgements

Assistance with the study: Gertrude Duncan for linguistic support.

References