Are Advertisement Claims in Anaesthetic Journals based on High-Quality Evidence ?

Background: Many medical journals contain advertisements for pharmaceuticals products. While the WHO demands that claims in pharmaceutical advertisements should be based on scientific evidence, past investigations demonstrated that some advertisements fail to fulfill these demands. As there is currently no investigation dealing with advertisements in anaesthetic journals, we aimed to find out whether claims in these journals are accompanied by references to external evidence, and whether these claims are supported by the evidence cited. Methods: We analysed all drug advertisements placed in four international and two German language anaesthetic journals published in the years 2000, 2004 and 2008, regarding presentation and occurrence of claims. For all claim-citing references to external evidence we tried to obtain these references, ranked their grade of evidence, and verified whether the claim was actually supported by evidence given. The analysis was primarily descriptive. Results: We found 5453 advertisements for 1063 pharmaceuticals. The advertisements contained 3797 marketing claims and 2489 references, of which 64% referred to articles published in a journal listed in MEDLINE. In total, 17% of all claims were supported by a high-quality study or guideline. This portion rose from 9% in 2000, up to 37% in 2008. Overall, 49% of the claims were not accompanied by a reference to an external proof, and 6% referenced a study which did not support the claim. Conclusions: These results imply that drug advertisements in anaesthetic journals are not yet sufficiently supported by highquality studies, and medical professionals therefore need to critically appraise the claims in advertisements.

Advertisements may also interfere with the scientific independence of a journal [6].
Pharmaceutical advertisements do have some effect on doctors' prescription behavior [7,8], and not all of them are aware of this influence [9].Supporters of pharmaceutical advertisements argue that they are a way to inform doctors about new products in a fast and direct fashion, and do not necessarily interfere with the need for balanced information [10,11].
Different authors as well as the WHO demand that claims in pharmaceutical advertisements should be based on scientific evidence [12][13][14].Past investigations demonstrated that a number of advertisements fail to meet these demands [7,[15][16][17][18][19], and ask journals to abstain from publishing pharmaceutical advertisements [20].At present there is no investigation dealing with advertisements in journals specializing in anaesthesia.As drugs commonly used in anaesthesia are quite specific, results from other investigations might not apply to them.
The aim of this study was to evaluate the quality of pharmaceutical advertisements in anaesthesia journals published between 2000 and 2008, and to identify fields for improvement.Furthermore, the influence of time and origin of the scientific journal will be investigated.

Introduction
Many medical journals contain advertisements placed by pharmaceutical companies [1].While they are an important source of funding for publishers [2], there is an ongoing discussion on how advertisements influence doctors' prescription behavior [3][4][5].
guidelines, and official publications were rated as high-quality evidence.Non-interventional studies, congress abstracts, and trials not blinded or not randomized were rated as low-quality evidence.All major medical journals are listed in MEDLINE; articles from journals not listed may be difficult to obtain for the regular reader of an advertisement.Therefore we rated any article published in a journal not listed in MEDLINE as low-quality evidence, regardless of the study type.References to monographs, non-systematic reviews, citations of the introduction of an article, letters and editorials were rated as expert opinion.

Evaluation
For each promotional claim accompanied by at least one reference we evaluated each reference separately.If some of the references were not available, we just assessed the available ones.In order to rate the claim as evidence-based or not evidence-based, we used a best-case scenario, [24] and rated it as evidence-based if at least one of the references substantiated the claim.Only if a reference of a higher level of evidence contradicted another substantiating reference of a lower level we considered the claim as not substantiated.For each claim we stated the level of evidence of the highest quality reference.
The primary assessment was performed by one investigator (CZ).If a claim was primarily found to be insufficiently substantiated, a second investigator assessed the claim independently.In case of disagreement between second and first investigator the claim was again evaluated by two senior investigators (LE, PK).
If at least one of the senior investigators found the claim inadequately substantiated, we regarded it as at least ambiguous, and therefore rated it as not supported by the given references.The two primary investigators were medical students close to their final exams, and the two senior investigators were consultants in anaesthetics.
As all medical claims should be based on high-quality evidence, we only regarded claims substantiated by at least one reference rated as high-quality evidence as sufficiently evidence-based.

Statistics
The statistical analysis was primarily descriptive.Furthermore, we compared the share of claims supported by high-quality evidence in 2000 with those supported in 2008, and performed a chi-square test.An additional explanatory analysis was performed using ordinal logistic regression analysis using JMP statistical software (version 8.0.1;SAS Institute Inc., Cary, NC, USA 27513).Scientific rating (high level of evidence, low level of evidence, expert opinion and no reference or reference not assessable) was used as the dependent variable and country of publication, year of publication and impact factor of the journal were used as independent variables.

General findings
We found 5453 advertisements (Figure 1), with 1063 advertising drugs or pharmaceutical companies (33%).Fewer pages were devoted to advertisements for scientific events (14%), medical equipment (10%) and classifieds marketplace (10%).The portion of journal pages occupied by pharmaceutical advertisements ranged between 2.2% and 9.6% (average 5.5%) depending on the journal.
Within the 1063 advertisements for pharmaceutical products, we identified 231 unique copies.Sixty-four advertisements were decided in accordance with the design of the page and the impression on an unbiased reader whether to include or to exclude it from our analysis of advertisement.We classified each advertisement according to product advertised.For pharmaceutical advertisements, we recorded the drug group and the size of the advertisement.
The evaluation of the advertisements followed a predefined protocol.
We searched for all promotional claims within each pharmaceutical advertisement.Phrases citing known medical facts without giving information about the promoted drug were classified as general knowledge statements, and were assessed separately.Text passages giving legally required information such as excerpts of the summary of product characteristics were not recorded.
We also did not further assess short slogans accompanying the drugs' or the companies' names.
Each promotional claim was classified as unambiguous clinical outcome, vague outcome, non-clinical outcome or emotive/ immeasurable outcome.This classification has been used by other authors before [21,22].
In addition, for all pharmaceutical claims we categorized the kind of sales argument used [18,23].The sales argument "emotional statement" corresponds with the outcome category "emotive/ immeasurable".
For all cited references we stated the kind of publication.We then attempted to obtain all cited references by searches via MEDLINE, general internet search engines, including Google Scholar or interlibrary loan.We did not search for references to summary of product characteristics and other legally required documents, as they are less suitable to verify the advantages of a drug.Likewise we did not obtain any data on file, as we believe it is very unlikely that the average reader of an advertisement will contact the manufacturer in order to get an extract of the cited data.These circumstances were recorded with the corresponding references, and the references were subsequently marked as not available.
All references were classified by their level of evidence.Three different [15] levels were used for this process.Blinded, randomizedcontrolled trials, meta-analyses, incidence/prevalence studies,  We found 64% (n = 1602) of all references referring to a paper in a MEDLINE-listed journal.Of these 62% (n = 988) cited a randomizedcontrolled trial as the most common type of referenced study.nonsystematic reviews (12%), incidence/prevalence studies (10%), non-RCT trials (10%) and others (6%) were cited less often.The average impact factor of the journals publishing the cited studies was 3.0 (SD 5.4).Only 42% (n = 676) of the papers published in a MEDLINElisted journal was written without any industrial financial support.
Comparing the advertisements published in 2008 to those published in 2000, the incidence of cited MEDLINE-papers rose from 51% to 79%.The ratio of congress abstracts and publications in NON-MEDLINE-listed journals inversely decreased from 21% to 6%.

Promotional claims supported by high-quality evidence
Of all 3797 claims just 17% (n = 657) were supported by highquality evidence such as randomized-controlled trials or guidelines.Forty-nine percent (n = 1876) of the claims were not accompanied by a reference to an external proof, 19% referenced data on file, data not useful as evidence or untraceable sources, and 8% were supported by low-quality studies, experts' opinions or similar.
We found 241 claims (6%) citing evidence not supporting or even contradicting the claim.Examples for non-supporting evidence are shown in table 4. While the senior reviewers coincided in most of the cases, they disagreed on six claims, which were therefore rated as not supported as well.
Comparing the three investigated years of publication, we found that the amount of claims supported by high-quality studies rose from 9% in 2000 up to 37% in 2008 (Table 5; p < 0.005 when applying the χ 2 -test).In contrast the amount of claims not supported by their references or not stating useful references declined.Assuming that all references to sources not evaluated in our study (data on file, data not useful as evidence or untraceable sources) were actually high-quality evidence, the amount of claims supported by high-quality evidence would still increase significantly from 24% in 2000 up to 48% in 2008 (p < 0.005 when applying the χ 2 -test).

Dependency on the journal
By comparing the advertisements of the different journals we found considerable differences between the journals, depending on the country of publication.In the two German journals we found more claims without any reference (A&I 79%, Anaesthesist 84%, on average 49%) when compared to the international journals.The advertisements in the American journals most often cited data excluded in our study (as data on file and summary of product characteristics), or data not obtainable (Anesthesiology 23%, A&A 28%, on average 19%).The two British journals had the highest ratio published in the chosen sample exactly once and three advertisements were published more than 20 times (up to 33).Pharmaceutical advertisements for pain medication (n = 452), especially opioids (n = 254), were the most common, followed by inhalational anaesthetics (n = 114), antiemetic drugs (n = 85), volume replacement fluids (n = 67) and muscle relaxants (n = 52).
The average size of a pharmaceutical advertisement was 1.8 standard pages (210 mm × 297 mm; SD 0.96, range 1/8 to 5 pages).
We found 3797 promotional claims within the 1063 pharmaceutical advertisements, resulting in a ratio of 3.6 claims per advertisement.There was no marked difference between advertisements published in 2000, 2004 and 2008 (with 3.4, 3.8 and 3.5 claims per advertisement).
The number of general knowledge statements was 655 in total, rising from 0.2 per advertisement in 2000 up to 2.0 in 2008.
A broad variety of sales arguments were used in the pharmaceutical advertisements.Apart from emotional statements (21% of all claims) most advertisements focused on aspects relevant to daily practise.Alongside financial aspects (2%) official recommendations (like guidelines) for a distinct treatment (1%) were the least used arguments.The portions of sales arguments used are presented in table 3.

Type of reference
On all 1063 pages containing pharmaceutical advertisements we found altogether 2489 references accompanying either general knowledge statements, or more frequent promotional claims.Sometimes one reference was used by more than one claim.On the other hand, we found one advertisement without a discrete claim with 12 references.More than 50% of all claims cited at least one reference.
We were able to obtain 1846 of these references.In 217 cases, data on file was referenced, which we did not generally consider for further analysis, apart from 18 references where substantial information was published within the advertisement itself.In 206 cases, references cited the summary of product characteristics, which were also excluded.Despite serious research efforts, 102 references could not be obtained.However, 45 of these may have been accessible at the time of publication of the advertisement (e.g.congress abstracts, official online resources). of references which did not prove the underlying claim (BJA 15%, Anesthesia 12%, on average 6%).
Ordinal logistic regression analysis revealed that references in German journals had significantly less quality (p < 0.0001) compared to British or US journals with no difference between the latter both (p = 0.21).This analysis also confirmed the results that quality improved over time in British and US journals (p < 0.0001) but to a lower extend in German journals (p < 0.04).Also impact factor of the journal was significantly associated with better reporting quality.However, additional graphic interaction analysis revealed that this effect was mainly due to the fact that both German journals had lower impact factors than the four international journals.Thus, limiting this analysis only to the British and US journals could not confirm the significant effect of the impact factor of a journal on reporting quality of published advertisements.
The funding source of the journal in which the cited study had been published had no influence on the fact if the study did or did not support the claim.

Dependency on the kind of claim and the promoted drug
We classified the claims depending on their characteristics, and evaluated how they were supported by references.Claims classified as emotional statements did not refer to any references.If a claim advertised a non-clinical outcome approximately an half of the cases pointed to a reference, but most often to data not freely available.Claims stating unambiguous or vague clinical outcomes were supported by clinical studies in about one third of the cases, but then often also referenced studies inappropriate for supporting the claim.
The portion of claims supported by a high-quality clinical study also depended on the kind of medication advertised by the claim.Advertisements for antiemetic drugs (27%), pain medication (24%) and drugs influencing the cardiovascular system (23%) were more often supported by high-class references than advertisements for other drugs (2-11%).

Evidence of general knowledge statements
In comparison to promotional claims we found general knowledge claims to be substantiated by high-quality evidence twice as often (30% vs. 17%) An expert's opinion was cited nearly five times more often (19% vs. 4%).Fewer general knowledge claims are lacking any reference (36% vs. 49%).

Discussion
We could show that 17% of all promotional claims in pharmaceutical advertisements were supported by high-quality scientific evidence.Furthermore, 6% stated evidence which did not support or even contradicted the claim.However, we found a temporal positive trend with an increase in both the number of claims accompanied by references and the quality of these references.Evaluating the reason for this trend is beyond the scope of our study.We suppose this effect is partly caused by an increased awareness for evidenced based medicine and partly caused by an easier access for medical professionals to original studies via electronic media.
In previous investigations even fewer claims were supported by good evidence [15,16,18].This partly may be the result of different study protocols; for instance one study did not consider monographs as a possible form of evidence [15] and another investigation only included references citing randomized controlled trials [16].The type of drug advertised may be also a reason.In 75% of advertisements we included in our study, narcotics, analgesics drugs or relaxants were promoted.Effects of these drugs can usually be observed directly and long term adverse effects are often just of secondary importance.
As we showed a positive trend towards fewer poorly supported claims the publication date of the advertisements included in the different studies may explain the differing results as well.

Limitations
Every year only a few new drugs are released.Since most of the advertisements refer to these few drugs and the same key selling arguments are used repeatedly, the influence of one advertising campaign on the results of our investigation is considerable.New drugs, legal regulations and differing journal topics did not permit any predictions about future drug advertisements.

Conclusion
A majority of all drug advertisement claims in anaesthetic journals are not supported by high-quality evidence.
Pharmaceutical companies could improve their credibility and increase the acceptance of pharmaceutical advertisements by substantiating any claim made within advertisements.Until then medical professionals dealing with advertisements need to critically appraise all given claims in advertisements.This is why academic training in reading and interpreting clinical studies is important.Likewise, it is equally important to draw attention to this issue with publications like the one you have just read here.

Figure 1 :
Figure 1: Number of advertisements, claims and supported claims

Table 2 :
Frequency of the different kinds of claims depending on the year of publication

Table 4 :
Examples for claims not supported by the cited evidence

Table 5 :
Proportion of advertisement claims supported or not supported by different kinds of evidence, distinguished by year