Quantitative pharmacodynamic (PD) methods are used to assess the effect of a drug on its target. The use of these methods in clinical cancer studies has increased enormously with the advent of targeted therapies in the past years. To ensure that quantitative PD methods meet their expectations, methods need to be validated according to internationally recognized standards for amongst others specificity, accuracy, sensitivity, precision and stability. We show that 87% of our selected clinical cancer studies, published in the last 7 years, used quantitative PD methods that were on average only validated for half of the 10 main validation parameters. Here, we offer our view on the quality of, and the challenges encountered in validation of quantitative PD methods. Finally, we provide future directions and considerations for improvement of the method validation quality.
Quantitative, Pharmacodynamic, Biomarker, Validation, Guidelines, Clinical, Cancer
The need for pharmacodynamic (PD) biomarker methods continues to grow with the explosive increase of targeted agents in clinical cancer therapy. PD biomarkers are endogenous biological indicators that can be measured objectively and provide insight about the target engagement, and proof of concept i.e. the extent of the biological effect of a drug. Poor target validation or insufficient target engagement is the main reason for failure of 42% of clinical phase I and 63% of phase II clinical cancer trials [1]. Early recognition of these failures is important, which can be accomplished by using PD biomarkers to provide proof of mechanism for the drug-target engagement. PD biomarkers may include molecular, histologic, radiographic, or physiologic characteristics. Most PD biomarkers are, however, molecular biomarkers, which encompass among others protein modifications, enzyme activities, receptor expression and occupancies, and cell counts. In early clinical trials, PD biomarkers can provide useful information for patient management, e.g., whether to continue treatment or to adjust dose (schedule) [2]. They can also help to guide drug dose individualization in targeted/personalized cancer therapy with the aim of treating cancer more effectively and with less toxicity [3]. An illustrative example of the potential usefulness of PD evaluations is the monitoring of B-lymphocyte suppression as PD biomarker in clinical trials to find doses of anti-CD20 monoclonal antibodies required to maximally reduce this cell population. The reduction of the amount of B-cells is presumed to underlie the clinical benefits of these drugs in treating cancer, and has become an established indicator for treatment success [4]. Another example is the monitoring of the inhibition of PARP (target) enzyme activity in white blood cells as PD biomarker for the target engagement of the anti-cancer drug Olaparib [5].
Quantitative PD biomarker methods are used to determine biomarker activity or concentration in different biological matrices such as blood, bile, serum, plasma, urine, tumor cells, and tissue such as skin. For this purpose, these methods need calibration standards prepared in biomarker-free surrogate matrices that are fully defined and representative for the measured biomarkers and their biological matrices. Full analytical validation of these methods is essential to assure accurate and specific biomarker quantification to prevent interference from other compounds in these matrices. Often biomarkers can fail not because of the underlying science, but because of poor choice of methods and lack of analytical validation [2]. The development and validation of quantitative biomarker methods is, however, more complicated than that of bioanalytical drug methods used for pharmacokinetics due to the following complications. Firstly, sometimes biomarkers need to be detected in more complex biological matrices such as tumor tissues. Secondly, the stability of biomarkers and other endogenous compounds in the biological matrix is often poor. Thirdly, biomarkers are endogenous substances, thus obtaining biomarker-free surrogate matrices for validation of method specificity, and preparation of the calibration curve may be more difficult. Lastly, the biomarker molecule is often not available to act as a certified calibration standard [6].
The last decade has seen significant progress in harmonisation of acceptance criteria, definitions, and guidelines for bioanalytical method validation (BMV) of chromatography and ligand binding methods [7]. For this purpose, the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) recently released BMV Guidance M10 [8]. Fundamentally, ICH-M10 validation involves demonstration of bioanalytical method specificity, accuracy (mean % deviation from nominal concentration), linearity, dilution integrity, sensitivity, limit of quantification, precision (% coefficient of variation (CV), and stability for drugs (Table 1). However, international harmonization of guidelines for biomarker method validation is lacking. In their BMV 2018 guideline, the Food and Drug Administration (FDA) recommends to "conduct a full validation of any new bioanalytical method for the analysis of a new drug entity, its metabolite(s), or biomarkers." [9]. The validation of biomarkers thus needs to be consistent with the validation principles applied to bioanalytical drug methods validation [9]. This guideline further states that exploratory quantitative biomarker methods, that are not used to support regulatory decision making, may not require such stringent validation. Exploratory biomarker methods are for instance used to better understand the mechanism of action of drugs, which may ultimately lead to improved future cancer treatments. However, in light of their importance, and to make sure that these methods life up to their expectations, we would argue that also exploratory quantitative biomarker methods, used in clinical trials, need to be fully validated.
Herein, we present our viewpoint on the degree of compliance with the FDA BMV 2018 guideline of quantitative PD biomarker methods used in our selection of clinical cancer trials. Challenges encountered in the process of PD biomarker method development and validation are discussed. The implications of poorly validated methods used for assessment of drug effects are described. In the final section, we offer some considerations and suggestions that may help to improve the quality of PD biomarker methods and consequently their usefulness in clinical cancer trials.
For the assessment of the status of analytical validation of quantitative PD methods used in clinical cancer trials, a focused PubMed literature query was performed on the 8th of December 2020, including the keywords "2013/04/12" [PDAT] : "2020/12/08" [PDAT] AND "humans" [MeSH Terms] AND ("Clinical Trials, Phase I as Topic" [Mesh] OR "Clinical Trials, Phase II as Topic" [Mesh] OR "Clinical Trials, Phase III as Topic" [Mesh] OR "Clinical Trials, Phase IV as Topic" [Mesh] OR phase I [tiab] OR phase 1 [tiab] OR phase II [tiab] OR phase 2 [tiab] OR phase III [tiab] OR phase 3 [tiab] OR phase IV [tiab] OR phase 4 [tiab] OR "Clinical Trials, Phase I as Topic" [Mesh] OR "Clinical Trials, Phase II as Topic" [Mesh] OR "Clinical Trials, Phase III as Topic" [Mesh] OR "Clinical Trials, Phase IV as Topic" [Mesh]) AND (pharmacodynamic* [tiab] AND method AND methods AND cancer NOT modeling [tiab] NOT review [tiab]). The query returned 199 articles. After review, 4 articles were excluded for failure to provide details about used validation criteria and guidelines. Another article was excluded for referencing to validation data in a submitted, but never published, other article from the authors. Yet another article was excluded for serial referencing. After applying additional exclusion criteria, 78 (39%) of articles were found suitable for our evaluation using quantitative PD methods in human clinical cancer studies (Figure 1) [10-87]. From each of these articles one quantitative PD method was evaluated and qualitatively scored for compliance with the 10 main validation criteria defined by the FDA BMV 2018 guideline (Table 1 and Table 2).
We found that, 10 PD biomarker methods (13%) were fully validated in accordance with all 10 main FDA BMV 2018 validation parameters. However, 53 PD methods (68%) were validated for just on average 5 parameters. Total absence of PD method validation was found for 17 (22%) of published clinical studies. Compared to chromatographic and ligand binding methods, flow cytometry (FC) methods were relatively less well validated. FC methods were validated for on average just 2 parameters and 50% lacked any validation. In order to effectively utilize biomarkers to draw meaningful conclusions, it is imperative that the measured concentration or activity of the biomarker in study samples is as close as possible to the actual value at the time of sample collection. The FDA guideline states that investigators should always evaluate their PD methods for stability themselves, referencing the limited sample stability data published in literature is not considered sufficient [9]. For this purpose, stability needs to be assessed for all stocks and reagents used, and for biomarkers during all different parts of the experimental procedure: 1) From sample collection in the clinic to the start of the sample work-up in the lab. 2) On the bench-top or in the autosampler after sample work-up. 3) Long term storage. 4) Freeze-thaw stability. Stability during the sample work-up procedure is by definition not part of the validation, but should be assessed during method development. Nevertheless, only 26 PD methods (33%) were evaluated for reagent and stock solution stability, and biomarker stability during one or more parts of the experimental procedure.
Commercial immuno-assay kits were used in 33 (77%) of PD methods. These kits are usually developed for general use and therefore may need some level of optimisation, such as adding more calibrators and/or replacing the kit controls in a buffer with controls that mimic the sample matrices. This may prevent accuracy problems from a lack of parallelism between standard curves prepared in buffer and matrix. Many reagents are "research purpose only" (RUO) with considerable batch to batch variation. The use of good manufacturing practice (GMP) reagents or pooled RUO batches should be preferred if possible. A common complaint from scientists is that suppliers do not provide the data required to evaluate a given antibody's specificity or its lot-to-lot variability [88]. Companies might ship a batch of antibodies with characterization information derived from a previous batch [88]. And any validation data that are given are often derived under ideal conditions that do not reflect typical experiments [89]. Investigators should be aware that certain suppliers are releasing with a fast pace antibodies and ELISA kits of questionable quality [88]. The largest vendors often buy antibodies from smaller suppliers, relabel them and offer them for sale [88]. Such products can lead to unfounded conclusions, waste of many months of research and publications that subsequently need to be retracted. In one illustrative example, the investigators spent 2 years and $500.000 because the commercial kit they were using did not recognize Zona Pellucida-like domains protein 1 but Cancer Antigen 125 [90]. Investigators should realise that validation of specificity is the basis of every reliable method. Ideally, researchers would refuse to buy antibodies and immune-assays without extensive validation and quality control data, or would perform the validation themselves. Antibodies should be tested for off-target binding using positive and negative controls e.g. control experiments that involve engineering cell lines to both express and stop expressing the protein of interest. We recommend additional in-house validation before using commercial antibodies and kits, also when obtained from suppliers with a proven track-record, and subsequent entry of the in-house validation data in centralized antibody validation registries such as CiteAb (www.citeab.com) [91]. CiteAb mines the scientific literature for publications citing the use of individual antibodies, to generate a comprehensive searchable database of literature citations, together with suppliers, blog and news.
Establishing methods "fit for purpose" requires extensive validation and is a continuous process. Scientists, however, often claim that their methods are validated in a very casual way, often without explaining which guidelines were followed or any reference to experimental data to back up their claim. The validation process should start with a description of the purpose of the method, followed by method development and the definition of the performance characteristics. It continues with documentation of the methodology and the validation results. Once the method has been validated, it is crucial that the method is consistently executed as it was validated. The same materials should be used, and system suitability and method controls are applied with the same specifications as during the validation process. During the in-use phase of the methods, there is continuous monitoring to assure that the method still generates results in accordance with the performance characteristics as originally determined. Revalidation is needed if the method is changed, or has been out of use for a while, when it is applied for another material or for a new purpose. For this, the validated method is carefully documented in a procedure or a standard protocol often called a Standard Operating Procedure or 'SOP'. The documentation is not limited to a general description of the methods, but preferably includes all relevant aspects that contribute to the performance of the method.
Validation of PD biomarker methods is a necessary component for obtaining high-quality PD biomarker data from clinical studies. For this purpose, FDA draft guidelines for the validation of quantitative PD biomarker methods have existed since 2013 [9]. In this light, the incomplete and sometimes total absence of validation for 87% of quantitative PD biomarker methods in phase 1-3 clinical cancer trials, published in the last 7 years, is quite disturbing. A possible explanation for this lack of validation may be that most quantitative biomarkers used in clinical trials originate from preclinical research in laboratories that are not used to working in accordance with good laboratory practice (GLP). GLP guidelines mandate full validation of quantitative bioanalytical drug methods. For quantitative biomarker methods these guidelines did not exist till the FDA recommended them in their 2013 draft BMV guideline, which became final in 2018 [9].
The FDA BMV 2018 guideline is, however, limited to recommendations for chromatographic and ligand binding method validation [9]. This may explain why we found that validation was most often lacking for quantitative FC biomarker methods [92]. These FC methods are used to quantify cell subpopulations and antigen expression per cell [93]. Quantitative FC methods are used in many clinical biomarker studies, and their importance is increasing [93]. Several factors complicate the validation of these FC methods [94]. One of the most important limitations is lack of standardization in methods, instrument setup, and quality controls. There is an urgent need to address these limitations by establishing internationally harmonized guidelines for quantitative FC biomarker methods used for clinical trials.
Currently, journals apparently do not mandate the use of properly validated quantitative PD methods for clinical studies. Investigators should, however, realize that quantitative PD methods used in clinical trials can only generate reliable and robust results if these methods are fully validated, reliable and robust. In that respect, it does not matter whether these methods are used for regulatory decision making, or for exploratory purposes. Full validation will increase the credibility of PD studies and may increase the discovery rate of qualified clinical PD biomarkers. For this purpose, we recommend that all quantitative PD methods used in clinical trials are fully validated as in the international guidelines for bioanalytical method validation [9].
None.
DP analyzed and interpreted the data. DP and JB were major contributors in writing the manuscript. JB supervised the study, provided critical suggestions to the study. All authors read and approved the final manuscript.
The authors received no funding for this study.
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Not applicable.
Not applicable.
The authors declare that they have no competing interests.