BEBPA Blog
Volume 1, Issue 7
Value-Driven Use of Assay Control Samples in the Relative Format Biological Assays
By Anton Stetsenko, M.D., M.B.A, Board Member, BEBPA
Abstract
The selection and implementation of Assay Control (AC) samples in bioassays require practical, risk-based considerations throughout the method lifecycle. Well-established, controlled, and validated methods adhering to ICH principles with robust statistical process control (SPC) may raise questions about the continued necessity of dedicated ACs in some cases. This might lead to considering AC samples fulfilling a similar role to QC samples within validated, controlled methods under GMP regulations.
However, for early-stage method development and other non-regulated studies like characterization, formulation, or comparability meticulous selection of AC material, acceptance criteria, and even configuration of AC preparation could be crucial for data generation and reliable results in the short term. In the long run, answers to these questions at the beginning of product development or an attempt to answer earlier can be considered significant and advantageous strategic decisions as they can reveal the important product or procedural properties required for a robust potency assurance strategy.
As a spin-off from the BEBPA White Paper (2015) on assay acceptance criteria (AAC) for multiwell-plate assays, this technical note delves into these complexities. Some questions addressed here include:
- Defining AC sample and acceptance limits
- Optimizing AC design and handling
- Using non-standard samples and out-of-specification lots to balance practicality with bioassay suitability requirements
- Leveraging AC sample and other reference-like materials to explore bioassay capabilities to support non-similarity assessment (lack of parallelism)
This technical note objectively evaluates the pros and cons of AC use across the method lifecycle, considering product, bioassay context, regulations, development stage, and desired confidence. It provides value-based insights through real-world scenarios to help analysts determine the specific value of AC for their assays.
Definition
The Assay Control sample serves as a control material designed to mimic the behavior of the reference standard and test samples. Ideally, it should be well-characterized and remain independent of the reference standard and test samples. However, when the diversity and number of manufacturing batches are limited in the early stages, it’s common to see the preparation of the AC sample from the current reference standard. It is important though to maintain such preparations independently. Otherwise, the value of such AC sample measurement will be limited to intra-replicate variability assessment, the same conundrum as with pseudo- vs true replicates. While this practice seems logical — if these independent preparations behave similarly and yield potency results close to 100%, it suggests flawless execution of the procedure — it has drawbacks. Namely, it may overlook potential issues with the reference material, which can behave differently due to inherent instability, handling, storage, preparation problems, and other factors. It is worth noting that while system suitability criteria are compliance matters and required by various guidelines (e.g., ICH Q14 and ICH Q2), the use of the AC sample in relative format bioassays is not required by regulators. The author believes that the stage-specific implementation of the AC sample(s) as an integral part of the bioassay’s system suitability algorithm including assay and sample acceptance criteria can be extremely valuable when properly designed and managed.
Pros and Cons
So, the AC sample should originate from a different lot of material produced using the same process. Its purpose extends beyond merely assessing assay validity; it also provides valuable insights into the reference standard, including its preparation and stability. We understand now that any abnormal behavior observed in an AC sample during a relative format bioassay analysis could indicate issues with the reference standard, such as stability concerns. Recognizing that anomalies in comparing an AC sample and a reference sample could stem from various factors is essential. These include incorrect preparation of either the reference or control sample or, for example, inaccuracies in the handling of cells (reconstitution, culturing, dilution, dispensing, seeding, passaging, etc.) or other procedural discrepancies influencing dose-dependent responses. A bioassayist must remember that under the same circumstances, there is a 50% chance that the reference sample is fine and the entire problem is with the AC sample. Still, it’s a good practice to explore and investigate abnormal results because a missed or ignored problem with the reference material can compromise the future of the entire program not to mention the risk to the patient. This can be an example of where the role of the independent from reference material AC sample is hard to overestimate. Table 1 below summarizes some of the pros and cons of the AC sample use. It is unlikely that everything is captured here, and it would be great to see what is missing in the comments section below this note.
Table 1. Pros and cons of AC sample use
PROS
- Provide insight into bioassay validity as part of the system suitability assessment, which is required by regulatory guidelines
- Aid in detecting systematic issues long-term, especially when combined with the routine tracking and trending algorithm to feed SPC analysis
- Offers potential for improved bioassay capability including similarity assessment especially when used along with other well-characterized reference-like preparations for diversification purposes
- Facilitates monitoring of the reference standard stability, and in-parallel characterization and qualification of a new reference standard candidate
NOTE: a substantial benefit in addition to bioassay validity assessment, which is well-recognized by numerous industry experts - Assists in maintaining the consistent long-term meaning of relative potency estimates providing method precision and accuracy characteristics in real-time
CONS
- Add additional and substantial cost when AC sample is routinely used in the bioassay
- Reduces bioassay throughput by up to 1/2 for a simple bioassay plate design with the reference standard and one test sample per plate (AC sample occupies the room for the 2nd test sample)
- Statistical inference based on within-assay variance may be unreliable and biased due to the presence of an AC sample and worsened by the poorly developed design structure of the bioassay
- Challenges in the identification, preparation, storage, and handling of diverse reference-like materials to be served as an AC sample(s)
- Reaction to isolated out-of-control events (AC failure due to lack of similarity or potency value outside of the acceptable range) may not be optimal, such events must be treated cautiously and preferably in the context of historical data
Acceptance Criteria
Establishing appropriate acceptance criteria is crucial when incorporating an Assay Control (AC) sample into your bioassay. These criteria differentiate between valid (acceptable, or successful) and invalid (unacceptable, or failed) bioassays. Typically, acceptance criteria are defined for the AC sample as if it were a standard test sample. Consequently, the same criteria used for regular test samples (Sample Acceptance Criteria or SAC) now serve as Assay Acceptance Criteria (AAC), determining the entire bioassay’s validity (refer to the BEBPA White Paper (2015), “Assay Acceptance Criteria for Multiwell-Plate–Based Biological Potency Assays,” for details).
These criteria may encompass equivalence boundaries for similarity assessment and the potency of the AC sample relative to the reference standard. In sensitive model systems, stricter criteria, like a ratio of upper asymptotes for similarity between the AC and reference, might be advisable for early detection and prompt addressing of potential procedural issues. For example, an out-of-trend upper asymptote ratio could indicate cell passage aging or heterogeneity in the working cell bank. This can affect the AC sample more than the reference, especially when the AC is a fresh preparation and the reference material is older and less sensitive to these changes, yet retains all other quality attributes. However, applying stricter criteria is most practical when analyzing trends in analytical parameters and characteristics. Reacting hastily to isolated events can be premature and impractical.
Design and Handling
Identifying suitable AC samples that closely match the behavior of the reference standard and test samples can be challenging, especially when the actual manufactured product has a limited quantity and/or short shelf-life (e.g., cell therapy product where cryostorage is prohibited). The AC sample could be unfeasible in this case, but these scenarios are rare. Typically, for most biopharmaceuticals, a sponsor sets aside a fraction of a drug product lot or drug substance to be used as an AC sample or as a source for AC sample preparation. Sometimes, a dedicated manufacturing campaign (e.g., small-scale preparation at a pilot plant) can be arranged to produce a desired amount of AC samples according to a controlled and qualified manufacturing process.
If AC sample preparation is desired but not feasible at the commercial scale manufacturing plant with a controlled and qualified process, it can be done as a small-scale stand-alone preparation, maintaining process comparability as close to the actual drug product. As in the case with Reference Standards, AC samples are not intended to be used in the clinic, and therefore, the GMP level scrutiny is not applicable since their purpose is to ensure consistency and validity of potency measurements for all drug product batches (test samples) intended for clinical application(s). According to the survey results from BEBPA’s 2023 Reference Material Webinar, preparations of the Reference Standards, and the following AC sample, can differ from drug substance or drug product in aliquot size, storage temperature, container configuration, formulation, aggregate state (liquid, frozen, lyophilized), and other. Obtaining regulatory feedback about preparing and implementing AC samples in the bioassay procedure could be helpful as part of system suitability before method validation activities. Nevertheless, it will be expected that a specific controlled procedure in the form of working instructions or SOP to ensure batch-to-batch consistency of AC samples across different batches is in place to maintain assay reliability. Implementing and enforcing standardized handling procedures are necessary to minimize variability in preparation methods (e.g., reconstitution, dilution, culture conditions) that can introduce bias and compromise assay results, ultimately diminishing the purpose of AC.
Regardless of how the AC sample was prepared, it requires thorough characterization and comparison to the actual product and its reference standard to ensure similarity in biological activity, composition, and stability. Ensuring the stability of AC samples over time is another essential factor for reliable assay performance. Factors such as temperature sensitivity, degradation, or microbial contamination can compromise the integrity of reference-like materials and lead to inaccurate results.
One of the most valuable advantages of using AC samples is the ability to collect a substantial amount of data throughout the assay, which can be used to qualify a new Working Reference Standard. In other words, with a 2-tier reference standard system (Primary and Working RS), any new batch or lot candidate for the next WRS can be introduced earlier as an AC sample. This approach works similarly in the one-tier reference system. Still, it does not alleviate the cascading error issue (known as the daisy chain effect) inherent to the one-tier system. Therefore, programs can benefit from extending AC sample use to qualify new reference standards, provided the preparation and characterization are comparable to the reference standard.
Non-Standard and Out-of-Specification Lots
In some cases, such as laboratory investigations or method optimization, non-standard AC samples can be valuable as part of a system suitability test with adjusted criteria. These samples could represent extreme manufacturing conditions, stressed materials with known residual activity, or different formulations. While not practical for low-throughput in vitro bioassays like 96-well plates, they can be arranged for 384-well plates if desired. The value of this unconventional approach lies in the increased detail of information obtained from each bioassay run, which can be valuable during early-stage method development or optimization or tuning up of similarity equivalence boundaries establishment.
Another scenario for a non-standard AC sample involves a manufactured lot of drug product that is Out-of-Expectations (OOE), Out-of-Trend (OOT), or even Out-of-Specification (OOS). Assuming the OOS potency value falls within the validated method’s range and there is sufficient room for the lower spec limit, this lot could be used as an AC sample throughout its shelf life instead of being discarded. It is a valid approach that holds merit for further exploration. However, this recommendation requires thorough exploration before implementation.
Non-similar AC Sample
This section builds on the previous discussion by considering the inclusion of a non-similar AC sample with known properties. Such a sample can be valuable in specific scenarios where the cause and impact of deviations from expected similarity are understood, at least to some extent, based on the known Mechanism of Action. These deviations can be incorporated into risk-based system suitability criteria to support normal (similar) AC samples to enhance system suitability assessment.
Specifically, this strategy can be used to assess the impact of these deviations on assay performance if:
- The biological property or reason causing a shift in key model parameters used for similarity assessment (e.g., A, D, or B in the 4-pl logistic model, or slope in the parallel-line linear model) is identified.
- This change is believed to be meaningful to the clinical efficacy, or attributable to a specific manufacturing process step, and allows prediction of its outcome.
- The approach is feasible from an operational standpoint (e.g., using a 384-well plate format would be more suitable than a 96-well plate format due to higher throughput).
Summary
This technical note emphasizes the significant value of Assay Control (AC) samples in bioassays, particularly when treated as a reference-like material and implemented with a thorough understanding of assay mechanics, biology, and plate design considerations. AC samples can effectively monitor and track assay performance over time, offering valuable real-time data on both assay characteristics: precision and accuracy. Additionally, they can serve as potential candidates for future reference standards. Ultimately, the decision to utilize AC samples rests with the sponsor. This note serves as a resource, drawing on the author’s experience and guiding analysts through a case-by-case evaluation of AC sample(s) implementation.
Acknowledgments
Special thanks to David Lansky, Ph.D. for his insightful feedback that helped improve the clarity and objectivity of this technical note.
References
- [1] The 4PL, Stanley N. Deming
- [2] BEBPA (2015). White Paper: Assay Acceptance Criteria for Multiwell-Plate-Based Biological Potency Assays [https://bebpa.org/whitepapers/2015-BEBPA-White-Paper-draft-for-consultation.pdf]
- [3] Future Science (2019). Bioassay Techniques: Development and Validation. [https://www.future-science.com/doi/10.2144/btn-2019-0031]
- [4] EDRA Services (2010). A Practical Approach to Biological Assay Validation by Joke Ederveen. [https://edraservices.nl/wp-content/uploads/2022/02/Assay-validation-P08090.pd]
- [5] Q7 Good Manufacturing Practice Guidance for Active Pharmaceutical Ingredients Guidance for Industry [https://www.fda.gov/files/drugs/published/Q7-Good-Manufacturing-Practice-Guidance-for-Active-Pharmaceutical-Ingredients-Guidance-for-Industry.pdf]
- [6] Q2(R2) Validation of Analytical Procedures Guidance for Industry [https://www.fda.gov/media/161201/download]
- [7] Q14 Analytical Procedure Development Guidance for Industry [https://www.fda.gov/media/161202/downloa
About The Author: Anton Stetsenko, M.D., M.B.A.
With 20 years of experience in the pharma/biotech industry, Anton Stetsenko is a consultant in CMC potency method development, validation, and troubleshooting. He specializes in in vitro enzyme activity, cell-based assays, immunoassays, and animal models. Anton has worked at Abbott Laboratories, Fresenius Kabi (formerly APP Pharmaceuticals), Dynavax Technologies, Orchard Therapeutics, ADC Therapeutics, 4D Molecular Therapeutics, and Orca Bio, where he was responsible for all aspects of the potency method lifecycle including comparability studies, method transfer, personnel training, data tracking and trending, customization of data analysis, preparation of regulatory submissions, reference standards, and qualification of critical reagents. This journey allowed him to learn about CMC potency aspects specific to different biopharmaceutical products such as small molecules, biosimilars, vaccines, antibody-drug conjugates, gene and cell therapies. He holds a Doctor of Medicine degree with an emphasis on clinical biochemistry from the Russian National Research Medical University and an MBA in General Business from Lake Forest Graduate School of Management.