Title:Reference Qualification for Relative Potency Assays: You Can’t Prove Not Different Only Not Too Different

Abstract: It is common practice to use statistical equivalence hypothesis testing to justify assignment of 100% relative potency when qualifying a new lot of reference material.  This is accomplished by predefining the maximum amount the mean measured value of the replacement can drift from the original 100% value assigned to the primary reference. Although an underestimate or overestimate of the true value of the new reference lot can introduce bias into test results when compared to measurements made against the primary, the equivalence bounds establish limits and corresponding lot release specifications can budget for such impact.  Because statistical equivalence can only be concluded when the mean measured value and its entire confidence interval (typically 90%) fit within the established bounds, greater statistical power allows for the measured value of a new reference to get closer to the bounds.  Smaller confidence intervals reflect low assay variability and/or high replication.  However, some organizations also require the confidence interval to include the 100% target value so that the new reference result is simultaneously considered equivalent to, and not different from, the primary.  Note that this additional latter requirement demands a greater measurement uncertainty (i.e. lower statistical power) as the true value of the new lot moves towards the equivalence bounds.  This presentation will highlight a common misunderstanding of statistical difference testing that underpins this counterintuitive conundrum.