ISPE Quality Metrics Pilot Program: Detailed Metrics Definitions
Following detailed evaluation of the various industry and regulatory considerations related to proposed pharmaceutical quality metrics, the ISPE Pilot Program metric set was developed to include 14 measures reflecting a mix of leading and lagging indicators collected by site and product. Twelve (12) of the metrics collect quantitative data responses and two (2) of the metrics (for Process Capability and Quality Culture) collect qualitative data responses using a survey based format. Click here for a description of each metric and its associated definition(s).
As the public discussion of quality metrics continues, the importance of how a given metric definition is established (both the numerator and the denominator) is being increasingly recognized. Early experience from Wave 1 of the ISPE Pilot Program is providing the following insights:
- Standardizing metrics and definitions across companies and sites is feasible, but challenging.
- Definitions and terminology must be exact; in particular, denominators can have a big impact on a company’s ability to report on the metric with either ease or difficulty.
- Frequency of data collection (e.g. quarterly, annually) has a major impact; analytic and reporting burden is proportional to the number of data points collected.
- Site-level metrics are more easily systematized for collection.
- Product-level metrics are significantly more complicated to collect; typically requiring a company to pull data from multiple systems and can rapidly result in large data management requirements.
The ISPE Pilot team is targeted to conclude the data collection period for Wave 1 on 30 November, 2014. This will enable the team to analyze the data and prepare reports for participating companies, as well as publishing an overall report for the benefit of industry and regulators in 1Q 2015.
ISPE and FDA are also working together to organize a jointly chaired workshop, targeted for April 2015, to discuss industry learnings from the Pilot Program as well as other ongoing efforts related to quality metrics.
In parallel, the ISPE team also is working to develop a Wave 2 for the Pilot Program in order to:
- Expand the number of participants
- Assess a potentially adjusted set of metrics based on learnings from Wave 1 and input from sub-teams working on Leading Quality Indicators / Quality Culture and Process Capability
- Increase focus on current vs retrospective data collection and analysis
- Provide a forum to encourage cooperation across industry groups in order to drive the dialogue and understanding on the use of pharmaceutical quality metrics