A 70-person PerkinElmer OneSource on-site team takes complete responsibility for maintaining and qualifying more than 50,000 Merck Research Laboratories assets in six facilities.
Practical advice for handling the second phase of out-of-specification investigationsIn the previous two articles, we discussed the background of laboratory out-of-specification (OOS) investigations and Phase I of the OOS investigation as outlined in the U.S. Food and Drug Administration (FDA) OOS guidance. If Phase I (see Figure 1) of the laboratory investigation does not identify an attributable laboratory error as the cause of the original OOS observation, the OOS guidance allows the organization to move into an expanded laboratory investigation (Phase IIB) in parallel with a review of the production activities (Phase IIA).
How the two parts of the investigation are coordinated depends on the corporate culture. Review of production records should be a priority, because there is a reasonable possibility that it will reveal that the OOS is attributable to an error in the operations area.
The Phase IIB (Step 11) laboratory investigation focuses on retesting to demonstrate that the original value does not represent the material. There are a number of conflicting opinions and expectations that must be considered as the organization develops its OOS retest policy, procedure, and protocols:
Unfortunately, many do not comprehend the impact of completing these tests with failing results. The laboratory might consider a policy or procedure that requires that the analyst inform laboratory management when moving to the second stage of testing so that management is aware of the potential problem. Because the rest of the organization does not understand that these procedures already include the retest, it will expect the laboratory to move to the OOS observation investigation mode. In reality, the organization should move into the investigation of a confirmed material failure, which will be driven by quality assurance and will involve the whole organization.
Any laboratory OOS standard operating procedure should clearly identify the company’s policy on the use of outlier testing and the expectation that the cause of the original OOS observation (a deviation) will be identified. The following observations in an FDA warning letter demonstrate that the investigators are looking for the identification of the cause of the deviation:
“Your OOS procedure contains no provision for conducting retest of new samples, yet your firm released these batches for distribution based on passing repeat test results without conducting a thorough investigation as required under 21 C.F.R. §211.192 to support your conclusion and rationale to release the affected lots … . Your firm’s OOS investigation relating to impurity levels for _____ concluded that the root cause was laboratory error, but the investigation did not identify what specific laboratory error occurred.”
The regulatory expectation requires that retesting be conducted according to an approved protocol. Since the landmark 1993 Barr ruling, industry representatives have asked the FDA and others for a recommendation on what should be in the protocol. This is not addressed in the OOS guidance. The following should be identified in the OOS procedure as required content for the retest protocol, and the points should be covered in each retest protocol:
The sample that will be used for the retesting. Retest replicates should come from the same sample that was used for the original test. Under circumstances in which the original sample is unstable, the sampling procedure will identify this and proceduralize resampling in the event of retest. If the original sample is inadequate, the sampling plan does not meet current regulatory expectations that the laboratory sample be adequate for the original test and any necessary retesting in the event of an OOS observation. If the laboratory continues, a resample will have to be justified and a resampling plan prepared and approved by the quality unit before resampling occurs and retesting is begun. The original, inadequate sampling plan should be investigated and corrected through the site deviation system.
The analysts who will perform the retesting. The draft OOS guidance stated that the original analyst should not be included in the retesting. This requirement could cause a hardship for some laboratories. The final guidance suggests that at least two analysts, not including the original analyst, be involved in the retesting. Some feel that there is value in having the original analyst participate. Any analyst, including the original, could have a procedural bias that impacts his/her results. That potential must be considered in the preparation of the retest protocol and the evaluation of the data.
Suggestion: Retesting should be performed by three analysts, including the original. The protocol should have acceptance criteria for intra- and inter-analyst result agreement, as well as the requirement that any lack of conformance to these criteria be resolved before the data analysis for an outlier begins.
The number of replicates that will be tested. The purpose of retesting is to demonstrate that the original value, the OOS observation, does not accurately reflect the material under test. This is done by demonstrating that the original value is a statistical outlier when compared to multiple, independent retest results of the same material. Two retests will not provide any statistical confidence. Five or six retests will provide borderline confidence. Several possible retest scenarios are shown in Figure 2. Scenario 1 has been published and accepted by many. It is based on a chromatographic procedure in which each injection is calculated as a separate result. However, the individual results lack the independence appropriate for the evaluation. In the other scenarios, the complete test procedure is followed for each retest sample preparation. This includes replicate injections and calculations defined by the procedure.
Suggestion: Use nine retests, three by each of three analysts. The protocol should require intra- and inter-analyst agreement as part of the overall outlier study.
How the results will be evaluated and what constitutes an outlier. The protocol should identify acceptance criteria for the test results before they are evaluated for the outlier. Where possible, the laboratory should use a statistician to lead it through this evaluation. If a statistician is not available, the laboratory can use one of the standard statistical packages. Test method system suitability criteria must be met. There should be acceptance criteria for intra- and inter-analyst agreement. For the determination that the original value is an outlier, a number of statistical routines have been used. The student T is one routine that is used often; however, the use of any routine to identify a value as an outlier must be justified. Any criteria for determining whether or not the original value is an outlier must be established in the protocol before retesting is initiated.
Suggestion: One possible criterion to consider is the following: The original is an outlier if it is outside of calculated average ±3σ for the retest results. When retesting is complete, the laboratory will report its complete investigation, results, and conclusions to quality assurance, which will be responsible for determining the disposition of the material.
Often the firm looks for ways to measure the performance of the laboratory OOS investigation process. Performance measures that should be considered include: