Why System Suitability Tests Are Not a Substitute for Analytical Instrument Qualification or Calibration (Part 1)

Can you qualify or calibrate your chromatograph using a system suitability test? No. However, for the doubters amongst you I'll spend the rest of this column and the next one explaining why!

In this two-part column I'll be discussing the role of system suitability tests (SSTs) in the context of analytical instrument qualification (AIQ) and the generation of quality analytical results. I'll look at this first from the perspective of regulatory requirements based upon the United States Pharmacopoeia (USP) general chapter 1058 (<1058>) on AIQ.¹ However, as much of this discussion is also good analytical science I will also look at this from the perspective of ISO 17025,² the interpretation of the ISO 9000 quality management system for testing and calibration laboratories. In this column I'll use the ISO 17025 term of calibration to be equivalent to AIQ and discuss the reason why I have taken this approach.

Specifically, I want to discuss the question why system suitability tests (SSTs) are not a substitute for AIQ for the initial qualification or re-qualification of a chromatograph (i.e., a periodic check or operational qualification) under GMP or calibration under ISO 17025. The reason for this is that there is a common misconception in some laboratories that SSTs can be used to qualify a chromatograph. This is wrong. Furthermore, using SSTs as the sole instrument qualification approach will leave any laboratory exposed to regulatory action or non-conformance as the chromatographs cannot be demonstrated as being fit for their intended purposes. Therefore, we will explore the reasons for this from both the regulatory and quality perspectives.

What is a System Suitability Test?

An SST is simply a check to see if the chromatographic system (instrument modules, column and mobile phase) is capable of undertaking the analysis you require. It is performed immediately before the samples you have prepared in the laboratory are committed for analysis. At it's simplest, an SST is the single injection of a standard solution of reference compound(s) to see if the separation is 'OK'. The only criterion used for assessing if the system is acceptable is a visual check by the chromatographer to assess if the separation is within expected boundaries. However, there are no measurements made: just a simple visual assessment that things are right.

At the other end of the scale are the formal SSTs described in the Pharmacopoeias used in pharmaceutical QC laboratories. Here five replicate injections of a standard are injected and the calculated peak areas and other chromatographic criteria (retention time window, peak shape, peak tailing, resolution between peaks etc.) are objectively compared with predefined specifications defined by the Pharmacopoeia (peak area repeatability) and the laboratory (all other chromatographic criteria). If the SST injections meet all the predefined criteria then the chromatographic system is deemed acceptable and the samples committed for analysis.

GMP and Pharmacopoeial Regulations

Good manufacturing practice (GMP) regulations from either the US or Europe state simply that equipment used in the manufacture of drug products should be fit for their intended use as can be seen in Table 1, which presents the US regulations.³ Also, shown in Table 1 are the requirements for ensuring that scientifically sound specifications for the qualification work are written and that qualification and calibration of instruments is documented correctly. Note also that if an instrument fails to meet its specification it must be taken out of service until the problem is rectified and retested to show that it meets its specification.

Table 1: Selected US good manufacturing practice (GMP) regulations for instruments and equipment.

However, for chromatographic equipment these GMP regulations are expanded by either the USP <621>⁴ or the European Pharmacopoeia (EP) chapter 2.2.46,⁵ which have both specified requirements for SST for chromatographic analysis to demonstrate that a chromatograph is fit for the analysis it will undertake on the day of analysis. System suitability tests must be run each time before an analysis is undertaken and each SST is specific for an individual method with pre-defined acceptance criteria, for example, precision, peak shape and resolution from other analytes. If an SST fails, then the samples cannot be assayed.

The principle of a point of use check, such as an SST, is applicable to any analytical instrument or system and this is performed just before an analysis to demonstrate correct performance, for example, a check to see if the chromatographic system or an analytical balance works before committing samples. This point of use check is used in other types of instrumental analysis, for example, balance and pH meter checks so it is not an unusual approach and is good analytical science.

I presented the main elements of USP <1058> on analytical instrument qualification (AIQ) in an earlier Questions of Quality column,⁶ where we looked at the impact of this on the chromatography laboratory. This informational general chapter consists of six major sections:

• Components of data quality.

• Analytical instrument qualification process.

• Roles and responsibilities.

• Software validation.

• Change control.

• Instrument categories.

The key sections from USP <1058> that concern our debate of AIQ versus SST are the components of data quality and the analytical instrument qualification process. The quotations from USP <1058> mentioned later in this column USP <1058> come from these two sections. I will focus my comments on the first section of USP <1058> that discusses the components of analytical data quality. In my earlier column⁶ I skipped over this section but I would like to go into further detail, as the content is good analytical science and is applicable to all laboratories.

ISO 17025 Accreditation

OK so what is ISO 170252 and what is accreditation? First question first, ISO 17025 is an international standard that both testing and calibration laboratories can work against; for the purpose of this column I will focus on testing laboratories mainly. A major difference between ISO 17025 and GMP is that compliance with the requirements of the ISO standard is voluntary rather than imposed by government agencies. However, the advantage of working to ISO 17025 is that a laboratory is working against an internationally recognized standard. To answer the second question, accreditation is the process of ensuring that the laboratory has implemented and complies with the requirements of the ISO 17025 standard and continues to do so. This is achieved by an independent audit by technical officers from the local accreditation body coupled with external experts in the specific areas offered by a laboratory. Each country in the European Union has an accreditation body (UKAS for the UK, COFRAC for France etc.) that is responsible for the accreditation of laboratories in their respective country. However, ISO 17025 goes further than GMP in that the test methods used by a laboratory and the performance of each one are also covered within the scope of a laboratory's accreditation and are included in the assessments made by accreditation bodies.

ISO 17025 Requirements for Equipment

When considering ISO 17025² we have to remember that this standard is a specific interpretation of ISO 9000 for testing and calibration laboratories. In this column I will focus mainly on the requirements for testing laboratories that use chromatographic equipment to perform their work. The applicable sections of ISO 17025 are presented in Table 2 and will now be discussed and interpreted in more detail below. Please note that key portions of the standard, as were the US GMP regulations, are highlighted for the purposes of this column and the full sections should be interpreted in practice.

Table 2: Selection of the main ISO 17025 sections covering instrument qualification.

The first requirement for equipment is in section 5.5.2 that notes that all laboratories are required to implement and operate a programme for the maintenance and calibration of equipment used in the laboratory. However, these calibration checks are only required where the equipment setting can significantly affect the test or analytical result (e.g., oven temperature, gas or liquid flow, detector wavelength accuracy and photometric linearity for GC or LC modules). Proper equipment use combined with periodic servicing and calibration does not necessarily ensure that an instrument, in this case a chromatograph, is performing adequately, there also has to be point of use checks of overall system performance that can be built into test methods (e.g., chromatographic system suitability tests). These checks should be documented and should be satisfactorily completed before the equipment is used or before results are accepted.

Reconciling GMP Regulations and ISO 17025 Requirements

How can we have a common understanding of both the GMP and ISO 17025 requirements so that we can have a debate that is independent of applicable regulations or requirements? This all comes down to definitions and what we mean by specific words. So here are the key terms used in either regulations or standards and let's see how we can reconcile them to help us in the debate:

• Equipment or Instrument? In GMP the term "equipment" is used to cover anything used in the manufacture of pharmaceutical products that includes the laboratory. Similarly, ISO 17025 uses the term equipment to cover the wide range of equipment, instruments and systems used in the testing laboratory. This is in contrast to USP <1058> that uses the term instrument. However, in practice the two terms are synonymous as we can refer to either chromatographic instruments or chromatographic equipment. I prefer the term instrument and will use that in the remainder of this column.

• Calibration or Qualification? Both GMP and ISO 17025 refer to calibration in contrast to qualification used by USP <1058>. Are the two terms equivalent? Yes in my view. Calibration is a simpler term to understand and has been defined by EU GMP as the set of operations which establish, under specified conditions, the relationship between values indicated by a measuring instrument or measuring system, or values represented by a material measure and the corresponding known values of a reference standard.⁷ I would argue that the work undertaken in qualification and calibration of an instrument is essentially the same: does it work for your purpose and can you demonstrate that by using calibrated test instruments and traceable reference standards?

• Maintenance? All regulations and guidance refer to these terms so there is full agreement and thus no need for interpretation

Figure 1

US FDA Warning Letter Citation

An example of how a laboratory can fall foul of a regulator is found in the US Food and Drug Administration (FDA) warning letters publicly available on the Agency website (www.fda.gov). Shaanxi Hanjiang Pharmaceutical Group, a Chinese company, received a warning letter in January 2008 following an inspection of its facilities in September 2007.8 Quoting from the warning letter we can see that:

3. Calibration of measuring devices was not traceable to the national standards.

Several balances and thermometers were not traceable to national standards. We acknowledge your firm's submission of a certificate of metrology <redacted> which expires on 16th April 2008. However, this certificate is deficient in that it does not include identification of the instrument for which it is meant for and does not include a statement of traceability.

So we can see the requirement for traceability outlined in ISO 17025 can be mirrored in GMP. The only problem is that there is no explicitly written requirement for this as you can see in Table 1. However, as you can see in 211.160(b) in Table 1 there is a need for the laboratory controls to be 'scientifically sound', as traceability of calibration is good science. Therefore, if you have not calibrated or qualified your instruments with appropriate and traceable reference standards then the data you produce from those instruments will be poor.

Conclusions

In this column, I've looked at the regulations and quality standards for qualification and calibration of chromatographic instruments. In part 2 we'll look at how the data quality triangle can be applied to this from a practical perspective.

"Questions of Quality" editor Bob McDowall is principal at McDowall Consulting, Bromley, Kent, UK. He is also a member of LCGC Europe's Editorial Advisory Board. Direct correspondence about this column to "Questions of Quality", LCGC Europe, Poplar House, Park West, Sealand Road, Chester CH1 4RN, UK, or e-mail Alasdair Matheson, the editor, at amatheson@advanstar.com

References

1. United States Pharmacopoeia general chapter <1058> Analytical Instrument Qualification.

2. ISO 17025 General requirements for the competence of testing and calibration laboratories, International Standards Organisation, 2005.

3. Good Manufacturing practice for finished pharmaceuticals (21 CFR 211), 2008.

4. United States Pharmacopoeia general chapter <621> Chromatography.

5. EP Chromatography chapter 2.2.46 Chromatography.

6. R.D. McDowall, Questions of Quality, LCGC Europe, 22(4), 204–210 (2009).

7. EU Good Manufacturing Practice Glossary.

8. Shaanxi Hanjiang Pharmaceutical Group, FDA Warning Letter, January 2008 (www.fda.gov).