Fat Finger, Falsification or Fraud?

When does human error or a fat finger moment slide down the slippery slope to falsification and fraud?

A central component of ensuring the integrity of data and results generated in any chromatography laboratory is the human element, that is, the chromatographer or the analytical chemist who will be involved with developing and validating methods or performing sample analysis. Mistakes or fat finger moments are part of human nature but where is the dividing line between this and falsification and fraud? We will discuss this further in this article.

Deviations From Data Integrity

In the title of this column I have suggested that there are three types of data integrity deviation: fat finger, falsification and fraud. Here are my definitions of the terms:

• Fat Finger: an inadvertent mistake made by an analyst during the course of their work that can be made either on paper or electronically.

• Falsification: an individual who deliberately writes or enters data or results with the intention to deceive.

• Fraud: collusion between two or more individuals who deliberately write or enter data or results with the intention to deceive.

I have drawn a clear distinction in the definitions of falsification and fraud: falsification is perpetrated by an individual and fraud by two or more people, however, the impact of both is the same — the intent to deceive.

In writing this column, I have made the following assumptions:

• Each chromatographer working in a laboratory has a minimum level of scientific and professional training to undertake their work. For example, in method development, method validation, analysis of samples, or review of work and the preparation of a report.

• All chromatographers will follow documented analytical methods and laboratory standard operating procedures (SOPs) or work instructions (WIs).

• The organization the individual works for has stated the ethical and professional standards expected of its staff at their induction and via regular training sessions thereafter.

To Err Is Human

Mistakes and fat finger moments? If we are honest, we all make them. That is why any quality system for laboratories (e.g. ISO 17025, Good Laboratory Practice and Good Manufacturing Practice) has the four eyes principle: one individual to perform the work and a second one to review the data produced to see that the procedure was performed correctly and that there are no typographical errors or mistakes with calculations. Errors are easy to make, you should see the number I'm making as I type this column using a new PC that has a different keyboard than I am used to.

Many of the errors and mistakes that we make are self-corrected. For example, as you enter a number into a spreadsheet cell, database field or report, often you will find that while the brain says enter 12.3 your fingers magically enter 13.2 instead. This is a fat finger moment, but before committing the number to the cell or database you can correct this as you can see and have realised your own error. The equivalent moment on paper is when you actually write the wrong numbers down in your laboratory notebook and then correct it by striking through the original entry so as not to obscure it and entering the correct value along with your initials and date and possibly the reason for change. This is the paper version of an audit trail.

Some other mistakes may not be noticed by a chromatographer but could be detected by the software application that they are using such as a spell checker, or by verification that the data entered fails to meets certain criteria, such as within a predefined range or specific format by a spreadsheet, Laboratory Information Management System (LIMS) or Chromatography Data System (CDS). So, using the example above, if the data verification range was 11.0 to 13.0, the software would have picked up the problem and warned you even if you had not noticed the error.

However, that still leaves the mistakes you don't realise you have made. For example, if the entry in the case above was 11.3, data verification would be useless and the error would have been entered without you or the software realizing that there was a problem.

Don't assume that you will spot all of your own mistakes because we are all human and error prone, which is why we need the second pair of eyes to check our analytical data and calculated results. From my experience as a laboratory manager and as an auditor, supervisors know which members of their staff are diligent about their work and how well they check it and those individuals who are slapdash. A supervisor will adjust their second person reviews accordingly. So if you don't want a dubious reputation to precede you, be diligent and try your best to find and correct your own errors before passing your work over to be checked.

What is the Fat Finger Rate in a Laboratory?

To a certain extent, this column is about airing dirty laundry which may not be a particular interesting problem to all but it is at the heart of any good laboratory quality management system: self audits coupled with effective corrective and preventative action planning. Quality is everybody's problem: it is not the sole responsibility of the quality assurance group to pick up the errors that the analytical laboratory has made. However, finding papers on how often we make mistakes in an analytical laboratory is hard to find — probably as we don't really want to go there. But this is the wrong approach to take and we should encourage studies to investigate this.

Help is at hand however from clinical chemists working in hospitals who have published many studies on error rates in laboratories. For those that do not know, clinical chemistry is involved in the analysis of blood, urine and other bodily outputs to help in the diagnosis and management of diseases. Mistakes in this area can have a critical impact on the health of a patient and therefore the reduction in errors is essential.

• One paper, entitled "The Blunder Rate in Clinical Chemistry", measured the rate of detected analytical errors before and after the introduction of a LIMS: it was reduced from about 5% to less than 0.3% following the introduction of computer system (1).

• Manual transcription errors in patient records were assessed for blood results recorded in a critical care setting by comparing the handwritten and printed laboratory results in 100 consecutive patients in the intensive care unit of a UK hospital. Out of 4664 individual values, there were 67.6% complete and accurate, 23.6% were not transcribed at all and 8.8% were inaccurate transcriptions of the results. Interestingly, this study found that the most accurate work was performed in the morning (2).

• An Australian study of transcribing hand-written pathology request forms to a computer system and chemical analysis of the samples found that error rates were both in the range 1–3% in the best laboratories. However, the worst laboratories had error rates of up to 39% in transcription and 26% in analytical results (3) which, in my view, would indicate a sloppy approach to quality by management in these organizations.

So, let us extrapolate from the clinical chemistry laboratory that the range of error rates in an average analytical laboratory is in the range of 0.3% to 3% depending on the degree of automation that exists there. The more manual input and transcription checking required, the greater the number of errors that need to be detected and captured. Therefore, laboratory errors are expected by external quality audits and regulatory inspections. Not finding these detectable errors raises suspicion of problems with the result being a further delving into laboratory records.

Therefore, depending on the level of automation involved in an individual laboratory, the quality system supervisors and chromatographers need to be aware of the existence of errors and the ways that can be used to reduce them, such as training, awareness and responsibility for error prevention and detection, mechanisms for manual review of data, design of pro-forma documents to capture laboratory data to ensure data integrity and objectives for performance review.

The Laboratory Notebook – Integrity or Falsification?

This brings us to a common issue that we all have experience with: the humble laboratory notebook. Typically this is a bound book with pre-numbered pages which are just there to prevent you tearing out a page to write down a shopping list or making a paper aeroplane from it, and, more importantly, as the first stage of ensuring data integrity in the laboratory. At the bottom of each page is space for you to sign and afterwards a reviewer/supervisor/witness/peer to sign after they have checked your work and accepted it as accurate.

Here's the situation: You are a supervisor and you are checking a laboratory notebook for some current work and in turning the page you notice that your signature is missing from when you reviewed some earlier work. Three out of four pages of the old work are signed and dated but you have omitted to sign one of the pages: so what do you do?

Temptation Time! You have the following options:

1. Ignore the problem and wait for somebody else to discover it?

2. Sign the page and date it the same as the other pages?

3. Sign the page but date it with the current date and add a note that you have just noticed the problem?

So what are you going to do? It is a pity that the paper and electronic versions of this magazine do not come with the prospect of having a large hammer hit you over the head if you pick the wrong option. Most chromatographers should reject option one, especially if you are working in a research environment where product development and especially patent protection can be crucially dependent on the date of discovery.

So we're down to options two and three. For option two is a little voice whispering in your ear "nobody will know if you put the same date that the other pages were signed on?" You are now on the brink of the abyss – on the sunny upland plateau is ethics and integrity and down the dark slippery slope is falsification and fraud. May I suggest to you that option three is the only option worth considering that will establish credibility for you and the laboratory? Reiterating the point in the section above and putting my auditor's hat on – I expect to see mistakes: if I don't find any I become suspicious.

Laboratory Fraud and Falsification

Let us move into the murkier world of falsification and fraud with the intent to deceive. Falsification and fraud can be classified into the following two main areas:

• Acts of omission — deliberately not revealing all data available and basing any decisions made on a selection of results that fit the desired outcome, for example, passing a test and ignoring the rest

• Acts of commission — deliberately altering data or fabricating data as shown with the substitution of the IR spectra or the Able Laboratories fraud case that we will discuss in a minute.

A good source for finding examples dealing with both these issues is the FDA warning letters section found on www.fda.gov. The agency posts warning letters on their website under the US Freedom of Information Act, with the intent to name and shame. I quote these examples from the pharmaceutical industry as the FDA openly publishes this information, as opposed to the European regulators or ISO 17025 accreditation agencies who usually keep such findings confidential. However, it is important to note that falsification and fraud could occur in any laboratory in any industry.

The following information is taken from five of the warning letters and regulatory issues that have emerged in this arena recently.

1. The classic fraud case involving laboratory data is that of Able Laboratories from 2005 (4). The company was engaged in a systematic laboratory fraud to pass batches of drug product that failed to meet specifications by changing weights, conversion factors and even cutting and pasting chromatograms. Results that failed were manipulated and faked until they passed: an original result for dissolution testing was ~ 30% versus a specification of > 85% but after the magic fingers were applied the final result was ~ 89%! The company had passed several regulatory inspections until a whistleblower alerted the FDA to these practices. After a detailed inspection, the company withdrew several drug applications, recalled over 3100 batches of product and eventually went bankrupt. There was a subsequent criminal prosecution of four members of the company for fraud.

2. During an inspection of Ohm Laboratories in 2009, suspicion was aroused in the stability testing laboratory about material that had been taken out from the stability chambers for analysis. The material had been signed out by the stability coordinator but the attendance record shows that your stability coordinator was absent from your firm during those dates in which the coordinator recorded the withdrawal of samples from the stability chambers (5). This is very similar to the laboratory notebook example we discussed above.

3. A Chinese company, Xian Libang Pharmaceutical Co, (6) was found to have used the IR spectra from one batch of material to support the release of two subsequent batches. The warning letter noted that this practice is unacceptable and raises serious concerns regarding the integrity and reliability of the laboratory analyses conducted by your firm. It is essential that at least one test be conducted to verify the identity of each lot of incoming material. In addition, the laboratory control records should include complete documentation of all raw data generated during each test, including graphs, charts and spectra from laboratory instrumentation. These records should be properly identified to demonstrate that each raw material batch was tested and met the release specification before its use in production. .... A cursory review of records is not sufficient to ensure that other personnel did not manipulate or inaccurately report test data.

It is interesting to note that finding falsification in one analysis, the agency, quite rightly, casts doubt on all data generated by the whole laboratory.

There was a further citation about the lack of controls to prevent manipulation of raw data during routine analytical testing and how measures would be put in place to stop unauthorized changes being made to data in the future. The agency wanted to see a process to prevent omissions in data, but also for recording any changes made to existing data, which should include the date of change, identity of person who made the change, and an explanation or reason for the change. All changes to existing data should be made in accordance with an established procedure.

4. A warning letter sent to Ranbaxy, an Indian generic drug manufacturer, in 2007 highlighted "untrue errors of material fact" made in drug submissions to the United States in the stability testing programmes run by the company (7). In item 5 of the warning letter it was noted that the results of an audit of existing drug submissions had found "2257 errors in entries for the dates of analysis; and errors in 1385 entries for stability test results, and tests for which corrections were made in specification limits". This is not a mass fat finger moment and I leave it to your interpretation if this is falsification or fraud.

5. My last example cost a European generic drug manufacturer a loss of $3.3 million in 2010 (8). Acino, a Swiss generic drug manufacturer, contracted Glochem, an Indian company, to supply clopidogrel which is the active ingredient of Plavix. Following a visit by European inspectors to the Indian company, they found more than 70 original batch records in a rubbish skip at the site — all the records had been rewritten to be perfect with no errors, in total contradiction of Good Manufacturing Practice (GMP). The inspectors, again quite rightly, classified this as fraud and this triggered a recall of the material.

The company thought that the inspectors response was too excessive and commissioned extensive third party analysis to demonstrate that the material met specifications. However, as the batch records had been copied and the originals were in the process of being destroyed the inspectors held to their original view.

You can see from these few examples, by being diligent, honest and professional you can avoid the problems faced by these companies. The fifth example also illustrates that if a company outsources to a third party the first company is still accountable for the quality of the material going into their own supply chain and pro-active auditing will help prevent these issues.

As a result of a number of cases of falsification and fraud, particularly the Able Laboratories case, the FDA have reappraised their approach to inspections, especially the preapproval inspection (PAI) and have issued a Compliance Policy Program (CPG) that becomes fully effective in May 2012. CPG 7243.836 (9) is entitled Pre-Approval Inspections and has three objectives for an inspection. In our discussion we are only interested in objective 3: data integrity.

Audit the raw data, hardcopy or electronic to authenticate the data submitted in the CMC (Chemistry, Manufacturing and Controls) section of the application. Verify that all relevant data were submitted in the CMC section such that CDER product reviewers can rely on the submitted data as complete and accurate (9).

The inspector will compare raw data, either paper or electronic files, laboratory analyst notebooks and additional information from the laboratory, with summary data filed in the CMC (Chemistry, Manufacturing and Controls) section. The CPG states explicitly:

Raw data files should support a conclusion that the data/information in the application is complete and enables an objective analysis by reflecting the full range of data/ information about the component or finished product known to the establishment. Examples of a lack of contextual integrity include absences in a submitted chromatographic sequence, suggesting that the application does not fully or accurately represent the components, process and finished product (9).

So to prevent data integrity problems being observed during manufacturing inspections, the FDA believe that if the submission data are suspect then a manufacturing licence will be withheld and the manufacturer will not be allowed to sell their product in the USA.

Quality and Integrity: How Should We Prevent Fraud and Falsification?

There are a number of ways that we can avoid the problems of fraud and falsification. The first is to develop clear written policies and procedures of what is expected when work is performed in any laboratory: the integrity of the data generated in the laboratory is paramount and must not be compromised. This is the "Quality" aspect of the Quality Management System that you work under.

There is the parallel need to provide initial and on-going training in this area. The training should start when a new chromatographer joins the laboratory and should continue as part of the individual's on-going training over the course of their career with the laboratory.

To help training staff we need to know the basics of laboratory data integrity. The main criteria are listed below:

• Attributable — who acquired the data or performed an action and when?

• Legible — can you read the data and any laboratory notebook entries?

• Contemporaneous — documented at the time of the activity

• Original — written printout or observation or a certified copy thereof

• Accurate — no errors or editing without documented amendments

• Complete — all data including any repeat or reanalysis performed on the sample

• Consistent — all elements of the chromatographic analysis such as the sequence of events follow on and dates or time stamped in expected sequence

• Enduring — not recorded on the back of envelopes, cigarette packets, post-it notes or the sleeves of a laboratory coat but in laboratory note books and/or electronically by the CMS and LIMS used in the laboratory

• Available — for review and audit or inspection over the lifetime of the record.

Chromatographers need to understand these criteria and apply them in their respective analytical methods regardless if working on paper, hybrid CDS or fully electronic CDS.

To support the human element we should also provide automation in the form of integrated laboratory instrumentation with data handling systems and LIMS as necessary to perform the work. In any laboratory this integration needs to include effective audit trails to help maintain data integrity and monitor changes to data. Supervisors and quality personnel need to monitor these audit trails to assess the quality of data being produced in a laboratory — if necessary a key performance indicator (KPI) or measurable metric could be produced. Finally, if all else fails, disciplinary procedures need to be in place and be used to resolve any problem as the reputation of the laboratory is of paramount importance.

Conclusions

In this article I have looked at errors caused by fat finger moments which are normal and unavoidable and why we need a second person to check our data and ensure that results generated are correct and can withstand quality scrutiny. The errors can be reduced greatly by using software to capture and transfer data automatically, eliminating the need for manual entry of data followed by transcription error checking. We have also looked at falsification and fraud, particularly at ways of ensuring that none occur in your laboratory.

If we do not ensure Quality by Integrity (QbI) for all data and results then all work towards Quality by Design (QbD) is wasted effort.

"Questions of Quality" editor Bob McDowall is Principal at McDowall Consulting, Bromley, Kent, UK. He is also a member of LCGC Europe's Editorial Advisory Board. Direct correspondence about this column should be addressed to "Questions of Quality", LCGC Europe, 4A Bridgegate Pavilion, Chester Business Park, Wrexham Road, Chester, CH4 9QH, UK or e-mail the editor Alasdair Matheson at amatheson@advanstar.com

References

(1) A.M. Chambers, J. Elder and D.StJ. O'Reilly, Annals Clinical Biochemistry, 23, 470–473 (1986).

(2) R. Black, P. Woolman and J. Kinsella, Presented at American Society of Anaesthesiologists Annual Meeting, New Orleans LA, October 2001.

(3) M. Khoury, L. Burnett and M.A. Mackay, The Medical Journal of Australia, 165, 128–130 (1996) (http://www.mja.com.au/).

(4) R.D. McDowall, Quality Assurance Journal, 10, 15–20 (2006).

(5) Ohm Laboratories, Warning Letter (December 2009).

(6) Xian Libang Pharmaceutical Company, Warning Letter (January 2010).

(7) Ranbaxy Laboratories, Warning Letter (February 2009).

(8) J-F. Tremblay, Chemical & Engineering News, 88(34) 23, August 23, (2010).

(9) FDA Compliance Program Guide 7346.832, Pre Approval Inspections, (2010).