Last week a BMJ investigation reported that one of the world’s leading producers of diagnostic tests has been falsely marketing one of their products. Oxoid, owned by US diagnostics giant Thermo Fisher, has been selling antimicrobial susceptibility test (AST) discs that do not always contain the advertised amount of antibiotic.

The $10bn a year company’s website claims that it is “renowned for quality, accuracy, reliability and innovation.” However, internal documents seen by the BMJ show that this reputation may not be entirely justified.

They show how the company used flawed data and misled the UK regulator, the Medicines and Healthcare Products Regulatory Agency (MHRA), when concerns were raised. The company say they comply with all legal and regulatory requirements.

The MHRA issued a statement in response to the BMJ investigation:

“We have investigated two separate issues involving test kits from this manufacturer. We took action to notify healthcare professionals in 2005 about problems with one batch of a specific test kit after receiving safety information from the manufacturer. This specific test kit was then removed from the market and reintroduced in 2010 after undergoing modifications to improve safety and performance. We were contacted by a former employee of the manufacturer in 2008 and 2011 raising concerns about the safety profile of this test kit.  We investigated these allegations and on each occasion we concluded that there was no evidence that the device did not perform as it should.
 
“The second issue relates to sporadic manufacturing problems with different test kits. In each case the manufacturer wrote to affected users and we published these letters on our website to ensure that healthcare professionals are aware of the problem. Healthcare professionals should continue to use these test kits as advised and we are continuing to work actively to ensure that these test kits remain safe and that the manufacturer is complying with the relevant safety regulations.”
 
A spokesperson confirmed that the MHRA checked the data provided by the manufacturer and they were unable to give any further details under European legislation.

However, the BMJ did not draw attention to two specific instances only and referred to a catalogue of bad practices.

A spokesperson from Thermo Fisher told the Daily Telegraph that the article does not “reflect the considerable factual information we provided to the BMJ” in response to the questions.

In the interests of transparency, the full on the record responses sent by Oxoid are linked to this blog. However, what they say is confusing and littered with contradictory statements. The BMJ sent two sets of questions, some of which we will attempt to unpick. 

1. Validation of assay methods is critical to check that tests are being used properly and work as they should. The FDA confirmed to the BMJ that validation is necessary. It’s not entirely clear what the MHRA thinks, as they have said they have dealt with past issues. But does Oxoid validate their assays?

Their first response says (document dated 09 January 2013) they use validated processes, but in a later response (15th January), the company states that they have no internal validation. Nor do they answer the question about whether they’ve given up trying to validate their assays.

Oxoid tries to caveat this by saying that internal validation is not required (response 15th Jan). But later the company admit that recommendations were made to ‘improve’ the R value., which included using A-grade glassware and other changes.

2. Microbes tend to respond in a non-linear way to antibiotic discs and again, Oxoid’s response is confused. When the BMJ asked about sudden improvements in R values (the methods became linear), Oxoid said this was due to improvements in techniques (15 January), yet they had just admitted that responses are curved (low R value).

3. A clear zone forms around the antibiotic disc if a microbe is susceptible. These are measured by callipers and have a degree of inaccuracy and a range of results tend to be recorded. Indeed, an expert said Oxoid’s ability to measure consistently to below 0.3mm for a single concentration was “too good to be true”.

Oxoid say this is due to the level of control they have over measurements (15 January), though previously they state (9 January) that: “All discs have an inherent level of inaccuracy because of the nature of the test” and also there is “variability of the assay in the real world”.

4. Discs are made to certain “standards”. This says how potent they are or what percentage of the stated amount they contain and are advertised as a range. Oxoid state that their discs are made to the higher DIN standard (9th January), though admit selling discs above or below DIN (9th January and 15th January) on an unspecified concessionary release.

Oxoid state that all their data on their certificates of analysis that appear online (9 January), but clearly in the case of voriconazole they did not publish the differing chemistry result. The MHRA has not stated whether they think this is acceptable—despite being asked.

Oxoid have agreed that they do not have any internal validation as required by the FDA. In addition, their responses indicate that they do not understand what validation means, often confusing performance (zone sizes against a published range) with assay (measurement of concentration). This confusion has worrying implications about the quality of Oxoid’s discs.

Oxoid’s responses are attached as pdfs. The ‘off the record’ comments have been redacted:

• Thermo Fisher Scientific: Response to Right to Reply email from Deborah Cohen 1 January 2013
• Thermo Fisher Scientific: Responses to Deborah Cohen’s email of January 11 2013

Deborah Cohen is Investigations editor, BMJ