An assessment of nucleic acid amplification testing for active mycobacterial infection

Meta-analysis of studies assessing the diagnostic accuracy of NAAT compared with culture

Forest plots showing the sensitivity and specificity for the 68 studies that compared the diagnostic accuracy of NAAT with culture in patients suspected of having TB are shown in Figure and Figure (Appendix ). Although the sensitivity ranged from 6% to 100%, it was less variable than for AFB microscopy (Appendix ), with only 12/68 (18%) having a sensitivity below 70%. Meta-analysis showed that the overall pooled sensitivity for NAAT compared with culture was 89% (95%CI 85, 92). There were no significant differences in the pooled sensitivities for in-house compared with the commercial NAAT (Xpert) for any subgroup investigated; however, there was a slight trend suggesting that in-house NAATs were more sensitive than the commercial NAAT for most comparisons (Figure ).

Figure Forest plot showing the pooled sensitivity and specificity values for NAAT compared with culture for studies grouped according to the NAAT comparator, specimen type and incidence of TB in the country in which the study was conducted

Incidence of TB based on WHO estimates from 2012: high incidence = > 100 cases per 100,000 people; medium incidence = 10–100 cases per 100,000 people; low incidence = ≤ 10 cases per 100,000 people

K = the number of studies; NAAT = nucleic acid amplification testing; TB = tuberculosis

There was greater variability in the specificity among studies, especially those that were conducted in countries with a high incidence of TB and that used in-house NAAT methodologies (Figure in Appendix ). Meta-analysis of this subgroup showed that the pooled specificity was 79% (95%CI 67, 88), which was significantly lower than for the Xpert NAAT in these countries (96%; 95%CI 92, 96). In fact, the overall pooled specificity for all studies using in-house NAATs (91%; 95%CI 85, 95) was significantly lower than for those using the Xpert NAAT (97%; 95%CI 95, 98). Similar differences in the pooled estimates were seen for both the sputum and non-sputum subgroups, but the difference between in-house and commercial NAATs did not reach statistical significance in the non-sputum subgroup due to the wide CIs (Figure ).

For specific non-sputum specimen types, the pooled specificity ranged from 90% to 97%, except for body fluids (such as synovial fluid and endometrial fluid), which had a pooled sensitivity of 69% (Figure in Appendix ). It should be noted that although the pooled sensitivity of NAAT compared with culture was 97% for CSF specimens (compared with only 11% for AFB microscopy), the wide 95%CI (21, 100) indicated uncertainty in this estimate. As expected, the pooled specificity among different specimen types was much more varied, ranging from 71% for body fluids to 97% for both bronchial specimens and urine specimens.

Overall, there was no observed publication bias based on the effective sample size between studies (p=0.23; Figure ). However, when the studies were separated according to the use of an in-house or commercial NAAT index test, the slope became significant (p=0.05 and 0.02, respectively). There was no observed publication bias for the most variable subgroup as the slope of the regression line for in-house NAAT studies conducted in high TB incidence countries was non-significant (p=0.30).

Figure Deek’s Funnel plot asymmetry test to assess publication bias for the diagnostic accuracy of NAAT compared with culture

Publication bias is assessed visually by using the inverse of the square root of the effective sample size (ESS) versus the log diagnostic odds ratio, which should have a symmetrical funnel shape when publication bias is absent (Light & Pillemer 1984). A regression slope coefficient, weighting by ESS, with p<0.05 indicates significant asymmetry (Deeks, Macaskill & Irwig 2005).

The LR scattergram in Figure shows that the summary LR+ and LR– values for all studies investigating the ability of NAAT to correctly identify patients with and without TB, compared with culture, were mostly within the upper right quadrant of the graph (Figure A). Thus, a positive NAAT result was likely to correctly confirm the presence of MTB (as diagnosed by culture). As the summary estimates for exclusion were within the green shaded area close to the upper left quadrant, NAAT provided strong diagnostic evidence suggesting that patients who tested negative were more likely not to have culture-positive TB than to be falsely negative. Similar results were seen when studies that reported data for either sputum or non-sputum specimens were analysed separately (Figure B and C).

Figure LR scattergram for diagnosis of MTB infection by NAAT compared with culture in studies using either in-house NAAT or commercial Xpert NAAT

LR = likelihood ratio; NAAT = nucleic acid amplification testing

The summary LR estimate for in-house NAATs was close to the border between the upper and lower quadrants or just below it, whereas the summary value for commercial NAAT was clearly in the upper half of the graph. This indicated that a positive NAAT result using commercial NAAT could predict culture-positivity with greater confidence than using an in-house NAAT for all specimen types. The summary estimates were also mostly within the green band in the upper right quadrant of the graph, indicating that although a negative result was likely to indicate a negative culture result, it could not rule out culture-positivity. For sputum specimens the summary estimate for in-house NAAT was just within the upper left quadrant, indicating more confidence in the specimen also being culture-negative.

The SROC curve, which depicts the relative trade-off between true-positive and false-positive results, indicated that NAAT performs well in predicting culture positivity, with an AUC of 0.97 (95%CI 0.95, 0.98) for all specimen types, 0.96 (95%CI 0.94, 0.97) for sputum specimens and 0.89 (95%CI 0.86, 0.91) for non-sputum specimens. The SROC curve showed some threshold effect, suggesting that in-house NAAT was less specific than the commercial Xpert NAAT when compared with culture, especially in countries with a high incidence of TB and when testing non-sputum specimens (Figure ).

Figure SROC curve for all studies investigating the sensitivity and specificity of NAAT versus culture in the diagnosis of TB for studies based on NAAT methodology (A), specimen type (B) and incidence of TB (C)

Incidence of TB based on WHO estimates from 2012: high incidence = > 100 cases per 100,000 people; medium incidence = 10–100 cases per 100,000 people; low incidence = ≤ 10 cases per 100,000 people

AUC = area under curve; NAAT = nucleic acid amplification testing; SROC = summary receiver–operator characteristic; TB = tuberculosis

In summary, the sensitivity and specificity for in-house NAATs and the commercial Xpert NAAT, when compared with culture, did not differ significantly. Nevertheless, the Xpert NAAT showed a trend suggesting that it may be less sensitive than in-house NAATs, especially when testing sputum specimens (83% versus 92%; Figure ). The summary LR values indicate that both in-house NAATs and the commercial Xpert NAAT have diagnostic value in confirming or excluding culture-positive disease. Patients with a positive commercial NAAT result were more likely to be culture-positive than those with a positive in-house NAAT result for all specimen types. Patients with a negative NAAT result in sputum specimens were more likely to be culture-negative than those with a negative Xpert NAAT result.