An assessment of nucleic acid amplification testing for active mycobacterial infection

Appendix Meta-analysis of studies assessing the diagnostic accuracy of AFB compared with culture

Of the 68 studies that compared the diagnostic accuracy of AFB microscopy to culture in patients suspected of having TB, 39 performed AFB microscopy using ZN staining, 23 used fluorescent stains such as auramine, 2 used alternative stains and 3 did not report the method used. Interestingly, 18/24 (75%) studies comparing AFB microscopy and the Xpert assay used fluorescent staining, whereas 34/44 (77%) of studies using in-house NAAT methods used ZN staining. Forest plots showing the sensitivity and specificity for these studies are shown in Figure and Figure (Appendix ). The sensitivity varied greatly between studies, ranging from 5% to 100% with a pooled sensitivity of 62% (95%CI 54, 69). There was less variability in the specificity, which was above 80% in all but 3 studies, with a pooled value of 98% (95%CI 97, 99). The proportion of culture-positive specimens that were AFB-positive is higher in these studies than that reported in the Tuberculosis notifications in Australia, 2010 Annual Report ³⁰, which reported that, of all MTB cases confirmed by culture, only 47% were AFB-positive.

Subgroup analysis was undertaken to determine the effects of AFB methodology, specimen type, incidence of TB in the country in which the study was conducted, and use of in-house or commercial NAAT index test on the accuracy of AFB microscopy (Figure ). There was a significant difference in sensitivity between studies investigating diagnostic accuracy in patients who provided sputum samples (71%; 95%CI 59, 81) compared with those that provided non-sputum samples (46%; 95%CI 37, 55), as the 95%CIs did not overlap. Non-sputum specimens included patients suspected of having either pulmonary TB (e.g. bronchial aspirates) or extrapulmonary TB (e.g. synovial fluid or tissue biopsy). Analysis of extrapulmonary specimens alone showed that the sensitivity and specificity of AFB compared with culture did not differ markedly from those for non-sputum samples (Table in Appendix ). For some specific specimen types there were sufficient studies for separate analysis (Figure in Appendix ). The pooled sensitivity for AFB microscopy compared with culture varied from 46% in urine to 62% in FNAs of lymph nodes. However, for CSF the pooled sensitivity was only 11%. Thus, AFB microscopy is not a useful tool for diagnosis of TB in CSF specimens. The pooled specificity was at least 94% in all specimen types.

There was an overall 11% difference in sensitivity of AFB microscopy compared with culture, favouring studies that used an in-house NAAT over those that used the commercial Xpert NAAT, which was not statistically significant. However, this difference was entirely due to the type of specimen tested. In studies that used sputum samples, AFB microscopy was 24% more sensitive compared with culture when an in-house NAAT was used as the index test instead of a commercial NAAT. Conversely, there was no difference in sensitivity in studies that used non-sputum samples (Figure ).

The reason for this is unclear, although there is likely to be some publication bias, as indicated by the significant asymmetry when comparing the effective sample size between studies (Figure ). This asymmetry was no longer significant (p>0.05) when the studies were separated according to AFB methodology, NAAT methodology or specimen type (data not shown). Other variables that may influence publication bias include funding, conflict of interest, prejudice against an observed association and sponsorship, but the effects of these parameters were not tested.

Figure Forest plot showing the pooled sensitivity and specificity values for AFB microscopy compared with culture for studies grouped according to NAAT comparator, AFB methodology and incidence of TB in the country in which the study was conducted

Incidence of TB based on WHO estimates from 2012: high incidence = > 100 cases per 100,000 people; medium incidence = 10–100 cases per 100,000 people; low incidence = ≤ 10 cases per 100,000 people

FL = fluorescent staining; K = the number of studies; NAAT = nucleic acid amplification testing; TB = tuberculosis; ZN = Ziehl-Neelsen staining

Figure Deek’s Funnel plot asymmetry test to assess publication bias for the diagnostic accuracy of AFB microscopy compared with culture

Publication bias is assessed visually by using the inverse of the square root of the effective sample size versus the diagnostic log odds ratio, which should have a symmetrical funnel shape when publication bias is absent (Light & Pillemer 1984). A regression slope coefficient, weighting by ESS, with p<0.05 indicates significant asymmetry (Deeks, Macaskill & Irwig 2005).

There was little to no difference in sensitivity and specificity between studies conducted in high-TB-incidence countries compared with low-incidence countries. The anomaly seen for medium-incidence countries was likely due to chance, given the variability between studies, as seen in Figure and Figure (Appendix ).

LR scattergrams plot LR+ against LR– where the likelihood of correctly identifying patients with MTB infections (as diagnosed by culture) increases along the x-axis and the likelihood of correctly eliminating the presence of MTB decreases along the y-axis. The summary LR+ and LR– values for studies investigating the ability of AFB microscopy to correctly diagnose patients with or without TB, compared with culture, were within the upper right quadrant of the graph (Figure ). This quadrant represents LR+ and LR– values that suggest that AFB microscopy is likely to correctly confirm the presence of MTB, but a negative test result does not eliminate the likelihood of a positive culture result in patients suspected of having TB. The observed difference in sensitivity of AFB microscopy compared with culture in sputum and non-sputum specimens did not affect the clinical utility of the AFB test. The LRs for both sputum (LR+ 27.0 [95% CI 15.9, 45.6]; LR– 0.29 [95%CI 0.20, 0.43] and non-sputum (LR+ 23.3 [95%CI 13.7, 39.7]; LR– 0.55 [95%CI 0.47, 0.65] specimens were also in the same upper right quadrant. Thus, AFB microscopy is useful for those patients with a positive AFB test result as it identifies those patients as having TB and requiring immediate treatment. However, the clinician gains no further knowledge if a patient has a negative AFB test result, as this patient may still have TB.

Figure LR scattergram for diagnosis of MTB infection by AFB microscopy compared with culture in studies using in-house NAAT or the Xpert NAAT

LR = likelihood ratio; NAAT = nucleic acid amplification testing

The SROC curve, which depicts the relative trade-off between true-positive and false-positive results, indicated that AFB microscopy performs well in predicting culture positivity, with an AUC of 0.94 (95%CI 0.92, 0.96). There was no threshold effect based on the AFB staining methodology, suggesting that it does not impact on the sensitivity or specificity of AFB microscopy when compared with culture (Figure ). This lack of threshold effect suggests that the observed differences in sensitivity between studies using in-house NAATs (which favoured ZN staining) and commercial NAATs (which favoured fluorescent staining) were due to other differences that have not been identified. However, there was a threshold effect based on specimen type, with sensitivity being higher when sputum specimens were tested (Figure ).

Figure SROC curve for all studies investigating the sensitivity and specificity of AFB microscopy versus culture in the diagnosis of TB

AFB = acid-fast bacilli; AUC = area under curve; FL = fluorescent staining; SROC = summary receiver–operator characteristic; NAAT = nucleic acid amplification testing; ZN = Ziehl-Neelsen staining