Proficiency Testing of Viral Marker Screening in African Blood Centers — Seven African Countries, 2017

A 2014 report evaluating accuracy of serologic testing for transfusion-transmissible viruses at African blood center laboratories found sensitivities of 92%, 87%, and 90% for detecting infections with human immunodeficiency virus (HIV), hepatitis B virus (HBV), and hepatitis C virus (HCV), respectively (1). Following substantial investments in national blood transfusion service (NBTS) laboratories, in 2017 investigators tested proficiency at 84 blood center laboratories (29 NBTS and 55 non-NBTS) in seven African countries. A blinded panel of 25 plasma samples was shipped to each participating laboratory for testing with their usual protocols based on rapid diagnostic tests (RDTs) (2) and third and fourth generation enzyme immunoassays (EIA-3 and EIA-4). Sensitivity and specificity were estimated using separate regression models that clustered assays by laboratory and adjusted for assay type and NBTS laboratory status. Mean specificities were ≥95% for all three viruses; however, mean sensitivities were 97% for HIV-positive, 76% for HBV-positive, and 80% for HCV-positive samples. Testing sensitivities for all viruses were high when EIA-3 assays were used (≥97%). Lower sensitivities for HBV-positive samples and HCV-positive samples were associated with assay types other than EIA-3, used primarily by non-NBTS laboratories. Proficiency for HIV testing has improved following international investments, but proficiency remains suboptimal for HBV and HCV testing. In sub-Saharan African blood centers, the quality of rapid tests used for HBV and HCV screening needs to be improved or their use discouraged in favor of EIA-3 tests.

Panels of 25 challenge specimens were prepared and characterized by the Institut National de la Transfusion Sanguine (Paris, France). Each panel included seven negative controls; seven specimens that contained HIV antigen and anti-HIV antibody (six HIV-1 and one HIV-2) (HIV-positive samples); six specimens containing hepatitis B surface antigen (confirmed by neutralization assay and quantified) (HBV-positive samples); and five specimens that contained HCV RNA and anti-HCV antibody (HCV-positive samples). All positive challenge specimens included viral genotypes that were specific to Africa. Plasma specimens were diluted with uninfected plasma to obtain specific antigen or antibody concentrations. The panels were confirmed to match their labels (Supplementary Table, https://stacks.cdc.gov/view/cdc/82012) at the Institut National de la Transfusion Sanguine, coded to allow for blinded testing, and sent to national coordinators who distributed them to participating laboratories while maintaining the cold chain.
Laboratories tested each challenge specimen in the panel using three assays, each designed to detect infection with HIV, HBV, or HCV, and reported findings for each assay. The primary study outcome was classification of each assay finding as correct or incorrect relative to each specimen's true infection status; classification was done at the unblinded data analysis center. Sensitivity (correct detection of infection-positive status whether by antibody, antigen, or RNA) was estimated using approximately 25% of specimens for which the challenge virus matched the assay virus (seven HIV, six HBV, and five HCV), and specificity (correct detection of infection-negative status) was estimated using approximately 75% of specimens for which the challenge virus (or control) did not match the assay virus (18 HIV,19 HBV,and 20 HCV).
The investigators used separate generalized estimating equation logit-binomial models to estimate mean sensitivity and specificity and 95% confidence intervals (CIs), each as a function of the three assay viruses (HIV, HBV, and HCV), clustering outcomes within laboratories. Multivariable models added NBTS status, assay type (RDT,, and all two-way interaction terms to the unadjusted model. The unadjusted model of specificity also included the identity of the challenge virus. All analyses were performed using SAS software (version 9.4; SAS Institute).

Proficiency Testing
Laboratory characteristics. Among the seven countries, the number of participating laboratories ranged from one (Rwanda) to 20 (Nigeria), and the proportion that were NBTS laboratories ranged from 9% (Malawi and Mozambique) to 100% (Rwanda) ( Table 1). Five non-NBTS laboratories (two each in Tanzania and Ghana and one in Kenya) did not participate, citing lack of reagents as the reason. Of 84 participating laboratories, 70 provided 100% of findings (25 specimens × three assays per laboratory), eight provided 93%, and six (all non-NBTS) provided 46%.
Sensitivity. Unadjusted mean sensitivity for detecting HIV-positivity was 97% (95% CI = 95%-98%); for detecting HBV-positivity was 76% (95% CI = 71%-81%); and for detecting HCV-positivity was 80% (95% CI = 75%-86%) ( Table 2). Sensitivity exceeded 90% for HIV-positive detection in all seven countries; however, this level of sensitivity for identifying HBV-positive specimens was reached only in Kenya and Rwanda, and for HCV-positive specimens, only in Kenya, Mozambique, and Rwanda (p<0.001). At NBTS laboratories, all three assays' sensitivities to their respective target viruses exceeded 92%; however, at non-NBTS laboratories, sensitivity to HBV-positive was 66% and to HCV-positive was 74% (p<0.001). Statistically significantly higher levels of testing sensitivity were observed in laboratories that tested more blood donations per year (p = 0.006), produced more components per year (p = 0.026), and had higher percentages of donors who were volunteers (p = 0.013). Testing sensitivity was not associated with the number of laboratory personnel.
Based on the multivariable model, adjusted sensitivities uniformly exceeded 96% when EIA-3 was used; however, the sensitivity of EIA-4 to detect HCV-positivity was <85%, and RDT assay sensitivities to detect HBV-and HCV-positivity were <71%. Sensitivity for detecting HIV-positivity was ≥95% regardless of laboratory or assay type. Sensitivity varied significantly among assay types (p = 0.011) but not among assay target viruses (p = 0.30) or between NBTS laboratory status (p = 0.81), and none of the three pairwise interaction effects was statistically significant (p≥0.25). These findings are reflected by observed sensitivity proportions ( Figure) that show that EIA-3 assays performed equally well or better than others for detecting HIV-, HBV-, and HCV-positivity, regardless of NBTS status.
Specificity. Unadjusted mean testing specificity was 95% (95% CI = 93%-97%) for HIV-negative specimens, 96% (95% CI = 93%-98%) for HBV-negative specimens, and 95% (90%-98%) for HCV-negative specimens. Across all assay target viruses, mean specificity was 90%-92% in three countries (Malawi, Mozambique, and Tanzania) and ≥98% in the other four countries.  Abbreviations: CI = confidence interval; EIA-3 = third generation enzyme immunoassay; EIA-4 = fourth generation enzyme immunoassay; NBTS = national blood transfusion service. * Based on univariate models. † Because HBV-and HCV-positive specimens were not assayed by three and five laboratories, respectively, sensitivity evaluations for assay targets HIV, HBV, and HCV were based on 84, 81, and 79 laboratories, respectively. § P-values report statistical significance of associations of sensitivity with the interaction between assay virus and laboratory characteristics. ¶ Model excluded Rwanda and excluded the interaction term. P-value reports statistical significance of association of sensitivity with country. ** Characteristic was analyzed on the log-10 scale. Mean sensitivity was estimated at the values shown.
Adjusted estimates based on the multivariable model showed that the targeted assays varied in specificity by assay type (p = 0.054) and interaction with NBTS status (p = 0.058). Specificity was relatively low at non-NBTS laboratories for RDT assays targeting HCV or HIV and at NBTS laboratories for EIA-4 assays targeting HIV (Figure).

Summary
What is already known about this topic? Substantial international investments have been made in African national blood transfusion services (NBTS) following reports of deficiencies in viral marker screening at African blood center laboratories.
What is added by this report?
Standardized proficiency testing conducted in seven African countries during 2017 found that proficiency in human immunodeficiency virus testing has improved, but testing proficiency for hepatitis B virus (HBV) and hepatitis C virus (HCV) needs to be improved.
What are the implications for public health practice?
Most poor performance in hepatitis virus testing can be attributed to the use of rapid tests rather than the non-NBTS setting of the laboratories. Remediation should be focused on improving the quality of rapid tests or avoiding their use.

Discussion
This investigation of testing proficiency of targeted assays for HIV, HBV, and HCV found specificities to be high overall, with clinically negligible variations by NBTS status or assay type. In contrast, clinically important variation in sensitivities within and between assay targets was found. The finding that non-EIA-3 tests had lower sensitivity than did other assay types for detecting HBV-and HCV-positive specimens but not HIV-positive specimens is consistent with findings from previous studies (1)(2)(3)(4). As noted, variation in testing proficiency for sensitivity among countries primarily reflects variation among assay types rather than between NBTS and non-NBTS laboratories.
This study found higher sensitivity for detecting HIVpositivity but lower sensitivity for detecting HBV-and HCVpositivity than is generally associated with the use of RDTs, compared with previous studies using similar methods (1,2). These results suggest that RDT assays targeting HIV perform better or have better quality assurance than do RDT assays targeting the hepatitis viruses. The poorer performance of RDT assays for detecting HBV-and HCV-positivity is most likely attributable to the quality of the assays themselves, because deficiency in performing the tests could have been signaled by lower mean accuracy at non-NBTS compared with NBTS laboratories. Of note, lower sensitivity to HCV-positivity using the EIA-4 was limited to a single reputable assay, suggesting a need to rule out poor technical performance or recording errors. After all laboratories had completed testing and the CDC International Laboratory Branch had evaluated the results, it conducted site visits at low-performing laboratories and developed recommendations for remediation.  The findings in this report are subject to at least four limitations. First, the numbers of positive-challenge specimens per assay target virus were small, which resulted in few response levels for sensitivity estimations. Second, the positive samples were diluted to approximate difficult samples, but this limits extrapolation of operational sensitivity. Third, the investigators attempted to overcome sampling bias by using a random sample of non-NBTS laboratories; however, five of these laboratories failed to participate in the study, and six others submitted incomplete data, which suggests problems with their supplies of assay kits. Finally, the unanticipated strong association of assay type with NBTS status and few NBTS laboratories per country precluded fully distinguishing the effects of assay type, NBTS status, and country.

RDT
Variation in blood center laboratory proficiency among sub-Saharan African countries has been reported previously and likely relates to both assay quality, representing a range of manufacturers, and organizational structures, resources, and training of technicians (5)(6)(7). Future studies of testing proficiency could be designed to study manufacturers in addition to assay type, with the aim of identifying products that perform poorly. Alternatively, future study protocols could provide high-accuracy assay kits targeting HIV, HBV, and HCV to better distinguish between assay quality and operator error.
To ensure that transfusion-transmitted viruses in donated blood are detected, the use of rapid diagnostic tests for HBV and HCV should be discouraged because of the general suboptimal performance of these assays. Where possible, scarce blood center resources should be allocated to enable all blood center laboratories to use EIA-based assays from selected manufacturers, improve the reliability of supply chains and implement standard quality assurance protocols for conducting the assays, and require technical staff members to participate in testing-proficiency training programs. However, quality improvements might be difficult to sustain if African national budgets are not supplemented by international funding (8).