NIOSH Conformity Assessment Notice

Subject: Intra-Laboratory Comparison of LabVIEW™ Data Acquisition Software to replace use of the Gould RS3200 Strip Chart Recorder

NIOSH CA 2018-1002
June 2018

1 – Summary

The National Personal Protective Technology Laboratory (NPPTL), a Division of the National Institute for Occupational Safety and Health (NIOSH), is responsible for the approval of respiratory protective devices in accordance with the regulations contained in Title 42 of the Code of Federal Regulations, Part 84 (42 CFR 84).

For more than 15 years, NPPTL has tested Respiratory Protective Devices (RPDs) using the Gould Instrument Systems, Inc. model RS3200 strip chart recorder (with carrier amplifier). Gould Instrument Systems, Inc. is no longer in business and replacement parts for the RS3200 are no longer available. This warranted the development of a new test system to replace the Gould RS3200 strip chart recorder.

To address this concern, NPPTL, under a contract with Data Science Automation (DSA), developed a personal computer (PC) based system running Microsoft Windows and using National Instruments’ LabVIEW™ data acquisition software. Data acquisition and carrier amplifier hardware were also incorporated into the new system.

NPPTL’s laboratories in Pittsburgh, Pennsylvania and Morgantown, West Virginia are equipped with the new LabVIEW™ based system. Only the Pittsburgh laboratory has the Gould RS3200 system. To validate the LabVIEW™ based system, testing was conducted in Pittsburgh and Morgantown using NIOSH-approved self-contained breathing apparatus (SCBAs). All units were tested using the Gould system in Pittsburgh as well as the LabVIEW™ systems in Pittsburgh and Morgantown. A statistician at NPPTL analyzed the results.

Results show that the LabVIEW™ strip chart system is capable of replacing the Gould strip chart recorder. Additional benefits of the LabVIEW™ system include automatic data logging and a reduction of the subjectivity associated with determining pass/fail results (using the strip recorder).

2 – Authority

42 C.F.R. Part 84, Respiratory Protective Devicesexternal icon

Subpart H, Self-contained breathing apparatus

Eight NIOSH Standard Test Procedures utilize the Gould RS3200 strip chart recorder:

  • RCT-APR-STP-0065, Determination of Airflow Resistance of Breath Responsive, Powered Air-Purifying Respirators (PAPRs)
  • RCT-ASR-STP-0105A, Determination of Airflow-Demand and Pressure-Demand, Type C and CE, Supplied-Air Respirators
  • RCT-ASR-STP-0106, Determination of Inhalation Airflow Resistance, Pressure-Demand, Type C and CE, Supplied-Air Respirators
  • RCT-ASR-STP-0117, Determination of Positive Pressure, Closed-Circuit, Pressure-Demand, Self-Contained Breathing Apparatus
  • RCT-ASR-STP-0120, Determination of Positive Pressure, Open-Circuit, Pressure-Demand, Self-Contained Breathing Apparatus
  • RCT-ASR-STP-0121, Determination of Rated Service Time, Open-Circuit, Demand and Pressure-Demand, Self-Contained Breathing Apparatus
  • RCT-ASR-STP-0135, Determination of Inhalation and Exhalation Breathing Resistance, Closed-Circuit, Demand and Pressure-Demand, Self-Contained Breathing Apparatus
  • RCT-ASR-STP-0138, Determination of Safety Relief Valve Operation, Closed-Circuit, Demand and Pressure-Demand, Self-Contained Breathing Apparatus

3 – Background and Supplemental Information – Intra-Laboratory Comparison

Gould Instrument Systems, Inc. is no longer in business, which prompted the question “What do we do when our RS3200 strip recorder stops working?” To address this concern, as well as NPPTL’s goal of updating test procedures to take advantage of new or emerging technology, NPPTL began development of a new test system that could replace the Gould test system. NPPTL awarded a contract to Data Science Automation (DSA) for development of a PC based system using National Instrument’s LabVIEW™ data acquisition software and LabVIEW™ compatible data acquisition hardware. Additional carrier amplifier hardware was also necessary. This new system was developed for a Microsoft Windows operating system.

Upon completion of the system development, DSA delivered the LabVIEW™ test setup to NPPTL. The system was initially setup at NPPTL’s Pittsburgh laboratory. To verify the LabVIEW™ system produced statistically similar results to the Gould system, testing was conducted using two readily available Self-Contained Breathing Apparatus (SCBAs), a Scott Air Pak 75 SCBA with a 60-minute cylinder and an MSA Firehawk® M7 SCBA with a 30 or 45-minute cylinder. Both units met NFPA 1981-2007. The units were tested on the Gould RS3200 system and the LabVIEW™ based system. Figures 1a and 1b show the corresponding test setup.

Flow chart - Figure 1a: NPPTL Gould Strip Chart Setup

Figure 1a: NPPTL Gould Strip Chart Setup

 

Flow chart - Figure 1b: NPPTL LabVIEW™ Strip Chart Setup

Figure 1b: NPPTL LabVIEW™ Strip Chart Setup

The units were tested under baseline conditions as well as under a condition where the zero point was adjusted to approximately +0.1 to +0.2 inches of water. This condition caused negative spikes to be displayed on the Gould RS3200 that had a minimum detectable area below the zero line. This is critical for accepting a negative spike on the Gould system at a speed of 20 mm/sec.

During the testing, the LabVIEW™ strip chart recorder display was adjusted to match the Gould speed. It was verified that the minimum detectable negative spikes produced on the Gould recorder were logged as negative spikes that met the criteria for being reported and logged as failures. The comparison verified that the minimum and maximum pressures recorded during the testing matched between the Gould recorder and the LabVIEW™ recorder at a tolerance of +/- 0.05 inches of water. The shapes of the breathing curves were also subjectively evaluated to ensure a match between the two methods. After this preliminary work was completed, several additional tests were performed on other approved makes and models of SCBAs.

All of the aforementioned tests were performed by running the Gould system and the LabVIEW™ system simultaneously. Results from the two recordings were compared for minimum and maximum pressures, and both measurements were within +/- 0.05 inches of water.

After the Gould system and LabVIEW™ system were validated at the NPPTL Pittsburgh laboratory, it was time to collect data at the NPPTL Morgantown laboratory. A second LabVIEW™ system, as shown in Figure 1b, was setup in the Morgantown laboratory. A detailed test plan was written which included the Gould system in Pittsburgh, the LabVIEW™ system in Pittsburgh, and the LabVIEW™ ™ system in Morgantown.

The test plan included eight SCBAs. Again, these SCBAs were chosen as they were readily available. The SCBAs chosen included Sperian-Honeywell (1 unit), Dräger (1 unit), Avon-ISI (1 unit), Interspiro (1 unit), Scott Health and Safety (3 units), and MSA (1 units). One of the Scott units was adjusted to allow the waveform to go slightly negative.

The test plan that was developed was a full comparison of the Pittsburgh laboratory to the Morgantown laboratory. Therefore, the test plan included all five of the current SCBA STPs (0120, 0121, 0122, 0123, and 0124). However, the information provided in this notice is focused on the comparison of the Gould system to the LabVIEW™ system. Therefore, the data presented is from STP-0120.

Each unit was tested three times. Each unit completed all of the tests at one laboratory before being moved to the other laboratory. All of the cylinders were charged using identical settings. The facepiece seal to the headform was tested by utilizing a vacuum delay method. The facepiece negative pressure was lowered, by vacuum, to below -1.0 inches of water. The time of the vacuum delay from -1.0 to -0.8 inches of water had to be less than 15.0 seconds. All tests were completed in the same order at both locations.

Table 1 shows the test results for the Positive Pressure Tests (STP-0120) for each unit tested. As mentioned previously, each unit was tested three times on each system. It is also worth mentioning again that the units were tested simultaneously on the Pittsburgh LabVIEW™ and Pittsburgh Gould systems. The data is shown by unit with the results of each run shown under each system.

The “Gould Difference” column shows the difference between the Pittsburgh systems only (Pittsburgh LabVIEW™ – Pittsburgh Gould). The “LabVIEW™ Difference” column shows the difference between the LabVIEW™ systems only (Morgantown LabVIEW™ – Pittsburgh LabVIEW™).

Table 1: Positive Pressure Test (STP-0120) Data (inches of water)

Positive Pressure Test: RCT-ASR-STP-0120 (results are inches of water)
Positive Pressure Test: RCT-ASR-STP-0120 (results are inches of water)
Sperian-Honeywell
Run Pittsburgh
Gould         LabVIEW™
Morgantown
LabVIEW™
Gould Difference
LabVIEW™ – Gould
LabVIEW™ Difference
Morgantown – Pittsburgh
1 0.15                     0.10 0.20 -0.05 0.10
2 0.15                     0.10 0.20 -0.05 0.10
3 0.15                     0.10 0.20 -0.05 0.10

 

Dräger
Run Pittsburgh
Gould         LabVIEW™
Morgantown
LabVIEW™
Gould Difference
LabVIEW™ – Gould
LabVIEW™ Difference
Morgantown – Pittsburgh
1 0.15                     0.10 0.10 -0.05 0.00
2 0.10                     0.10 0.05 0.00 -0.05
3 0.05                     0.10 0.10 0.05 0.00

 

Avon-ISI
Run Pittsburgh
Gould         LabVIEW™
Morgantown
LabVIEW™
Gould Difference
LabVIEW™ – Gould
LabVIEW™ Difference
Morgantown – Pittsburgh
1 0.50                     0.50 0.50 0.00 0.00
2 0.50                     0.50 0.50 0.00 0.00
3 0.50                     0.50 0.50 0.00 0.00

 

Interspiro
Run Pittsburgh
Gould         LabVIEW™
Morgantown
LabVIEW™
Gould Difference
LabVIEW™ – Gould
LabVIEW™ Difference
Morgantown – Pittsburgh
1 0.40                     0.35 0.40 -0.05 0.05
2 0.40                     0.40 0.40 0.00 0.00
3 0.35                     0.35 0.35 0.00 0.00

 

Scott NXG
Run Pittsburgh
Gould         LabVIEW™
Morgantown
LabVIEW™
Gould Difference
LabVIEW™ – Gould
LabVIEW™ Difference
Morgantown – Pittsburgh
1 0.25                     0.25 0.25 0.00 0.00
2 0.20                     0.20 0.25 0.00 0.05
3 0.20                     0.20 0.20 0.00 0.00

 

Scott 2.2
Run Pittsburgh
Gould         LabVIEW™
Morgantown
LabVIEW™
Gould Difference
LabVIEW™ – Gould
LabVIEW™ Difference
Morgantown – Pittsburgh
1 0.10                     0.10 0.10 0.00 0.00
2 0.05                     0.05 0.05 0.00 0.00
3 0.05                     0.05 0.05 0.00 0.00

 

Scott 4.5
Run Pittsburgh
Gould         LabVIEW™
Morgantown
LabVIEW™
Gould Difference
LabVIEW™ – Gould
LabVIEW™ Difference
Morgantown – Pittsburgh
1 0.00                     -0.05 -0.10 -0.05 -0.05
2 -0.20                     -0.20 -0.15 0.00 0.05
3 -0.25                     -0.25 -0.10 0.00 0.15

 

MSA
Run Pittsburgh
Gould         LabVIEW™
Morgantown
LabVIEW™
Gould Difference
LabVIEW™ – Gould
LabVIEW™ Difference
Morgantown – Pittsburgh
1 0.10                     0.10 0.10 0.00 0.00
2 0.05                     0.05 0.10 0.00 0.05
3 0.05                     0.05 0.10 0.00 0.05

A statistician at NPPTL analyzed all data. Since each system was tested three times in Morgantown and three times in Pittsburgh, this type of data is referred to as “paired replicates” data. The difference within each run was used for statistical analysis. For comparing the LabVIEW™ data, the difference measure (Morgantown LabVIEW™ Data – Pittsburgh LabVIEW™ Data) was used for statistical testing. To compare the Gould Data, the difference measure (Pittsburgh LabVIEW™ Data – Pittsburgh Gould Data) was used for statistical testing.

Because of the small sample sizes (n=3) for each unit, the Wilcoxon Signed Rank (WSR) test was used to test if the difference was zero (p-value is shown in the tables). The WSR test is preferable to a parametric test (i.e. Student’s t-test with degrees of freedom = 2) when the sample size is small, because the WSR uses exact probabilities and does not require a random sample. Note that the smallest possible p-value for this test (with n=3) is 0.25.

There are four possible p-values for the WSR when the sample size is three: 0.25, 0.50, 0.75, or 1.0. There can be no differences with a traditionally statistically significant test result (with α=0.05) and if one or more of the differences are the same (tied observations) the p-value cannot be calculated. However, p-values with the smallest value of 0.25 should be considered as possibly significant differences. The estimate of the difference is a non-parametric location estimator known as the pseudo-median or Hodges-Lehmann estimator calculated for each unit separately and for all the data combined.

For all practical purposes, the estimate of the difference, whether nearly statistically significant or not, may be too small to represent any important difference or may be within the measurement error of the laboratory. The estimates of the difference and tests of significance can only provide information to aid the determination.

The Kruskal-Wallis test can be used to test if the systems are different. To determine which of the systems are different from each other, typically the confidence intervals of the differences can be compared. Again, with the small sample size of n=3, the WSR test provides (if there are no ties in the data) a 75% confidence interval instead of the more familiar 95% confidence interval. In fact, given only three observations, the 75% confidence interval is the same as the range of a sample size of 3, so the minimum and maximum of the sample are the same as the 75% confidence limits. This way, there is some indication if a particular system tends to have higher measurements in one particular location compared to the other systems.

The WSR test on the differences, using all the data for the systems combined, tests if the Morgantown laboratory values are different from the Pittsburgh laboratory values. It also estimates the magnitude of the difference. This statistic is reported at the bottom of the tables for each test. It is also possible to compare the number of positive differences to the number of negative differences to determine if there is a tendency for measurements to be higher in one location or the other. This is known as a binomial test or sign test. This result should generally agree with the WSR test; but is generally less efficient than the WSR test.

Table 2 shows the statistical analysis for the Pittsburgh data only (LabVIEW™ vs Gould). Using the WSR test, the results indicate there is no difference between the LabVIEW™ data and the Gould data (p=0.1094). The WSR test on the median estimate for the combined data (all units) is 0.00, which indicates that the LabVIEW™ data is not statistically different from the Gould data.

Table 2: Positive Pressure Test (STP-0120) Statistical Analysis (LabVIEW™ vs Gould)

Table 2: Positive Pressure Test (STP-0120) Statistical Analysis (LabVIEW™ vs Gould)
Unit p-value Median Estimate C.I. or Min, Max
Sperian-Honeywell 0.2500 -0.0500 -0.050, -0.050
Dräger 1.0000 0.0000 -0.050, 0.050
Avon-ISI NA 0.0000 0.000, 0.000
Interspiro 1.0000 -0.0125 -0.050, 0.000
Scott NXG NA 0.0000 0.000, 0.000
Scott 2.2 NA 0.0000 0.000, 0.000
Scott 4.5 1.0000 -0.0125 -0.050, 0.000
MSA NA 0.0000 0.000, 0.000
Wilcoxon Signed Rank Test 0.1094 0.0000 95% C.I. = -0.050, 0.050
Sign Test (exact Binomial) 0.1300
Kruskal-Wallis Test 0.2100

Out of the 24 difference estimates (LabVIEW™ – Gould) there were six negative values, 17 zero values, and one positive value suggesting that there is a tendency for higher measurements with the Gould system.

Examining each of the eight systems separately, the median estimate of the difference was zero (indicating no difference between the LabVIEW™ data and Gould data) for five systems (Dräger, Avon-ISI, Scott NXG, Scott 2.2, and MSA). The median estimate of the difference was negative (indicating higher results for the Gould recorder) for three of the systems (Sperian-Honeywell, Interspiro, and Scott 4.5). In order, from smallest to largest the system estimates are -0.05, -0.0125, -0.0125, 0, 0, 0, 0, and 0 for the respective systems: Sperian-Honeywell, Interspiro, Scott 4.5, Avon-ISI, Scott NXG, Scott 2.2, MSA. and Dräger.

A Kruskal-Wallis test comparing the difference between the LabVIEW™ data and the Gould data indicates there is no statistically significant difference between the two systems (p=0.2100).

Table 3 shows the statistical analysis for the LabVIEW™ data only (Morgantown vs Pittsburgh). Using the WSR test, the results indicate that the Morgantown results are higher than the Pittsburgh results (p=0.0224). The WSR test on the median estimate for the combined data (all units) is 0.0250, which indicates that the Morgantown measurements are higher than the Pittsburgh measurements.

Table 3: Positive Pressure Test (STP-0120) Statistical Analysis (LabVIEW™ Only)

Table 3: Positive Pressure Test (STP-0120) Statistical Analysis (LabVIEW™ Only)
Unit p-value Median Estimate C.I. or Min, Max
Sperian-Honeywell 0.2500 0.1000 0.100, 0.100
Dräger 1.0000 -0.0125 -0.050, 0.000
Avon-ISI NA 0.0000 0.000, 0.000
Interspiro 1.0000 0.0125 0.000, 0.050
Scott NXG 1.0000 0.0125 0.000, 0.050
Scott 2.2 NA 0.0000 0.000, 0.000
Scott 4.5 0.5000 0.0500 -0.050, 0.150
MSA 0.5000 0.0375 0.000, 0.050
Wilcoxon Signed Rank Test 0.0224 0.0250 95% C.I. = -0.000, 0.010
Sign Test (exact Binomial) 0.0654
Kruskal-Wallis Test 0.1102

Out of the 24 difference estimates (Morgantown – Pittsburgh) there were two negative values, 13 zero values, and nine positive values suggesting that there is a tendency for higher measurements in Morgantown.

Examining each of the eight systems separately, the median estimate of the difference was negative (which would indicate higher results for Pittsburgh) for only one of the systems (Dräger), for which two of the three data points were zero. The median estimate was zero (indicating no difference between Morgantown and Pittsburgh estimates) for two of the systems (Avon-ISI and Scott 2.2) for which all three data points were zero. The median estimate of the difference was positive (indicating higher results for Morgantown) for five of the systems (Sperian-Honeywell, Interspiro, Scott NXG, Scott 4.5, and MSA).

A Kruskal-Wallis test comparing the difference between Morgantown and Pittsburgh indicates there is no statistically significant difference among the systems (p=0.1102).

To address the need to potentially replace a piece of aging test equipment (Gould RS3200) that can no longer be purchased or repaired, NPPTL had a PC based LabVIEW™ program developed. During this time, NPPTL also setup a new laboratory in Morgantown, WV. Eight SCBAs were used to conduct a correlation study between the LabVIEW™ system and the Gould system, as well as the LabVIEW™ systems in Pittsburgh and Morgantown. Statistical analysis of the data indicates that there is no statistically significant difference (p=0.2100) in the data comparing the LabVIEW™ system to the Gould system in Pittsburgh. Likewise, there was no statistically significant difference (p=0.1102) in the data comparing the LabVIEW™ system in Pittsburgh to the LabVIEW™ system in Morgantown.

4 – References

Approval of Respiratory Protective Devices, 42 C.F.R, Part 84external icon

December 7, 2017 NIOSH NPPTL Meeting for All Respirator Manufacturers

Page last reviewed: June 12, 2018